Reading through PDF2.0 while building a PDF by hand

1 Getting the document

In order to get the document, I go to order page and go through the steps. They request my information but I fill in some bogus data, no need to give my real info for a free document.

2 Navigating the document

I opened the document and was greeted by 1003 pages.

Since reading this thing is not an option, I went to look for something that could help me writing a minimal PDF-document.

In the document outline I notice 7.5 File structure. Aha!

Figure 2 on page 54 shows us the basic structure. We need 4 parts in our PDF-document: a header, a body, a cross-reference table and a trailer. The first two seem obvious. The cross-reference table gives me a vague idea of what it is but I am not sure. No idea about the trailer. Is that a reverse header?

I see the headers in chapter 7.5.2 File header, and choose %PDF-2.0 since the document also refers so version 2.0.

7.5.3 File Body does not give much info on the body, so lets just skip that for now. Onto the cross-reference table— which takes 3 pages. No, lets look at the trailer first!

Ah, that is why PDF needs a trailer, nice to know!

%PDF-2.0
BODY SHOULD COME HERE
CROSS REFERENCE TABLE SHOULD COME HERE
trailer
KEY/VALUE-PAIRS SHOULD COME HERE
startxref
BYTE OFFSET SHOULD COME HERE
%%EOF

I still do not know what the body, cross-reference table, key-value pairs and the byte-offset are...

3 The body

The body consists of objects, so I take a look at 7.3 Objects.

Strings and streams sound interesting for pieces of text. Maybe we can output Hello world in our PDF!

I scroll to 7.3.4.2 Literal strings, which seems to be what we need!

I add (Hello world!) into the document

%PDF-2.0
(Hello world!)
CROSS REFERENCE TABLE SHOULD COME HERE
trailer
KEY/VALUE-PAIRS SHOULD COME HERE
startxref
BYTE OFFSET SHOULD COME HERE
%%EOF

4 The cross-reference table

At this point I want to know what the minimal table should be. I mean, we already have the text in the body, right?

Why is there no basic example in the chapter for the document structure? Maybe in the Annexes!

I go to H.3 Basic text string example, and... that is not minimal at all! How is it possible that so much is needed for some basic text?

Table H.2 reveals why: fonts, document catalog, page thingies,... Is it really impossible to show text without all that?

I go back to H.2 Minimal PDF file to see what their definition of minimal is.

So not a minimal PDF-file at all. No examples for me to use it seems.

I go back to 7.5.4 Cross-reference table hoping to find some info on a minimal, or empty cross-reference table.

The description is vague, but I get the feeling maybe I can just leave it out completely by not mentioning any xrefs. That would yield me the following:

%PDF-2.0
(Hello world!)
trailer
startxref
0
%%EOF

Now only the byte offset left! Wait... what can the byte offset to the xref-table be when there is none. Something is not right here... For now I just enter a 0 and see if some PDF-reader wants to open it.

Not a single PDF-reader wants to open it. I guess that xref-table was in fact needed. Back to the minimal PDF-examples.

5 Rereading the minimal examples

Sigh... lets go back to the minimal examples, to see if I do not some extra things to get the document to work.

I already see ... a lot of time, where I would not know what should go there. Do I really have to go outside de PDF-standards-document for my information?

I look at some small PDF's I have on my computer and notice I do not even see the structure that is mentioned until now. This is the first moment I am starting to doubt if I should just stop trying to figure this out myself.

6 Rereading

I start rereading the parts I already read, and noticed 7.5.3 File Body mentioning indirect objects. Does that mean (Hello world) is not direct? Might that be the mistake?

Searching for indirect objects brings me to 7.3.10 Indirect Objects. It seems I need to add some numbers, obj and endobj.

%PDF-2.0
1 0 obj
(Hello world!)
endobj
trailer
startxref
0
%%EOF

Alas, it is still considered an invalid PDF-document. Lets continue reading further. I feel we are on the right track!

Next I noticed something else about indirect objects in 7.5.4 Cross-reference table:

So it is here we need to add a reference to our Hello world. I continue reading chapter 7.5.4 Cross-reference table.

I need a xref, give the lowest given number (0), give the number of objects (1), and follow it with some kind of code. The code is a 10-digit number of bytes until the object (9), some zeroes since that is the value the objects should initially have, the letter n and a space because the line-ending must be two characters and I like my single-character line-endings. I also set the byte count in the startxref to 39, which is the byte count before xref.

%PDF-2.0
1 0 obj
(Hello world!)
endobj
xref
0 2
0000000000 65535 f 
0000000009 00000 n 
trailer
startxref
39
%%EOF

Still no success, dang it!

I fire up a hex-editor and check if the byte-offsets are correct.

0000000 25 50 44 46 2d 32 2e 30 0a 31 20 30 20 6f 62 6a
0000020 0a 28 48 65 6c 6c 6f 20 77 6f 72 6c 64 21 29 0a
0000040 65 6e 64 6f 62 6a 0a 78 72 65 66 0a 30 20 31 0a
0000060 30 30 30 30 30 30 30 30 30 39 20 30 30 30 30 30
0000100 20 6e 0a 74 72 61 69 6c 65 72 0a 09 3c 3c 0a 09
0000120 3e 3e 0a 73 74 61 72 74 78 72 65 66 0a 33 39 0a
0000140 25 25 45 4f 46 0a

It seems those are correct. Could it be that it needs a page in order to print the text?

7 The trailer

I remember the trailer, the thing I kinda skipped at the start, and start reading that in more detail too. I notice that, between double angle-brackets I need to specify some properties. It is not mentioned that the properties need to have a slash in front of them, but it is shown in the example beneath the table.

%PDF-2.0
1 0 obj
(Hello world!)
endobj
xref
0 2
0000000000 65535 f 
0000000009 00000 n 
trailer
<<
	  /Size 2
	  /Root ???
	  /ID [??? ???]
	  >>
startxref
39
%%EOF

Now I need to figure out what a root is. As for the ID, I update it to use the same one as in the standards-document. Maybe that will just work!

8 Root

So that might be where I need to specify the pages! The link brings me to 7.7.2 Document catalog dictionary.

%PDF-2.0
1 0 obj
(Hello world!)
endobj
2 0 obj
<<
	/Type /Catalog
	/Pages 3 0 R
	>>
endobj
3 0 obj
<<
	/Type /Pages
	/Kids [4 0 R]
	/Count 1
	>>
endobj
4 0 obj
<<
	/Type /Page
	/Parent 3 0 R
	/MediaBox [0 0 300 300]
	/Resources << >>
	/Contents 1 0 R
	>>
endobj
xref
0 5
0000000000 65535 f 
0000000009 00000 n 
0000000039 00000 n 
0000000091 00000 n 
0000000152 00000 n 
trailer
<<
	  /Size 2
	  /Root 2 0 R
	  /ID [<81b14aafa313db63dbd6f981e49f94f4> <81b14aafa313db63dbd6f981e49f94f4>]
	  >>
startxref
264
%%EOF

Woohoo! The PDF-document finally opens! Still no text though. Could it be the empty resources causing that?

9 Resource dictionary

In Table 34 under 7.8.3 Resource dictionaries it mentions the different things you can specify, but all are optional. I half expected Font to be required since I see no text, but that is not the case.

10 Reviewing the type of Contents

Going back to the properties of Page I notice that Contents expects a content-stream. Currently I use a string, so that might be the problem here.

Looking at 7.8.2 Content streams I get the following information:

That... does not really help. Does that mean that content stream is the same thing as stream object as mentioned in 7.3.8 Stream objects?

The description given in 7.3.8 Stream objects is vague, but I guess this is how a stream object should look:

X Y obj
<<
	Length Z
	>>
stream
	ABC
endstream

I look through the document outline to see if I can find some more info on streams, since this part is rather short. I come across 7.5.7 Object streams.

I think the PDF-standards-document is taking a piss with me. How can I differentiate object stream and stream object?

I go back a bit and read from the start of the chapter.

It is confirmed, this document is taking a piss with me. Onwards we go.

11 Text

I check the document outline again, and see 9.2.2 Basics of showing text. It shows a basic usages of showing some text which I guess should go between stream and endstream.

X Y obj
<<
	Length 30
	>>
stream
BT
	100 100 Td
	( test ) Tj
ET
endstream

I paste it into the rest of the document

%PDF-2.0
1 0 obj
(Hello world!)
endobj
2 0 obj
<<
	/Type /Catalog
	/Pages 3 0 R
	>>
endobj
3 0 obj
<<
	/Type /Pages
	/Kids [4 0 R]
	/Count 1
	>>
endobj
4 0 obj
<<
	/Type /Page
	/Parent 3 0 R
	/MediaBox [0 0 300 300]
	/Resources << >>
	/Contents 5 0 R
	>>
5 0 obj
<<
	Length 30
	>>
stream
BT
	100 100 Td
	( test ) Tj
ET
endstream
endobj
xref
0 6
0000000000 65535 f 
0000000009 00000 n 
0000000039 00000 n 
0000000091 00000 n 
0000000152 00000 n 
0000000255 00000 n 
trailer
<<
	  /Size 2
	  /Root 2 0 R
	  /ID [<81b14aafa313db63dbd6f981e49f94f4> <81b14aafa313db63dbd6f981e49f94f4>]
	  >>
startxref
337
%%EOF

Still nothing but an empty page. I check the examples given in the next few chapters, and notice example EXAMPLE 1 at the end of page 298, start of page 299 which does not selects a font. Good, seems selecting a font is not required. Or maybe the example is not complete. But lets hope fonts are not needed.

9.2.2 Basics of showing text does mention To paint glyphs, a content stream shall first identify the font to be used. The Tf operator shall specify the name of a font resource, but this document has been vague before, so maybe it does that, but it is not required for me to specify it?

Reading a bit further I see the Table 103 — Text state operators:

Ok, font is required. Got it.

12 Fonts

Sigh... I really hoped I could just skip Fonts. It seems like such a hassle. Do I have to save the entire font with the PDF? It probably has to, how else will a PDF work on another machine without the font installed? Some applications might save just the characters in that font as images or something, but not me, because I am doing it by hand.

Oh well... lets take a look at 7.8.3 Resource dictionaries. The description is incomplete. What do F5 to F8 mean in EXAMPLE 2 on page 114? Lets see if there is some useful info elsewhere...

I now remember somewhere I read Helvetica in a previous part of the documentation. So lets search for that font and see the rest of the information around it. Doing that brings me to page 295 which shows how to create a font-object and reference it.

But how does F13 get made? When searching for it I only get 7 search-results. Maybe I can just reference it directly to get Helvetica? Lets just try it

Adding /F13 12 Tf as the first line of the stream (after BT) and changing the Length still yields no text... I am starting to get annoyed.

No mention of /F13 (or any other number) in 9.6 Simple fonts.

No mention of /F13 (or any other number) in the examples in part H. I do notice, in H.2 Minimal PDF file no Resources is given. Wasn't that required? Oh well.

I do notice the Font dictionary being mentioned. Does that mean that the number after the F is the index in that dictionary?

Searching for Font dictionary reveals that it is not a PDF dictionary. But when looking the word up I do get only results for PDF and postscript. So something PDF-specific after al?

Do I look like I know what a font dictionary is? I just want a document with a god dang Hello World!

I am frustrated. Why is this so confusing?

At this point I close all chapters not underneath 7, 9 or H. The solution to this problem should be somewhere in these three chapters.

Oh wait, looking back at the Helvetica example I am starting to get it. A font objects is made, in the resources it is linked via /F13 and from then on I can use /F13 in the stream! At least, I think that should do it. Lets try it out.

%PDF-2.0
1 0 obj
(Hello world!)
endobj
2 0 obj
<<
	/Type /Catalog
	/Pages 3 0 R
	>>
endobj
3 0 obj
<<
	/Type /Pages
	/Kids [4 0 R]
	/Count 1
	>>
endobj
4 0 obj
<<
	/Type /Page
	/Parent 3 0 R
	/MediaBox [0 0 300 300]
	/Resources 
		<< 
			/Font <</F13 6 0 R>>
			>>
	/Contents 5 0 R
	>>
endobj
5 0 obj
<<
	Length 44
	>>
stream
BT
	/F13 12 Tf
	100 100 Td
	( test ) Tj
ET
endstream
endobj
6 0 obj
<<
	/Type /Font
	/Subtype /Type1
	/BaseFont /Helvetica
	>>
endobj
xref
0 7
0000000000 65535 f 
0000000009 00000 n 
0000000039 00000 n 
0000000091 00000 n 
0000000152 00000 n 
0000000293 00000 n 
0000000387 00000 n 
trailer
<<
	  /Size 2
	  /Root 2 0 R
	  /ID [<81b14aafa313db63dbd6f981e49f94f4> <81b14aafa313db63dbd6f981e49f94f4>]
	  >>
startxref
732
%%EOF

It works! Wait... why is it... at that weird place?

This is the moment where my partner tells me PDF coordinates work from the bottom-left instead of the top-left. Makes sense!

I try to change F13 to Fab to check if I can use any name, apparently I can! So why does the documentation insist on 13? Lucky number? I will probably never know, and at this point I do not care anymore either.

Now to get the string-object we initially defined as the text, instead of the one currently in use. Changing ( test ) Tj to 1 0 R Tj does not work. I see an empty page again.

I look for examples using Tj but only see string-objects being used directly. I do come across A.2 PDF content stream operators, which does not seem to have an operator for indirect string objects. But that would be weird, why would I be able to create an indirect object with string objects if I cannot use them?

At this point I give up the documentation and go search the web for an answer. I do not find an answer there either. So probably not allowed. That is sad... but at least I can change test to Hello World! and call it done.

Oh wait I also wanted a smiley.

13 The smiley

I now put my focus on 8 Graphics. Of the things mentioned in 8.1 General, it is mostly the second and third options that interest me: the path operations. But lets read all of it instead of skipping anything.

8.2 Graphics objects seems to tell me that I can reuse the stream I have for Hello World! for drawing. That would be nice! The operators look like the operators for SVG-paths to me, which is nice. They are different of course but still.

Ok, that is a lot of text, lets skip some of it anyway and search for what I need.

My current plan is to draw a yellow circle, a red half circle as the mouth and two black circles as the eyes.

So I guess I first need to set the painting color to yellow or something. I see Table 73 — Colour operators which might include what I need. I find RG, which sounds a good way to set yellow. I guess it expects three values between 0 and 1. Strange the command is called RG instead of RGB, but I guess PDF just likes two-character names.

Should RG come before or after the parameters? The operators we used for text had them at the end, so I guess it is the same here. So yellow, red and black are 1.0 1.0 0.0 rg, 1.0 0.0 0.0 rg and 0.0 0.0 0.0 rg respectively.

Onto the circles, Table 58 — Path construction operators does not include arcs or circles. I don't like to work with bézier curves, so maybe I need to look at the other operations outside of paths.

Mmm, no such operations it seems, lets take a quick look at the examples to see which operations they use. Awh, it seems the half circle is actually a half-oval made with a bézier curve. Lets look online for a formula to get a half circle with a bézier curve.

I find How to create circle with Bézier curves? on Stackoverflow, which gives me the following answer:

Just my luck. I search some examples with circle, but it seems the PDF-standards also uses four bézier curves.

The answer given by Blindman67 seems to be what I need:

I check the syntax of c to make a bézier curve. Three points... damn it. But wait... it appends a bézier curve, so a point is already given!

So I guess a circle can be drawn this way:

x1 y1 m
handle1_1_x handle1_1_y handle1_2_x handle1_2_y x2 y2 c
handle2_1_x handle2_1_y handle2_2_x handle2_2_y x3 y3 c
handle3_1_x handle3_1_y handle3_2_x handle3_2_y x4 y4 c
handle4_1_x handle4_1_y handle4_2_x handle4_2_y x1 y1 c
B

It seems I only only need to move exactly horizontally or vertically for the handles, and the offset from the point should be ~0.55.

Ok, so how big should the smiley be? Lets just fill the space between the corner and the text, which is 100x100. That means the coordinates we need for the circle are (50, 0), (0, 50), (50, 100) and (100, 50).

1.0 1.0 0.0 rg
50 0 m
handle1_1_x handle1_1_y handle1_2_x handle1_2_y 0 50 c
handle2_1_x handle2_1_y handle2_2_x handle2_2_y 50 100 c
handle3_1_x handle3_1_y handle3_2_x handle3_2_y 100 50 c
handle4_1_x handle4_1_y handle4_2_x handle4_2_y 50 0 c
B

Now we can set the handles, adding and subtracting the offset. For a unit circle that offset was 0.55, we need to multiply it with 50 here, giving us 27,5. I don't know the order of the handles, but lets just hope for the best!

1.0 1.0 0.0 rg
50 0 m
0    22.5 0    77.5 0   50  c
22.5 100  77.5 100  50  100 c
100  22.5 100  77.5 100 50  c
22.5 0    77.5 0    50  0   c
B

Awh, guessing the handles did not work. The bottom-left part looks correct though!

Since I still don't feel like figuring it out myself, I am going to look up the example in the PDF-standards again, for drawing a circle with the curves.

99.92 49.92 m %Start new path
99.92 77.52 77.52 99.92 49.92 99.92 c %Construct lower-left circle
22.32 99.92 -0.08 77.52 -0.08 49.92 c
-0.08 22.32 22.32 -0.08 49.92 -0.08 c
77.52 -0.08 99.92 22.32 99.92 49.92 c
B

I notice an offset of 0.08 which is annoying me so lets get rid of that:

100 50 m %Start new path
100   77.60 77.60 100   50  100 c %Construct lower-left circle
22.40 100   0     77.60 0   50 c
0     22.40 22.40 0     50  0 c
77.60 0     100   22.40 100 50 c
B

Only now I notice it uses the same coordinates and almost the same offset for the handles... but the coordinates of the handles are all different. Looking at the coordinates, I do not understand at all how the handles are defined... they do not move solely horizontally or vertically. I am going to stick with the handles only moving that way. What might be wrong is the order of the coordinates, so I am going to take the order as shown in the example.

1.0 1.0 0.0 rg
50 0 m
100  77.5 100  22.5 100 50  c
22.5 100  77.5 100  50  100 c
0    22.5 0    77.5 0   50  c
77.5 0    22.5 0    50  0   c
B

Ok, that just rotated the problem. Lets take a closer look at the handles...

Could it be that I need to look at the first point instead of the second point? So the first bézier curve needs to look at the point placed with m?

1.0 1.0 0.0 rg
50 0 m
77.5 0    22.5 0    100 50  c
100  77.5 100  22.5 50  100 c
22.5 100  77.5 100  0   50  c
0    22.5 0    77.5 50  0   c
B

That just flips it! Arrrrgggg!

Lets take a look at 8.5.2.2 Cubic Bézier curves. Ah, there is the answer already: the first handle is for the previous point and the second is for the next point. That explains why the coordinates were all different in the example.

1.0 1.0 0.0 rg
50 0 m
77.5 0    100    22.5 100 50  c
100  77.5 77.5   100  50  100 c
22.5 100  0      77.5 0   50  c
0    22.5 22.5   0    50  0   c
B

Finally, circle one is drawn!

Stroking is done with S, so use that and a red color for the lips

1.0 0.0 0.0 RG
10 50 m
10   22.5 22.5 10   50 10 c
77.5 10   90   22.5 90 50 c
S

Yes, the handles are not correct which gives the mouth a weird shape. No, I do not care. Next up: the eyes! Almost finished!!!

I want the eyes in the center of the top quadrants. That means the centers will be at (25, 75) and (75,75). Lets give them a size of 10. Size of 10 means an offset for the handles of 5.5. For the left eye I will take the data from the yellow circle, for the right eye I will copy the left eye and move it 50.

The points for the left circle will be (25,65), (35,75), (25, 85), (15, 75).

0.0 0.0 0.0 rg
0.0 0.0 0.0 RG
25 65 m
30.5 65   35   69.5 35 75 c
35   80.5 30.5 85   25 85 c
19.5 85   15   80.5 15 75 c
15   69.5 19.5 65   25 65 c
B

75 65 m
80.5 65   85   69.5 85 75 c
85   80.5 80.5 85   75 85 c
69.5 85   65   80.5 65 75 c
65   69.5 69.5 65   75 65 c
B

Victory!

14 Final PDF

14.1 How it looks

%PDF-2.0
1 0 obj
<<
	/Type /Catalog
	/Pages 2 0 R
	>>
endobj
2 0 obj
<<
	/Type /Pages
	/Kids [3 0 R]
	/Count 1
	>>
endobj
3 0 obj
<<
	/Type /Page
	/Parent 2 0 R
	/MediaBox [0 0 300 300]
	/Resources 
		<< 
			/Font <</Fab 5 0 R>>
			>>
	/Contents 4 0 R
	>>
endobj
4 0 obj
<<
	Length 587
	>>
stream
BT
	/Fab 12 Tf
	100 100 Td
	( Hello World! ) Tj

	1.0 1.0 0.0 rg
	50 0 m
	77.5 0    100    22.5 100 50  c
	100  77.5 77.5   100  50  100 c
	22.5 100  0      77.5 0   50  c
	0    22.5 22.5   0    50  0   c
	B

	1.0 0.0 0.0 RG
	10 50 m
	10   22.5 22.5 10   50 10 c
	77.5 10   90   22.5 90 50 c
	S

	0.0 0.0 0.0 rg
	0.0 0.0 0.0 RG
	25 65 m
	30.5 65   35   69.5 35 75 c
	35   80.5 30.5 85   25 85 c
	19.5 85   15   80.5 15 75 c
	15   69.5 19.5 65   25 65 c
	B

	75 65 m
	80.5 65   85   69.5 85 75 c
	85   80.5 80.5 85   75 85 c
	69.5 85   65   80.5 65 75 c
	65   69.5 69.5 65   75 65 c
	B
ET
endstream
endobj
5 0 obj
<<
	/Type /Font
	/Subtype /Type1
	/BaseFont /Helvetica
	>>
endobj
xref
0 6
0000000000 65535 f 
0000000009 00000 n 
0000000061 00000 n 
0000000122 00000 n 
0000000263 00000 n 
0000000365 00000 n 
trailer
<<
	  /Size 2
	  /Root 1 0 R
	  /ID [<81b14aafa313db63dbd6f981e49f94f4> <81b14aafa313db63dbd6f981e49f94f4>]
	  >>
startxref
732
%%EOF

15 Thoughts

It was a fun exercise. Way more difficult and time-taking than I imagined before I started.

The order in which I found the information confused me a lot. At first I assumed the body of the PDF is where you can place text on your PDF like in HTML, but that is not the case. It took me way to long before I realised that the root is the entry point from which all the rest is defined. The body seems to be more of a storage for the objects you want to use with the reference table being the dictionary so you can find the objects.

I still feel bad about the string-object I had to remove since I could not use it in the stream. I had to remove the first object I created!

I think prior knowledge of PDF is expected before reading the document, which I lacked.

The goal was to have Hello World! printed on a PDF with a smiley. I succeeded in that, although I had it a bit different in mind before I started. I could play around with positioning and sizes, but that is not the goal and would just cause me to procrastinate publishing this article, so I am leaving it as it is!

16 Tools used

• Default PDF-reader
• Vim
• vxHexEditor (to quickly find the offsets I needed)
• od (to get the hex to show in this article)

17 Sources

• Announcing no-cost access to the latest PDF standard: ISO 32000-2 (PDF 2.0), by Duff Johnson
• How to create circle with Bézier curves?, asked by Rella on Stackoverflow