Extracting text from real documents

An answer comes back citing a document, you open the document to check, and the sentence the system quoted is not in it. Not paraphrased — not there. The words exist on the page, scattered across two columns and a table, and something joined them into a sentence nobody wrote.

Nothing errored. The extractor ran, produced text, and reported success. That is the normal case, and it is why extraction is worth treating as a stage with its own verification rather than a library call.

A page is not a document

The central problem: most document formats describe how a page looks, not what it says.

A PDF is closer to a set of drawing instructions than to a text file. It records that a glyph appears at a coordinate, and another glyph appears nearby, and it has no obligation to record them in the order a human would read them. Reading order is something an extractor infers, and inference has failure modes.

The consequence is that the question “did extraction work” has no yes-or-no answer from inside the extractor. It produced characters. Whether they are in the right order is a judgement it cannot make about itself.

The failure signatures worth recognising

You will diagnose these faster by their shape than by their cause.

Interleaved columns. A two-column page comes out as one line from the left column, one line from the right, alternating. Each individual line is intact, which is why it reads as text rather than as garbage, and every sentence spanning a line break is nonsense. This is the single most common extraction failure in reports, papers and anything typeset for print.

Headers and footers inline. A running header appears in the middle of a sentence every time a page breaks — the sentence stops, a document title and page number appear, and the sentence resumes. Easy to spot once you know to look, invisible in an aggregate quality metric.

Table collapse. A table becomes a stream of cell values with no row or column structure. The numbers are all present and no longer attached to anything. Worse than losing the table, because the values are now retrievable and meaningless.

Ligature and encoding damage. fi and fl vanish or become replacement characters, so find becomes nd. Smart quotes and dashes turn into mojibake. Words silently lose letters, which breaks lexical matching in a way that is very hard to notice by reading.

Empty output from a scan. A page-image PDF produces little or no text. This one at least announces itself, if anybody looks at the output length.

Reading order following the layout, not the argument. Sidebars, pull quotes and captions get inserted mid-paragraph. The text is all real; the sequence is not the author’s.

Scanned pages and what OCR actually gives you

If a document is an image of a page, extraction is recognition, and recognition has an error rate that is never zero.

The useful thing to know is the error signature, because it is distinctive: OCR errors are character-level confusions between visually similar glyphs rather than word-level nonsense. Digits and letters swap — 0 for O, 1 for l, 5 for S. rn becomes m. Accents disappear. Line breaks land mid-word.

Two properties follow, and both matter more than the raw accuracy number.

Errors concentrate in exactly the content that is most often the reason someone is searching: reference numbers, dates, amounts, part codes, names. Prose survives OCR noise well because there is redundancy in it. An invoice number does not.

And OCR output usually loses structure entirely. Headings stop being headings, tables stop being tables, and everything arrives as a flat stream — which removes the structural signal that anything downstream might have used.

Scanned documents are worth marking as scanned in metadata at ingest, so their content can be treated as lower-confidence later. That is a cheap decision that is impossible to make retrospectively.

The other formats, briefly

Word processor files carry real structure and extract well, with one trap: tracked changes and comments. Depending on the extractor you may get deleted text, comment bodies, or both, mixed into the document as though they were content. A draft with rejected edits can extract as a document that contradicts itself.

HTML extracts easily and brings the entire page furniture with it, which is a separate problem.

Spreadsheets have no reading order at all. Extraction has to be told what shape the data is, and a sheet where meaning lives in the position of a cell relative to a header row does not survive being flattened. Multiple sheets, hidden rows and formula results versus formula text are all decisions, not defaults.

Slides are boxes on a canvas. Extraction gives you the boxes, often not in presentation order, with speaker notes either included as body text or dropped.

Email brings quoted reply chains, so the same message body is extracted once per reply in the thread. A ten-message thread can contain ten copies of the first message.

The pipeline

THE PIPELINE — extraction

  · Digital document with a clean text layer
                    → extracts reliably. Verify anyway.

  · Multi-column or print-typeset page
                    → FAILS SILENTLY. Columns interleave
                      line by line. Every cross-line
                      sentence becomes text nobody wrote.

  · Table
                    → FAILS SILENTLY. Cell values survive
                      as a flat stream, detached from their
                      rows and headers, and stay
                      retrievable.

  · Page image with no text layer
                    → needs OCR. Errors concentrate in
                      identifiers, dates and amounts, and
                      structure is lost.

  · Tracked changes and comments
                    → FAILS SILENTLY. Deleted text and
                      reviewer notes can arrive as content.

  · Whether a given corpus needs layout-aware
    extraction or OCR at all
                    → CORPUS-DEPENDENT. Decided by
                      sampling, not by assumption.

Detecting failure instead of hoping

You cannot inspect every document, and you do not have to. A handful of cheap automatic checks catch most of the damage, and each one is a signal to route a document for attention rather than a reason to reject it.

Character count against page count. A page yielding almost nothing is a scan or a failure. A page yielding an implausible amount is often a duplicated text layer.

Proportion of characters that are alphabetic. Very low means structure or symbol soup rather than prose.

Dictionary hit rate on words. Catches encoding damage and OCR noise in one number. Low rates on documents that should be ordinary prose are the strongest single signal available.

Average sentence length and unterminated-line rate. Interleaved columns produce short fragments that never terminate; so does OCR line breaking.

Repeated short lines across page boundaries. Running headers and footers announce themselves as the same string appearing once per page.

None of these judge quality. They flag documents that deserve a human’s thirty seconds, and thirty seconds of looking at extracted text is worth more than any amount of reasoning about which extractor to use.

What this stage hands on

Text, in the order a reader would read it, with the furniture removed and with metadata recording where it came from, what format it was, whether it was recognised rather than read, and how confident the extraction looked.

That last part is the point of doing extraction as a stage rather than a call. A document that arrives flagged as a low-confidence scan can be treated differently forever after. A document that arrives as plain text with no provenance is indistinguishable from a good one, and the first time anybody finds out is when it answers a question wrongly.