Deciding which documents need OCR

A contract is in the corpus, extraction reported success, and the text is thirty-one characters long: a header, a page number, and a footer. The eleven pages of terms in between are an image of a signed document, and the extractor read exactly the parts that were not.

The document was not skipped and did not fail. It produced text. There was simply almost none of it, and nothing in the pipeline treats “very little text” as different from “a short document”.

Recognition is a routing decision, not a fallback

Optical recognition is usually the single most expensive stage available: slow, often the only stage that wants specialised hardware, and priced per page whether it is bought or run. Which means it cannot be applied to everything, and it cannot be applied to nothing.

So something has to decide, per document — really per page — whether the bytes contain text that can be read or an image that has to be recognised. That decision is an inference from the extraction output, because no format reliably declares it.

The decision is per page, not per document. Mixed documents are the common case in any real corpus: a born-digital report with three scanned appendices, a form with a scanned signature block, a wrapper page in front of a faxed original. Deciding at the document level gets those wrong in both directions — either paying to recognise pages that already had clean text, or skipping recognition on a document that had some.

Signals that a page was not read

None is conclusive alone. Together they route reliably.

Characters per page. The strongest single signal. A page yielding a handful of characters is a scan with running headers, or a failure. Set the threshold by looking at the distribution across the corpus rather than by picking a number in the abstract — the point of a threshold here is to separate two clusters that are usually very far apart.

Presence of a large image covering most of the page. A full-page image plus almost no text is diagnostic. A small image plus plenty of text is a figure.

Text present but not word-like. A page whose extracted text is mostly punctuation, single letters or non-words. This is the case that looks like success and is not, and it comes up when a document has a text layer that does not correspond to the visible content.

Consistency across pages. A document where most pages yield a thousand characters and four yield none has four scanned pages. Comparing pages within a document is more reliable than comparing a page against a global threshold, because document density varies enormously.

The text layer that lies

The case worth planning for, because it defeats the character-count signal entirely: a page that has both an image and a text layer, where the text layer is wrong.

This arises when a document was recognised once already, badly, and the result was embedded. It also arises from redaction done by drawing rectangles over text that remains in the file underneath, and from files assembled by tools that copied a text layer from the wrong source.

The visible page and the extracted text disagree, and extraction has no way to know. The page yields a plausible volume of plausible text, so every volume-based check passes. What gives it away is the error signature — the character-level confusions described in the extraction post — showing up in a document that was supposed to be born-digital. If recognition errors appear in text that no recognition step produced, the text layer came from somewhere else.

There is no cheap general fix. What there is: recognising the pattern, flagging the source that produces it, and deciding whether to trust the existing layer or overwrite it with a fresh pass. That is a per-source decision, and the sources that do it tend to do it consistently.

The pipeline

THE PIPELINE — recognition routing

  · Page with a substantial, word-like
    text layer
                    → read it. Do not pay to recognise it.

  · Page image with no text layer
                    → route to recognition. Mark the output
                      as recognised, permanently.

  · Page with a header and footer and nothing
    between them
                    → FAILS SILENTLY. Reports success with
                      thirty characters of furniture. Looks
                      like a short document.

  · Page with both an image and a wrong
    text layer
                    → FAILS SILENTLY. Volume checks pass and
                      the text does not match the page. Only
                      the error signature gives it away.

  · Recognised text stored without a
    recognised flag
                    → FAILS SILENTLY. Indistinguishable from
                      read text forever after.

  · Whether recognition is worth its cost
                    → CORPUS-DEPENDENT. Decided by how much
                      of the corpus is image-only and what
                      those documents are worth.

Handwriting, and knowing when to stop

Print recognition and handwriting recognition are different problems with different reliability, and a corpus of forms contains both — printed labels and handwritten values in the boxes beside them.

The failure is specific and worth anticipating: the printed structure recognises well, the handwritten content recognises poorly or not at all, and the result is a document full of field names with no or wrong values. That reads as a coherent document. It is worse than an empty page, because an empty page gets noticed.

Where a corpus is substantially handwritten, the honest answer is often that this content is not going into a retrieval system as text. Marking those documents as present-but-unreadable, so they can be found and looked at by a person, is a better outcome than admitting low-confidence transcriptions into the corpus, and it is a decision better made deliberately at ingest than discovered later from a wrong answer.

What to record, beyond the text

Recognition output needs provenance more than any other extraction path, because it is the one whose content is a guess.

That it was recognised at all. The single most valuable flag in the pipeline, and impossible to reconstruct later. Everything else about handling low-confidence content depends on it.

Confidence, where the engine reports it, at the finest granularity available. A per-page or per-region figure is far more useful than a document average, because recognition quality varies wildly within one document — a clean printed page and a photographed page at an angle are not the same.

Which engine and which version. So a re-recognition pass after an upgrade can be scoped to the affected documents rather than run across everything.

Whether the page had a pre-existing text layer that was overwritten. The one fact that makes the lying-text-layer problem diagnosable in retrospect.

What this stage hands on

Text for every page that has any, with the recognised pages marked as recognised and carrying whatever confidence the engine gave, and the pages nothing could read recorded as unreadable rather than as empty.

The distinction in that last clause is the one that matters. An unreadable page recorded as unreadable is a known gap that somebody can act on. An unreadable page recorded as a document with thirty characters of text is a document that will be retrieved, cited, and answered from — and the answer will be confidently constructed out of a page number and a running header.