Tagged “boilerplate”
-
The page that checks who is asking
Four hundred documents in the corpus say 'Verify you are human'. Every fetch succeeded; the source served the barrier instead of the page.
-
Boilerplate is not content
Navigation, footers and disclaimers extract perfectly and index as fact. Repeated across a corpus, they crowd out the pages they were attached to.
-
Pages that are empty until something runs them
A web source ingested cleanly and every page holds the same nav and a loading message. The fetch returned a shell that assembles its content elsewhere.
-
A thread is not a document
A twenty-message email thread contains the first message twenty times. Ingest it naively and the oldest, most-quoted text becomes the loudest claim.