Menu

Batch ingestion

A package of documents moving through intake, classification and extraction, stage by stage.

Demonstration project - no figure here describes a real facilityChanges are not saved
Why

Stage progress is derived from each document's recorded state. No worker is running in this environment, so nothing is in progress.

Writes are validated against the engineering rules and then refused, because no database has been configured for this environment.

Documents in the package

25

Through every stage

22

Waiting

2

Refused

1

Nothing is running

No worker in this environment. Each row is a recorded state.

By stage

A batch that reports one percentage hides which stage it is stuck at, and the stages fail for entirely different reasons.

IntakeThe file is accepted, checksummed and stored
Intake22 of 25 — failed

1 refused at or before this stage

ClassificationWhat kind of document it is, which decides how it is read
Classification22 of 25 — failed

1 refused at or before this stage

Text extractionThe native text layer, where there is one
Text extraction14 of 15
RasterisationPage images, for a scan or a mixed sheet
Rasterisation2 of 3 — failed

1 refused at or before this stage

RecognitionCharacters from pixels, where there is no text layer
Recognition2 of 3 — failed

1 refused at or before this stage

GeometryVector paths, runs and symbols on a drawing
Geometry15 of 17 — failed

1 refused at or before this stage

AssociationTags attached to the thing they label
Association15 of 17 — failed

1 refused at or before this stage

ReconciliationWhat the new document says against what the project already holds
Reconciliation22 of 25 — failed

1 refused at or before this stage

Per document

Documents in the package
DocumentTitleRead byState
ZRF-PID-1201P&ID - Condensate Export Pumps and Dischargenative_textComplete
ZRF-PID-1202P&ID - Condensate Export Meteringnative_textComplete
ZRF-PID-1101P&ID - Inlet Separation (scanned)ocrComplete
ZRF-PID-3101P&ID - Cooling Water Circulation (mixed)bothComplete
ZRF-PID-3201P&ID - Instrument Air and Nitrogennative_textComplete
ZRF-PFD-1200PFD - Condensate Exportnative_textComplete
ZRF-PFD-1100PFD - Inlet Receptionnative_textComplete
ZRF-SLD-6001Single Line Diagram - Utilities MCCnative_textComplete
ZRF-DS-P-1201AEquipment Datasheet - Condensate Export Pump P-1201Anative_textComplete
ZRF-DS-PT-1201Instrument Datasheet - Pressure Transmitter PT-1201native_textComplete
ZRF-DS-PSV-1210Relief Device Datasheet - PSV-1210native_textComplete
ZRF-REG-EQ-001Equipment RegisterstructuredComplete
ZRF-REG-LL-001Piping Line ListstructuredComplete
ZRF-REG-II-001Instrument IndexstructuredComplete
ZRF-REG-VR-001Valve RegisterstructuredComplete
ZRF-REG-CS-001Cable SchedulestructuredComplete
ZRF-REG-MDR-001Master Document RegisterstructuredComplete
ZRF-SPC-3300Piping Material and Pressure Testing Specificationnative_textComplete
ZRF-SPC-4100Instrumentation and Control Specificationnative_textComplete
ZRF-CTR-APX-AContract Appendix A - Scope of Worknative_textComplete
ZRF-CTR-APX-DContract Appendix D - Quality and Inspection Requirementsnative_textComplete
ZRF-CTR-APX-FContract Appendix F - Documentation and DeliverablesnoneWaiting
ZRF-VND-P-1201Vendor Manual - Condensate Export Pumpnative_textWaiting
ZRF-SCH-0001Project Schedule - Detailed Design and ConstructionstructuredComplete
ZRF-PID-9001P&ID - Flare Header (unreadable scan)ocr
RefusedA scan too faint to read. Processing refuses it and states why; nothing is extracted.

Queue a batch

Requeuing a document measured as unreadable is refused.

Unread documents, previously refused documents, or the whole package

Each stage fails differently

  • Intake fails on a file type nothing can read.
  • Classification fails on a title block it cannot parse.
  • Recognition fails on a scan too poor to resolve characters.
  • Reconciliation does not fail: it reports a disagreement, which is a finding rather than a fault.

Nothing arrives verified

Readings, not facts, until a person checks them. Why