Aller au contenu principal

Intake Table Read

Category: Data
Node id: intake-table-read
Maturity: beta
Risk tier: B

Stream one table of an accepted intake delivery in batches with a resumable cursor. Refuses anything outside accepted/ or whose latest verdict is not accepted.

What it does​

  • phi

Use cases​

  • Feed verified HGUGM tables to the FHIR mapping in batches
  • Page through an accepted Parquet export of the Oracle staging restore

Runtime​

  • Performance tier: medium

Input ports​

NameTypeRequiredDescription
cursoranyNoRow offset to resume from: the cursor output of the previous batch (string or {cursor}). Empty starts at the first row.

Output ports​

NameTypeRequiredDescription
recordsjsonNoThe batch: one object per row keyed by column name; nulls are null.
cursortextNoOffset of the next batch; empty when done.
donebooleanNoTrue when this batch reached the end of the table.
countnumberNoRows in this batch.

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Intake Rootintake_rootstringYesAbsolute path of the intake volume holding incoming/, accepted/ and quarantine/ (pilot: /srv/hgugm/intake).
Delivery IDdelivery_idstringYesThe accepted delivery to read (from intake-verify's accepted[] output).
TabletablestringYesTable name as listed in the delivery manifest, e.g. EPISODIOS.

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Delivery Pathdelivery_pathstringNoOptional settled path of the delivery (intake-verify accepted[].path). When set it must resolve to <intake_root>/accepted/<delivery_id>.
Batch Sizebatch_sizenumberNo500min 1; max 10000Rows per batch.

Errors​

CodeDescription
DELIVERY_NOT_ACCEPTEDThe delivery is not under accepted/, its latest verdict is not accepted, or no verdict store is configured
INTAKE_ROOT_INVALIDintake_root is not an absolute directory with an incoming/ subdirectory
TABLE_NOT_FOUNDThe delivery carries no tabular file for the table
DELIVERY_ALTEREDThe accepted delivery no longer matches its manifest

Runtime notes​

  • Refuses with DELIVERY_NOT_ACCEPTED unless the delivery sits under accepted/ (symlinks resolved) AND its latest recorded verdict is accepted; no store also refuses.
  • Re-checks the manifest sidecar and file size on every batch and the full SHA-256 on the first, so a file edited after acceptance is refused.
  • CSV pages re-scan from the start of the file; Parquet pages seek to the offset.

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.