Intake Table Read
Category: Data
Node id: intake-table-read
Maturity: beta
Risk tier: B
Stream one table of an accepted intake delivery in batches with a resumable cursor. Refuses anything outside accepted/ or whose latest verdict is not accepted.
What it does
- phi
Use cases
- Feed verified HGUGM tables to the FHIR mapping in batches
- Page through an accepted Parquet export of the Oracle staging restore
Runtime
- Performance tier:
medium
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
cursor | any | No | Row offset to resume from: the cursor output of the previous batch (string or {cursor}). Empty starts at the first row. |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
records | json | No | The batch: one object per row keyed by column name; nulls are null. |
cursor | text | No | Offset of the next batch; empty when done. |
done | boolean | No | True when this batch reached the end of the table. |
count | number | No | Rows in this batch. |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Intake Root | intake_root | string | Yes | Absolute path of the intake volume holding incoming/, accepted/ and quarantine/ (pilot: /srv/hgugm/intake). | ||
| Delivery ID | delivery_id | string | Yes | The accepted delivery to read (from intake-verify's accepted[] output). | ||
| Table | table | string | Yes | Table name as listed in the delivery manifest, e.g. EPISODIOS. |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Delivery Path | delivery_path | string | No | Optional settled path of the delivery (intake-verify accepted[].path). When set it must resolve to <intake_root>/accepted/<delivery_id>. | ||
| Batch Size | batch_size | number | No | 500 | min 1; max 10000 | Rows per batch. |
Errors
| Code | Description |
|---|---|
DELIVERY_NOT_ACCEPTED | The delivery is not under accepted/, its latest verdict is not accepted, or no verdict store is configured |
INTAKE_ROOT_INVALID | intake_root is not an absolute directory with an incoming/ subdirectory |
TABLE_NOT_FOUND | The delivery carries no tabular file for the table |
DELIVERY_ALTERED | The accepted delivery no longer matches its manifest |
Runtime notes
- Refuses with DELIVERY_NOT_ACCEPTED unless the delivery sits under accepted/ (symlinks resolved) AND its latest recorded verdict is accepted; no store also refuses.
- Re-checks the manifest sidecar and file size on every batch and the full SHA-256 on the first, so a file edited after acceptance is refused.
- CSV pages re-scan from the start of the file; Parquet pages seek to the offset.
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.