Intake SQL Export
Category: Data
Node id: intake-sql-export
Maturity: beta
Risk tier: B
Export every table of a restored Oracle dump delivery from the read-only staging database into Parquet, reconciling COUNT(*) against the manifest and the exported rows, and hand the generated delivery to intake-verify.
What it does
- phi
- database
Use cases
- Turn an Oracle Data Pump delivery restored on the hospital staging container into a verifiable Parquet delivery
- Prove every restored table holds exactly the rows the hospital manifest declared
Runtime
- Performance tier:
slow
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
trigger | any | No | Optional upstream signal; its content is not read. |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
generated_delivery_id | text | No | Id of the generated delivery now waiting in incoming/: <delivery_id>-export. |
tables | json | No | Per table: staging COUNT(*), exported rows and the manifest row count. |
success | boolean | No | True when every table reconciled and the delivery reached incoming/. |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Staging Connector | connector_id | string | Yes | Saved oracle, mssql or postgres connector for the hospital-side staging database, connecting as the read-only user. | ||
| Source Delivery ID | delivery_id | string | Yes | The accepted dump delivery whose tables were restored by the runbook. | ||
| Intake Root | intake_root | string | Yes | Absolute path of the intake volume holding incoming/, accepted/ and quarantine/ (pilot: /srv/hgugm/intake). | ||
| Intake Profile | profile | string | No | hgugm-cardiology-v1 | Intake profile whose table contracts give the columns to export and their declared types. |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Expected DB User | expected_user | string | No | HYDRA_RO | The staging read-only user. The export refuses to run when the connection is any other user. | |
| Row Group Size | batch_size | number | No | 5000 | min 100; max 100000 | Rows per Parquet row group. |
Errors
| Code | Description |
|---|---|
COUNT_MISMATCH | Staging COUNT(*), exported rows and the manifest row_count are not all equal |
IDENTIFIER_INVALID | A manifest or profile identifier failed the allowlist; no SQL was executed |
NOT_READ_ONLY | The connector is not an oracle/mssql/postgres connector configured read_only and connected as expected_user |
DELIVERY_NOT_ACCEPTED | The source dump delivery is not accepted |
EXPORT_EXISTS | The generated delivery id is already in incoming/, accepted/ or quarantine/ |
EXPORT_LOCKED | Another run is exporting the same dump delivery; retry after it finishes |
STORE | No verdict store or connector resolver is configured |
Runtime notes
- Reads only: COUNT(*) goes through the connector's guarded sql_query service, rows stream through a read-only transaction, and the connector must be configured read_only and connect as expected_user.
- Every identifier is checked against the allowlist before any SQL is built; nothing runs when one fails.
- Output is written under staging-export/ and renamed into incoming/ only when every table reconciled; on failure nothing appears in incoming/.
- Values are exported as text in the profile's declared layouts; intake-verify then parses every one against the declared type.
- The dump itself is restored by scripts/hgugm/restore_oracle_dump.sh, never by Hydra.
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Etwas unklar? Senden Sie Feedback zur Dokumentation per E-Mail..