Catalog DB Introspect
Category: Data
Node id: estate-db-introspect
Maturity: beta
Risk tier: none
Enumerate the schemas, tables and columns of any supported database through a saved connector, emitting catalogue assets ready for estate-register. Pages large schemas and reports truncation rather than silently stopping.
What it does
- Thirteen SQL dialects including Oracle, SQL Server, HANA and Db2
- Read-only: every query is gated and run in a read-only transaction
- Keyset paging with a resumable cursor
- Explicit truncation reporting
Use cases
- Catalogue a hospital information system before an integration project
- Enumerate one schema of a data warehouse for FHIR mapping
- Re-run nightly to detect new or removed tables
Runtime
- Performance tier:
slow
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
cursor | json | No | Optional {cursor} from a previous run's next_cursor, to continue a paged sweep |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
assets | json | No | Assets shaped for estate-register: {fqn, kind, system, name, fields[]} |
next_cursor | text | No | Opaque cursor when more tables remain; empty when the sweep is complete |
truncated | boolean | No | True when more tables remain after this page |
table_count | number | No | Tables enumerated in this page |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Connector | connector_id | connector-select | Yes | Saved database connector supplying the connection and its credentials. Credentials never appear in the pipeline definition. | ||
| Schema | schema_filter | string | No | Restrict the sweep to one schema. Strongly recommended on a large database: it is the difference between a targeted catalogue and a full crawl. | ||
| Table | table_filter | string | No | Restrict the sweep to a single table. | ||
| Tables Per Page | page_size | number | No | 500 | Maximum tables to enumerate per run. Bounds both the query and the memory one page holds. |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Cursor | cursor | string | No | Continue a previous sweep. Usually fed from the cursor input port inside a loop rather than set by hand. | ||
| Timeout | timeout_seconds | number | No | 20 | Per-page query timeout. Raise it for a slow or heavily loaded source. | |
| System Name | system | string | No | Name used in generated FQNs (table://<system>/<schema>.<table>). Defaults to the connector id, which keeps FQNs stable across runs. |
Errors
| Code | Description |
|---|---|
connector_unavailable | Connector registry not configured on this server |
no_connector | connector_id is missing or does not resolve |
unsupported_driver | The connector's driver does not expose schema exploration |
Runtime notes
- Reads catalogue metadata only — no table rows are read, so it is safe against a production system.
- Emits assets shaped for estate-register; wire the two together and register before profiling so a failure part-way through loses nothing.
- When truncated is true, feed next_cursor back into the cursor input to continue; the sweep is complete when truncated is false.
- FQNs are table://<system>/<schema>.<table> and are stable across runs, which is what makes registration idempotent.
- Without schema_filter this enumerates every non-system schema, which on a large HIS is a long sweep — set it unless a full crawl is genuinely intended.
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Etwas unklar? Senden Sie Feedback zur Dokumentation per E-Mail..