NDJSON Reader
Category: Data
Node id: ndjson-reader
Standards: EHDS, FHIR-Bulk-Data
Reads FHIR resources from NDJSON files with resource type filtering, pagination, bulk directory reading, and schema inference
What it does
- Three read modes: single file, bulk directory, schema inference
- FHIR resource type filtering
- Pagination with row limit and offset
- Gzip-compressed NDJSON file support
- Resource type counting across files
Use cases
- Read FHIR Bulk Data Export files for pipeline processing
- Infer schema from NDJSON files before transformation
- Bulk-read all patient resources from an export directory
Runtime
- Performance tier:
medium
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
resources | json | No | Array of parsed FHIR resources |
total_read | number | No | Total lines read from the file |
total_matched | number | No | Total lines matching the resource type filter |
resource_counts | json | No | Map of resourceType to count |
files_read | json | No | List of files read (read_bulk) |
schema | json | No | Inferred schema from first resource (read_schema) |
resource_type | text | No | Resource type from first resource (read_schema) |
success | boolean | No | Whether the operation succeeded |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Operation | operation | enum | Yes | read | read, read_bulk, read_schema | Read operation mode |
| File Path | file_path | string | No | NDJSON file path (for read and read_schema) | ||
| Input Directory | input_directory | string | No | Directory containing NDJSON files (for read_bulk) | ||
| Resource Type Filter | resource_type_filter | string | No | Only return resources of this FHIR type (e.g., Patient) |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Row Limit | row_limit | number | No | 0 | min 0 | Maximum number of resources to return (0 = unlimited) |
| Row Offset | row_offset | number | No | 0 | min 0 | Number of matching resources to skip |
| Base Directory | base_directory | string | No | Base directory for path security validation |
Errors
| Code | Description |
|---|---|
FILE_NOT_FOUND | The specified NDJSON file does not exist |
INVALID_JSON_LINE | A line in the NDJSON file is not valid JSON |
PATH_SECURITY_VIOLATION | File path is outside the allowed base directory |
Runtime notes
- Output keys vary by operation: read returns resources/total_read/total_matched, read_bulk adds files_read, read_schema returns schema/resource_type
- Supports gzip-compressed .ndjson.gz files transparently
- Malformed JSON lines are silently skipped; total_read still counts them
- resource_counts tallies ALL resource types in the file, even those excluded by resource_type_filter
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
¿Algo no está claro? Envíe sus comentarios sobre la documentación por correo electrónico..