NDJSON Reader
Category: Data
Node id: ndjson-reader
Standards: EHDS, FHIR-Bulk-Data
Reads FHIR resources from NDJSON files with resource type filtering, pagination, bulk directory reading, and schema inference
What it does
- Three read modes: single file, bulk directory, schema inference
- FHIR resource type filtering
- Pagination with row limit and offset
- Gzip-compressed NDJSON file support
- Resource type counting across files
Use cases
- Read FHIR Bulk Data Export files for pipeline processing
- Infer schema from NDJSON files before transformation
- Bulk-read all patient resources from an export directory
Runtime
- Performance tier:
medium
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
resources | json | No | Array of parsed FHIR resources |
total_read | number | No | Total lines read from the file |
total_matched | number | No | Total lines matching the resource type filter |
resource_counts | json | No | Map of resourceType to count |
files_read | json | No | List of files read (read_bulk) |
schema | json | No | Inferred schema from first resource (read_schema) |
resource_type | text | No | Resource type from first resource (read_schema) |
success | boolean | No | Whether the operation succeeded |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Operation | operation | enum | Yes | read | read, read_bulk, read_schema | Read operation mode |
| File Path | file_path | string | No | NDJSON file path (for read and read_schema) | ||
| Input Directory | input_directory | string | No | Directory containing NDJSON files (for read_bulk) | ||
| Resource Type Filter | resource_type_filter | string | No | Only return resources of this FHIR type (e.g., Patient) |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Row Limit | row_limit | number | No | 0 | min 0 | Maximum number of resources to return (0 = unlimited) |
| Row Offset | row_offset | number | No | 0 | min 0 | Number of matching resources to skip |
| Base Directory | base_directory | string | No | Base directory for path security validation |
Errors
| Code | Description |
|---|---|
FILE_NOT_FOUND | The specified NDJSON file does not exist |
INVALID_JSON_LINE | A line in the NDJSON file is not valid JSON |
PATH_SECURITY_VIOLATION | File path is outside the allowed base directory |
Runtime notes
- Output keys vary by operation: read returns resources/total_read/total_matched, read_bulk adds files_read, read_schema returns schema/resource_type
- Supports gzip-compressed .ndjson.gz files transparently
- Malformed JSON lines are silently skipped; total_read still counts them
- resource_counts tallies ALL resource types in the file, even those excluded by resource_type_filter
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Something unclear? Send documentation feedback by email.