Saltar al contenido principal

NDJSON Reader

Category: Data
Node id: ndjson-reader

Standards: EHDS, FHIR-Bulk-Data

Reads FHIR resources from NDJSON files with resource type filtering, pagination, bulk directory reading, and schema inference

What it does​

  • Three read modes: single file, bulk directory, schema inference
  • FHIR resource type filtering
  • Pagination with row limit and offset
  • Gzip-compressed NDJSON file support
  • Resource type counting across files

Use cases​

  • Read FHIR Bulk Data Export files for pipeline processing
  • Infer schema from NDJSON files before transformation
  • Bulk-read all patient resources from an export directory

Runtime​

  • Performance tier: medium

Output ports​

NameTypeRequiredDescription
resourcesjsonNoArray of parsed FHIR resources
total_readnumberNoTotal lines read from the file
total_matchednumberNoTotal lines matching the resource type filter
resource_countsjsonNoMap of resourceType to count
files_readjsonNoList of files read (read_bulk)
schemajsonNoInferred schema from first resource (read_schema)
resource_typetextNoResource type from first resource (read_schema)
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
OperationoperationenumYesreadread, read_bulk, read_schemaRead operation mode
File Pathfile_pathstringNoNDJSON file path (for read and read_schema)
Input Directoryinput_directorystringNoDirectory containing NDJSON files (for read_bulk)
Resource Type Filterresource_type_filterstringNoOnly return resources of this FHIR type (e.g., Patient)

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Row Limitrow_limitnumberNo0min 0Maximum number of resources to return (0 = unlimited)
Row Offsetrow_offsetnumberNo0min 0Number of matching resources to skip
Base Directorybase_directorystringNoBase directory for path security validation

Errors​

CodeDescription
FILE_NOT_FOUNDThe specified NDJSON file does not exist
INVALID_JSON_LINEA line in the NDJSON file is not valid JSON
PATH_SECURITY_VIOLATIONFile path is outside the allowed base directory

Runtime notes​

  • Output keys vary by operation: read returns resources/total_read/total_matched, read_bulk adds files_read, read_schema returns schema/resource_type
  • Supports gzip-compressed .ndjson.gz files transparently
  • Malformed JSON lines are silently skipped; total_read still counts them
  • resource_counts tallies ALL resource types in the file, even those excluded by resource_type_filter

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

¿Algo no está claro? Envíe sus comentarios sobre la documentación por correo electrónico..