Parquet Reader
Category: Data
Node id: parquet-reader
Standards: EHDS
Reads data from Apache Parquet files — a local path or bytes handed over by another step — with pagination, schema inspection, and metadata extraction
What it does
- Three read modes: data, schema inspection, metadata extraction
- Pagination with row limit and offset for large files
- Parquet schema-to-JSON type mapping
- Row group and key-value metadata extraction
- Path security validation
Use cases
- Read columnar healthcare analytics data from Parquet files
- Inspect Parquet file schema before data transformation
- Extract metadata from Parquet exports for audit logging
Runtime
- Performance tier:
medium
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
file_path | text | No | Parquet file path (overrides config) |
content | any | No | Parquet bytes when source is content: []byte, base64 text, an object-store download record (download with format: raw, which keeps data_base64) or a read_file result |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
data | json | No | Array of record objects (read operation) |
schema | json | No | Parquet schema as field-type map |
row_count | number | No | Total number of rows in the file |
returned | number | No | Number of rows returned after pagination |
columns | json | No | Column detail array (read_schema) |
num_columns | number | No | Number of columns |
num_rows | number | No | Total rows in file metadata |
num_row_groups | number | No | Number of row groups (read_metadata) |
key_value_meta | json | No | File key-value metadata (read_metadata) |
success | boolean | No | Whether the operation succeeded |
Settings
Set at least one of: file_path, source.
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Operation | operation | enum | No | read | read, read_schema, read_metadata | Read operation mode |
| Source | source | enum | No | file | file, content | Where the Parquet bytes come from: file (file_path on this host) or content (the content input, e.g. an S3 download) |
| File Path | file_path | string | No | Parquet file path (source: file) |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Content Encoding | content_encoding | enum | No | auto | auto, text, base64 | How a string content input is encoded; auto = base64 (a read_file result's own encoding field always wins) |
| Row Limit | row_limit | number | No | 0 | min 0 | Maximum rows to return (0 = unlimited) |
| Row Offset | row_offset | number | No | 0 | min 0 | Number of rows to skip |
| Base Directory | base_directory | string | No | Base directory for path security validation |
Errors
| Code | Description |
|---|---|
FILE_NOT_FOUND | The specified Parquet file does not exist |
INVALID_PARQUET | File is not a valid Parquet file or is corrupted |
PATH_SECURITY_VIOLATION | File path is outside the allowed base directory |
Runtime notes
- Output keys differ by operation: read returns data/schema/row_count/returned; read_schema returns schema/columns/num_columns; read_metadata returns num_rows/row_groups/key_value_meta etc.
- file_path input overrides the config value, allowing dynamic path injection from upstream nodes
- source: content reads the content input (bytes, base64, an S3/GCS download record or a read_file result) in memory, so no local hop is needed; a truncated read_file result is refused
- Row offset/limit operate at the read level; row_count always reflects the total rows in the file
- schema is always included regardless of operation and maps column names to Parquet type strings
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Something unclear? Send documentation feedback by email.