File
Category: Data
Node id: file
Performs file system operations (read, write, append, delete, list, exists) and parses file content handed over by another step (parse); CSV and Excel (.xlsx) become records keyed by the header row
What it does
- Eight operations: read, write, append, delete, list, exists, read_all (a folder in one run) and parse (content from another step)
- Multi-format support: text, JSON, CSV, Excel (.xlsx read), binary
- CSV/xlsx to records: delimiter auto-detection, UTF-8/Latin-1/Windows-1252, header row, sheet selection, row and size caps
- Path security validation with base directory restriction
- Automatic parent directory creation
- Glob pattern filtering for directory listings
Use cases
- Read FHIR export files for pipeline processing
- Write transformed data to JSON or CSV files
- List files in a bulk export directory for batch ingestion
- Turn a CSV read over SFTP, or an Excel workbook downloaded from a bucket, into records for a database load
Runtime
- Performance tier:
medium
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
file_path | text | No | File path (overrides config) |
content | any | No | Content to write or append; for parse, the file to parse: text, bytes, base64, or a read_file / object-store download result |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
result | json | No | Full file operation result with path, type, size, and metadata |
content | any | No | File content (read and parse), or for read_all an array of {path, name, size, modified, content} in name order |
records | json | No | CSV / xlsx only: one object per data row, keyed by the header row |
columns | json | No | CSV / xlsx only: the record keys in file order |
success | boolean | No | Whether the operation succeeded |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Operation | operation | enum | Yes | read | read, write, append, delete, list, exists, read_all, parse | File operation to perform; parse reads the content input instead of a local file |
| File Path | file_path | string | No | Path to the file | ||
| File Type | file_type | enum | No | text | text, json, csv, xlsx, binary | Type of file for read/write/parse parsing |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Base Directory | base_directory | string | No | Directory every path must stay inside (symbolic links included); a relative file_path resolves against it | ||
| Create Directory | create_directory | boolean | No | false | Create parent directories if they do not exist | |
| Glob Pattern | pattern | string | No | File name pattern filter for list operation | ||
| Only Files | only_files | boolean | No | false | Exclude directories from list results | |
| Add Newline | add_newline | boolean | No | true | Append a newline after content in append operation | |
| Max Files | max_files | number | No | 1000 | min 1; max 10000 | read_all: hard cap on files read per run |
| Max File Bytes | max_file_bytes | number | No | 10485760 | min 1; max 67108864 | read_all: a larger file is skipped (listed in result.skipped), never cut |
| Max Total Bytes | max_total_bytes | number | No | 67108864 | min 1; max 268435456 | read_all: hard cap on bytes read per run |
| Content Encoding | content_encoding | enum | No | auto | auto, text, base64 | parse: how a string content input is encoded; auto = text for csv/json/text, base64 for xlsx (a read_file result's own encoding field always wins) |
| CSV Delimiter | delimiter | string | No | auto | auto (detects , ; tab |) or one character; "tab" for tab | |
| CSV Encoding | encoding | enum | No | utf-8 | utf-8, latin-1, windows-1252 | Character encoding of CSV bytes; utf-8 must be valid UTF-8 (a BOM is dropped) |
| Header Row | header_row | number | No | 1 | min 0 | 1-based row holding the column names (rows above are skipped); 0 = no header, columns are column_1..column_N |
| Sheet Name | sheet | string | No | xlsx: sheet to read by name (default: the sheet at sheet_index) | ||
| Sheet Index | sheet_index | number | No | 1 | min 1 | xlsx: 1-based sheet position when sheet is not set |
| Max Rows | max_rows | number | No | 100000 | min 1 | CSV/xlsx: fail when the file has more data rows than this (memory guard; hard limit 2,000,000) |
| Max Bytes | max_bytes | number | No | 67108864 | min 1 | CSV/xlsx: fail when the file is larger than this many bytes (hard limit 512 MiB) |
| Empty As Null | empty_as_null | boolean | No | false | CSV/xlsx: empty cells become null instead of an empty string | |
| Normalise Headers | normalize_headers | boolean | No | false | CSV/xlsx: lower-case column names, strip accents, replace other characters with _ ("Data Naixement" -> data_naixement) | |
| Raw Cell Values | raw_values | boolean | No | false | xlsx: return unformatted cell values (date serial numbers, full-precision numbers) | |
| Source Field | source_field | string | No | CSV/xlsx: add this field to every record holding the file path (read) or the upstream path/key (parse), for lineage |
Errors
| Code | Description |
|---|---|
FILE_NOT_FOUND | The specified file does not exist |
PERMISSION_DENIED | Insufficient permissions to access the file |
PATH_SECURITY_VIOLATION | File path is outside the allowed base directory |
INVALID_OPERATION | The specified file operation is not recognized |
Runtime notes
- content output is only populated for read, read_all and parse; write/append/delete/list/exists return nil
- read_all reads every regular file directly in file_path (no subdirectories, no symlinks) matching pattern, in name order, bounded by max_files, max_file_bytes and max_total_bytes; result.truncated/truncated_reason report a cap that stopped it, result.skipped lists files not read, and success is false unless every matching file was read
- result shape varies by operation — read includes size/modified_time, write includes bytes_written, list includes files array
- Path traversal (..) is rejected; use base_directory config for sandboxed access
- With base_directory, a relative file_path resolves against it; containment uses filepath.Rel (base /x/in does not admit /x/inbox) and symbolic links may not lead outside it
- CSV read returns [][]string (rows of columns, header included) in content, as it always has, and the records in records; xlsx read and every parse return the records in content too
- CSV/xlsx values are never type-guessed: every cell is text as the file holds it (leading zeros kept); empty_as_null turns empty cells into null
- A CSV row with more or fewer fields than the header, bytes invalid in the declared encoding, an xlsx value right of the last header column, or more than max_rows rows fails the node — nothing is truncated or dropped silently
- xlsx: cells with Excel's built-in short-date format come back as YYYY-MM-DD; custom date formats come back as displayed (normalise them with transform parse_date)
- parse with a read_file result whose truncated flag is set fails: raise the reader's size cap instead of parsing a partial file
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.