Saltar al contenido principal

File

Category: Data
Node id: file

Performs file system operations (read, write, append, delete, list, exists) and parses file content handed over by another step (parse); CSV and Excel (.xlsx) become records keyed by the header row

What it does​

  • Eight operations: read, write, append, delete, list, exists, read_all (a folder in one run) and parse (content from another step)
  • Multi-format support: text, JSON, CSV, Excel (.xlsx read), binary
  • CSV/xlsx to records: delimiter auto-detection, UTF-8/Latin-1/Windows-1252, header row, sheet selection, row and size caps
  • Path security validation with base directory restriction
  • Automatic parent directory creation
  • Glob pattern filtering for directory listings

Use cases​

  • Read FHIR export files for pipeline processing
  • Write transformed data to JSON or CSV files
  • List files in a bulk export directory for batch ingestion
  • Turn a CSV read over SFTP, or an Excel workbook downloaded from a bucket, into records for a database load

Runtime​

  • Performance tier: medium

Input ports​

NameTypeRequiredDescription
file_pathtextNoFile path (overrides config)
contentanyNoContent to write or append; for parse, the file to parse: text, bytes, base64, or a read_file / object-store download result

Output ports​

NameTypeRequiredDescription
resultjsonNoFull file operation result with path, type, size, and metadata
contentanyNoFile content (read and parse), or for read_all an array of {path, name, size, modified, content} in name order
recordsjsonNoCSV / xlsx only: one object per data row, keyed by the header row
columnsjsonNoCSV / xlsx only: the record keys in file order
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
OperationoperationenumYesreadread, write, append, delete, list, exists, read_all, parseFile operation to perform; parse reads the content input instead of a local file
File Pathfile_pathstringNoPath to the file
File Typefile_typeenumNotexttext, json, csv, xlsx, binaryType of file for read/write/parse parsing

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Base Directorybase_directorystringNoDirectory every path must stay inside (symbolic links included); a relative file_path resolves against it
Create Directorycreate_directorybooleanNofalseCreate parent directories if they do not exist
Glob PatternpatternstringNoFile name pattern filter for list operation
Only Filesonly_filesbooleanNofalseExclude directories from list results
Add Newlineadd_newlinebooleanNotrueAppend a newline after content in append operation
Max Filesmax_filesnumberNo1000min 1; max 10000read_all: hard cap on files read per run
Max File Bytesmax_file_bytesnumberNo10485760min 1; max 67108864read_all: a larger file is skipped (listed in result.skipped), never cut
Max Total Bytesmax_total_bytesnumberNo67108864min 1; max 268435456read_all: hard cap on bytes read per run
Content Encodingcontent_encodingenumNoautoauto, text, base64parse: how a string content input is encoded; auto = text for csv/json/text, base64 for xlsx (a read_file result's own encoding field always wins)
CSV DelimiterdelimiterstringNoautoauto (detects , ; tab |) or one character; "tab" for tab
CSV EncodingencodingenumNoutf-8utf-8, latin-1, windows-1252Character encoding of CSV bytes; utf-8 must be valid UTF-8 (a BOM is dropped)
Header Rowheader_rownumberNo1min 01-based row holding the column names (rows above are skipped); 0 = no header, columns are column_1..column_N
Sheet NamesheetstringNoxlsx: sheet to read by name (default: the sheet at sheet_index)
Sheet Indexsheet_indexnumberNo1min 1xlsx: 1-based sheet position when sheet is not set
Max Rowsmax_rowsnumberNo100000min 1CSV/xlsx: fail when the file has more data rows than this (memory guard; hard limit 2,000,000)
Max Bytesmax_bytesnumberNo67108864min 1CSV/xlsx: fail when the file is larger than this many bytes (hard limit 512 MiB)
Empty As Nullempty_as_nullbooleanNofalseCSV/xlsx: empty cells become null instead of an empty string
Normalise Headersnormalize_headersbooleanNofalseCSV/xlsx: lower-case column names, strip accents, replace other characters with _ ("Data Naixement" -> data_naixement)
Raw Cell Valuesraw_valuesbooleanNofalsexlsx: return unformatted cell values (date serial numbers, full-precision numbers)
Source Fieldsource_fieldstringNoCSV/xlsx: add this field to every record holding the file path (read) or the upstream path/key (parse), for lineage

Errors​

CodeDescription
FILE_NOT_FOUNDThe specified file does not exist
PERMISSION_DENIEDInsufficient permissions to access the file
PATH_SECURITY_VIOLATIONFile path is outside the allowed base directory
INVALID_OPERATIONThe specified file operation is not recognized

Runtime notes​

  • content output is only populated for read, read_all and parse; write/append/delete/list/exists return nil
  • read_all reads every regular file directly in file_path (no subdirectories, no symlinks) matching pattern, in name order, bounded by max_files, max_file_bytes and max_total_bytes; result.truncated/truncated_reason report a cap that stopped it, result.skipped lists files not read, and success is false unless every matching file was read
  • result shape varies by operation — read includes size/modified_time, write includes bytes_written, list includes files array
  • Path traversal (..) is rejected; use base_directory config for sandboxed access
  • With base_directory, a relative file_path resolves against it; containment uses filepath.Rel (base /x/in does not admit /x/inbox) and symbolic links may not lead outside it
  • CSV read returns [][]string (rows of columns, header included) in content, as it always has, and the records in records; xlsx read and every parse return the records in content too
  • CSV/xlsx values are never type-guessed: every cell is text as the file holds it (leading zeros kept); empty_as_null turns empty cells into null
  • A CSV row with more or fewer fields than the header, bytes invalid in the declared encoding, an xlsx value right of the last header column, or more than max_rows rows fails the node — nothing is truncated or dropped silently
  • xlsx: cells with Excel's built-in short-date format come back as YYYY-MM-DD; custom date formats come back as displayed (normalise them with transform parse_date)
  • parse with a read_file result whose truncated flag is set fails: raise the reader's size cap instead of parsing a partial file

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

¿Algo no está claro? Envíe sus comentarios sobre la documentación por correo electrónico..