Skip to main content

Transform

Category: Transform
Node id: transform

Handles data transformation and mapping with support for field mapping, filtering, merging, extraction, and date normalisation to ISO-8601 (parse_date)

What it does​

  • Five transformation types: map, filter, merge, extract, parse_date
  • Date normalisation to ISO-8601 UTC with explicit formats, source time zone and DST checks; unparseable values listed, never guessed
  • Dot-notation field path access for nested data
  • Field renaming via source-to-target mappings
  • Deep merge and selective field extraction

Use cases​

  • Map FHIR resource fields to a flat analytics schema
  • Extract specific clinical data elements from complex objects
  • Merge patient demographics with clinical observations
  • Normalise admission and birth dates from a HIS export (dd/mm/yyyy, local time) to ISO-8601 before loading

Runtime​

  • Performance tier: fast

Input ports​

NameTypeRequiredDescription
inputjsonYesData to transform. The edge target_handle determines the key name in the inputs map.

Output ports​

NameTypeRequiredDescription
resultjsonNoTransformed data — the shape is defined by the mappings config (parse_date: the input record or array with the listed fields normalised)
errorsjsonNoparse_date only: one entry per value that could not be normalised (it is null in result)
error_countnumberNoparse_date only: number of entries in errors
successbooleanNoWhether the transformation completed without error

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Transformation Typetransformation_typeenumYesmapmap, filter, merge, extract, parse_dateType of transformation to apply
Mappingsmappingsfield-mappingYesWhich input fields land where. map: target <- source (+ optional transform, default); extract: target <- dot path; filter/merge: the fields to keep; parse_date: the date fields to normalise
Date FormatsformatsarrayNoparse_date: accepted formats, all tried for every value — ISO8601, RFC3339, token patterns (DD/MM/YYYY, YYYY-MM-DD HH:mm:ss) or Go layouts
Source Time Zonesource_timezonestringNoEurope/Madridparse_date: IANA zone for values without an offset

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Dates Without Timedate_onlyenumNodatedate, utcparse_date: date keeps YYYY-MM-DD (a date is not an instant); utc converts local midnight to a UTC instant
Repeated DST Hourambiguous_timeenumNoerrorerror, earlier, laterparse_date: a local time that occurs twice when clocks go back is an error, or the earlier / later instant when the source's convention is known
Fail On Unparseable Datefail_on_errorbooleanNofalseparse_date: fail the node when any value cannot be normalised instead of returning it as null in errors

Errors​

CodeDescription
UNSUPPORTED_TRANSFORMATIONThe specified transformation_type is not recognized
INVALID_MAPPINGSMappings configuration is missing or invalid
FIELD_NOT_FOUNDA referenced source field does not exist in the input data

Runtime notes​

  • The executor delivers upstream data into the inputs map keyed by the edge's target_handle. If the edge is source_handle='result' target_handle='input', then inputs['input'] contains the upstream result.
  • MAP mode: mappings is {"output_key": "input_key_or_dot.path"}. Supports dot-notation for nested access (e.g. 'input.data.first_name').
  • EXTRACT mode: mappings is {"output_key": "dot.notation.path"}. Always uses dot-notation to traverse nested objects.
  • For both map and extract, the dot-notation path starts from the inputs map root. If your edge target_handle is 'input', paths start with 'input.' (e.g. 'input.data.first_name').
  • PARSE_DATE mode: mappings is the list of top-level date fields; input is a record or an array of records (e.g. connector records); result has the same shape with each listed field as ISO-8601: date-times as UTC instants (2026-03-01T09:15:00Z), dates without a time kept as dates (2026-03-01) unless date_only is utc.
  • PARSE_DATE tries every listed format: a value no format reads, or that two formats read differently (01/03/2026 under DD/MM/YYYY and MM/DD/YYYY), or a local time in a DST gap/overlap, becomes null and is listed in errors with record, field, value and reason — never guessed. A missing field is listed too. Empty strings and nulls stay null without an error.
  • PARSE_DATE formats: presets ISO8601 (with/without offset, T or space, optional seconds/fraction, or a bare date) and RFC3339; token patterns DD/MM/YYYY, D/M/YYYY, YYYY-MM-DD HH:mm:ss, DD.MM.YYYY HH:mm:ss.SSS, DD-MMM-YYYY, …Z (offset); or Go layouts containing 2006. Two-digit years (YY) and zone abbreviations are refused.

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Something unclear? Send documentation feedback by email.