HDFS Put (WebHDFS / httpFS)
Category: Data
Node id: cloudera-webhdfs-put
Standards: HIPAA, GDPR
Upload files straight to HDFS over WebHDFS/httpFS with its own connection (no Hydra Connect proxy). Direct + Kerberos (SPNEGO) to the cluster httpFS/NameNode when Knox is token-gated, or Knox Bearer token / HTTP Basic (LDAP). Single-file (file_path/data) or batch: wire a dicom-deid result (instances) to land one de-identified .dcm per instance.
What it does
- Direct WebHDFS/httpFS + Kerberos (SPNEGO)
- Knox Bearer / HTTP Basic (LDAP)
- Batch: one de-identified .dcm per instance
- Batch: per-study series index (_series.json)
- Handles the CREATE datanode redirect + re-auth
Runtime
- Performance tier:
variable
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
instances | json | No | A dicom-deid result envelope ({instances:[{file_path|deid_data, regenerated_uids,...}]}). Each emitted instance is written to <hdfs_path>/<deidStudyUID>/<deidSopUID>.dcm. Wire dicom_deid:result here. |
file_path | text | No | Single-file mode: local file to upload (e.g. a parquet-writer result.file_path on the same container). |
data | json | No | Single-file mode: inline bytes/string to write when no file_path is given. |
hdfs_path | text | No | Destination HDFS path (single-file) or directory (batch); overrides config. A trailing '/' is treated as a directory and the source basename appended. |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
result | json | No | Parsed JSON response payload from the vendor API |
metadata | json | No | Operation, HTTP status code, and content length |
success | boolean | No | True if HTTP 2xx |
Settings
Set at least one of: hdfs_url, knox_url.