Skip to main content

HDFS Read

Category: Data
Node id: cloudera-hdfs-read

Standards: HIPAA, GDPR

Read a single object's bytes from Cloudera HDFS (WebHDFS/httpFS). Server connection is configured on this node.

Runtime​

  • Performance tier: variable

Input ports​

NameTypeRequiredDescription
pathtextNoOptional HDFS path overriding config.path

Output ports​

NameTypeRequiredDescription
filebinaryNoThe object bytes
metajsonNo{path, size}
successbooleanNoTrue on HTTP 2xx

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
HDFS PathpathstringNoPath of the object to read (absolute, or relative to HDFS base).

Connection​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Impala Hostimpala_hoststringNoImpala coordinator host (Flight SQL / HS2). Required for Impala operations.
Impala Portimpala_portnumberNoImpala port (Flight SQL default 443/tls; HS2 21050).
Impala Protocolimpala_protocolenumNoflightsqlflightsql, hs2Wire protocol: Arrow Flight SQL (preferred) or HiveServer2 Thrift.
Databaseimpala_databasestringNoDefault Impala database/schema for unqualified table names.
Impala TLSimpala_tlsbooleanNotrueUse TLS for the Impala connection (grpc+tls for Flight SQL, https for HS2).
HS2 HTTP Pathimpala_http_pathstringNocliserviceHiveServer2 HTTP path for the hs2 protocol (Cloudera Data Warehouse uses 'cliservice').
HDFS / httpFS URLhdfs_urlstringNoWebHDFS/httpFS base URL, e.g. https://httpfs.cluster:14000. Required for HDFS operations.
HDFS Base Pathhdfs_basestringNoBase path that relative HDFS paths resolve against, e.g. /data/dicom.

Auth​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Impala Username (LDAP)impala_usernamestringNoLDAP user for Impala basic-auth (Cloudera Data Warehouse). Leave blank to use Kerberos.
Impala Password (LDAP)impala_passwordsecretNoLDAP password for the Impala user. Stored as a secret.
Kerberos Principalkerberos_principalstringNoService principal, e.g. tietai@REALM. Leave blank for a non-Kerberized cluster.
Kerberos Realmkerberos_realmstringNoRealm (defaults to the part after @ in the principal).
KDC Hostkerberos_kdcstringNoKey Distribution Center host, e.g. kdc.cluster.
Keytab (base64)keytab_base64secretNoBase64-encoded keytab for the principal. Stored as a secret; sent to Connect over the call.

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Mock (dev)mockbooleanNofalseUse an in-memory mock backend (no cluster/Kerberos) for local development and testing.

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Something unclear? Send documentation feedback by email.