Zum Hauptinhalt springen

ML Inference

Category: AI & ML
Node id: ml-inference

Calls external ML model endpoints for predictions (supports TensorFlow Serving, Triton, and generic JSON)

What it does​

  • Generic ML model endpoint integration via HTTP
  • Support for TensorFlow Serving, Triton, and JSON formats
  • Single and batch prediction modes
  • Custom HTTP headers for authentication
  • Configurable request timeout

Use cases​

  • Running predictions against hosted ML models
  • Batch inference for patient risk scoring
  • Integrating custom NLP or image classification models
  • Calling TensorFlow Serving or Triton endpoints

Runtime​

  • Performance tier: variable

Input ports​

NameTypeRequiredDescription
featuresjsonNoFeature map for a single prediction
feature_arrayjsonNoArray of feature sets for batch prediction
raw_inputtextNoRaw text input for text-based models

Output ports​

NameTypeRequiredDescription
resultjsonNoMain result containing predictions, model_name, http_status, and any additional endpoint response fields
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Model Namemodel_namestringNo""Name of the model to invoke
Input Formatinput_formatenumNojsonjson, tensorflow_serving, tritonRequest body format

Auth​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Model Endpointmodel_endpointstringYesURL of the ML model serving endpoint
API Keyapi_keysecretNoBearer token for the model endpoint
Custom Headerscustom_headersjsonNoAdditional HTTP headers as key-value pairs

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Timeout (seconds)timeout_secondsnumberNo120min 1Request timeout in seconds

Errors​

CodeDescription
ENDPOINT_REQUIREDmodel_endpoint configuration is missing
INPUT_REQUIREDfeatures, feature_array, or raw_input must be provided
INFERENCE_FAILEDHTTP request to the ML endpoint failed
HTTP_ERRORML endpoint returned an HTTP error status (4xx/5xx)

Runtime notes​

  • Result shape depends on the ML endpoint; predictions key is common but not guaranteed
  • If the response is not valid JSON, predictions is returned as a raw string
  • model_name and http_status are injected into the result map alongside endpoint fields
  • At least one of features, feature_array, or raw_input must be provided

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Etwas unklar? Senden Sie Feedback zur Dokumentation per E-Mail..