Aller au contenu principal

ML Inference

Category: AI & ML
Node id: ml-inference

Calls external ML model endpoints for predictions (supports TensorFlow Serving, Triton, and generic JSON)

What it does​

  • Generic ML model endpoint integration via HTTP
  • Support for TensorFlow Serving, Triton, and JSON formats
  • Single and batch prediction modes
  • Custom HTTP headers for authentication
  • Configurable request timeout

Use cases​

  • Running predictions against hosted ML models
  • Batch inference for patient risk scoring
  • Integrating custom NLP or image classification models
  • Calling TensorFlow Serving or Triton endpoints

Runtime​

  • Performance tier: variable

Input ports​

NameTypeRequiredDescription
featuresjsonNoFeature map for a single prediction
feature_arrayjsonNoArray of feature sets for batch prediction
raw_inputtextNoRaw text input for text-based models

Output ports​

NameTypeRequiredDescription
resultjsonNoMain result containing predictions, model_name, http_status, and any additional endpoint response fields
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Model Namemodel_namestringNo""Name of the model to invoke
Input Formatinput_formatenumNojsonjson, tensorflow_serving, tritonRequest body format

Auth​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Model Endpointmodel_endpointstringYesURL of the ML model serving endpoint
API Keyapi_keysecretNoBearer token for the model endpoint
Custom Headerscustom_headersjsonNoAdditional HTTP headers as key-value pairs

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
Timeout (seconds)timeout_secondsnumberNo120min 1Request timeout in seconds

Errors​

CodeDescription
ENDPOINT_REQUIREDmodel_endpoint configuration is missing
INPUT_REQUIREDfeatures, feature_array, or raw_input must be provided
INFERENCE_FAILEDHTTP request to the ML endpoint failed
HTTP_ERRORML endpoint returned an HTTP error status (4xx/5xx)

Runtime notes​

  • Result shape depends on the ML endpoint; predictions key is common but not guaranteed
  • If the response is not valid JSON, predictions is returned as a raw string
  • model_name and http_status are injected into the result map alongside endpoint fields
  • At least one of features, feature_array, or raw_input must be provided

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.