ML Inference
Category: AI & ML
Node id: ml-inference
Calls external ML model endpoints for predictions (supports TensorFlow Serving, Triton, and generic JSON)
What it does
- Generic ML model endpoint integration via HTTP
- Support for TensorFlow Serving, Triton, and JSON formats
- Single and batch prediction modes
- Custom HTTP headers for authentication
- Configurable request timeout
Use cases
- Running predictions against hosted ML models
- Batch inference for patient risk scoring
- Integrating custom NLP or image classification models
- Calling TensorFlow Serving or Triton endpoints
Runtime
- Performance tier:
variable
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
features | json | No | Feature map for a single prediction |
feature_array | json | No | Array of feature sets for batch prediction |
raw_input | text | No | Raw text input for text-based models |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
result | json | No | Main result containing predictions, model_name, http_status, and any additional endpoint response fields |
success | boolean | No | Whether the operation succeeded |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Model Name | model_name | string | No | "" | Name of the model to invoke | |
| Input Format | input_format | enum | No | json | json, tensorflow_serving, triton | Request body format |
Auth
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Model Endpoint | model_endpoint | string | Yes | URL of the ML model serving endpoint | ||
| API Key | api_key | secret | No | Bearer token for the model endpoint | ||
| Custom Headers | custom_headers | json | No | Additional HTTP headers as key-value pairs |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Timeout (seconds) | timeout_seconds | number | No | 120 | min 1 | Request timeout in seconds |
Errors
| Code | Description |
|---|---|
ENDPOINT_REQUIRED | model_endpoint configuration is missing |
INPUT_REQUIRED | features, feature_array, or raw_input must be provided |
INFERENCE_FAILED | HTTP request to the ML endpoint failed |
HTTP_ERROR | ML endpoint returned an HTTP error status (4xx/5xx) |
Runtime notes
- Result shape depends on the ML endpoint; predictions key is common but not guaranteed
- If the response is not valid JSON, predictions is returned as a raw string
- model_name and http_status are injected into the result map alongside endpoint fields
- At least one of features, feature_array, or raw_input must be provided
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
¿Algo no está claro? Envíe sus comentarios sobre la documentación por correo electrónico..