Saltar al contenido principal

OpenAI LLM

Category: AI & ML
Node id: openai-llm

OpenAI LLM inference with streaming support and full parameter control

What it does​

  • OpenAI API integration with full parameter control
  • Streaming and non-streaming response modes
  • Multi-turn conversation support
  • Custom base URL for API-compatible endpoints
  • Token usage and chunk-level metadata tracking
  • Frequency and presence penalty tuning

Use cases​

  • Production LLM inference via OpenAI models (GPT-4, GPT-4o)
  • Streaming chat interfaces with real-time token delivery
  • Multi-turn conversational agents
  • Using OpenAI-compatible endpoints (Azure, local proxies)

Runtime​

  • Performance tier: variable

Input ports​

NameTypeRequiredDescription
prompttextNoSingle user prompt
messagesjsonNoMulti-turn conversation messages array

Output ports​

NameTypeRequiredDescription
resultjsonNoMain result containing response and finish_reason
metadatajsonNoExecution metadata including tokens, chunk_count, and stream
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
ModelmodelstringYesgpt-4OpenAI model name
System Promptsystem_prompttextNo""System prompt to prepend
StreamstreambooleanNofalseEnable streaming responses

Auth​

SettingNameTypeRequiredDefaultAllowed valuesDescription
API Keyapi_keysecretNoOpenAI API key (or set OPENAI_API_KEY env var)
Base URLbase_urlstringNo""Custom API base URL

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
TemperaturetemperaturenumberNo0.7min 0; max 2Sampling temperature
Max Tokensmax_tokensnumberNo1000min 1Maximum tokens to generate
Top Ptop_pnumberNo1min 0; max 1Nucleus sampling parameter
Frequency Penaltyfrequency_penaltynumberNo0min -2; max 2Penalize frequent tokens
Presence Penaltypresence_penaltynumberNo0min -2; max 2Penalize already-present tokens

Errors​

CodeDescription
API_KEY_MISSINGOpenAI API key not found in config or OPENAI_API_KEY env var
MODEL_REQUIREDModel name must be specified
API_CALL_FAILEDOpenAI API request returned an error
INPUT_REQUIREDEither prompt or messages input must be provided

Runtime notes​

  • result.response contains the full text; for streaming, chunks are in metadata.chunks
  • API key falls back to the OPENAI_API_KEY environment variable when not set in config
  • Provide either 'prompt' (single turn) or 'messages' (multi-turn); both are optional but one is required
  • When streaming, metadata.chunk_count > 0 and metadata.stream = true

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

¿Algo no está claro? Envíe sus comentarios sobre la documentación por correo electrónico..