Aller au contenu principal

OpenAI LLM

Category: AI & ML
Node id: openai-llm

OpenAI LLM inference with streaming support and full parameter control

What it does​

  • OpenAI API integration with full parameter control
  • Streaming and non-streaming response modes
  • Multi-turn conversation support
  • Custom base URL for API-compatible endpoints
  • Token usage and chunk-level metadata tracking
  • Frequency and presence penalty tuning

Use cases​

  • Production LLM inference via OpenAI models (GPT-4, GPT-4o)
  • Streaming chat interfaces with real-time token delivery
  • Multi-turn conversational agents
  • Using OpenAI-compatible endpoints (Azure, local proxies)

Runtime​

  • Performance tier: variable

Input ports​

NameTypeRequiredDescription
prompttextNoSingle user prompt
messagesjsonNoMulti-turn conversation messages array

Output ports​

NameTypeRequiredDescription
resultjsonNoMain result containing response and finish_reason
metadatajsonNoExecution metadata including tokens, chunk_count, and stream
successbooleanNoWhether the operation succeeded

Settings​

Basic​

SettingNameTypeRequiredDefaultAllowed valuesDescription
ModelmodelstringYesgpt-4OpenAI model name
System Promptsystem_prompttextNo""System prompt to prepend
StreamstreambooleanNofalseEnable streaming responses

Auth​

SettingNameTypeRequiredDefaultAllowed valuesDescription
API Keyapi_keysecretNoOpenAI API key (or set OPENAI_API_KEY env var)
Base URLbase_urlstringNo""Custom API base URL

Advanced​

SettingNameTypeRequiredDefaultAllowed valuesDescription
TemperaturetemperaturenumberNo0.7min 0; max 2Sampling temperature
Max Tokensmax_tokensnumberNo1000min 1Maximum tokens to generate
Top Ptop_pnumberNo1min 0; max 1Nucleus sampling parameter
Frequency Penaltyfrequency_penaltynumberNo0min -2; max 2Penalize frequent tokens
Presence Penaltypresence_penaltynumberNo0min -2; max 2Penalize already-present tokens

Errors​

CodeDescription
API_KEY_MISSINGOpenAI API key not found in config or OPENAI_API_KEY env var
MODEL_REQUIREDModel name must be specified
API_CALL_FAILEDOpenAI API request returned an error
INPUT_REQUIREDEither prompt or messages input must be provided

Runtime notes​

  • result.response contains the full text; for streaming, chunks are in metadata.chunks
  • API key falls back to the OPENAI_API_KEY environment variable when not set in config
  • Provide either 'prompt' (single turn) or 'messages' (multi-turn); both are optional but one is required
  • When streaming, metadata.chunk_count > 0 and metadata.stream = true

See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.

Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.