RLM Query
Category: AI & ML
Node id: rlm-query
Maturity: beta
Risk tier: B
Executes Recursive Language Model queries: context is offloaded into a persistent Python REPL and the root model iteratively probes, chunks, and delegates it to sub-LLM calls instead of receiving it in the prompt
What it does
- Near-arbitrary context length via REPL offloading (context never enters the root prompt)
- Multi-turn orchestration with in-REPL sub-LLM calls (llm_query, llm_query_batched)
- Dual-model architecture (root orchestrator + cheaper sub-call model)
- Answer-dict sentinel termination with partial answers on turn/time/budget exhaustion
- FHIR navigation helpers inside the REPL for structured clinical contexts
- Optional per-turn execution trace for auditability
Use cases
- Whole-bundle FHIR analysis that exceeds a single context window
- Cross-document aggregation over large batches (e.g. many patient summaries)
- Complex question answering requiring programmatic decomposition and delegation
Runtime
- Performance tier:
slow
Input ports
| Name | Type | Required | Description |
|---|---|---|---|
query | text | No | Query to execute (overrides config query) |
context | any | No | Context data offloaded into the REPL (text or JSON; JSON stays structured) |
Output ports
| Name | Type | Required | Description |
|---|---|---|---|
result | json | No | RLM execution result: answer, status (completed | max_turns | timeout | budget_exhausted | error), turns, sub_calls, tokens_in, tokens_out, duration_ms, and trace when include_trace is set |
success | boolean | No | True when the model submitted an answer via the answer-dict sentinel (status completed) |
Settings
Basic
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Provider | provider | enum | No | openai | openai, anthropic, vllm, local | Model provider for in-process execution |
| Root Model | root_model | string | No | gpt-4o | Orchestrating model that writes REPL code | |
| Sub Model | sub_model | string | No | gpt-4o-mini | Model used by llm_query sub-calls (defaults to root_model when empty) | |
| Query | query | text | No | Static query (overridden by input) |
Auth
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| API Key | api_key | secret | No | API key for the provider (or the external service in legacy mode) | ||
| API Base URL | api_base_url | string | No | OpenAI-compatible base URL; required for vllm/local providers |
Advanced
| Setting | Name | Type | Required | Default | Allowed values | Description |
|---|---|---|---|---|---|---|
| Context Type | context_type | string | No | text | How the context input is loaded: text or json | |
| Max Turns | max_turns | number | No | 20 | min 1; max 100 | Maximum root-model reasoning turns |
| Max Execution Time (s) | max_execution_time | number | No | 300 | min 10; max 600 | Wall-clock budget for the whole run |
| Temperature | temperature | number | No | 0.7 | min 0; max 2 | Root model sampling temperature |
| Include Trace | include_trace | boolean | No | false | Attach the per-turn execution trace (code + truncated outputs; never the raw context) to the result | |
| External RLM Endpoint (legacy) | llm_endpoint | string | No | When set, delegates execution to POST {llm_endpoint}/rlm/execute instead of running in-process | ||
| Prompt Type (legacy) | prompt_type | string | No | general | Forwarded to the external service in legacy mode; the in-process executor uses the baseline orchestrator prompt |
Errors
| Code | Description |
|---|---|
QUERY_REQUIRED | No query provided via input or config |
CLIENT_BUILD_FAILED | Model client could not be constructed from provider/model/api settings |
ENV_START_FAILED | The Python REPL environment could not be started (no python3/python on PATH) |
RLM_EXECUTION_FAILED | The RLM run failed before producing a result |
HTTP_ERROR | Legacy mode: external RLM service returned an HTTP error status |
Runtime notes
- In-process mode (default) runs a persistent local Python subprocess as the REPL: model-generated code executes with OS-process isolation only. Do not point it at PHI-bearing contexts until the sandboxed environment ships; see docs/architecture/rlm-on-hydra-design.md
- Setting llm_endpoint switches to legacy external delegation (POST {llm_endpoint}/rlm/execute) and ignores provider/api_base_url
- Termination is the REPL answer-dict sentinel, never string parsing of model output; non-completed statuses still return partial answers
- Sub-LLM calls are capped per turn and bounded by max_execution_time overall
- The query input takes precedence over the static query in config
See Build any workflow for how to put nodes together and Ports, types and data flow for how port types connect.
Quelque chose n'est pas clair ? Envoyez vos commentaires sur la documentation par courriel.