Introduction

rho is a Rust coding agent that runs against local and remote LLMs.

It connects through native OpenAI Responses or OpenAI-compatible Chat Completions endpoints (LM Studio, Ollama, Groq, OpenRouter, DeepInfra, Z.ai, and more), gives the model access to tools for reading and writing files, executing shell commands, and running Rust tooling—then runs an autonomous agent loop supervised through an approval gate.

rho is a headless JSON-RPC 2.0 agent over stdin/stdout, suitable for embedding in editors, bots, and custom UIs.

See RPC Mode for the full protocol reference.

Design priorities:

• Local first — your code stays on your machine by default
• Remote ready — external providers (OpenAI, Groq, etc.) with consent warnings
• Safe by default — destructive actions require your approval
• Rust-native — structured compiler diagnostics, tree-sitter syntax analysis, not text scraping
• Extensible — custom tools and hooks via TypeScript extensions (V8/deno-core), hot-reloadable at runtime

Platform support

rho runs on Windows, macOS, and Linux. PowerShell 7+ (pwsh) is the primary shell on all platforms; Windows PowerShell 5.1 (powershell) is the fallback on Windows only.

What can rho do?

• Read, write, and edit files within a sandboxed project directory
• Execute shell commands with a safety denylist
• Run cargo check, cargo clippy, cargo test, cargo fix, and rustc --explain with structured output
• Detect syntax node splits during edits via tree-sitter
• Manage conversation state across sessions with tree-structured persistence and session discovery
• Observe agent activity in real time (reasoning, tool calls, errors) via the AgentObserver trait
• Get structured results from run_loop via AgentResult (reply, iterations, token usage, tool call history, duration, finish reason) without custom observers
• Extend rho with TypeScript extensions that add tools, hooks, and commands — hot-reloadable at runtime
• Compact old conversation turns to stay within context limits
• Monitor context window usage with a live status bar, getSessionStats RPC method, and enhanced token distribution stats
• Graduated resolution (Outlined/Summarized) with selective per-entry eviction before turn-level eviction
• Phase-aware and LLM-assisted compaction for high-fidelity context summaries
• Run headless via JSON-RPC 2.0 over stdin/stdout for integration with editors, bots, and custom UIs

This book documents the architecture, security model, configuration, and development of rho.

Getting Started

Prerequisites

• Rust toolchain (edition 2024)
• PowerShell 7+ (pwsh) — required for shell command execution
• A local LLM server (LM Studio, Ollama) running on localhost:1234, or an API key for an external provider (OpenAI, Groq, OpenRouter, DeepInfra, etc.)

Local model (default)

No configuration needed — just start a local server on localhost:1234 and run rho. Set the model explicitly with --model:

rho --model qwen3-8b

Or configure a default model in ~/.rho/config.toml:

[[providers]]
preset = "lm-studio"
default_model = "qwen3-8b"

External provider

Create ~/.rho/config.toml with a provider preset and default model. See External Providers for worked examples.

[[providers]]
preset = "openrouter"
api_key_env = "OPENROUTER_API_KEY"
default_model = "deepseek-v4-flash"

[agent]
provider = "openrouter"

No configuration?

If you run rho with no local server and no configured provider, rho falls back to a built-in default model (anthropic/claude-sonnet-4 via OpenRouter). This requires an OpenRouter API key — set OPENROUTER_API_KEY in your environment or configure a provider explicitly. Set the model in config or pass --endpoint <url> --model <id> on the command line.

Build

cargo build --release -p rho        # headless RPC agent

Run

rho                      # persist session to disk (headless RPC)
rho -m my-model          # specify a model
rho -c                   # resume the most recent session for this project
rho --session <path>     # resume a specific session
rho --ephemeral          # in-memory mode, no session file

rho runs as a headless JSON-RPC 2.0 agent over stdin/stdout. Send prompts and receive streaming responses via the protocol. See RPC Mode for the full protocol reference.

Example scripted use:

echo '{"jsonrpc":"2.0","method":"prompt","params":{"message":"fix the bug"},"id":1}' | rho --model <id>

On startup, rho:

1. Auto-detects the project root by walking up from the current directory looking for markers.
2. Loads configuration from ~/.rho/config.toml and .rho/config.toml.
3. Scans for project context files (AGENTS.md, etc.) and auto-denies untrusted files in headless mode.
4. Checks provider consent for external endpoints.
5. Resolves the model from config (agent.model, agent.provider, or a provider's default_model) or CLI --model. Falls back to a built-in default model if none configured. No network calls at startup.
6. Creates a session (persisted to ~/.rho/sessions/ by default, or in-memory with --ephemeral). If previous sessions exist, a hint is printed.
7. Enters the RPC loop and waits for commands.

CLI flags (rho)

The rho binary runs in headless JSON-RPC 2.0 mode.

Session persistence

By default, every rho invocation creates a new JSONL session file under ~/.rho/sessions/<project-hash>/. Sessions survive process restarts — use rho -c to resume the latest session or --session <path> to resume a specific one.

The listSessions RPC method returns the 10 most recent sessions with timestamps, sizes, and entry counts. The latest session is marked so you know which one rho -c will pick.

See Sessions for details.

RPC methods

rho communicates via JSON-RPC 2.0 over stdin/stdout. See RPC Mode for the full protocol reference, including all methods, notifications, and the approval flow.

Next Steps

• Architecture — how the crates fit together
• Core Concepts — sessions, agent loop, context management
• Security — the threat model and defenses
• Configuration — customizing behavior
• RPC Mode — headless JSONL integration

Architecture

rho is organized as a layered workspace where dependencies flow downward only.

┌─────────────────────────────────────────────────┐
│                   rho (binary)                   │  ← Headless JSON-RPC 2.0 agent
├─────────────────────────────────────────────────┤
│  rho-ext          rho-tools                       │  ← Extensions   /  Built-in tools
├─────────────────────────────────────────────────┤
│            rho-memory      rho-highlight           │  ← Siblings of rho-tools
├─────────────────────────────────────────────────┤
│                   rho-core                       │  ← Agent kernel (loop, types, traits, data model)
├─────────────────────────────────────────────────┤
│                   rho-ai                         │  ← Unified LLM provider abstraction
└─────────────────────────────────────────────────┘

rho-ai is the lowest layer; it depends only on external libraries. rho-core depends on rho-ai for the LlmService trait. Every other crate eventually depends on rho-core.

rho-ext, rho-tools, rho-memory, and rho-highlight are all siblings sitting directly on rho-core. rho-tools additionally depends on rho-memory and rho-highlight; the others each depend only on rho-core.

See also:

• Workspace Layout
• Crate Responsibilities
• Dependency Flow

Workspace Layout

rho-coding-agent/
├── Cargo.toml              # Workspace root — version, edition, lints
├── CHANGELOG.md             # Generated via git-cliff
├── cliff.toml               # git-cliff configuration
├── AGENTS.md                # Project instructions for AI assistants
├── docs/                    # This book (mdBook)
│   └── rpc-schema/
│       └── openrpc.json    # OpenRPC 1.3.1 schema (machine-readable API spec)
├── rho/                     # Binary entry point (headless JSON-RPC 2.0)
│   └── src/
│       ├── main.rs           # Thin: parse CLI, build App, run
│       ├── lib.rs            # Module declarations
│       ├── cli.rs            # `Cli` — CLI flags with clap
│       ├── app.rs            # `App` — runtime state, build/run orchestration, extension loading
│       ├── model.rs          # Model resolution (config-as-truth, provider-first)
│       ├── ext_observer.rs   # `CompositeObserver` — fans out to RPC observer + extension observers
│       ├── rpc.rs            # JSON-RPC 2.0: `run_rpc`, `run_rpc_on`, observer, approval gate
│       ├── rpc_wire.rs        # Typed wire-format structs (params, results, notifications)
│       ├── transport.rs        # `Transport` trait, `StdioTransport` (newline-delimited JSON)
│       └── presenter/
│           └── rpc.rs        # `RpcPresenter` — diagnostic output to stderr
├── rho-ai/                 # Unified LLM provider abstraction
│   └── src/
│       ├── lib.rs           # Re-exports: `LlmService`, `EventStream`, unified types
│       ├── service.rs       # `LlmService` trait
│       ├── openai.rs        # `OpenAiService` — Chat Completions HTTP + SSE
│       ├── responses.rs     # `ResponsesService` — OpenAI Responses HTTP + SSE
│       ├── sse.rs           # Server-sent event parser
│       ├── retry.rs         # Exponential backoff retry logic
│       ├── catalog.rs       # `Catalog`, `Model`, `ModelCost` — model registry + pricing
│       ├── catalog_generated.rs # Generated model list (`cargo xtask generate-models`)
│       ├── types.rs         # `LlmMessage`, `LlmRequest`, `StreamEvent`, `AccumulatedResponse`
│       └── error.rs         # `ProviderError`
│   └── model-overrides.json # Manual catalog overrides (thinking flags, compatibility)
├── rho-ext/                # TypeScript extension runtime (V8/deno-core)
│   └── src/
│       ├── lib.rs          # Re-exports: `ExtensionLoader`, `DenoTool`, `DenoObserver`, etc.
│       ├── runtime.rs      # `ExtensionRuntime` — V8 isolate on dedicated OS thread
│       ├── async_dispatcher.rs # Async bridge between V8 isolate and tokio
│       ├── loader.rs       # `ExtensionLoader` — discovery, filtering, spawning, hot reload
│       ├── deno_tool.rs    # `DenoTool` — `Tool` trait wrapper
│       ├── deno_observer.rs# `DenoObserver` — `AgentObserver` wrapper
│       ├── manifest.rs     # `LoadedExtension` — manifest parsing and validation
│       ├── discover.rs     # Extension discovery (single-file and multi-file)
│       ├── transpile.rs    # TS → JS transpilation
│       ├── module_loader.rs# `RhoModuleLoader` — ESM module resolution
│       ├── host.rs         # `rho.*` host ops (log, readFile, writeFile, runCommand, getModel, getCwd)
│       ├── ops.rs          # Helper macros for host op boilerplate
│       ├── error.rs        # `ExtensionError`
│       ├── host_shim.js    # ESM shim for extension module loading
│       └── std_shim.js    # Standard library shim for extensions
│   └── types/
│       └── rho.d.ts        # TypeScript type definitions for extension authors
├── rho-core/                # Agent kernel
│   └── src/
│       ├── lib.rs           # Module declarations, convenience re-exports
│       ├── agent.rs         # Agent loop state machine, `run_loop`, `AgentResult`, `CollectingObserver`, `LoopFinishReason`, phase tracking, auto-compact
│       ├── approval.rs      # `ApprovalPolicy`, `ApprovalGate` traits
│       ├── client.rs        # `RhoAiClient`, `resolve_api_key`, `is_local_endpoint`, `ModelInfo`, `ModelList`
│       ├── client/          # Module directory
│       │   └── error.rs      # `ClientError`
│       ├── config.rs        # `RhoConfig`, `ConfigLoader`, two-tier TOML loading
│       ├── context.rs       # `ContextManager`, `SlidingWindowContextManager`, `TokenBudget`
│       ├── context_files.rs # Project context file scanner, `TrustStore`, prompt composition
│       ├── conversation.rs  # `Conversation`, `AssistantResponse`
│       ├── denylist.rs      # `CommandDenylist` — shell command denylist
│       ├── diagnostic.rs    # Structured compiler diagnostic types
│       ├── error.rs         # `RhoError` and `Result`
│       ├── message.rs       # `ChatMessage`, `ContentBlock`, `ModelToolCall`
│       ├── model_match.rs   # Model name matching / fuzzy matching
│       ├── newtypes.rs      # `FilePath`, `ToolName`, `ToolCallId`, `EntryId`, `DiagnosticCode`
│       ├── prompts.rs       # `base_prompt()`, `compact_prompt()` (embedded from `prompts/`)
│       ├── redact.rs        # `Redactor` — secret pattern matching
│       ├── request.rs       # `ChatRequest`
│       ├── response.rs      # `ModelResponse`, `FinishReason`, `ModelUsage`
│       ├── sandbox.rs       # `SandboxRoot` — file sandbox validation
│       ├── schema.rs        # `ToolSchema` — wire-format tool definitions
│       ├── provider.rs      # `Provider` trait, `OpenAiCompatibleProvider`, `ProviderRegistry`
│       ├── session.rs       # `Session` struct, module declarations, re-exports
│       ├── session/
│       │   ├── accessors.rs    # Header, leaf, entry, flush, path, getters/setters
│       │   ├── append.rs       # Append ops, close, extension typed methods, append_entry core
│       │   ├── builder.rs      # `new`, `in_memory`, `with_*` builder methods
│       │   ├── compaction.rs   # `CompactionStrategy`, `MechanicalCompactionStrategy`, `LlmCompactionStrategy`
│       │   ├── context.rs      # `path_messages`, `send_current`, `compact_older_than`, `context_stats`, `prepare_context`
│       │   ├── context_stats.rs # `ContextStats`, `RoleTokenDistribution`, `ResolutionTokenDistribution`, `PhaseTokenDistribution`
│       │   ├── entry.rs        # `Entry`, `EntryPayload`, `EntryResolution`, `CompactionPhase`
│       │   ├── error.rs        # `SessionError`
│       │   ├── estimator.rs    # `TokenEstimator`, `HeuristicEstimator`
│       │   ├── eviction.rs     # Selective downgrade planner (`plan_downgrades`)
│       │   ├── extensions.rs   # `ExtensionEntry`, `ExtensionMessageEntry` traits
│       │   ├── header.rs       # `SessionHeader`
│       │   ├── outliner.rs     # Tool-specific structural summary formatters
│       │   ├── persist.rs      # JSONL persistence, `SessionMetadata`
│       │   ├── phase.rs        # `SessionPhase` enum, `classify_tool_phase()`, `transition_phase()`
│       │   ├── tree.rs         # `path_to_root`, `branch_to`, `branch_with_summary`
│       │   └── truncation.rs   # Bounded truncation, char boundary, footer
│       ├── shell.rs         # `ShellExecutor` trait, `ShellOutput`
│       ├── stream.rs        # `StreamChunk` — unified streaming event type
│       └── tool.rs          # `Tool` trait, `ToolRegistry`, `ToolRisk`, `ToolOutcome`
├── rho-memory/             # Persistent knowledge base (SQLite/FTS5)
│   ├── src/
│   │   ├── lib.rs           # Re-exports: `Memory`, `Document`, `SearchResult`
│   │   ├── brain.rs         # `Memory` — public knowledge-base API
│   │   ├── db.rs            # `Database` — raw SQLite operations
│   │   ├── models.rs        # `Document`, `SearchResult`, `CreateRequest`, `Stats`
│   │   └── error.rs         # Knowledge-base errors
│   └── migrations/
│       └── 001_initial.sql  # Schema: documents, FTS5, triggers
├── rho-highlight/           # Tree-sitter syntax analysis
│   └── src/
│       ├── lib.rs           # Re-exports
│       ├── lang.rs          # `Language` enum
│       ├── parse.rs         # Tree-sitter parsing
│       ├── highlight.rs     # Token classification and highlighting
│       ├── query.rs         # `node_at()` — AST node lookup by position
│       └── error.rs         # `HighlightError`
├── rho-tools/               # Built-in tool implementations
│   └── src/
│       ├── lib.rs           # `register_all()`
│       ├── files.rs         # Re-exports file tools from `file_ops` + `edit`
│       ├── file_ops.rs      # `ReadFile`, `WriteFile`, `BatchRead`, `ListDir`
│       ├── edit.rs          # `EditFile` — hashline-anchored editing
│       ├── hashline.rs      # Hashline content-addressed editing
│       ├── shell.rs         # `RunCommand`, `PowerShellExecutor`, `CommandDenylist`
│       ├── crates_io.rs     # `CratesIoLookup`
│       ├── session_summary.rs # `SessionSummary` — compressed session history tool
│       ├── memory.rs        # `MemoryTool` — knowledge base (via `rho-memory`)
│       ├── error.rs         # Tool error types
│       └── rust/            # Rust tooling (module root: `rust.rs`)
│           ├── tools.rs     # `CargoCheck`, `CargoClippy`, `CargoTest`, `CargoFix`, `RustcExplain`
│           ├── rustdoc.rs   # `RustdocTool` — local rustdoc HTML parsing
│           ├── parse.rs     # NDJSON output parsing
│           ├── convert.rs   # Diagnostic conversion
│           ├── format.rs    # Diagnostic formatting
│           └── types.rs     # Shared Rust tool types
├── rho-test-helpers/        # Shared test infrastructure (dev-only)
│   └── src/lib.rs           # `MockChatClient`, `TestProvider`, response builders, helpers
└── xtask/                   # Dev task runner
    └── src/
        ├── main.rs          # CLI dispatch
        └── tasks.rs         # `ci`, `test`, `build`, `release`, `changelog`, `fmt`, `fmt-fix`, `lint`, `run`, `clean`, `status`, `schema`, `generate-models` tasks

Crate Responsibilities

rho (binary)

The headless JSON-RPC 2.0 agent: CLI argument parsing, tool wiring, extension loading, and system prompt assembly. Reads newline-delimited JSON-RPC 2.0 requests from stdin and writes responses/notifications to stdout. Diagnostic output goes to stderr.

Constructs a Session, connects to the model via a Provider, and drives the agent loop. Handles provider consent checks, model resolution, startup budget diagnostics, shell-specific prompt extensions, session discovery, extension loading, and live observer output via RpcObserver.

The RPC implementation is decoupled from I/O via the Transport trait (rho/src/transport.rs); StdioTransport (newline-delimited JSON over stdin/stdout) is the default, and in-process tests inject a StdioTransport wired to canned readers and captured writers. Wire-format types in rpc_wire.rs give every method param, result, and notification a typed Rust struct. See RPC Mode for the full protocol reference and OpenRPC schema for machine-readable API discovery.

rho-core

The kernel: agent loop state machine (with AgentObserver for live output, CollectingObserver for structured tool call recording, phase tracking, auto-compaction) returning AgentResult with full structured output (reply, iterations, token usage, tool call history, duration, finish reason, context stats), session management (tree persistence, graduated resolution, phase-aware compaction, LLM compaction, selective eviction, branching, session discovery), context window management (sliding window, token estimation, enhanced ContextStats with role/resolution/phase token distributions), tool registry and traits, approval gates, secret redaction, file sandbox, provider abstraction (Provider trait, ProviderRegistry with named presets and additive config merge), LLM client (RhoAiClient), configuration loading (two-tier TOML with provider preset resolution), and all shared types.

Key types: Session, AgentConfig, AgentObserver, NopObserver, CollectingObserver, AgentResult, TokenUsage, ToolCallRecord, ToolCallOutcome, LoopFinishReason, LoopParams, ContextStats, RoleTokenDistribution, ResolutionTokenDistribution, PhaseTokenDistribution, SessionMetadata, ToolRegistry, Tool, Provider, ProviderRegistry, ContextManager, TokenBudget, SandboxRoot, Redactor, RhoConfig, SessionPhase, CompactionPhase, CompactionStrategy, LlmCompactionStrategy, MechanicalCompactionStrategy, CommandDenylist.

rho-highlight

Tree-sitter-based syntax analysis. Provides parse(), highlight(), and node_at() for Rust source code. Used by EditFile for node-splitting validation and by tools that need structural code awareness.

Key types: Language, HighlightSpan, HighlightTag, NodeInfo.

rho-tools

Built-in tool implementations:

• File tools: ReadFile, BatchRead, WriteFile, ListDir, EditFile
• Shell tools: RunCommand, PowerShellExecutor, CommandDenylist
• Rust tools: CargoCheck, CargoClippy, CargoTest, CargoFix, RustcExplain
• Lookup tools: RustdocTool, CratesIoLookup
• Session tools: SessionSummary (compressed turn history for context recovery)
• Knowledge base: MemoryTool (project-local docs via rho-memory, gated by [memory] enabled)

Each implements the Tool trait from rho-core.

rho-memory

Persistent knowledge base for project-local docs: full-text search (SQLite FTS5), content deduplication, and soft deletes. Exposed to the agent through the MemoryTool in rho-tools. Storage lives at <project-root>/.rho/memory.db. Gated by the [memory] config section (enabled = true).

Key types: Memory, Database, Document, SearchResult.

rho-ext

TypeScript extension runtime (powered by deno_core and V8). Enables user-authored extensions that add tools, hooks, and commands to rho.

• ExtensionRuntime — owns a V8 isolate on a dedicated OS thread, handles async tool/hook/command calls via tokio channels
• ExtensionLoader — orchestrates discovery (single-file and multi-file), config-driven filtering, spawning, tool registration, and hot reload (mtime-based change detection)
• DenoTool — wraps extension tool functions as Box<dyn Tool> for the ToolRegistry
• DenoObserver — wraps extension hooks as AgentObserver for the agent loop (onLoad, onToolCall, onToolResult, onBeforeModel)
• TypeScript transpilation — deno_ast transpiles .ts to JS at load time
• Host functions — rho.log(), rho.readFile(), rho.writeFile(), rho.runCommand(), rho.getModel(), rho.getCwd() with permission gating
• Type definitions — ships rho.d.ts for extension author IntelliSense

Key types: ExtensionRuntime, ExtensionLoader, DenoTool, DenoObserver, LoadedExtension, ExtensionError.

See Extensions for the full extension system documentation.

rho-test-helpers

Shared test infrastructure (dev-only): MockChatClient, TestProvider, MockShellExecutor, AutoApproveGate, AutoDenyGate, FixedResponseTool, FailingTool, FileTestEnv, in_memory_session, empty_trust_store, detect_shell, tempdir_with_sandbox, assert_no_orphan_tool_results. Depends on rho-core and rho-ai. Never published.

TestProvider wraps a MockChatClient as a Provider impl, enabling integration tests that need a ProviderRegistry without a live model server.

xtask

Dev task runner: cargo xtask ci (fmt → lint → build → test), cargo xtask test, cargo xtask build, cargo xtask release, cargo xtask changelog, cargo xtask fmt, cargo xtask fmt-fix, cargo xtask run, cargo xtask clean, cargo xtask status, cargo xtask schema, cargo xtask generate-models. Tests use cargo-nextest when available, falling back to cargo test.

Dependency Flow

Dependencies flow downward only. A crate may depend on crates below it in the stack but never above.

rho (binary) ──────────────────────────────────────────
    │
    ├── rho-ext ──── rho-core ──── rho-ai ──────────────
    │
    ├── rho-tools ── rho-highlight ── rho-core ─────────
    │           └──── rho-memory ── rho-core ────────────
    │
    └── rho-core ──── rho-ai ──────────────────────────

rho-test-helpers ── rho-core, rho-ai ────────────────
xtask ──────────────────────────────────────────────

Key external dependencies

What this buys you

The layered structure means:

• rho-core is UI-agnostic — no shell, no filesystem, no terminal. Any frontend (REPL, TUI, editor integration) consumes it through the JSON-RPC 2.0 protocol.
• Tools are pluggable — ToolRegistry stores Box<dyn Tool>, so adding a tool is registry.register(Box::new(MyTool)).
• Testing is isolated — rho-test-helpers mocks the core traits (MockChatClient, MockShellExecutor, ApprovalGate) so tool and agent loop tests run without a model server.

Core Concepts

The agent kernel in rho-core is built on a small set of interlocking concepts.

• Agent Loop — the state machine that drives the conversation, returning AgentResult with structured output
• Tool Trait — the interface all tools implement
• Provider Architecture — provider-agnostic model access
• Approval Policy — the gate between tool calls and execution
• Context Management — keeping the conversation within the context window
• Sessions — persistent, tree-structured conversation state
• Project Context Files — loading and trusting project-level instructions

Agent Loop

The agent loop is the state machine at the heart of rho. It drives the conversation forward by sending messages to the model, executing tool calls, and feeding results back — tracking session phase and managing context utilization throughout.

State machine

  ┌──────────────────────────────────────────────┐
  │                                              │
  ▼                                              │
Thinking ──► AwaitingApproval ──► ExecutingTool ──┘
  │
  ▼
Idle (done)

• Thinking: send the session to the model and wait for a response. Stream timeouts (first_token_timeout_secs, stream_idle_timeout_secs) convert hung streams into retryable errors.
• AwaitingApproval: if a tool call requires human confirmation, ask the user. On denial, feed a synthetic "denied" result and return to Thinking. On redirect (approved: false with a message), inject the user's instructions as a new turn and return to Thinking — the tool batch is abandoned.
• ExecutingTool: run the tool and append the result to the session.
• Idle: the model returned a text reply — the loop is done.

The run_loop function

#![allow(unused)]
fn main() {
pub async fn run_loop(
    session: &mut Session,
    message: &str,
    params: &LoopParams<'_>,
) -> Result<AgentResult>
}

run_loop takes a Session and a LoopParams (bundle of LLM service, tool registry, config, cancellation token, approval gate, observer, and optional compaction client). A CollectingObserver is always active internally — it records tool call events into AgentResult.tool_calls automatically, so consumers don't need custom observers for structured output.

It returns Result<AgentResult>, which captures:

The loop:

1. Appends message as a user turn via session.append_user_message().
2. Enters the Thinking state and sends the session to the model via session.send_current().
3. Processes the model's response:
   • Text reply → record phase as Conclusion, populate AgentResult and return it (Idle).
   • Tool calls → for each call, check approval, execute, record phase transition, append the result, then loop back to Thinking.

All tool calls in a single model response are executed sequentially. Each result is appended before the session is re-sent to the model. Parallel execution is a future optimisation.

Mid-turn steering

A SteeringSource can be provided via LoopParams.steering. At the tool-batch seam — after all tool calls in a batch have executed and before returning to Thinking — the loop drains any queued steering messages and injects each as a user message. This lets the user provide course corrections mid-turn without interrupting the active execution. In RPC mode, the concurrent reader task routes prompt messages with steer: true to a SteeringQueue that backs the SteeringSource.

Stream timeouts

The loop applies two configurable timeouts when consuming the model's stream:

• first_token_timeout_secs (default 90) — maximum wait for the first stream event. Guards against hung requests where the server accepts the connection but never emits a token.
• stream_idle_timeout_secs (default 60) — maximum gap between consecutive stream events once streaming has started. Guards against mid-stream stalls.

Both return a retryable ClientError::StreamTimeout so the existing exponential-backoff retry loop handles them. Set to 0 to disable either timeout. Compaction's LLM call always uses StreamTimeouts::none() (it has its own mechanical fallback).

finish_reason persistence

Every assistant message records an optional FinishReason (Stop, ToolCalls, Length, ContentFilter), serialized OpenAI-style (snake_case) and omitted when None for backward compatibility with existing session files. Terminal errors (e.g. retry budget exhausted) are recorded as Attached custom-state entries (kind rho.turn_error.v1) so silent failures leave a diagnostic trace on disk.

Phase tracking

The loop tracks the current SessionPhase (Exploration, Execution, Verification, Conclusion) and transitions on each tool execution:

• Reading files / listing directories → Exploration
• Editing files / writing files → Execution
• Running cargo check / cargo test / cargo clippy → Verification
• Final text reply → Conclusion

Phase detection uses classify_tool_phase() with has_had_edits context for ambiguous tools. See Context Management for details.

Auto-compact

When context utilization crosses the auto_compact_threshold, the loop proactively compacts older entries using the configured compaction strategy:

• Mechanical (default) — deterministic, structured summaries with phase-aware narratives
• LLM (opt-in via compaction_mode = "llm") — adds narrative notes generated by the model

On LLM failure, falls back to mechanical compaction. See Context Management for details.

AgentObserver

The AgentObserver trait receives live events from the agent loop so that a REPL, TUI, or test harness can render progress as it happens.

#![allow(unused)]
fn main() {
pub trait AgentObserver: Send + Sync {
    async fn on_state_change(&self, _state: AgentState) {}
    async fn on_text_delta(&self, _delta: &str) {}
    async fn on_reasoning_delta(&self, _delta: &str) {}
    async fn on_tool_call(&self, _name: &str, _arguments: &str) {}
    async fn on_tool_result(&self, _name: &str, _result: &ToolResult) {}
    async fn on_tool_denied(&self, _name: &str) {}
    async fn on_approval_requested(&self, _tool_name: &str, _risk: ToolRisk) {}

    /// Called before a tool executes. Return Block to prevent execution.
    /// Used by extensions to implement approval gates and safety filters.
    /// Synchronous because it is a pure policy decision (no I/O).
    fn on_tool_call_intercept(&self, _name: &str, _args: &str) -> Option<InterceptResult> {
        None
    }

    /// A model response completed and its token usage was accumulated.
    /// Fires once per iteration that hit the model, carrying the per-iteration
    /// delta and a live context snapshot — useful for rendering a live
    /// context/cost gauge during long multi-iteration turns.
    async fn on_usage(
        &self,
        _iteration: u32,
        _usage: &IterationUsage,
        _context: &crate::session::ContextStats,
    ) {
    }
}
}

CollectingObserver

run_loop always creates a CollectingObserver internally — it records tool call events (name, arguments, outcome, duration) into ToolCallRecords that populate AgentResult.tool_calls. This is independent of the observer in LoopParams and requires no action from consumers. Any observer passed via LoopParams still receives all callbacks as before; the CollectingObserver runs alongside it.

When extensions are loaded, the CompositeObserver fans out every call to both the RPC observer and the extension DenoObservers. For on_tool_call_intercept, the first Block wins — if any extension blocks a tool call, execution is denied immediately.

The observer is called:

• At every state transition (on_state_change)
• As streaming deltas arrive (on_text_delta, on_reasoning_delta)
• Before and after each tool call (on_tool_call, on_tool_result, on_tool_denied)
• When approval is requested (on_approval_requested)

Retry with backoff

Transient HTTP errors (503, 429, connection refused) are retried with exponential backoff. The retry budget is configurable via AgentConfig. If the budget is exhausted, the loop terminates with AgentResult.finish_reason = LoopFinishReason::RetryBudgetExhausted.

Cancellation

A CancellationToken is checked at the top of each loop iteration and between tool calls in a batch. When cancelled, the loop terminates with AgentResult.finish_reason = LoopFinishReason::Cancelled immediately — no pending tool execution is aborted mid-flight, but no new ones are started.

Iteration guard

The loop terminates after config.max_iterations tool-call rounds (default: 32). This prevents infinite loops from misbehaving models. Terminates with AgentResult.finish_reason = LoopFinishReason::MaxIterations.

Stuck-loop detection

If the model produces the same output stuck_loop_threshold times consecutively (default: 3), a nudge is injected prompting the model to try a different approach.

Session persistence

Since run_loop operates on a Session, all appends (user messages, tool results) are automatically flushed to the JSONL session file if persistence is enabled. If the process crashes mid-loop, the session file contains everything up to the last successful append — at most one entry is lost.

Tool Trait

The Tool trait is the interface every tool implements. It defines a name, description, JSON Schema for parameters, risk classification, and an async execute method.

Definition

#![allow(unused)]
fn main() {
#[async_trait]
pub trait Tool: Send + Sync {
    /// The tool's registered name (used by the model to invoke it).
    fn name(&self) -> ToolName;

    /// Human-readable description shown to the model.
    fn description(&self) -> &str;

    /// JSON Schema object describing this tool's parameters.
    fn parameters_schema(&self) -> serde_json::Value;

    /// Risk classification, used by the approval policy.
    fn risk(&self) -> ToolRisk;

    /// Execute the tool. Check `cancel.is_cancelled()` at I/O boundaries.
    async fn execute(
        &self,
        arguments: serde_json::Value,
        cancel: CancellationToken,
    ) -> Result<ToolOutcome>;
}
}

Risk levels

Every tool has a ToolRisk that determines whether it needs human approval:

The approval policy can override these defaults per-tool via config.

Tool result

execute returns ToolOutcome::Immediate(ToolResult) containing:

• output: String — the text result (sent to the model and session)
• is_error: bool — whether the tool reported an error (e.g., non-zero exit code)
• details: ToolResultDetails — structured data not sent to the model

The details field carries richer information for tools and extensions:

Tool registry

ToolRegistry maps tool names to Box<dyn Tool> implementations:

#![allow(unused)]
fn main() {
let mut registry = ToolRegistry::new();
registry.register(Box::new(ReadFile { root: root.clone() }));
registry.register(Box::new(WriteFile { root: root.clone() }));

// Execute a model-issued tool call
let result = registry.execute(&call, cancel).await?;
}

Duplicate tool names panic at registration (fast-fail, programming error). Execution returns RhoError::ToolNotFound if the model requests an unregistered tool.

Built-in tools

rho ships with 14 built-in tools, all registered in rho-tools::register_all() (plus memory, registered only when [memory] enabled = true):

Extension tools

TypeScript extensions can register additional tools via rho-ext. Each extension tool is wrapped in a DenoTool that implements the same Tool trait. Extension tools appear alongside built-in tools in the ToolRegistry and are visible to the model via the tool schema.

Extension tools are discovered at startup from ~/.rho/extensions/ and .rho/extensions/. They can be hot-reloaded at runtime via the reloadExtensions RPC method.

See Extensions for details.

Provider Architecture

rho communicates with LLMs through a layered provider abstraction in rho-ai.

LlmService trait

The core trait in rho-ai defines provider-agnostic LLM communication:

#![allow(unused)]
fn main() {
#[async_trait]
pub trait LlmService: Send + Sync {
    fn chat_stream(
        &self,
        request: LlmRequest,
    ) -> Result<EventStream, ProviderError>;
}
}

All providers implement this trait. The agent loop consumes the EventStream (a Pin<Box<dyn Stream<Item = Result<StreamEvent>>>>) for streaming responses.

Transport services

OpenAiService targets Chat Completions-compatible endpoints:

#![allow(unused)]
fn main() {
use rho_ai::openai::OpenAiService;

let service = OpenAiService::new(rho_ai::ProviderConfig::new(
    "", // no API key for local
    "http://localhost:1234/v1/chat/completions",
));
}

ResponsesService targets native OpenAI Responses:

#![allow(unused)]
fn main() {
use rho_ai::responses::ResponsesService;

let service = ResponsesService::new(rho_ai::ProviderConfig::new(
    "sk-...",
    "https://api.openai.com/v1/responses",
));
}

Both implement LlmService, consume the same LlmRequest, and emit the same StreamEvent contract. RhoAiClient selects between them from each provider's ApiProtocol. The model is specified per request, never in the provider transport config.

Authentication

When an API key is provided via api_key_env in config, it is sent as an Authorization: Bearer <key> header with every request. Local endpoints typically don't need this. See External Providers for setup details.

Provider consent

The binary (rho) checks whether the configured endpoint is local before connecting. Non-local endpoints (e.g., api.openai.com) trigger a consent warning that lists the external provider(s) by name and exits with an error unless the user passes --accept-external-provider or --endpoint (which implies consent since the user explicitly chose the target). This is a headless consent gate — there is no interactive prompt.

Model picker

When no model can be resolved from config or CLI, rho falls back to a built-in default model (anthropic/claude-sonnet-4 via OpenRouter). Set --model <id>, agent.model, or agent.provider with a default_model on the provider to use a specific model. The built-in default requires an OpenRouter API key or a configured provider.

Model resolution

The model identifier is resolved in priority order:

1. CLI flag — --model <name> (highest priority)
2. Config model — agent.model in .rho/config.toml or ~/.rho/config.toml
3. Config provider — agent.provider selects a provider and uses its default_model
4. Provider default — the first provider's default_model if configured
5. RHO_MODEL environment variable
6. Built-in default — anthropic/claude-sonnet-4 (via OpenRouter; synthesizes an OpenRouter provider if no external provider is configured)

rho does not call /v1/models at startup. The config file is the source of truth — model names are passed directly to the provider, and misconfiguration surfaces as a clear HTTP error at request time.

Request / response

Streaming

Both services use SSE streaming. Chat Completions deltas and Responses output/reasoning/function-call events are normalized into one EventStream. StreamEvent::Text and StreamEvent::Reasoning are forwarded to the AgentObserver in real time; tool events use the same approval and execution path regardless of transport.

Multi-provider support

rho can manage multiple providers simultaneously via the ProviderRegistry and Provider trait. Each provider encapsulates identity, externality, model discovery, and access to an LlmService.

• Each provider has a name, endpoint, protocol, and optional API key (configured via [[providers]] with optional preset). Missing protocol values use Chat Completions; the OpenAI preset uses Responses.
• Providers can set models_endpoint when model listing uses a different path prefix. When unset, the models URL is derived from a trailing /chat/completions or /responses endpoint.
• Model list responses are parsed flexibly via serde untagged enums: ModelList accepts both {"data": [...]} (standard OpenAI) and {"models": [...]} (Z.ai). ModelInfo accepts both id and slug fields.
• The registry scans all providers to find which one serves a given model.
• listModels lists all models across all configured providers.
• setModel switches to a model, automatically selecting the right provider based on which provider serves that model.
• Project-level providers merge with user-level providers by name, so you can configure providers once globally and override selectively per project.

Example with multiple providers:

# ~/.rho/config.toml
[[providers]]
preset = "lm-studio"

[[providers]]
preset = "openrouter"
api_key_env = "OPENROUTER_API_KEY"

[agent]
model = "deepseek-v4-flash"  # served by openrouter

See Configuration for preset details and External Providers for provider setup.

Approval Policy

The approval policy sits between the model's tool call and its execution. It decides whether a tool call needs human confirmation, and the approval gate presents that decision to the user.

Two-part system

Policy: ConfigApprovalPolicy

The default policy uses per-tool overrides from config, falling back to risk-based defaults:

[approval.per_tool]
read_file = "auto"    # never ask
write_file = "ask"    # always ask
run_command = "ask"   # always ask

When a tool is not listed in config, the risk-based default applies:

Gate: RpcApprovalGate

The RPC implementation reads an approvalResponse JSON-RPC 2.0 method from stdin and returns the approved boolean. See RPC Mode for the approval flow.

Example gate

In RPC mode, the RpcApprovalGate reads an approvalResponse method from stdin (see RPC Mode). A REPL, TUI, or editor integration can implement the ApprovalGate trait to present approvals however it chooses.

On denial, a synthetic error result is fed back to the model as if the tool had failed. The model can then adjust its approach — it does not crash or get stuck.

Custom gates

The ApprovalGate trait is async, so any UI can implement it:

#![allow(unused)]
fn main() {
#[async_trait]
pub trait ApprovalGate: Send + Sync {
    async fn request_approval(&self, call: &ModelToolCall, risk: ToolRisk) -> ApprovalDecision;
}
}

ApprovalDecision has three variants:

• Approved — proceed with execution.
• Denied — inject a generic denial error and continue.
• Redirect { message } — inject the user's alternative instructions and return to thinking so the model can re-plan.

Context Management

The context manager is responsible for fitting the conversation within the model's context window. It decides which messages to keep, which to evict, and how to render compacted entries — using graduated resolution, phase-aware summaries, and selective eviction to maximize useful context.

The problem

Language models have finite context windows. A long coding session — especially one involving tool calls with large outputs — can easily exceed the window. When that happens, something has to give. The naive approach (drop the oldest message) silently discards the user's original request, causing the amnesia bug where the model forgets what it was asked to do.

Token budget

TokenBudget controls how much of the context window the agent is allowed to use for the conversation history:

#![allow(unused)]
fn main() {
TokenBudget {
    context_window: 32768,     // total window size
    completion_reserve: 4096,  // reserved for the model's reply
}
// prompt_budget() = context_window - completion_reserve = 28672
}

The completion_reserve ensures the model always has room to generate a reply. The prompt_budget() is the hard limit on how many tokens of conversation history are sent.

Sliding window

SlidingWindowContextManager is the default implementation. It:

1. Takes the message chain from the session's current branch (path_messages()).
2. Estimates token counts using the configured TokenEstimator.
3. Evicts the oldest middle messages to fit within prompt_budget().
4. Pins the system message, the first user turn, and the last turn — these are never evicted.

The pinning of the first user turn is the fix for the amnesia bug: no matter how long the conversation, the model always sees what the user originally asked.

Graduated resolution

Entries can exist at multiple fidelity levels, not just binary Full/Compacted. The context manager renders each entry according to its resolution:

This is the adaptive mesh refinement metaphor: full fidelity where the model needs it, coarsened where it doesn't. The system can downgrade entries through Full → Outlined → Summarized before resorting to eviction.

Selective turn-internal eviction

Instead of evicting entire turns when the context window fills up, the eviction planner (plan_downgrades) analyzes the entry path and produces a plan that downgrades individual entries within turns. The prepare_context() method applies this plan before building messages, reducing the need for turn-level eviction.

Invariants: the system message, first user turn, last turn, and pinned entries are never downgraded. Tool-pair integrity (assistant + tool result) is maintained.

Phase detection

The agent loop tracks which phase of work is currently active:

• Exploration — reading files, listing directories, gathering context
• Execution — writing files, editing code
• Verification — running cargo check, cargo test, cargo clippy
• Conclusion — final text reply from the model

Phase transitions follow a state machine: Exploration → Execution → Verification → Conclusion, with back-transitions allowed (e.g., Verification → Execution when a test fails).

Compaction

When older entries are compacted (see Sessions), the compaction system produces structured summaries.

Phase-aware compaction

Compaction summaries are grouped by phase, producing a narrative that tells the model what happened in each phase:

**Exploration:** Read main.rs, lib.rs (2 reads).
**Execution:** Edited src/parser.rs (3 edits).
**Verification:** cargo_check, cargo_test. Found: 1 error, 0 warnings.

LLM compaction (optional)

When compaction_mode = "llm" is configured, the compaction system calls the LLM to generate a narrative summary in the notes field of CompactionSummary. This runs a two-stage process:

1. Mechanical compaction — structured fields (tool calls, findings, phases) are always computed deterministically.
2. LLM narrative — the model writes a concise paragraph summarizing what was accomplished, stored in notes.

On LLM failure, the system falls back to mechanical-only compaction — no error propagation to the agent loop.

Auto-compact

When context utilization crosses the auto_compact_threshold (configurable), the agent loop proactively compacts older entries before eviction is needed. This prevents the abrupt context loss that occurs when turn-level eviction kicks in.

Tool schema overhead

Tool schemas (the JSON descriptions sent to the model so it knows what tools are available) consume tokens. The context manager subtracts an estimate of this overhead from the available budget before fitting messages, preventing the combined total from exceeding the window.

Token estimation

TokenEstimator is a trait for estimating token counts without calling the model's tokenizer. The default implementation (HeuristicEstimator) uses:

• Per-model calibration: an exponential moving average (α=0.3) that learns the real tokens-per-character ratio from model responses.
• Bootstrap ratios: built-in starting ratios for common model families (Gemma, Qwen, Claude, GPT-4, Llama).
• Substring matching: falls back to character-level estimation for non-ASCII text.

The estimator converges to within 10% accuracy by the third model call in most cases.

Enhanced context stats

ContextStats provides rich token distribution diagnostics:

#![allow(unused)]
fn main() {
pub struct ContextStats {
    // Basic stats
    pub context_window: usize,
    pub completion_reserve: usize,
    pub estimated_used: usize,
    pub message_count: usize,
    pub entry_count: usize,
    pub path_entry_count: usize,

    // Token distribution by message role
    pub role_tokens: RoleTokenDistribution,      // system, user, assistant, tool

    // Token distribution by resolution level
    pub resolution_tokens: ResolutionTokenDistribution,  // full, outlined, summarized, pinned

    // Token distribution by session phase
    pub phase_tokens: PhaseTokenDistribution,    // exploration, execution, verification, conclusion, unclassified

    // Compaction tracking
    pub compaction_tokens: usize,
    pub compacted_entry_count: usize,
}
}

Key methods:

• utilization_percent() — context utilization as 0–100% (based on prompt budget)
• estimated_remaining() — remaining tokens in the prompt budget (saturates at 0)
• prompt_budget() — context window minus completion reserve

Context status bar

A consumer (TUI, editor integration) can display a compact one-line status bar after every turn showing estimated token usage:

[████████████░░░░░░░░] 12.3k/32k tokens (50%) │ 12.3k remaining │ 10 messages

The bar can be color-coded by utilization: green (<60%), yellow (60–80%), red (>80%). This gives a quick at-a-glance sense of how much context window remains.

For a detailed breakdown, the getSessionStats RPC method shows:

• Context window size and completion reserve
• Prompt budget (context window minus reserve)
• System prompt and tool schema overhead
• Conversation tokens
• Estimated used/remaining/percentage
• Message count, path entry count, total entries
• Token distribution by role (system, user, assistant, tool)
• Token distribution by resolution (full, outlined, summarized, pinned)
• Compaction tokens and compacted entry count

Sessions

A session is rho's unit of persistent conversation state. It replaces the older flat Conversation type with a tree-structured model where every turn, tool call, and compaction is an entry in an append-only log.

Why sessions?

The original Conversation type stored messages in a Vec<ChatMessage>. When the context window filled up, the sliding window evicted old messages — including, in some cases, the user's original request. The session model fixes this by:

1. Tree structure — branching, not deletion. /clear and compaction create new branches; the old tree is preserved.
2. Typed entries — user messages, assistant messages, tool calls, tool results, compaction summaries, and extension entries are all first-class nodes.
3. Resolution levels — each entry carries a resolution (Full, Outlined, Summarized, Pinned, Compacted, or Attached) that tells the context builder how to render it.
4. Persistence — sessions auto-flush to JSONL files, surviving process restarts and crashes.

Tree structure

A session is a tree of Entry nodes, each identified by an EntryId (a UUID). Every entry has a parent_id pointing to its predecessor, forming a single chain (the trunk) with possible branches.

[System] ──→ [User: "fix the bug"] ──→ [Assistant: "ok"] ──→ [User: "read another file"]
                                         │
                                         └──→ [Compaction: "user asked to fix the bug"]

The leaf is the entry the agent loop is currently building from. branch_to(id) moves the leaf to any existing entry, creating a divergence point. The old branch stays in the tree for later inspection or resumption.

Persistence

Sessions are stored as append-only JSONL files under ~/.rho/sessions/:

~/.rho/sessions/
  └── <project-hash>/
        └── <unix-timestamp>_<session-id>.jsonl

• Project hash: first 16 hex characters of SHA-256 of the canonical project root path. This groups sessions by project.
• Auto-flush: every append writes to disk immediately. A crash loses at most the last entry.
• Format: each line is a JSON object with a type discriminator (Header or Entry). The first line is always a Header with session metadata.

Example JSONL file:

{"type":"Header","id":"4b021d5d","version":1,"created_at_secs":1777859536,"cwd":"/home/user/project","parent_session":null}
{"type":"Entry","id":"a1b2c3d4","parent_id":null,"timestamp":{...},"resolution":"Full","payload":{"Message":{"content":"you are a coding assistant","role":"system"}}}
{"type":"Entry","id":"e5f6a7b8","parent_id":"a1b2c3d4","timestamp":{...},"resolution":"Full","payload":{"Message":{"content":"fix the bug","role":"user"}}}

Session discovery

rho can scan saved sessions without loading the full entry tree. This enables session listing and quick-resume features.

find_latest_session(cwd)

Returns the path to the most recently modified JSONL file for the given project directory. Used by rho -c to auto-resume. Reads only the header line — does not parse entry lines.

list_sessions(cwd)

Returns all saved sessions for the project, sorted by filesystem modification time (most recent first). Each result includes:

The listSessions RPC method uses this to return the 10 most recent sessions, marking which one rho -c will resume.

Resuming a session

Use rho -c to resume the most recent session for the current project:

rho -c

Or resume a specific session with --session <path>:

rho --session ~/.rho/sessions/abcdef1234567890/1777859536_4b021d5d.jsonl

The -c, --session, and --ephemeral flags are mutually exclusive (enforced at parse time by clap).

The resumed session picks up where it left off. The model, token budget, redactor, and tool set are updated from the current configuration so a session started with one model can be continued with another.

Ephemeral mode

Use --ephemeral to run without any disk persistence:

rho --ephemeral

All conversation state lives only in memory and is lost when rho exits. Useful for one-shot commands, CI pipelines, or when you don't want session files accumulating.

Startup hint

When rho creates a new persisted session, it checks for previous sessions for the same project. If any exist, it prints a hint:

session: /home/user/.rho/sessions/abcdef12/1777859536_4b021d5d.jsonl
  (3 previous session(s) for this project — use rho -c to resume the latest)

Context stats

The ContextStats struct provides a rich snapshot of context window usage, available through Session::context_stats():

#![allow(unused)]
fn main() {
pub struct ContextStats {
    // Basic stats
    pub context_window: usize,
    pub completion_reserve: usize,
    pub estimated_used: usize,
    pub message_count: usize,
    pub entry_count: usize,
    pub path_entry_count: usize,

    // Token distribution
    pub role_tokens: RoleTokenDistribution,           // system, user, assistant, tool
    pub resolution_tokens: ResolutionTokenDistribution, // full, outlined, summarized, pinned
    pub phase_tokens: PhaseTokenDistribution,        // exploration, execution, verification, conclusion, unclassified

    // Compaction tracking
    pub compaction_tokens: usize,
    pub compacted_entry_count: usize,
}
}

Key methods:

• utilization_percent() — context utilization as 0–100% (based on prompt budget)
• estimated_remaining() — remaining tokens in the prompt budget (saturates at 0)
• prompt_budget() — context window minus completion reserve

A consumer can display a compact one-line status bar after every turn:

[████████████░░░░░░░░] 12.3k/32k tokens (50%) │ 12.3k remaining │ 10 messages

Color-coded green (<60%), yellow (60–80%), red (>80%). The getSessionStats RPC method shows a detailed multi-line breakdown including token distribution by role, resolution level, compaction stats, system prompt overhead, tool schema overhead, and conversation token breakdown.

Entry types

Each entry in the tree has a payload that describes what it represents:

Resolution levels

Every entry carries a resolution field:

• Full — complete, verbatim content. Used for recent entries that fit within the context window.
• Outlined — structural summary: tool name, key arguments, truncated output. Consumes ~10-20% of original tokens.
• Summarized — prose summary: a one-line description of what the entry represents. Consumes ~5-10% of original tokens.
• Pinned — protected from eviction and downgrade. Rendered at its native resolution.
• Compacted — replaced by a Compaction summary. The original content is still in the tree but the context builder renders the summary instead.
• Attached — lightweight reference. Rendered as a brief mention rather than full content.

Entries are downgraded through Full → Outlined → Summarized by the selective eviction planner before turn-level eviction is needed. Pinned entries are protected from all downgrade and eviction.

The context builder (Context Management) uses resolution levels to decide what to send to the model — full detail where it matters, summaries where it doesn't.

Bounded tool results

Tool results can be very large (entire file contents, command output). Sessions enforce a bounded output size:

• Tool output exceeding a fraction of the prompt budget is truncated.
• The full output is preserved in ToolResultDetails::FullOutput for later retrieval.
• Truncation splits on the last newline boundary to avoid breaking UTF-8 sequences.

This prevents a single verbose tool call from consuming the entire context window.

The /clear command

The clear RPC method branches back to the system message entry. This has the same practical effect as clearing the conversation, but the old tree is preserved on disk. You can resume the old branch later with rho -c or --session.

Session API

The Session type in rho-core exposes:

• Constructors: Session::new() (persisted), Session::in_memory() (no disk I/O), Session::open(path) (resume from JSONL)
• Appenders: append_user_message(), append_assistant_message(), append_tool_result(), append_tool_call()
• Navigation: path_to_root(), children(), leaf(), branch_to(), branch_with_summary()
• Context: path_messages() (messages along the current branch), send_current() (send to model with context fitting), context_stats() (usage snapshot), prepare_context() (selective downgrade)
• Compaction: compact_older_than() (compact entries older than a threshold)
• Discovery: find_latest_session(cwd), list_sessions(cwd) (header-only session scanning)
• Extensions: write_custom_state(), read_custom_state(), write_custom_message(), read_custom_message()
• Persistence: save_path(), flush()

Project Context Files

rho scans for project-level instruction files and injects them into the system prompt. This lets projects tell the agent how to work with their codebase — without modifying rho itself.

Scan list

rho looks for the following files in the project root (sandbox root), in priority order:

The scan list can be overridden in config:

[context]
scan_list = ["AGENTS.md", "CLAUDE.md"]

Trust model

Project context files are a prompt injection vector — anyone with write access to the repository can modify them. rho addresses this with a trust store:

1. First encounter: rho prompts the user to review the file contents and confirm trust.
2. Trust stored: the SHA-256 hash of the accepted file is stored in ~/.rho/trusted_projects.toml.
3. Subsequent loads: if the hash matches, the file is loaded silently. If it changed since the last trust decision, rho prompts the user again.

This ensures:

• A malicious PR that silently modifies AGENTS.md will trigger a re-review prompt.
• Legitimate updates (new project instructions) are confirmed once, then loaded automatically.
• The user can always audit what's in trusted_projects.toml.

Prompt composition

Context files are appended to the system prompt after the base prompt, in scan-list order. The final system prompt structure is:

[base prompt]           ← prompts/base.md (or compact prompt)
[project context files] ← AGENTS.md, CLAUDE.md, etc.
[environment block]     ← working directory, shell info
[Rust tooling block]    ← cargo check/clippy/test guidance

In practice

The file you're reading right now (AGENTS.md) is a context file. It tells rho about the project layout, conventions, testing approach, and commit format. Without it, rho would still work — but it wouldn't know about cargo xtask ci, the conventional commit format, or the forbid(unsafe_code) lint.

Security

rho takes untrusted input (LLM output), interprets it as instructions, and executes those instructions with the full privileges of the user. The security model is defense-in-depth — no single layer is sufficient, but each layer raises the bar.

See also:

• Threat Model
• File Sandbox
• Secret Redaction
• Prompt Injection Defense
• Egress Control (historical)

Threat Model

rho takes untrusted input (LLM output), interprets it as instructions, and executes those instructions with the full privileges of the user. The security model is defense-in-depth — no single layer is sufficient, but each layer raises the bar.

Threats and defenses

Scope and limitations

• Trusted user: The human at the keyboard is trusted. Approval gates protect against model-initiated actions, not user-initiated ones.
• No OS-level isolation: Tools execute as the same user/process as rho. The sandbox validates paths but cannot prevent a determined model from exploiting a zero-day in a dependency.
• Best-effort redaction: The Redactor matches known secret patterns (API keys, tokens, passwords). It cannot detect secrets that don't match its patterns.
• Command denylist coverage: The denylist catches common exfiltration vectors (PowerShell networking cmdlets, curl, wget, .NET HTTP). A creative model can construct network requests using .NET APIs that aren't in the substring list. The approval gate is the primary defense.

See the individual pages for detailed design of each defense layer:

• File Sandbox
• Secret Redaction
• Prompt Injection Defense
• Egress Control (historical — removed in v0.33.3)

File Sandbox

All file tools (ReadFile, WriteFile, ListDir, EditFile) operate within a sandbox root — a canonical directory that no tool operation can escape.

SandboxRoot

#![allow(unused)]
fn main() {
let root = SandboxRoot::new("/home/user/project")?;
root.validate("src/main.rs")?;       // Ok — within sandbox
root.validate("../../etc/passwd")?;  // Err — escapes sandbox
}

The sandbox root is the project root by default, auto-detected by walking up from the current directory looking for markers (.rho/config.toml, .git/, Cargo.toml, package.json, etc.). It can be overridden with --root <path>.

Validation strategies

Existing paths

validate() resolves symlinks and .. components via std::fs::canonicalize, then checks the canonical path starts with the canonical root. This prevents path traversal through symlinks or directory climbing.

Not-yet-existing paths

validate_for_write() handles new files (where canonicalize would fail). It:

1. Walks up from the path until it finds an existing ancestor.
2. Canonicalises that ancestor.
3. Re-appends the remaining components.
4. Rejects any .. component in the not-yet-existing suffix immediately.
5. Verifies the resolved path stays within the sandbox root.

This allows WriteFile to create new files within the project while still preventing escapes like new_dir/../../../etc/passwd.

Opt-out

The sandbox is always enabled and cannot be disabled.

TOCTOU note

There is a theoretical time-of-check-time-of-use race between validate_for_write and the subsequent create_dir_all + write. This is accepted because:

• The threat model does not include concurrent adversarial filesystem modification.
• If the model has shell access, it can write directly via the shell — the sandbox only constrains the tool interface.
• The approval gate is the primary defense against model-initiated writes.

Secret Redaction

The Redactor scans tool output for known secret patterns and replaces matches with [REDACTED]. This prevents the model from seeing (and potentially leaking) secrets that appear in file contents, command output, or configuration files.

How it works

Redaction is applied to tool results before they enter the conversation history. The model never sees the original secret — only [REDACTED].

Built-in patterns

Custom patterns

Additional regex patterns can be added in config:

[redaction]
enabled = true
custom_patterns = [
    "my-service-key-[a-zA-Z0-9]{32}",
    "token: \\S+",
]

Custom patterns are applied after the built-in patterns. Invalid regex patterns are silently ignored (a warning is printed to stderr at startup).

Vec replacement semantics

Custom patterns follow the same replacement rule as all Vec fields: the project-level list replaces the user-level list (not appended). This avoids surprising composition effects.

Configuration

Redaction is enabled by default. It can be disabled in config:

[redaction]
enabled = false

Limitations

• Best-effort: The redactor matches known patterns. It cannot detect secrets that don't match any pattern (e.g., a novel token format).
• No reconstruction: Redaction is one-way — the original value is discarded. Tool result details (ToolResultDetails::FullOutput) preserve the un-redacted content for tools that need it, but the model-facing text is always redacted.
• Pattern overlap: A string matching multiple patterns is replaced once. The [REDACTED] sentinel is not itself matched by subsequent patterns.

Prompt Injection Defense

File contents are untrusted data. When the model reads a file, the contents could contain instructions designed to manipulate the model's behaviour. rho mitigates this with framing.

The <context> / <context:end> convention

File contents returned by ReadFile are wrapped in framing tags:

<context>
[actual file contents here]
<context:end>

The system prompt instructs the model that content between <context> and <context:end> is data to be processed, not instructions to be followed. This is a form of role-based compartmentalisation — the model is told to adopt a different role when reading framed content.

Why <context:end> and not </context>?

The original closing tag </context> was ambiguous — it could appear in legitimate file contents (e.g., XML files, HTML comments). The <context:end> sentinel is a unique string that will not appear naturally in source code, making it a reliable boundary marker.

Where framing is applied

System prompt primacy

The system prompt is loaded before any user interaction and is never evicted from the context window (it's pinned by the context manager). This means the framing instructions persist even if the conversation grows long.

Limitations

• Framing is advisory, not enforced. The model could choose to ignore the <context> tags and follow injected instructions. The framing relies on the model's instruction-following behaviour, not a technical barrier.
• No content scanning. rho does not scan file contents for injection patterns. The framing is applied uniformly regardless of content.
• Approval gate is the real defense. If the model attempts to execute a destructive action based on injected instructions, the approval gate catches it. Framing is a defense-in-depth layer, not a substitute for human oversight.

Egress Control

Removed in v0.33.3. Egress enforcement was removed because it only gated one HTTP client (LocalChatClient) while shell commands had unrestricted network access via curl, Invoke-WebRequest, etc. The provider consent warning is the real defense for external providers.

What was removed

• EgressConfig struct and [egress] config section
• RhoError::EgressBlocked error variant
• check_egress() method on LocalChatClient
• with_endpoint_and_egress() and with_endpoint_egress_and_key() constructors
• Egress allowlist checks before every chat() and list_models() call

Why it was removed

The egress allowlist created a false sense of security:

1. Shell bypass: The model (or a prompt injection attack) could exfiltrate data via shell commands (curl, Invoke-WebRequest, etc.) regardless of the egress config.
2. Limited scope: Egress only gated LocalChatClient HTTP requests. It did not cover any other network access path.
3. The consent warning already covers this: When connecting to a non-local endpoint, rho displays an interactive consent prompt warning that data will leave the machine. This is the meaningful defense — the user explicitly approves where their data goes.

What remains

The command denylist in RunCommand blocks common exfiltration tools (curl, wget, Invoke-WebRequest, Invoke-RestMethod, etc.). This is still active and provides practical protection against the most common exfiltration vectors.

The provider consent warning fires before any connection to a non-local endpoint. Use --accept-external-provider to skip it in automated workflows, or --endpoint (which implies consent since the user explicitly chose the target).

Configuration

rho reads configuration from two TOML files, merged with project-level overrides taking precedence over user-level defaults.

File locations

Both files are optional. Missing files are silently skipped; all fields have sensible defaults.

Merging strategy

Project-level config overrides user-level config on a per-field basis. For struct fields: if the project sets a field, it wins; if not, the user-level value applies; if neither sets it, the hardcoded default applies.

For Vec fields (denylist commands, context scan list, custom redaction patterns): the project-level list replaces the user-level list. It does not append. This avoids surprising composition effects.

Provider merging

Project-level [[providers]] merge with user-level [[providers]] by name (not replaced). This lets you configure providers once globally and override selectively per project:

• Same name → project provider overrides user provider entirely (all fields from project)
• New name → project provider is appended
• No match → user provider preserved in original order
• No project providers → user providers preserved unchanged

Provider presets

The preset field auto-fills endpoint, name, and request api from a built-in registry of known providers. This reduces config boilerplate and eliminates typo-prone URLs.

# Instead of writing out the full endpoint URL:
[[providers]]
preset = "openrouter"
api_key_env = "OPENROUTER_API_KEY"

Built-in presets:

Resolution rules:

• If preset is set and endpoint is also set → endpoint wins; the preset's protocol still applies unless api is also explicit
• If preset is set and name is also set → name wins
• If preset is set and api is also set → explicit api wins
• If api and preset are both absent → chat_completions is the safe default
• If preset is set and api_key_env is not set → a hint is logged at startup (not auto-injected)
• If preset is set and models_endpoint is also not set and the preset provides one → the preset's models_endpoint is used
• If preset is set and models_endpoint is also set → models_endpoint wins (preset is informational)
• If preset is set and endpoint is overridden to a different URL than the preset's → the preset's models_endpoint is skipped (it would target the wrong API), and the models URL is derived from the chat endpoint instead
• Unknown presets → warning logged, treated as if no preset was set

Full configuration reference

[agent]
# Model identifier (highest priority; overrides provider and default_model)
model = "qwen3-8b"

# Provider to use at startup (uses that provider's default_model if model not set)
# provider = "openrouter"

# Maximum model-tool-model round trips before the loop fails (default: 32)
max_iterations = 32

# Maximum retry attempts on transient errors (default: 4)
retry_budget = 4

# Base backoff in milliseconds (default: 500)
initial_backoff_ms = 500

# Context window token budget (default: 32768)
token_budget = 32768

# Stuck-loop detection: consecutive identical outputs before nudge (default: 3, 0 = off)
stuck_loop_threshold = 3

# Maximum consecutive empty model responses before aborting (default: 5, 0 = unlimited)
# Prevents infinite retry spirals with models that can't produce output.
max_consecutive_empty = 5

# Show full chain-of-thought reasoning in output (default: false)
show_reasoning = false

# Reasoning effort for thinking-capable models (optional)
# Common values: "low", "medium", "high".
# Sent as `reasoning_effort` for Chat Completions. For Responses, sent as
# reasoning = { effort = ..., summary = "auto" }, with summary deltas streamed.
# Use only values supported by the selected model.
# reasoning_effort = "medium"

# Context utilization threshold for auto-compaction (default: 0 = disabled)
# When utilization >= this %, older entries are proactively compacted.
auto_compact_threshold = 0

# Compaction strategy: "mechanical" (default) or "llm" (opt-in)
# When "llm", the model generates narrative notes for compacted entries.
compaction_mode = "mechanical"

# Maximum seconds to wait for the FIRST stream event before aborting and
# retrying (default: 90, 0 = disabled).
# Guards against hung requests where the server (or an upstream proxy) accepts
# the request but never emits a token — otherwise the loop blocks until the
# upstream's own timeout drops the connection, often as a silent empty reply.
# On timeout the request is retried with backoff. Raise this for slow
# reasoning models.
first_token_timeout_secs = 90

# Maximum gap (seconds) allowed between stream events once streaming has
# started before aborting and retrying (default: 60, 0 = disabled).
# A long gap mid-stream indicates a dead connection.
stream_idle_timeout_secs = 60

# ── Providers ────────────────────────────────────────────────────
# Use the [[providers]] array format for single or multiple providers.
# The legacy [provider] single-table format still works but [[providers]]
# is preferred.
#
# Each provider has:
#   name           — display name (shown in /models, consent prompt)
#   preset         — built-in preset (fills endpoint/name automatically)
#   endpoint       — explicit endpoint URL (overrides preset)
#   api             — "chat_completions" or "responses"; missing defaults to
#                     Chat Completions unless a preset supplies a value
#   api_key_env    — env var holding the API key (not auto-injected by presets)
#   default_model  — model to use when selected via agent.provider
#   models_endpoint — explicit models endpoint URL (overrides preset; used when
#                   the provider serves models at a different path prefix)
#   type           — informational label, has no effect on behavior

# Example: LM Studio via preset (local, no API key needed)
[[providers]]
preset = "lm-studio"

# Example: OpenAI Responses via preset
# [[providers]]
# preset = "openai"
# api_key_env = "OPENAI_API_KEY"
#
# Example: OpenRouter Chat Completions via preset
# [[providers]]
# preset = "openrouter"
# api_key_env = "OPENROUTER_API_KEY"

# Example: custom endpoint (preset as label, endpoint overrides)
# [[providers]]
# name = "my-proxy"
# preset = "openrouter"
# endpoint = "https://my-proxy.example.com/v1/chat/completions"
# api_key_env = "MY_PROXY_KEY"

# Example: custom Responses-compatible endpoint (explicit opt-in)
# [[providers]]
# name = "custom-responses"
# endpoint = "https://llm.example.com/v1/responses"
# api = "responses"
# api_key_env = "CUSTOM_API_KEY"
#
# Example: multiple providers (local + remote)
# [[providers]]
# preset = "lm-studio"
#
# [[providers]]
# preset = "openrouter"
# api_key_env = "OPENROUTER_API_KEY"

# Legacy single-provider format (still supported):
# [provider]
# endpoint = "http://localhost:1234/v1/chat/completions"
# api_key_env = "OPENAI_API_KEY"

[approval.per_tool]
# Per-tool approval overrides: "auto", "ask", "deny"
read_file = "auto"
write_file = "ask"
edit_file = "ask"
run_command = "ask"

[shell]
# Additional commands to deny (appended to the built-in denylist)
denied_commands = ["Remove-Item", "Invoke-WebRequest"]

# Additional denied flag combinations (all flags must be present to deny)
denied_flag_combos = [["-Quiet", "-Force"]]

[context]
# Override the default context file scan list
scan_list = ["AGENTS.md", "CLAUDE.md"]

[redaction]
# Secret redaction toggle (default: true)
enabled = true

# Additional regex patterns to redact
custom_patterns = ["my-key-[a-zA-Z0-9]{32}"]

[memory]
# Enable the project-local knowledge base (MemoryTool). Default: false.
# When enabled, rho-memory opens <project-root>/.rho/memory.db (SQLite + FTS5)
# and the `memory` tool becomes available to the model.
enabled = false

[system_prompt]
# Additional prompt fragments appended after the base prompt
extensions = ["Always use Rust idioms."]

[extensions]
# Extension allowlist — only load these extensions (omit to load all)
enabled = ["hello", "crates-search"]

# Extension denylist — never load these extensions
# disabled = ["experimental-thing"]

[extensions.defaults]
# Default permissions for all extensions
network = true           # allow fetch() by default
max_memory_mb = 64       # V8 heap limit per isolate
max_execution_time_s = 30

[extensions.per_extension."rust-docs"]
# Override defaults for a specific extension
max_memory_mb = 128      # needs more for HTML parsing

[extensions.per_extension."dangerous-tool"]
network = false          # deny fetch()
commands = false         # deny rho.runCommand()

API key handling

API keys are never stored in plaintext in config files. Instead, config references environment variable names:

[[providers]]
preset = "openai"
api_key_env = "OPENAI_API_KEY"

The provider reads the key from the environment variable at runtime. If the variable is not set, the key is silently omitted (local endpoints typically don't need one).

Provider protocols

The OpenAI preset uses native Responses with stateless requests and store: false. OpenRouter, Groq, Z.ai, Ollama, LM Studio, and custom unspecified providers retain Chat Completions. Set api = "responses" for a custom endpoint only when it implements the Responses request and SSE formats. See External Providers for details and limitations.

CLI overrides

All config values can be overridden by CLI flags. CLI flags take highest priority:

External Providers

rho supports two OpenAI-shaped streaming protocols behind the same internal LlmService contract:

• Responses (POST /v1/responses) through ResponsesService, used by the built-in openai preset. This supports OpenAI reasoning and function tools in the same turn.
• Chat Completions (POST /v1/chat/completions) through OpenAiService, used by every other built-in preset and by custom providers unless explicitly overridden.

The agent loop, tools, sessions, and UIs receive the same text, reasoning, tool-call, usage, and completion events from either transport.

Chat Completions compatibility

A Chat Completions request uses messages and nested function tools:

{
  "model": "gpt-4o",
  "messages": [...],
  "tools": [...]
}

Any endpoint that implements the OpenAI Chat Completions request, SSE, tool-call, and usage shapes can be used. This remains the safe default for unspecified custom providers.

Native OpenAI Responses

A Responses request uses input, instructions, flat function tools, and max_output_tokens. rho operates statelessly: it sends the fitted conversation on every turn and explicitly sets store: false; it does not use previous_response_id or server-side Conversations. When reasoning_effort is configured, rho requests an automatic reasoning summary and streams summary deltas through the existing reasoning event path.

Function calls and outputs round-trip as function_call and function_call_output items correlated by call_id. Encrypted reasoning items and assistant phase metadata are not persisted in this first implementation, so full hidden-reasoning continuity for zero-data-retention/Codex-style workflows is deferred.

Supported providers and defaults

OpenRouter, Groq, Z.ai, Ollama, LM Studio, and unspecified custom endpoints do not switch to Responses as a side effect of this feature.

Providers with unrelated native formats

Native Anthropic Messages, Google GenerateContent, AWS Bedrock, Cohere, and other unrelated formats are not implemented directly. Use an OpenAI-compatible proxy such as LiteLLM or OpenRouter if it exposes a tested Chat Completions interface.

Prerequisites

To use an external provider you need two things:

1. An API key from the provider
2. The provider's endpoint URL (matching the configured api protocol)

That's it. Configure via config file or CLI flags — see below.

Quick start: OpenAI

Via CLI flags

A CLI-only endpoint uses the backward-compatible Chat Completions default:

export OPENAI_API_KEY="sk-..."

rho --endpoint https://api.openai.com/v1/chat/completions \
    --api-key-env OPENAI_API_KEY \
    --model gpt-4o

--endpoint implies consent. There is no separate --api flag; use provider config below to select Responses.

Via config

# ~/.rho/config.toml
[agent]
model = "gpt-5"
provider = "openai"
token_budget = 131072
reasoning_effort = "medium"

[[providers]]
preset = "openai" # supplies api = "responses" and /v1/responses
api_key_env = "OPENAI_API_KEY"
default_model = "gpt-5"

export OPENAI_API_KEY="sk-..."
rho --accept-external-provider

Provider consent

When connecting to a non-local endpoint, rho checks for external providers before starting. If external providers are configured and neither --accept-external-provider nor --endpoint is passed, rho prints a warning to stderr and exits with an error:

  ⚠  No local model server detected
  ⚠  External provider(s) configured:
      - openrouter

      Your prompts and code will be sent to external servers.
      This may expose proprietary code, secrets, or other
      sensitive data to the providers and any intermediaries.

  Aborting. Use --accept-external-provider to skip this prompt.

If rho also has a local provider (e.g. LM Studio is running alongside an external provider), the "No local model server detected" header is omitted.

The consent gate fires automatically for config-driven external endpoints. It is skipped when you use --endpoint on the CLI (explicit endpoint implies consent) or --accept-external-provider.

Specifying the model

There are several ways to set the model, in priority order:

1. CLI flag — --model gpt-4o (highest priority)
2. Config model — [agent] model = "gpt-4o"
3. Config provider — [agent] provider = "openrouter" uses that provider's default_model
4. Provider default — the first provider's default_model if configured
5. RHO_MODEL environment variable
6. Built-in default — anthropic/claude-sonnet-4 (synthesizes an OpenRouter provider)

rho does not call /v1/models at startup. The config file is the source of truth — model names are accepted as-is, and misconfiguration surfaces as a clear HTTP error at request time.

listProviders RPC method

The listProviders RPC method shows all configured providers with their reachability status:

  configured providers:
  * lm-studio (local, ok)
    openrouter (remote, ok)
    ollama (remote, down)
  * = active

Each entry shows:

• * — active provider (where the next prompt will go)
• local/remote — whether the endpoint is on localhost
• ok/down — whether the provider's models endpoint responded

Mid-session provider switching

The /model command accepts provider:model syntax to switch to a specific provider without model discovery. This is useful when a provider is slow to respond or its models endpoint is unavailable:

/model openrouter:deepseek/deepseek-v4-flash

With a bare model ID, rho discovers which provider serves it by querying each provider's models endpoint and switches automatically:

/model qwen3-8b

CLI flags for provider configuration

Priority: CLI flag → project config → user config → hardcoded default.

Provider examples

OpenAI

Use the preset for native Responses, including reasoning plus function tools:

[agent]
model = "gpt-5"
provider = "openai"
reasoning_effort = "medium"

[[providers]]
preset = "openai"
api_key_env = "OPENAI_API_KEY"
default_model = "gpt-5"

To force OpenAI Chat Completions for an older workflow, override both fields explicitly:

[[providers]]
name = "openai-chat"
endpoint = "https://api.openai.com/v1/chat/completions"
api = "chat_completions"
api_key_env = "OPENAI_API_KEY"

Custom Responses-compatible endpoint

Responses is opt-in for custom servers and proxies:

[[providers]]
name = "custom-responses"
endpoint = "https://llm.example.com/v1/responses"
api = "responses"
api_key_env = "CUSTOM_API_KEY"

Only select this when the endpoint implements OpenAI's Responses request items and SSE event taxonomy. Merely accepting Chat Completions is not sufficient.

OpenRouter

OpenRouter proxies many models behind a single endpoint. Model names include the provider prefix (e.g. anthropic/claude-sonnet-4-20250514).

export OPENROUTER_API_KEY="sk-or-..."
rho --endpoint https://openrouter.ai/api/v1/chat/completions \
    --api-key-env OPENROUTER_API_KEY \
    --model anthropic/claude-sonnet-4-20250514

Or via config with a preset:

[agent]
model = "anthropic/claude-sonnet-4-20250514"
provider = "openrouter"

[[providers]]
preset = "openrouter"
api_key_env = "OPENROUTER_API_KEY"
default_model = "anthropic/claude-sonnet-4-20250514"

Groq

export GROQ_API_KEY="gsk_..."
rho --endpoint https://api.groq.com/openai/v1/chat/completions \
    --api-key-env GROQ_API_KEY \
    --model llama-3.3-70b-versatile

DeepInfra

export DEEPINFRA_API_KEY="di-..."
rho --endpoint https://api.deepinfra.com/v1/openai/chat/completions \
    --api-key-env DEEPINFRA_API_KEY \
    --model meta-llama/Llama-3.3-70B-Instruct

Z.ai

Z.ai provides GLM models (e.g. GLM-5.2) via an OpenAI-compatible API. The endpoint is at api.z.ai/api/paas/v4/chat/completions — note the api. subdomain and /api/paas/v4/ path, which differs from the marketing site at z.ai.

The zai preset also sets a models_endpoint at https://api.z.ai/api/v1/models (a different path prefix than the chat completions endpoint). This is used automatically — no manual configuration needed.

export ZAI_API_KEY="..."
rho --endpoint https://api.z.ai/api/paas/v4/chat/completions \
    --api-key-env ZAI_API_KEY \
    --model glm-5.2

Or via config with a preset:

[agent]
model = "glm-5.2"
provider = "zai"

[[providers]]
preset = "zai"
api_key_env = "ZAI_API_KEY"

Z.ai Coding Plan (subscription)

If you have a z.ai Coding Plan subscription (the one built for Claude Code / Cline / similar), use the zai-coding preset instead. It targets the subscription endpoint (/api/coding/paas/v4/...) and derives the models URL from it (/api/coding/paas/v4/models):

[agent]
model = "glm-5-turbo"
provider = "zai"

[[providers]]
preset = "zai-coding"
api_key_env = "ZAI_API_KEY"

Don't mix preset = "zai" with a Coding Plan endpoint override. If you do, rho now detects the mismatch and derives the models URL from your chat endpoint instead of the platform /api/v1/models — so it still works — but zai-coding is the cleaner choice.

Ollama (remote)

If you run Ollama on a different machine, point the endpoint at it:

rho --endpoint http://192.168.1.100:11434/v1/chat/completions --model llama3

API key handling

API keys are never stored in config files. Instead, config references an environment variable name:

[[providers]]
preset = "openai"
api_key_env = "OPENAI_API_KEY"

At runtime, rho reads the environment variable and sends the key as an Authorization: Bearer <key> header with every request. If the variable is not set, the key is silently omitted (local endpoints don't need one).

You can set the key in your shell, in .bashrc/.zshrc, or in a .env file:

# One-time (current shell)
export OPENAI_API_KEY="sk-..."

# Persistent (add to shell profile)
echo 'export OPENAI_API_KEY="sk-..."' >> ~/.bashrc

# Via .env file (load with direnv or source manually)
echo 'export OPENAI_API_KEY="sk-..."' >> .env
source .env

Token budget tuning

External models often have much larger context windows than local 8K models:

rho defaults to token_budget = 32768. For models with larger windows, increase it:

[agent]
token_budget = 131072

Or via CLI: --token-budget 131072.

The budget is split into a prompt budget (conversation + system prompt + tool schemas) and a completion reserve (room for the model's reply). The default reserve is 8192 tokens.

Run rho with RUST_LOG=info to see budget diagnostics at startup:

budget: 131072T context, 8192T reserve, 122880T prompt (4700T system + 2000T schema = 6700T overhead, 116180T for conversation)

The type, name, preset, and api fields

The type field in a provider config is informational only — it has no effect on behavior. rho uses endpoint, api, and api_key_env to determine how to connect; it doesn't branch on type.

The name field is used as the display name in the consent prompt, /models output, and provider switching. If not set, rho uses the type field, then the endpoint hostname, then the provider index.

The preset field auto-fills endpoint, name, and api from a built-in registry. The openai preset selects Responses; all other presets select Chat Completions. An explicit api value overrides the preset. See the Configuration page for the full list.

[[providers]]
# Explicit name (optional, set automatically by preset)
name = "my-openrouter"

# Preset fills in endpoint and display name
preset = "openrouter"

# Explicit endpoint overrides preset (optional)
endpoint = "https://..."

# Protocol: "chat_completions" (default) or "responses"
api = "chat_completions"

# API key env var (required for remote providers)
api_key_env = "OPENROUTER_API_KEY"

# Default model for this provider (used when agent.provider points here)
default_model = "anthropic/claude-sonnet-4-20250514"

# Informational type label (has no effect on behavior)
type = "openrouter"

If type is set to a known unsupported provider name, rho prints a warning at startup. This is a safety net—the real requirement is that the endpoint implements the selected Responses or Chat Completions wire format.

Connection timeouts

The HTTP client uses sensible defaults to prevent indefinite hangs when a provider is unreachable:

• Connect timeout: 30 seconds — how long to wait for the initial TCP/TLS connection
• Request timeout: 2 minutes — total time allowed for the entire request (including streaming response)

These are hardcoded and not currently configurable. If you frequently hit these timeouts with a slow local server, ensure the server is running before starting rho.

Small-model robustness

rho includes safeguards for models that struggle with tool-use tasks:

• max_consecutive_empty (default 5) — aborts the agent loop when the model returns too many consecutive empty responses, preventing infinite retry spirals. Set to 0 to disable.
• Sandbox path hints — when a tool call fails with a sandbox path error, rho appends an actionable hint (sandbox root, correct path format) so the model can self-correct instead of repeating the same mistake.

Both features are configured under [agent] — see Configuration for details.

Privacy considerations

When you use an external provider, your entire conversation — including file contents read by the agent, shell command output, and your instructions — is sent to the provider's servers. Review the provider's privacy policy before use.

The consent prompt reminds you of this on every startup for config-driven external endpoints.

RPC Mode

rho runs as a headless agent communicating via JSON-RPC 2.0 over stdin/stdout. All requests must include "jsonrpc": "2.0", a method field, optional params, and a numeric or string id for response correlation. Streaming events are delivered as JSON-RPC notifications (no id field).

Diagnostic output (warnings, budget info, session status) is written to stderr, keeping stdout exclusively for the protocol.

Machine-readable schema. An OpenRPC 1.3.1 schema documents every method, its params, and result types. Use it with any OpenRPC-compatible code generator to produce type-safe client stubs in your language of choice.

Usage

echo '{"jsonrpc":"2.0","method":"prompt","params":{"message":"fix the bug"},"id":1}' | \
  rho --model my-model

Or with an external provider:

echo '{"jsonrpc":"2.0","method":"prompt","params":{"message":"explain this function"},"id":1}' | \
  rho --endpoint https://openrouter.ai/api/v1/chat/completions \
    --api-key-env OPENROUTER_API_KEY \
    --model deepseek/deepseek-v4-flash \
    --accept-external-provider

Protocol

Methods (stdin → rho)

Notifications (rho → stdout, no id)

Error codes

Approval flow

When rho emits an approval/request notification, it blocks until it reads an approvalResponse method from stdin:

{"jsonrpc": "2.0", "method": "approvalResponse", "params": {"approved": true}, "id": 2}

Sending approved: false denies the tool call and lets the agent continue.

When approved: false and message is provided, the agent treats it as a redirect: the user's alternative instructions are injected as a new conversation turn and the model returns to thinking to re-plan. No tool/denied notification is emitted for a redirect — the tool batch is abandoned and the model gets another turn.

Example interaction with a destructive tool:

→ {"jsonrpc":"2.0","method":"prompt","params":{"message":"delete the temp files"},"id":1}
← {"jsonrpc":"2.0","method":"agent/start"}
← {"jsonrpc":"2.0","method":"state/change","params":{"state":"thinking"}}
← {"jsonrpc":"2.0","method":"tool/call","params":{"name":"run_command","arguments":"{\"command\":\"Remove-Item temp/*\"}"}}
← {"jsonrpc":"2.0","method":"approval/request","params":{"tool":"run_command","arguments":"{\"command\":\"Remove-Item temp/*\"}","risk":"destructive"}}
→ {"jsonrpc":"2.0","method":"approvalResponse","params":{"approved":true},"id":2}
← {"jsonrpc":"2.0","method":"tool/result","params":{"name":"run_command","is_error":false,"output":""}}
← {"jsonrpc":"2.0","method":"state/change","params":{"state":"idle"}}
← {"jsonrpc":"2.0","method":"agent/end","params":{"reply":"Done — the temp files have been removed.","iterations":1,"usage":{"inputTokens":420,"outputTokens":28,"totalTokens":448,"totalCost":0.0,"requestCount":1},"toolCalls":[{"name":"run_command","arguments":"{\"command\":\"Remove-Item temp/*\"}","outcome":{"kind":"success"},"durationMs":45}],"durationMs":1200,"finishReason":"stop"}}
← {"jsonrpc":"2.0","result":{"reply":"Done..."},"id":1}

Headless startup policy

In RPC mode there is no interactive terminal, so the startup phases that normally prompt the user use safe defaults instead:

Session management

RPC mode supports the same session options:

• --ephemeral — in-memory session, no disk I/O (recommended for scripts)
• --continue / -c — resume the most recent session
• --session <path> — resume a specific session file

Use getSessionStats to monitor context usage and compact to free space when the context fills up.

Mid-turn steering

A prompt with steer: true is a steering nudge — it is queued and injected at the next tool-batch seam (between the last tool execution and the next LLM call), rather than starting a new turn. This lets the user provide course corrections while the agent is mid-turn without interrupting the active execution.

→ {"jsonrpc":"2.0","method":"prompt","params":{"message":"also check the tests","steer":true},"id":3}
← {"jsonrpc":"2.0","result":{},"id":3}

The steering message is drained at the tool-batch seam and appended as a user message before the next LLM call, so the model re-plans with the user's mid-turn input. If no tool batch is in flight, the message is still queued and will be drained at the next seam.

Steering is backed by a concurrent reader architecture: a spawned reader task owns the transport and demuxes every inbound message. Time-sensitive messages (steering prompts, approvalResponse, abort) are routed inline so they arrive even while a turn is running on the main task. Ordinary prompts and all other methods are queued for serial processing.

Testing

The RPC dispatch loop uses a concurrent reader architecture decoupled from I/O via the Transport trait (rho/src/transport.rs). StdioTransport (newline-delimited JSON over stdin/stdout) is the default; in-process integration tests use ChannelTransport (mpsc-backed, controls when each message becomes readable) and SyncTool (a tool that blocks until released) to inject messages deterministically mid-turn. Tests use TestProvider (from rho-test-helpers) to wrap a MockChatClient as a Provider and construct App directly, bypassing CLI startup. See Testing for details.

Run cargo xtask schema to update the version in the OpenRPC schema after a release.

Example test session

stdin:  {"jsonrpc":"2.0","method":"prompt","params":{"message":"say hello"},"id":1}
stdout: {"jsonrpc":"2.0","method":"ready"}
stdout: {"jsonrpc":"2.0","method":"agent/start"}
stdout: {"jsonrpc":"2.0","method":"state/change","params":{"state":"thinking"}}
stdout: {"jsonrpc":"2.0","method":"message/delta","params":{"delta":"hello"}}
stdout: {"jsonrpc":"2.0","method":"state/change","params":{"state":"idle"}}
stdout: {"jsonrpc":"2.0","method":"agent/end","params":{"reply":"hello","iterations":1,"usage":{"inputTokens":32,"outputTokens":5,"totalTokens":37,"totalCost":0.0,"requestCount":1},"toolCalls":[],"durationMs":800,"finishReason":"stop"}}
stdout: {"jsonrpc":"2.0","result":{"reply":"hello"},"id":1}

Wire-format types

All wire-format structs live in rho/src/rpc_wire.rs. Every method param, result, and notification has a typed Rust struct with Serialize and (where applicable) Deserialize, so the dispatch layer catches malformed requests early and produces well-formed responses. The mdBook source for this page (docs/src/rpc-mode.md) duplicates the information in prose form; the single source of truth is the OpenRPC schema.

Extensions

rho supports TypeScript extensions that add custom tools, hooks, and commands without modifying the core codebase. Extensions run in isolated V8 isolates via rho-ext (powered by deno_core).

Overview

Extensions are TypeScript files placed in extension directories:

~/.rho/extensions/           # User-level (available in all projects)
  ├── hello.ts               # Single-file extension
  └── rust_docs/
      ├── mod.ts             # Entry point for multi-file extension
      ├── parser.ts          # Helper module
      └── cache.ts           # In-memory cache

.rho/extensions/             # Project-level (overrides user-level by name)
  └── project_tool.ts

Each extension gets its own V8 isolate on a dedicated OS thread. The extension's export default { ... } manifest declares tools, hooks, and commands. TypeScript is transpiled to JavaScript at load time via deno_ast — no external Deno CLI or build step needed.

Extension format

/// <reference path="../types/rho.d.ts" />

export default {
  name: "my-extension",
  version: "1.0.0",

  tools: [{
    name: "search",
    description: "Search documentation",
    risk: "read" as const,
    parameters: {
      query: { type: "string", description: "Search query", required: true },
    },
    async execute(args: string) {
      const { query } = JSON.parse(args);
      const resp = await fetch(`https://docs.example.com/search?q=${encodeURIComponent(query)}`);
      if (!resp.ok) return JSON.stringify({ error: `HTTP ${resp.status}` });
      const text = await resp.text();
      return text;
    },
  }],

  hooks: {
    async onLoad() {
      rho.log("info", "my-extension loaded");
    },
    async onToolCall(args: string) {
      // Notification-only in the current implementation.
      // Return-value interception (block/modify) is planned.
      rho.log("debug", `tool called`);
    },
    async onToolResult(args: string) {
      // Notification-only.
    },
  },

  commands: [{
    name: "deploy",
    description: "Deploy to production",
    async handler(args: string) {
      rho.log("info", `Deploying: ${args || "default"}`);
      return "deployed";
    },
  }],
};

Available APIs

Built-in (from V8)

• Web APIs: fetch(), URL, URLSearchParams, TextEncoder/TextDecoder, setTimeout, setInterval, console.log
• JavaScript built-ins: Promise, Array, Map, Set, RegExp, JSON, Date, crypto.subtle

rho host functions (the rho global)

declare const rho: {
  function log(level: "trace" | "debug" | "info" | "warn" | "error", message: string): void;
  function readFile(path: string): string;
  function writeFile(path: string, content: string): void;
  function runCommand(command: string, args?: string[]): {
    stdout: string;
    stderr: string;
    exitCode: number;
  };
  function getCwd(): string;
  function getModel(): string;
};

Not available (by design)

• No Deno.* namespace
• No Node.js APIs (require, node:fs, child_process)
• No arbitrary filesystem access (must go through rho.readFile()/rho.writeFile())
• No npm packages (extensions are self-contained)

Configuration

Extensions are configured in .rho/config.toml or ~/.rho/config.toml:

[extensions]
# Allowlist: only load these extensions (omit to load all)
enabled = ["hello", "crates-search"]

[extensions.defaults]
# Default permissions for all extensions
network = true           # allow fetch()
max_memory_mb = 64       # V8 heap limit per isolate
max_execution_time_s = 30

[extensions.per_extension."rust-docs"]
# Override defaults for a specific extension
max_memory_mb = 128      # needs more for HTML parsing

[extensions.per_extension."dangerous-tool"]
network = false          # deny fetch()
commands = false         # deny rho.runCommand()

Hot reload

Use the reloadExtensions RPC method to hot-reload extensions without restarting rho:

• Extensions with changed file mtimes are destroyed and re-spawned
• New extensions are loaded; removed extensions are shut down
• Tool registry is updated (stale tools unregistered, new tools registered)
• onLoad hooks fire on new/reloaded extensions

V8 isolate creation takes ~20–50ms per extension. For 5 extensions, full reload is ~100–250ms.

RPC methods

Extension types

Type definitions ship at rho-ext/types/rho.d.ts. Extension authors can reference them for IntelliSense:

/// <reference path="~/.rho/types/rho.d.ts" />

Architecture

The extension system is implemented in the rho-ext crate:

Tool-call interception

Extensions can observe tool calls via the onToolCall hook. In the current implementation this hook is notification-only — it cannot block, modify, or rewrite tool calls. The InterceptResult enum (Block/Allow) exists at the Rust AgentObserver level, but DenoObserver does not yet bridge return values from JS hooks to the Rust interception path. Return-value interception (block/modify) is planned for a future release.

Model propagation

When the model is switched via /model or RPC set_model, the new model identifier is propagated to all loaded extensions so rho.getModel() returns the current model.

Performance

For rho's deployment model (long-running service, 3–8 extensions typical), this overhead is negligible. Extension bridge overhead per tool call is ~100µs–1ms, imperceptible compared to LLM round-trip times.

Examples

Error code lookup

export default {
  name: "rust-error-codes",
  tools: [{
    name: "error_code",
    description: "Look up a Rust compiler error code (e.g. E0277)",
    risk: "read" as const,
    parameters: {
      code: { type: "string", description: "Error code (e.g. E0277)", required: true },
    },
    async execute(args: string) {
      const { code } = JSON.parse(args);
      const upper = code.toUpperCase();
      if (!/^E\d+$/.test(upper)) {
        return JSON.stringify({ error: "Invalid format. Use E followed by digits." });
      }
      const resp = await fetch(`https://doc.rust-lang.org/stable/error_codes/${upper}.html`);
      if (!resp.ok) return JSON.stringify({ error: `${upper} not found` });
      const html = await resp.text();
      const text = html.replace(/<[^>]+>/g, "").replace(/\s+/g, " ").trim();
      return text;
    },
  }],
};

Approval gate

export default {
  name: "no-force",
  hooks: {
    async onToolCall(args: string) {
      const { toolName, arguments } = JSON.parse(args);
      if (toolName !== "run_command") return;
      const cmd = (arguments as string) ?? "";
      if (cmd.includes("-Force") || cmd.includes("--force")) {
        rho.log("warn", "Force flags blocked by no-force extension");
        // Note: onToolCall is notification-only in the current implementation.
        // Return-value interception (Block) is planned but not yet wired.
      }
    },
  },
};

Session extension entries

rho's session tree supports typed extension entries that are versioned and schema-skew-safe:

#![allow(unused)]
fn main() {
// Write typed state (never sent to the model)
session.write_custom_state("rho.diagnostics.v1", &json!({ "errors": 3 }))?;

// Read back
if let Some(state) = session.read_custom_state("rho.diagnostics.v1")? {
    println!("errors: {}", state["errors"]);
}
}

Extension entry kinds use the format <author>.<feature>.v<n>. When rho encounters an unknown kind version, it returns None (graceful degradation).

Future work

Development

How to build, test, and extend rho.

• Testing — test-driven approach, fixtures, naming conventions
• Dependency Philosophy — when to write it vs. depend on it
• Roadmap — phase-by-phase development plan

Testing

rho uses a layered test strategy: unit tests within each source file, integration tests at the crate level, and end-to-end scenario tests that exercise the full agent loop.

Running tests

cargo xtask ci        # Full CI pipeline: fmt → lint → build → test
ctest                 # All tests via cargo-nextest (falls back to cargo test)
cargo xtask test -p rho-core -- --nocapture  # Single crate

Test structure

RPC integration tests live in rho/src/rpc.rs (not rho/tests/) because App's fields are pub(crate). They use TestProvider to wrap a MockChatClient as a Provider and construct App directly, bypassing the full CLI startup sequence. The RPC dispatch loop is decoupled from I/O via the Transport trait (rho/src/transport.rs); tests inject a StdioTransport wired to Cursor<Vec<u8>> readers and captured writers for both stdin and stdout.

Test helpers (rho-test-helpers)

rho-test-helpers provides the shared infrastructure that all test layers use:

Mocks

Builders

Naming conventions

• Test functions: descriptive_snake_case — e.g., system_message_is_always_retained
• Test modules: match the function under test — e.g., tests::tool_result_success_has_no_details
• Fixtures: tests/fixtures/*.json — externalised test data for deserialization tests

Coverage priorities

Dependency Philosophy

rho is conservative about adding external dependencies. Every new dependency is a supply-chain commitment that affects build times, binary size, audit surface, and maintenance burden.

Decision framework

Before adding a dependency, answer these questions:

1. Can we write it ourselves in < 100 lines? → Write it. rho prefers small, purpose-built implementations over pulling in a crate for something trivial.

2. Is it already a transitive dependency? → If reqwest already pulls in serde_urlencoded, we can use it directly without adding a new top-level dependency.

3. Is it the de-facto standard for the problem? → serde, tokio, clap, tracing are widely adopted, well-maintained, and unlikely to cause issues. Standard choices are preferred over novel alternatives.

4. Is the maintenance track record good? → Check last release date, open issues, contributor count. A crate that hasn't been updated in 2 years is a risk.

Current dependencies

rho's direct (non-transitive) dependencies are:

What we don't depend on

Roadmap

rho is developed in phases, each building on the last. The current version is 0.84.7.

Phase summary

Context Management Enhancement (0.62–0.64)

A series of incremental improvements to context management, addressing structural issues that caused context exhaustion and amnesia:

Upcoming phases

Phase 4 — Terminal UI

Rich TUI replacing the bare REPL. Streaming output, approval prompts with rich previews, diagnostic panels, syntax highlighting. Built on ratatui/crossterm.

Phase 6 — LSP

rust-analyzer integration for real-time diagnostics, go-to-definition, and refactoring support. Deferred until after the TUI is stable.