Tool Dispatch

⚠️

This is a v2 design — not yet implemented. See Architecture v2 for context.

The problem

When the LLM returns tool calls, something must: find the tool, check policy, validate arguments, execute in the right sandbox, handle errors, emit events, call hooks, and format results. Without a centralized processor, this logic is scattered across agent cores and diverges between implementations.

ToolDispatchEngine


pub struct ToolDispatchEngine;
impl ToolDispatchEngine {
    /// Dispatch a single tool call with full lifecycle.
    pub async fn dispatch(call: &ToolCall, ctx: &DispatchContext<'_>) -> Option<ToolExecution>;
    /// Dispatch multiple tool calls sequentially.
    pub async fn dispatch_all(calls: &[ToolCall], ctx: &DispatchContext<'_>) -> Vec<ToolExecution>;
    /// Build ToolResult ChatMessages from call/result pairs.
    pub fn build_tool_result_messages(calls: &[ToolCall], results: &[ToolExecution])
        -> (ChatMessage, ChatMessage);
}
pub struct DispatchContext<'a> {
    pub registry: &'a PluginRegistry,
    pub policy: &'a dyn RuntimePolicy,
    pub hooks: &'a dyn AgentHooks,
    pub events: &'a EventSender,
    pub session_id: SessionId,
}

Dispatch flow


ToolDispatchEngine::dispatch(call, ctx)
│
├─ 1. HOOK GATE
│     hooks.on_tool_call() → may abort (return None)
│
├─ 2. TOOL LOOKUP
│     registry.tool(name) → not found = ToolExecution { success: false }
│
├─ 3. ARGUMENT PARSING
│     serde_json::from_str(args) → parse error = ToolExecution { success: false }
│
├─ 4. POLICY CHECK
│     policy.check_permission(name, args) → denied = ToolExecution { success: false }
│
├─ 5. PRE-EXECUTION
│     hooks.on_tool_start() + emit ToolCallStart event
│
├─ 6. EXECUTION (sandbox-routed)
│     match grant.sandbox {
│         Host    → native execution
│         Wasm    → wasmtime sandbox
│         MicroVm → microsandbox
│         Remote  → HTTP forward
│     }
│
└─ 7. RESULT HANDLING
      success → emit ToolCallEnd, return ToolExecution { success: true }
      failure → emit ToolCallFailed, return ToolExecution { success: false }

Tool errors are conversation data

Tool failures are never Rust Errs that terminate the turn. They are wrapped as ToolExecution { success: false }:


pub struct ToolExecution {
    pub call_id: String,
    pub tool_name: String,
    pub arguments: Value,
    pub success: bool,
    pub output: Value,     // result or error description
    pub duration_ms: u64,
    pub sandbox: SandboxMode,
}

The LLM sees errors in its next context and can retry, fall back, or explain:

Error	What LLM Sees
Tool not found	`"Tool 'X' not found. Available: [a, b, c]"`
Invalid arguments	`"Invalid JSON: unexpected token at position 5"`
Permission denied	`"Permission denied: tool 'X' is disabled by policy"`
Execution failure	`"HTTP 500: connection refused"`
Sandbox timeout	`"Tool exceeded 30s wall time limit"`

None of these terminate the turn.

LLM conversation format

After dispatch, results are formatted as two messages:


// Message 1: Assistant's tool use request
ChatMessage { role: Assistant, message_type: ToolUse(calls), content: text }
// Message 2: Tool results
ChatMessage { role: Tool, message_type: ToolResult(results), content: "" }

build_tool_result_messages() constructs these, preserving call IDs for LLM correlation.

Policy integration

The dispatch is where tool registration (Phase 2) meets runtime policy (Phase 5):


// Phase 2: stub policy — everything runs on host
impl RuntimePolicy for HostPolicy {
    fn check_permission(&self, _tool: &str, _args: &Value) -> Result<ExecutionGrant, PolicyError> {
        Ok(ExecutionGrant { sandbox: SandboxMode::Host, limits: ResourceLimits::unlimited() })
    }
}
// Phase 5: real policy — routes by config
impl RuntimePolicy for DefaultPolicy {
    fn check_permission(&self, tool: &str, args: &Value) -> Result<ExecutionGrant, PolicyError> {
        let rule = self.rules.get(tool).unwrap_or(&self.default_rule);
        if !rule.allowed { return Err(PolicyError::Denied(tool)); }
        Ok(ExecutionGrant { sandbox: rule.sandbox, limits: rule.limits.clone() })
    }
}

The ToolDispatchEngine doesn't change between phases — only the policy implementation does.

Input & Output Traits Turn Engine