Architecture v2 — Complete

This is the complete architecture for Jan Agent Framework, building on top of Architecture v1 (MVP). It adds the subsystems designed in plan documents 10–17: centralized tool dispatch, shared turn engine, lifecycle hooks, typed event protocol, LLM abstraction refinements, builder pattern, benchmark harness, and structured error strategy.

⚠️

v2 subsystems are in design phase — none are implemented yet. Everything in v1 remains unchanged.

Component architecture (v2 additions)

Layered view


┌─────────────────────────────────────────────────────────────────────┐
│                         CONSUMER LAYER                               │
│                                                                      │
│       jan-cli        │     Jan Desktop     │     API Server          │
│                      │                     │                        │
│   AgentRuntimeBuilder::new()               (NEW: builder pattern)   │
│     .core(ReActCore::new())                                         │
│     .provider(OpenAiProvider::new(...))                              │
│     .hooks(chain)                          (NEW: lifecycle hooks)   │
│     .build()?                                                       │
│     → AgentRuntime                                                  │
└──────────────────────────────────┬──────────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────────┐
│                       ORCHESTRATION LAYER                            │
│                                                                      │
│   AgentRuntime                                                       │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  Owns: core, provider, tools, memory, policy,              │   │
│   │        hooks, event_tx                     (NEW: hooks)     │   │
│   │                                                             │   │
│   │  run_turn():                                                │   │
│   │    1. hooks.pre_turn(input)                (NEW)            │   │
│   │    2. Memory: pre_turn_context()                            │   │
│   │    3. take_context() → AgentContext                          │   │
│   │    4. core.run_turn(input, ctx)                             │   │
│   │    5. restore_context()                                     │   │
│   │    6. Memory: post_turn_observe()                           │   │
│   │    7. hooks.post_turn(input, output)       (NEW)            │   │
│   └─────────────────────────────────────────────────────────────┘   │
└──────────────────────────────────┬──────────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────────┐
│                        CORE LAYER                                    │
│                                                                      │
│   AgentCore (trait)                                                  │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  Receives: AgentContext { provider, tools, memory,          │   │
│   │              policy, hooks, events, config }  (NEW: hooks)  │   │
│   │                                                             │   │
│   │  ReActCore / PlanExecuteCore / EmbodiedCore                 │   │
│   │    │                                                        │   │
│   │    └── delegates to TurnEngine              (NEW)           │   │
│   │        ┌─────────────────────────────────────────┐          │   │
│   │        │ TurnEngine (shared single-turn runner)  │          │   │
│   │        │                                         │          │   │
│   │        │  1. Build messages                      │          │   │
│   │        │  2. hooks.pre_llm_call()                │          │   │
│   │        │  3. provider.chat_completion()           │          │   │
│   │        │  4. hooks.post_llm_call()               │          │   │
│   │        │  5. ToolDispatchEngine.execute_all()     │          │   │
│   │        │  6. Emit events                         │          │   │
│   │        └─────────────────────────────────────────┘          │   │
│   └─────────────────────────────────────────────────────────────┘   │
│                                                                      │
│   ToolDispatchEngine (centralized)               (NEW)              │
│   ┌─────────────────────────────────────────────────────────────┐   │
│   │  For each tool call:                                        │   │
│   │    1. Lookup tool in registry                               │   │
│   │    2. policy.check_permission() → ExecutionGrant            │   │
│   │    3. hooks.pre_tool_call(name, args)                       │   │
│   │    4. Route to sandbox (Host/WASM/MicroVM/Remote)           │   │
│   │    5. hooks.post_tool_call(name, result)                    │   │
│   │    6. Emit AgentEvent::ToolResult                           │   │
│   │    7. On error → wrap as ToolExecution{success:false}       │   │
│   └─────────────────────────────────────────────────────────────┘   │
└──────────────────────────────────┬──────────────────────────────────┘
                                   │
┌──────────────────────────────────▼──────────────────────────────────┐
│                      EXECUTION LAYER                                 │
│                                                                      │
│   RuntimePolicy ──▶ ExecutionGrant ──▶ Sandbox Selection             │
│                                                                      │
│   ┌──────────┐    ┌──────────────┐    ┌──────────────┐              │
│   │   Host   │    │  WASM        │    │  MicroVM     │              │
│   │ (native) │    │  (wasmtime,  │    │ (microsandbox│              │
│   │          │    │   fuel-based) │    │   server)    │              │
│   └──────────┘    └──────────────┘    └──────────────┘              │
└─────────────────────────────────────────────────────────────────────┘

Component interaction (v2)


                    ┌──────────────┐
                    │  Consumer    │
                    │  (CLI/App)   │
                    └──────┬───────┘
                           │ AgentRuntimeBuilder
                           │   .core().provider().hooks().build()?
                           ▼
                    ┌──────────────┐
              ┌────▶│ AgentRuntime │◀────┐
              │     └──────┬───────┘     │
              │            │             │
         events()     take_context()   with_config()
              │            │
              ▼            ▼
     ┌────────────┐  ┌─────────────┐
     │ EventBus   │  │AgentContext  │
     │ (broadcast)│  │             │
     │            │  │ .provider ──┼──▶ Box<dyn LlmProvider>
     │ SessionId  │  │ .tools ─────┼──▶ Box<dyn ToolDispatcher>
     │ correlation│  │ .memory ────┼──▶ Box<dyn MemoryPlugin>
     │            │  │ .policy ────┼──▶ Box<dyn RuntimePolicy>
     └────────────┘  │ .hooks ─────┼──▶ HookChain           (NEW)
                     │ .events ────┼──▶ EventSender (clone)
                     │ .config ────┼──▶ FrameworkConfig
                     └──────┬──────┘
                            │
                            ▼
                     ┌─────────────┐
                     │ AgentCore   │──▶ TurnEngine     (NEW)
                     │ (ReActCore) │──▶ ToolDispatch   (NEW)
                     └─────────────┘

New trait additions

Hook trait

The AgentHook trait defines 10 lifecycle hook points — 2 gate hooks (can abort) and 8 observation hooks (can transform):


AgentHook
├── name() -> &str
├── priority() -> u32
├── fail_open() -> bool
│
├── on_session_start/end(ctx)                 ← gate hooks (can abort)
├── on_tool_call(name, args, ctx)             ← gate hook (can abort)
│
├── pre_turn(input, ctx)           -> HookAction<TurnInput>
├── post_turn(input, output, ctx)  -> HookAction<TurnOutput>
├── on_turn_error(input, err, ctx) -> HookAction<TurnOutput>
├── pre_llm_call(messages, tools)  -> HookAction<LlmCallInput>
├── post_llm_call(response)        -> HookAction<LlmResponse>
├── on_llm_error(error, attempt)   -> HookAction<RetryDecision>
├── pre_tool_call(name, args)      -> HookAction<Value>
├── post_tool_call(name, result)   -> HookAction<DispatchResult>
└── pre/post_compaction(messages)  -> HookAction<CompactionHint>
HookAction<T> = Continue | Transform(T) | Reject(String) | Replace(T)

See Lifecycle Hooks for full design details.

I/O traits

Typed input/output with type erasure at the runtime boundary:


AgentInput: Send + Sync + Serialize + DeserializeOwned
└── input_schema() -> Value
AgentOutput: Send + Sync + Serialize + DeserializeOwned
├── output_schema() -> Value
└── structured_output_format() -> Option<Value>
AgentCore (updated with associated types)
├── type Input: AgentInput = TurnInput
├── type Output: AgentOutput = TurnOutput
└── run_turn(Self::Input, ctx) -> Result<Self::Output, AgentError>
ErasedAgentCore (internal)
└── run_turn_erased(Value, ctx) -> Result<Value, AgentError>
    // Blanket impl handles serde for type erasure at runtime boundary

See Input & Output Traits for full design details.

LLM provider refinement

Single chat() + chat_stream() replaces the current chat_completion():


LlmProvider (updated)
├── name() -> &str
├── chat(messages, options) -> Result<LlmResponse, ProviderError>
├── chat_stream(messages, options) -> Result<Stream<LlmChunk>, ProviderError>
└── supports_streaming() -> bool
ChatOptions {
    tools: Vec<ToolMeta>,
    temperature: Option<f32>,
    max_tokens: Option<u32>,
    structured_output: Option<Value>,
    ...
}

See LLM Abstractions for full design details.

New subsystems

Turn engine

Shared single-turn execution engine that all cores delegate to. Handles the inner loop mechanics so cores only define the loop strategy:


TurnEngine::execute(messages, ctx) -> TurnStepResult
    │
    ├── Build final message list
    ├── hooks.pre_llm_call(messages, tools)
    ├── provider.chat(messages, options)
    ├── hooks.post_llm_call(response)
    │
    ├── If tool calls present:
    │   └── ToolDispatchEngine.execute_all(tool_calls, ctx)
    │
    ├── Emit events (Thinking, ToolCall, ToolResult, etc.)
    └── Return TurnStepResult { response, tool_results, done }

Cores use TurnEngine like this:

ReActCore: loop calling engine.execute() until done or max_steps
PlanExecuteCore: plan phase, then loop execute phase
EmbodiedCore: inject sensor frames, then loop

See Turn Engine for full design details.

Tool dispatch engine

Centralized tool call processor replacing inline tool dispatch scattered across cores:


ToolDispatchEngine::execute_all(tool_calls, ctx) -> Vec<ToolResult>
    │
    For each tool call:
    ├── 1. Lookup in registry → ToolMeta
    ├── 2. policy.check_permission(tool, args) → ExecutionGrant
    ├── 3. hooks.pre_tool_call(name, args)
    ├── 4. Match grant:
    │      ├── Host   → native execution
    │      ├── Wasm   → wasmtime sandbox
    │      ├── MicroVm → microsandbox
    │      └── Remote → HTTP forward
    ├── 5. hooks.post_tool_call(name, result)
    ├── 6. Emit AgentEvent::ToolResult
    └── 7. On error → ToolExecution { success: false, error }
           (tool failures become conversation data, never exceptions)

See Tool Dispatch for full design details.

Event protocol

Typed event system with session correlation:


AgentEvent {
    session_id: SessionId,          ← correlates events to sessions
    timestamp: Instant,
    payload: AgentEventPayload,
}
Event ownership:
  AgentRuntime    → SessionStarted, SessionEnded, TurnStarted, TurnCompleted
  TurnEngine      → Thinking, LlmCall, LlmResponse, Retrying, ContextCompacted
  ToolDispatch    → ToolCall, ToolResult, ToolLog
  HookChain       → HookRejected, HookTransformed

See Event Protocol for full design details.

Builder pattern

Non-generic fluent builder with sensible defaults:


let runtime = AgentRuntimeBuilder::new()
    .core(ReActCore::new(agent_loop))
    .provider(OpenAiProvider::new(url, key))
    .memory(WorkingMemory::new())           // default: NullMemory
    .policy(StandardPolicy)                  // default: HostPolicy
    .hooks(chain)                            // default: NoOpHooks
    .config(FrameworkConfig { max_steps: 50, .. })
    .build()?;                               // validates required fields

See Builder Pattern for full design details.

Error strategy

Layered error handling — failures are categorized by severity:


LLM failures     → fatal (bubble up as AgentError)
Tool failures    → wrapped as ToolExecution { success: false }
                   (LLM sees the error and adapts)
Memory failures  → logged and degraded (turn continues without memory)
Event failures   → silent (broadcast send errors are ignored)
Hook failures    → depends on hook.fail_open():
                   true  → log and continue
                   false → abort turn with HookRejected error

See Error Strategy for full design details.

Security model (v2)

v2 adds hook-based guardrails at multiple points in the security pipeline:


User input
    │
    ▼
hooks.pre_turn(input) ──▶ Reject if blocked      (NEW: guardrail hook)
    │
    ▼
LLM inference (external call)
    │
    ▼
hooks.post_llm_call(response) ──▶ Transform/Reject (NEW: output guardrail)
    │
    ▼
Tool call requested
    │
    ▼
RuntimePolicy::check_permission()
    │
    ├── Denied → AgentError, turn continues without tool
    │
    ├── Granted(Host) → native execution, full access
    ├── Granted(Wasm) → wasmtime sandbox, fuel-limited, no filesystem
    ├── Granted(MicroVm) → microsandbox container, network-isolated
    └── Granted(Remote) → forwarded to remote execution endpoint
    │
    ▼
hooks.pre_tool_call(name, args) ──▶ Human approval, arg rewriting (NEW)
    │
    ▼
Tool execution (in granted sandbox)
    │
    ▼
hooks.post_tool_call(name, result) ──▶ Audit logging, result filtering (NEW)

Crate layout (v2 additions)

New crates

Crate	Purpose
`jan-providers`	Additional LlmProvider impls (Anthropic, local)
`jan-memory`	ConversationMemory (jan-data), SemanticMemory
`jan-tools`	Tool plugin SDK, manifest validation, WASM helpers
`jan-hooks`	Built-in hooks (audit, guardrails, cost tracking)
`jan-bench`	Benchmark harness (criterion, MockLlmProvider)

Updated dependency graph


jan-cli
  ├── jan-agent-core
  │     ├── jan-framework        ← trait crate (the spine)
  │     └── jan-hooks            ← built-in hooks    (NEW)
  ├── jan-framework (direct)
  ├── jan-providers              ← provider impls    (NEW)
  ├── jan-memory                 ← memory impls      (NEW)
  ├── jan-tools                  ← tool SDK          (NEW)
  ├── jan-data
  ├── jan-llamacpp
  └── jan-utils
jan-bench (dev-dependency only)
  ├── jan-framework
  ├── jan-agent-core
  └── criterion

Benchmark targets

Benchmark	Target
Registry lookup (100 tools)	< 1ms
Policy check (ProfilePolicy, 50 rules)	< 0.1ms
Hook chain dispatch (10 hooks)	< 0.5ms
Full run_turn (mock LLM, 1 tool call)	< 5ms
Event broadcast (1000 events)	< 10ms

Key design decisions (v2)

Decision	Rationale
Hook chain inside AgentContext	Cores call `ctx.hooks.run_pre_llm_call()` at appropriate points. Hooks are opt-in — a minimal core can ignore them. Zero overhead when chain is empty.
Type erasure at runtime boundary	`AgentCore` gains associated types `Input`/`Output`. `AgentRuntime` wraps the core in `ErasedAgentCore` that serde round-trips to `Value`. No generic parameters leak into the runtime.
Tool failures as conversation data	Tool errors are wrapped as `ToolExecution { success: false }` — never exceptions. The LLM sees the error and can retry or explain to the user.
SessionId on events	Every `AgentEvent` carries a `SessionId` for correlation. Consumers (TUI, desktop, API) can filter and route events by session.

Relationship to v1

Everything in Architecture v1 remains unchanged. v2 builds on top:

v1 Component	v2 Addition
`AgentRuntime.run_turn()`	Adds hook calls at steps 1 and 7
`AgentContext`	Adds `.hooks` field
Inline tool dispatch in cores	Extracted to `ToolDispatchEngine`
Per-core LLM call logic	Extracted to `TurnEngine`
`AgentRuntime::new()`	Replaced by `AgentRuntimeBuilder`
Simple `AgentEvent` enum	`AgentEvent` struct with `SessionId` + typed payload
`AgentCore::run_turn(TurnInput)`	`AgentCore::run_turn(Self::Input)` with associated types

v1 — MVP Agent System