Core Concept

Jan Agent Framework is built on one idea: every component is a trait. The reasoning loop, the LLM backend, the tools, the memory, and the execution policy are all defined as Rust traits. Swap one implementation for another and the agent behaves completely differently — without changing the framework itself.

This makes the same framework power a chat assistant, a coding agent, or a robot brain. They differ only in which traits they plug in.

Why traits

Agents are not one-size-fits-all:

A chat assistant needs conversation memory and web tools
A coding agent needs file tools, a sandbox, and project-scoped memory
A robot needs vision sensors, spatial memory, and motor commands

Building each from scratch duplicates the common infrastructure — LLM communication, tool dispatch, context management, event streaming. The framework extracts that infrastructure into reusable contracts (traits) and lets you plug in the pieces you need.

The five components

Every Jan agent is assembled from five pluggable components, wired together by AgentRuntime:


┌───────────────────────────────────────────────────┐
│                  AgentRuntime                      │
│                                                    │
│  ┌───────────┐  ┌───────────┐  ┌───────────────┐ │
│  │ Agent Core│  │ LLM       │  │ Runtime       │ │
│  │ (trait)   │  │ Provider  │  │ Policy        │ │
│  │           │  │ (trait)   │  │ (trait)       │ │
│  └─────┬─────┘  └─────┬─────┘  └───────┬───────┘ │
│        │              │                 │          │
│  ┌─────▼──────────────▼─────────────────▼───────┐ │
│  │            Plugin Registry                    │ │
│  │  ┌──────────┐  ┌──────────┐  ┌────────────┐ │ │
│  │  │  Tools   │  │  Memory  │  │  Sensors   │ │ │
│  │  │ (trait)  │  │  (trait) │  │  (trait)   │ │ │
│  │  └──────────┘  └──────────┘  └────────────┘ │ │
│  └──────────────────────────────────────────────┘ │
└───────────────────────────────────────────────────┘

Component	What it does	Why it's a trait
Agent Core	The reasoning loop — how the agent thinks and acts	A chat agent uses ReAct. A robot uses sense-plan-act. Same interface, different logic.
LLM Provider	Talks to the inference backend	LlamaCPP locally, OpenAI-compatible remotely, or any custom endpoint. The agent doesn't care which.
Tools	Executable capabilities — web search, code execution, file I/O	WASM modules, JS scripts, native Rust, HTTP endpoints. All implement the same trait.
Memory	Persistent state across turns and sessions	Conversation history for chat, spatial maps for robots, vector stores for RAG.
Runtime Policy	Controls where and how tools execute	Host for trusted tools, WASM sandbox for untrusted ones, microVM for code execution.

How the runtime glues them together

The AgentRuntime orchestrates every agent turn through the same flow, regardless of which components are plugged in:


User message
    │
    ▼
AgentRuntime
    │
    ├── 1. memory.pre_turn_context()
    │      Retrieve relevant context from memory
    │
    ├── 2. core.run_turn()
    │      │
    │      ├── provider.chat_completion()
    │      │    Call the LLM with tools + context
    │      │
    │      ├── policy.check_permission()
    │      │    Can this tool run? In what sandbox?
    │      │
    │      └── registry.dispatch(tool, args)
    │           Execute the tool, return result
    │           (loop until LLM stops calling tools)
    │
    ├── 3. memory.post_turn_observe()
    │      Store what happened for future recall
    │
    └── 4. Emit AgentEvent stream
           For TUI / Desktop / API consumers

The runtime doesn't know or care about implementation details — it only calls trait methods. This is what makes the framework extensible: adding a new tool, a new memory backend, or a new agent core requires zero changes to the runtime.

Same skeleton, different agents

The framework's power is composition. The same AgentRuntime produces different agents by swapping components:

Chat assistant


AgentRuntime::new(
    ReActCore::new(system_prompt),
    OpenAiProvider::new("http://localhost:6767/v1"),
    registry.with_tools([web_search, http_fetch, code_exec]),
    CompositeMemory::new([conversation, working]),
    DefaultPolicy::standard(),
)

Coding agent


AgentRuntime::new(
    ReActCore::new(coding_prompt),
    OpenAiProvider::new(api_url),
    registry.with_tools([read_file, write_file, edit_file, bash, web_search]),
    CompositeMemory::new([conversation, working]),
    DefaultPolicy::privileged(),  // code.exec in microVM
)

Robot agent


AgentRuntime::new(
    EmbodiedCore::new(vec![CameraCapture::new()], Duration::from_millis(100)),
    OpenAiProvider::new(cloud_vlm_endpoint),
    registry.with_tools([robot_move, robot_look, robot_stop]),
    CompositeMemory::new([spatial, episodic]),
    DefaultPolicy::host(),  // low latency, no sandbox
)

In all three cases, the AgentRuntime, PluginRegistry, and event system are identical. Only the components plugged into them change.

Use cases

Jan Desktop App with agent

Jan Desktop embeds the framework as a Rust library through Tauri. The desktop app provides the UI; the framework provides the brain. Models and data are shared between Jan Desktop and Jan CLI — a model downloaded via the desktop UI is immediately available to the CLI agent.

Coding agent via CLI

The agent runs via the TUI (jan agent chat --mount /path/to/project), has access to the project directory, and can read, write, and execute code in a sandboxed environment. Memory persists across sessions so the agent remembers project context.

Robot agent on cloud platform

The EmbodiedCore runs a continuous sense-plan-act loop: capture a camera frame, send it to a vision-language model, receive a movement command, execute it, repeat. The framework handles LLM communication, tool dispatch, memory, and event streaming — the core only implements the reasoning pattern. The agent can run on a cloud platform (the robot's brain) while the hardware runs locally, connected via HTTP tool endpoints.

Decoupled from inference

The framework does not embed any LLM. It talks to inference providers through the LlmProvider trait:


jan serve <model>                    ← LlamaCPP serves on localhost:6767
    │
    ▼
OpenAiProvider("localhost:6767/v1")  ← Provider talks OpenAI format
    │
    ▼
AgentRuntime                         ← Framework is inference-agnostic

LlamaCPP is the default local backend because it already speaks OpenAI-compatible format. Any endpoint that implements /v1/chat/completions works as a drop-in replacement — vLLM, Ollama, OpenRouter, or a remote API.

First citizens

The framework is shaped by concrete, working implementations — not hypothetical future needs:

Dimension	First citizen (ships first)	Enabled for later
Agent Core	ReAct (current `AgentLoop`)	Plan-Execute, Embodied/Robot
LLM Provider	LlamaCPP local + OpenAI-compatible	Anthropic, Gemini, custom
Tools	Existing CLI tools (web.search, http.fetch, code.exec)	Community plugins
Memory	Conversation history + working scratchpad	Semantic/vector, spatial
Policy	Host execution + existing WASM/microVM sandbox	Remote execution, approval flows
UI	Current TUI (Ratatui)	Jan Desktop, web

Design principles

Traits over inheritance — Every extension point is a Rust trait. Composition over inheritance. Implement the trait, register it, done.
First citizens drive abstractions — Traits are extracted from working code (ReAct loop, WASM sandbox, LlamaCPP), not designed from theory. New implementations slot into abstractions shaped by real use.
No Tauri dependency — Every crate builds without Tauri. State is passed as Arc<Mutex<...>> instead of tauri::State, so the framework embeds in any Rust application.
Shared data folder — Jan Agent reads from the same data folder as Jan Desktop. Models and backends installed via either interface are available to both.
Progressive enhancement — Works with zero config (interactive pickers), but supports full CLI flags and config files for automation. Memory, policy, and tools are all opt-in with sensible defaults.

Quickstart Architecture