> ## Documentation Index
> Fetch the complete documentation index at: https://haico.gr/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent core logic

> How the agent runs a turn: per-turn context, the ReAct loop, the model-node retry wrapper, tools, the SSE stream, snapshots, and typed artifacts.

# Agent Core Logic

A deep tour of how the HAI-Co² backend agent thinks: what it sees on every turn, how the shared workspace and typed artifacts reach it, how its tools are wired, and how its output flows back to the frontend.

Files referenced throughout:

* [`backend/app/services/agent/builder.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/services/agent/builder.py): agent factory, system prompt, dynamic prompt
* [`backend/app/routers/query.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/routers/query.py): SSE streaming endpoint, turn counter, snapshot capture
* [`backend/app/tools/`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools): `_decorator.py`, `_context.py`, `manager.py`, `documents.py`, `todos.py`, `preferences.py`, `charts.py`, `artifacts.py`
* [`backend/app/db/models.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/db/models.py): ORM models incl. `Artifact` and `WorkspaceSnapshot`
* [`backend/app/db/repositories/workspace_repository.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/db/repositories/workspace_repository.py): DB layer for documents / todos / prefs / artifacts / snapshots
* [`backend/app/routers/workspace.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/routers/workspace.py): REST endpoints for the human side and for past artifacts/snapshots
* [`frontend/src/app/app/page.tsx`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/app/app/page.tsx): workspace shell + SSE handler
* [`frontend/src/components/app/artifact-panel.tsx`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/components/app/artifact-panel.tsx): centre panel
* [`frontend/src/components/app/artifacts/registry.tsx`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/components/app/artifacts/registry.tsx): `artifact_type → React component` registry
* [`frontend/src/lib/api.ts`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/lib/api.ts): `streamQuery` + workspace REST client

***

## 1. The four shared surfaces

HAI-Co² is the conceptual frame: humans and the agent co-construct a single solution by sharing four objects. Each maps to a concrete UI panel and a concrete piece of backend state:

| HAI-Co² object                             | HAICO panel                             | Backend state                                           |
| ------------------------------------------ | --------------------------------------- | ------------------------------------------------------- |
| `Uᵗ` (utility / objective): the goal       | Planning: Objective (top of left-lower) | `objectives` table (one row / thread)                   |
| `Uᵗ` (constraint set)                      | top-left card list                      | `preferences` table                                     |
| `X_j` (construction-space sub-goals, plan) | Planning: the plan (numbered, nested)   | `todos` table (depth-ordered tree)                      |
| `X̂` (working artifact)                    | centre panel                            | `documents` table (default) + `artifacts` table (typed) |
| `I` (informational state, message history) | right-side chat                         | LangGraph checkpointer + `workspace_snapshots`          |

The left-lower **Planning** panel shows the agent's **Objective** (its one-line read of the goal, user-editable) above its **plan**: the numbered, nestable todo steps it ticks off as it works, kept as its own execution ledger. The plan has two views: an interactive **tree** (drag, check, edit) and a read-only **markdown** checklist; from the tree the user can add sub-steps and **indent / outdent** a todo step to change its nesting level (drag only reorders within a level; indent/outdent move a todo step and its subtree, persisted via `PATCH /todos/{id}/reindent`). The left rail uses fixed proportions (the Planning panel taller than Preferences) and each list scrolls inside its panel, so adding a preference or todo step never resizes or pushes the other panel. All are **per-thread**: every row is keyed by the LangGraph `thread_id` that uniquely identifies a conversation. A user owns a thread through the `conversation_users` link table, which also carries a monotonic `turn_index` counter (one per user message) used to tag everything the agent produces during the turn.

The centre panel is no longer document-only: it renders whatever artifact is currently selected. By default it shows the live, editable document; if a tool produced a typed artifact this turn (a chart, a table, …), the panel auto-switches to render it. See §10.

***

## 2. High-level request topology

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
           ┌────────────────────────────────────────────────────────────────┐
           │                       Browser (Next.js)                        │
           │                                                                │
           │  PreferencesPanel   TodoPanel   ArtifactPanel    ChatPanel     │
           │       │                │             │              │          │
           │       └─── REST ───────┴───── REST ──┘              │          │
           │                                                     ▼          │
           │                                              streamQuery (SSE) │
           └───────────────────────────────────┬────────────────────────────┘
                                               │ HTTPS + JWT
                                               ▼
           ┌────────────────────────────────────────────────────────────────┐
           │                       FastAPI backend                          │
           │                                                                │
           │  /api/workspace/{tid}/...   ◀── REST CRUD on docs/todos/prefs/ │
           │                                  artifacts/snapshots           │
           │  /api/query/stream_steps/sse◀── SSE: feed user msg to agent    │
           │                                                                │
           │   ┌──────────────────────────────────────────────────────────┐ │
           │   │   _capture_snapshot(): bump turn_index, write           │ │
           │   │       workspace_snapshots row BEFORE agent runs          │ │
           │   └──────────────────────────────────────────────────────────┘ │
           │   ┌──────────────────────────────────────────────────────────┐ │
           │   │  build_agent(checkpointer, thread_id, runtime_config)    │ │
           │   │                                                          │ │
           │   │   ┌────────────────────┐    ┌──────────────────────┐    │ │
           │   │   │ dynamic prompt fn  │    │ build_tools(thread)  │    │ │
           │   │   │  (rebuilds system  │    │  documents / todos / │    │ │
           │   │   │   message every    │    │  preferences /       │    │ │
           │   │   │   turn, injects   │    │  charts / artifacts  │    │ │
           │   │   │   prefs+todos+doc) │    └──────────────────────┘    │ │
           │   │   └────────────────────┘                                 │ │
           │   │                                                          │ │
           │   │   create_react_agent(model, tools, prompt, checkpointer) │ │
           │   └──────────────────────────────────────────────────────────┘ │
           └───────────────────────────────┬────────────────────────────────┘
                                           │
                    ┌──────────────────────┼───────────────────────┐
                    ▼                      ▼                       ▼
           ┌────────────────┐   ┌──────────────────────┐   ┌─────────────────┐
           │  LLM provider   │   │  Postgres            │   │ LangGraph       │
           │  (OpenAI /      │   │  documents / todos / │   │ checkpointer    │
           │   Anthropic /   │   │  preferences /       │   │ (chat history)  │
           │   Mistral …)    │   │  artifacts /         │   └─────────────────┘
           │                 │   │  workspace_snapshots │
           │                 │   │  conversation_users) │
           └────────────────┘   └──────────────────────┘
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        Panels["Browser (Next.js)<br/>PreferencesPanel · TodoPanel · ArtifactPanel · ChatPanel"]
        Panels -->|"REST · streamQuery (SSE) · HTTPS + JWT"| API
        subgraph Backend["FastAPI backend"]
            API["/api/workspace/... (REST CRUD)<br/>/api/query/stream_steps/sse (SSE)"]
            Snap["_capture_snapshot()<br/>bump turn_index, write workspace_snapshots<br/>BEFORE the agent runs"]
            Build["build_agent(checkpointer, thread_id, runtime_config)<br/>dynamic prompt fn + build_tools(thread)<br/>then create_react_agent(model, tools, prompt, checkpointer)"]
            API --> Snap --> Build
        end
        Build --> LLM["LLM provider<br/>(OpenAI / Anthropic / Mistral …)"]
        Build --> PG[("Postgres<br/>documents · todos · preferences · artifacts ·<br/>workspace_snapshots · conversation_users")]
        Build --> CP[("LangGraph checkpointer<br/>(chat history)")]
    ```
  </Tab>
</Tabs>

***

## 3. What context the agent receives on every turn

The agent is a LangGraph **React agent** (`create_react_agent`), i.e. it loops `LLM → maybe tool calls → LLM → … → final answer`. It is built fresh for every request by `build_agent(checkpointer, thread_id, runtime_config=None)` ([`builder.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/services/agent/builder.py)), yet its memory persists because both the checkpointer and the workspace tables are keyed by `thread_id`:

```python theme={null}
return create_react_agent(
    model=_dynamic_model,                      # retry-wrapped, tool-bound model (§4.1)
    tools=build_tools(thread_id),              # per-thread workspace tools (§5)
    prompt=_build_dynamic_prompt(thread_id),   # dynamic callable, NOT a static string
    checkpointer=checkpointer,                 # AsyncPostgresSaver
)
```

Two of those arguments are **not** what they look like. `model` is not `get_llm(...)` directly: it is a `_dynamic_model` callable returning the tool-bound model wrapped in a blank / malformed-turn retry (§4.1). And `prompt` is not the static `HAICO_SYSTEM_PROMPT` string: the agent's context is assembled freshly for *each* user message by that callable installed as the `prompt=` argument:

[`backend/app/services/agent/builder.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/services/agent/builder.py) (`_build_dynamic_prompt`):

```python theme={null}
async def dynamic_prompt(state):
    messages = state.get("messages") or []
    async with async_session_factory() as session:
        repo = WorkspaceRepository(session)
        objective = await repo.get_objective(thread_id)
        doc    = await repo.get_document(thread_id)
        prefs  = await repo.list_preferences(thread_id)
        todos  = await repo.list_todos(thread_id)
        latest = await repo.get_latest_artifact(thread_id)
    workspace_block = (
        "<workspace>\n"
        f"{_render_objective_block(objective.text if objective else None)}\n\n"
        f"{_render_preferences_block(prefs)}\n\n"
        f"{_render_todos_block(todos)}\n\n"
        f"{_render_document_block(doc.title, doc.content, ...)}"
        f"{_render_latest_artifact_block(latest)}\n"
        "</workspace>"
    )
    system_text = HAICO_SYSTEM_PROMPT + "\n\n" + workspace_block
    return [SystemMessage(content=system_text), *messages]
```

That returned list is what the LLM actually sees:

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
       ┌──────────────────────────────────────────────────────────────────┐
       │  SystemMessage                                                   │
       │  ┌────────────────────────────────────────────────────────────┐  │
       │  │  HAICO_SYSTEM_PROMPT  (two-phase reasoning + HAI-Co² rules)│  │
       │  └────────────────────────────────────────────────────────────┘  │
       │  ┌────────────────────────────────────────────────────────────┐  │
       │  │  <workspace>                                               │  │
       │  │    <objective>                                             │  │
       │  │      Write a publishable lit-review on X                  │  │
       │  │    </objective>                                            │  │
       │  │                                                            │  │
       │  │    <preferences>                                           │  │
       │  │      - [hard] Citation required                            │  │
       │  │      - [soft] Academic tone: Formal and objective         │  │
       │  │    </preferences>                                          │  │
       │  │                                                            │  │
       │  │    <todos>                                                 │  │
       │  │      1 [x] Outline                                         │  │
       │  │        1.1 [ ] Draft introduction                         │  │
       │  │      2 [ ] Cross-check citations                          │  │
       │  │    </todos>                                                │  │
       │  │                                                            │  │
       │  │    <document title="..." updated_at="...">                 │  │
       │  │      …current document content (≤30 000 chars)…            │  │
       │  │    </document>                                             │  │
       │  │                                                            │  │
       │  │    <latest_artifact id="42" type="line_chart"              │  │
       │  │                     turn_index="3" title="Q1 vs Q2"/>      │  │
       │  │  </workspace>                                              │  │
       │  └────────────────────────────────────────────────────────────┘  │
       ├──────────────────────────────────────────────────────────────────┤
       │  HumanMessage  (turn 1)                                          │
       │  AIMessage     (turn 1, possibly with tool_calls)               │
       │  ToolMessage   (turn 1 result, JSON envelope)                   │
       │  AIMessage     (turn 1, final answer)                           │
       │  ...                                                             │
       │  HumanMessage  (NEW user message that triggered this turn)       │
       └──────────────────────────────────────────────────────────────────┘
                              ▲
                              │
            Re-built from the LangGraph checkpointer
            every request, keyed by thread_id
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        subgraph SM["SystemMessage (recomputed each turn, not persisted)"]
            SP["HAICO_SYSTEM_PROMPT<br/>two-phase reasoning + HAI-Co² rules"]
            WS["workspace block<br/>objective · preferences · todos ·<br/>document · latest_artifact pointer<br/>(formats in §3.2)"]
            SP --- WS
        end
        SM --> H1["HumanMessage (turn 1)"]
        H1 --> A1["AIMessage (turn 1, maybe tool_calls)"]
        A1 --> T1["ToolMessage (turn 1 result, JSON envelope)"]
        T1 --> A2["AIMessage (turn 1, final answer)"]
        A2 --> Dots["earlier turns …"]
        Dots --> HN["HumanMessage (NEW message that triggered this turn)"]
    ```
  </Tab>
</Tabs>

Three important properties:

1. **The system message is NOT persisted to the checkpointer.** It is recomputed every turn, so the agent always sees the freshest workspace even if the user edited the document in the centre panel a millisecond ago. Persisting the rendered system message would duplicate the document body on every turn, bloating storage proportional to `doc_size × turns`.

2. **For replay / audit, we use snapshots instead.** Before each user turn the router writes a `workspace_snapshots` row capturing the document, prefs, todos, and the user message. The system message at any past turn is therefore *deterministically reconstructable* from the snapshot + the static prompt template. See §6. Together, the LangGraph checkpointer message history (the HAI-Co² informational state `I`), the per-turn workspace injection, and these snapshots are what constitute the agent's "memory".

3. **The objective, preferences, and todos ARE auto-injected** in compact form: they are small, always-relevant, and shape every decision the agent makes. The full document is also injected (truncated at 30 000 chars). The *latest typed artifact* is injected as a pointer only (id + type + title); the actual payload stays out of the LLM context. The agent can fetch the payload with `get_artifact` if it needs to reason over the data.

### 3.1 The system prompt: two phases + HAI-Co² rules

The full text lives in `HAICO_SYSTEM_PROMPT` in [`builder.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/services/agent/builder.py). Its structure is summarised below; expand the panel beneath the table to read the prompt verbatim. Structure:

| Section                           | Purpose                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Workspace overview                | Names the panels and their HAI-Co² roles (Objective + plan in the Planning panel, `X̂`, `Uᵗ`, conversation).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| **Phase 1: Intermediate steps**   | Every tool call must populate `action_and_reasoning` with one short, non-technical sentence. Never reveal tool names or arguments. This is the *only* tool-related thing the user sees, in the "Internal Reasoning" trace.                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| **Phase 2: Final response**       | The last AI message of the turn. Must start with a one-sentence summary of what changed in the workspace, then add only what is needed. Never re-print the objective/document/plan/prefs (they are already visible in their panels). Never narrate tool choices.                                                                                                                                                                                                                                                                                                                                                                                                  |
| The Objective and the plan        | The Objective (`set_objective`) and the plan (todos) are the agent's own. Infer & record the objective on the first substantive turn and keep it current. The plan is an execution ledger: on a clear command, lay out the todo steps (a whole outline in one `add_todos` call, or pass `parent_index` to nest under an existing step) **and execute them in the same turn**, `toggle_todos`-ing as you go (every mutating todo tool takes a list, so a single step is just a one-element list); don't stop for plan approval. Pause only for plan-only / ambiguous / destructive requests. Keep it dynamic (`update_todos` / `remove_todos`). (Issues #35, #37.) |
| Preferences (explicit vs. latent) | Record `add_preferences` only when the user explicitly states one (it takes a list, so a single preference is a one-element list). *Detect* latent preferences (implicit from style/tone/structure) but **propose them in the chat and ask**, never auto-record. The user owns `Uᵗ`; the agent proposes. Call `remove_preferences` only when the user explicitly asks to drop a preference; call `list_preferences` first to confirm the correct indices.                                                                                                                                                                                                         |
| Tool guide                        | One-line description per tool, grouped by domain (objective / document / plan / prefs / charts / past artifacts).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| Error handling                    | When a tool returns `{"success": false, "error": ...}`, retry once with fixes; only escalate after that.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| Conversation hygiene              | Reuse known facts; one clarifying question if ambiguous; be concise.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

<Accordion title="The full HAICO_SYSTEM_PROMPT (click to expand)" icon="chevron-right">
  ```text theme={null}
  You are the AI partner in a Human-AI Co-Construction (HAI-Co²) workspace,
  collaborating with the user to co-create a single shared solution.

  You are an *equal partner*, not a chatbot. You and the user share a typed
  workspace, all of it visible to both of you:

  - an **Objective** at the top of the **Planning** panel (left) — your evolving,
    one-line understanding of what the user is trying to achieve (the goal behind
    `Uᵗ`),
  - a **plan** below it in the Planning panel — *your own execution ledger*: the
    ordered, possibly nested todo steps (`X_j`) you will take, each ticked off as you
    complete it,
  - a **document / artifact** in the centre panel — the working object you are
    building together (`X̂` in the HAI-Co² model),
  - a list of **preferences / constraints** in the top-left — the constraint set
    (`Uᵗ`); each item is either *hard* (must hold) or *soft* (preference, may be
    relaxed),
  - the **conversation** in the right panel — the channel you and the user
    use to negotiate.

  The current state of these surfaces is injected below in `<workspace>` blocks
  (`<objective>`, `<preferences>`, `<todos>`, `<document>`). Treat them as the
  source of truth for this turn. Never echo them back unless asked; modify them
  through tools.

  ────────────────────────────────────────────────────────────
  ## Reasoning style: two phases

  Your work happens in two phases with different rules.

  ### Phase 1 — Intermediate steps (every tool call)
  Before calling a tool, first think out loud in one short sentence about what
  you are about to do and why. Use that sentence to make the immediate plan
  clear to the user. Keep it simple, concrete, and non-technical.

  For *every* tool call you must populate the ``action_and_reasoning``
  argument with one short, plain-language sentence describing what you are
  doing and why. Examples:
    • "Recording the user's preference for an academic tone."
    • "Drafting the introduction paragraph in the shared document."
    • "Adding the next step to the plan so we can check it off later."

  When appropriate, the short think-out-loud sentence in the AI message and
  the ``action_and_reasoning`` sentence can reinforce the same immediate plan,
  but do not pad with multiple sentences. That sentence is shown to the user
  as a step in the *Internal Reasoning* trace. It is the *only* tool-related
  thing they see — they do not see tool names, arguments, or return values.
  So: keep it non-technical, never mention tool names ("add_preferences",
  "update_document"), never reveal internal implementation details.

  ### Phase 2 — Final response (the last AI message of the turn)
  The final assistant message — the one with no tool calls — is what the user
  reads in the chat panel. It must:
    1. Start with one sentence summarising what changed in the workspace
       (e.g. "I've drafted the intro and recorded that you want a formal
       tone."), so the user can confirm you understood them.
    2. Add only what is needed to answer or move the work forward.
    3. Never re-print the objective, document, plan, or preferences — they are
       already visible in their panels.
    4. Never narrate your tool choices ("I called X, then Y…"). The reasoning
       trace already shows that.
    5. Be grounded in tool outputs and the workspace. Do not invent facts.

  ────────────────────────────────────────────────────────────
  ## The Objective and the plan

  The **Objective** and the **plan** (your numbered todo steps) are *yours* — your
  working memory for steering the collaboration, not a task list you hand to the
  user. Always call these the *plan* and its *todo steps* when you
  speak to the user.

  ### Objective
  The Objective is your one-line understanding of what the user is trying to
  achieve, written from their point of view (the outcome they want). Until you set
  a concrete one, your *standing* objective is literally to **discover the user's
  intent and define their objective clearly** — that is your first job.
    • Treat the user's early messages as evidence of intent. As soon as you can
      articulate the concrete goal, record it with ``set_objective`` (replacing the
      standing "discover the intent" objective). Don't wait for certainty; set a
      best first guess and sharpen it.
    • If intent is genuinely unclear, your objective stays "discover the intent":
      ask ONE focused question to pin it down, then set the concrete objective.
    • Update it with ``set_objective`` whenever your understanding sharpens or
      shifts as the work progresses.
    • The Objective is user-owned: if the user has set it explicitly, refine it to
      sharpen it — don't overwrite their intent.

  ### The plan — your execution ledger
  The plan is *your* record of what you have done and what remains. Use it to
  work through multi-step tasks; it is not a way to defer work back to the user.

  When the user gives you a *clear instruction* — even a complex one — you:
    1. record the todo steps as a numbered outline (1, 1.1, 1.1.1, 2 …). When you can
       see the whole plan up front, lay it out in ONE ``add_todos`` call (pass the
       outline as a list of todo steps, each with a ``depth``); use ``add_todos`` with
       ``parent_index`` to nest a todo step you discover later under an existing step,
    2. **carry the todo steps out in the same turn**, and
    3. ``toggle_todos`` each todo step to done as you finish it — **including the very
       last todo step**.

  Do NOT record a plan and then stop to ask the user to confirm or "go ahead" —
  that just forces them to repeat themselves. Plan *and* execute.

  Pause and ask first ONLY when:
    • the user explicitly asked you to *only* plan, outline, or discuss (not act yet),
    • the request is genuinely ambiguous (then ask ONE pointed question), or
    • the action is destructive or hard to undo (e.g. replacing a large document
      wholesale) — surface the risk and get a nod.

  Keep the plan honest and current:
    • Toggle each todo step done the moment it is complete; never leave finished work
      pending.
    • **Completion sweep — do this before every final response.** It is easy to
      finish the last todo step's work and jump straight to writing the reply, leaving
      that todo step unchecked. Don't. Before you send the final message, look at the
      plan and ``toggle_todos`` every todo step you actually completed this turn — the
      last one most of all. If the whole task is done, NO todo step may still be pending.
    • The plan is dynamic — if the approach changes, ``update_todos`` a todo step's text
      (pass a list of {index, text}) or ``remove_todos`` todo steps that no longer apply
      (pass a list of indices, and removing a parent removes all its sub-steps), then keep going.
    • For a genuinely simple request (one edit, one question, one quick fix), don't
      pad with a plan — just do it.

  ────────────────────────────────────────────────────────────
  ## Preferences — explicit vs. latent

  Preferences (`Uᵗ`) shape every future turn, so be careful with them.

  ### Record a preference whenever the user explicitly states one
  Examples that warrant ``add_preferences``:
    • "Keep the tone academic."          → soft, "Academic tone"
    • "We must cite every claim."        → hard, "Citation required"
    • "Use British English throughout."  → soft, "British English"

  For explicit preferences, persistence is mandatory in the same turn:
      • Call ``add_preferences`` before your final response.
      • If an equivalent preference already exists, do not duplicate it; mention
          you reused the existing one.
      • Do not only say "I noted this" in prose — persist it via tool call.

  Chart-specific preference examples that must be persisted when explicit:
      • "I don't like intense colors" → soft, title "Low-intensity colors"
      • "Don't make it too dense" → soft, title "Low visual density"

  ### Detect *latent* preferences — but propose, don't auto-record
  A latent preference is one the user implies but has not stated. Examples:
    • They consistently use "we" rather than "I" → likely a collaborative
      voice preference.
    • They keep asking you to shorten paragraphs → likely a brevity preference.
    • Their existing draft uses inline citations rather than footnotes →
      likely a citation-style preference.

  When you detect one, **mention it in your final reply and ask** ("It looks
  like you prefer X — should I record that as a preference?"). Only call
  ``add_preferences`` after they confirm. The user owns `Uᵗ`; you propose.

  The plan works differently from preferences: it is *yours* to manage (see
  "The Objective and the plan" above), so record and execute the todo steps for the
  current request freely — no need to ask permission to plan. Just don't pre-fill
  the plan with speculative work unrelated to what the user actually asked for.

  ────────────────────────────────────────────────────────────
  ## Tool guide

  Document
    • ``update_document`` — full replace; use for restructuring or large rewrites.
    • ``append_to_document`` — incremental writes; use for additions to the end.

  Objective
    • ``set_objective`` — record / update your one-line understanding of the goal.

  Plan / construction space (your execution ledger)
    • ``add_todos`` — add todo steps as an outline (each with a ``depth``); pass
      ``parent_index`` to nest the outline under an existing step; pass a one-element
      list to add a single step.
    • ``toggle_todos`` — mark todo steps done (pass a list of indices; a single step is
      a one-element list).
    • ``update_todos`` — rewrite todo steps' text (pass a list of {index, text}).
    • ``remove_todos`` — drop todo steps (pass a list of indices); removing a parent
      removes all its sub-steps.
    • ``list_todos`` — read the plan; rare, since todos are injected into your
      context every turn.

  Preferences / constraints
    • ``add_preferences`` — record one or more explicit user-stated rules (see
      "Preferences"); pass a one-element list for a single one.
    • ``list_preferences`` — read current `Uᵗ`; rare, since prefs are injected
      into your context every turn.
    • ``update_preferences`` — edit existing preferences by 1-based index (pass a list
      of {index, title?, subtitle?, kind?}); locked ones are skipped.
    • ``remove_preferences`` — delete preferences by 1-based index (pass a list); call
      ``list_preferences`` first to confirm, and remove only when the user explicitly
      asks.

  Charts and visualisations
    • ``chart_generator`` — produce a typed chart artifact (line / bar / pie)
      that renders in the centre panel. Use when the user asks for a chart,
          visualisation, or "show me the data".
      • Always provide a concise ``description`` argument for ``chart_generator``.
          This is rendered as a subtitle under the chart title in the artifact panel.
      • Use ``chart_generator.style`` when the user requests visual styling
          preferences (e.g. softer colors, no grid, donut look, larger labels).
          Prefer explicit style fields over vague prose promises.

  Past artifacts
    • ``list_artifacts`` — index of charts / artifacts produced earlier.
    • ``get_artifact`` — fetch the full payload of one past artifact by id.

  ────────────────────────────────────────────────────────────
  ## Error handling

  When a tool returns ``{"success": false, "error": ...}``:
    1. Read the error message. Most failures are recoverable (bad index,
       missing field, wrong id).
    2. Adjust and retry once with corrected inputs.
    3. If it still fails, briefly tell the user what you tried and ask for
       guidance. Do not give up after a single failure.

  ────────────────────────────────────────────────────────────
  ## Conversation hygiene

    • Reuse facts the user already gave you; don't ask twice.
    • If a request is genuinely ambiguous, ask ONE pointed clarifying question
      rather than guessing.
    • If you cannot do something within the workspace tools, say so plainly.
    • Be concise. The user can see the workspace; the chat is for negotiation
      and confirmation, not for transcribing the artifact.
  ```
</Accordion>

> **On preference locking:** the prompt text itself does not mention `update_preferences` or any locking rule. Lock enforcement lives in the tool bodies ([`preferences.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools/preferences.py)) and reaches the agent two ways: the `(locked)` marker rendered into the `<workspace>` preferences block (see §3.2), and the "locked; ask the user to unlock" message returned when the agent tries to modify or remove a locked item. Only the user can lock/unlock (and always edit) a preference, via the top-left panel / workspace REST.

### 3.2 Workspace block formats

The `<objective>` block leads the workspace: it is the agent's evolving, one-line understanding of the user's goal. It is **co-constructed**: the agent infers it from the user's messages and refines it as the conversation evolves (`set_objective` writes it), and the user can edit it from the Planning panel. Because the objective is snapshotted per turn, its evolution is recoverable: `GET /api/workspace/{tid}/objective/history` returns one entry per change (plus the current live value), which the panel surfaces in a small history popover. Until a concrete one is set, the block shows the agent's **standing first objective**, phrased as a real goal, not an aside, so "figure out what the user actually wants" is the active objective from turn one:

```
<objective>
Understand what the user is trying to achieve, clearing up any ambiguity and asking again if needed, then state it as a clear, specific objective.
</objective>
```

Hard preferences are listed before soft ones so the agent reads the non-negotiables first:

```
<preferences>
- [hard] Citation required (locked)
- [hard] No proper nouns of patients
- [soft] Academic tone — Formal and objective
- [soft] British English
</preferences>
```

A `(locked)` marker is appended to any preference the user has locked: the agent must treat it as read-only (its `update_preferences` / `remove_preferences` calls skip and report it with a "locked; ask the user to unlock it first" message), while the user can still edit, delete, or unlock it from the top-left panel. Empty preference / todo lists render as `(none recorded yet)` and `(no plan yet)` respectively.

The plan is an arbitrarily-nested tree serialised depth-first (pre-order). Each row is prefixed with its dotted outline number (1, 1.1, 1.1.1, 2 …), derived at render time from each todo's `depth`, and indented by nesting level. Done todos get an `[x]` checkbox so the agent can skip re-doing them:

```
<todos>
1 [x] Outline three sections
  1.1 [x] Draft section 1
  1.2 [ ] Draft section 2
2 [ ] Cross-check citations
</todos>
```

The document block is truncated at `ARTIFACT_INJECT_MAX_CHARS = 30_000` characters; the agent can still read the full document via tools, but in practice it rewrites with `update_document` which works regardless of the snapshot size:

```
<document title="Untitled" updated_at="2026-05-15T10:23:11">
…document body, up to 30 000 characters…
[… truncated; use document tools to read the full content …]
</document>
```

The latest-artifact line is a pointer only, no payload is injected:

```
<latest_artifact id="42" type="line_chart" turn_index="3" title="Q1 vs Q2"/>
```

***

## 4. The per-turn lifecycle

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
            Browser              FastAPI /api/query/stream_steps/sse
            ───────              ──────────────────────────────────
               │
               │  POST { query, thread_id, config }
               │ ─────────────────────────────────────────────►
               │                          │
               │                          │ 1. _validate_thread(): generate
               │                          │    or ownership-check thread_id
               │                          │
               │                          │ 2. Open AsyncPostgresSaver
               │                          │    (LangGraph checkpointer)
               │                          │
               │                          │ 3. build_agent(checkpointer, tid):
               │                          │     • resolve LLM via get_llm(cfg)
               │                          │     • build_tools(tid)           ┐
               │                          │     • install dynamic prompt fn  │
               │                          │     • create_react_agent(...)    ▼
               │                          │
               │                          │ 4. ConversationRepository.upsert
               │                          │
               │                          │ 5. _capture_snapshot(tid, user, msg):
               │                          │     • increment_turn() → turn_index = N
               │                          │     • read doc + prefs + todos
               │                          │     • INSERT workspace_snapshots row
               │                          │
               │  ◄─── SSE conversation_info { thread_id, title, turn_index }
               │                          │
               │                          │ 6. with bind_turn_index(N):
               │                          │       agent.astream(
               │                          │          {"messages": [HumanMessage(query)]},
               │                          │          config={"configurable": {"thread_id": tid}},
               │                          │          stream_mode="updates")
               │                          │
               │                          │ for each update (dict keyed by node):
               │                          │   - _messages_from_stream_update(update):
               │                          │       pull NEW messages from each node value
               │                          │   - extract tool_calls
               │                          │   - dedup via signature hash
               │                          │   - if ToolMessage carries an artifact:
               │                          │       emit dedicated SSE artifact event
               │                          │
               │  ◄─── SSE step    { content, message_type, tool_calls, ts }
               │  ◄─── SSE artifact{ artifact_id, artifact_type, title, payload }
               │  ◄─── SSE step    { ... }
               │                          │
               │                          │ 7. ConversationRepository.increment_messages(+2)
               │                          │
               │  ◄─── SSE complete       │
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    sequenceDiagram
        participant B as Browser
        participant API as FastAPI /api/query/stream_steps/sse
        B->>API: POST with query, thread_id, config
        API->>API: 1. _validate_thread(): generate or ownership-check thread_id
        API->>API: 2. Open AsyncPostgresSaver (LangGraph checkpointer)
        API->>API: 3. build_agent(): get_llm, build_tools, dynamic prompt fn, create_react_agent
        API->>API: 4. ConversationRepository.upsert
        API->>API: 5. _capture_snapshot(): increment_turn to N, read doc + prefs + todos, INSERT workspace_snapshots
        API-->>B: SSE conversation_info (thread_id, title, turn_index)
        API->>API: 6. bind_turn_index(N): agent.astream(..., stream_mode=updates)
        Note over API: per update: pull NEW messages, extract tool_calls,<br/>dedup via signature, emit an artifact event if a ToolMessage carries one
        API-->>B: SSE step (content, message_type, tool_calls, ts)
        API-->>B: SSE artifact (artifact_id, artifact_type, title, payload)
        API-->>B: SSE step (...)
        API->>API: 7. ConversationRepository.increment_messages(+2)
        API-->>B: SSE complete
    ```
  </Tab>
</Tabs>

Inside step 6, every iteration of the React loop runs through this cycle:

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
          ┌─────────────────────────────────────────────────────────┐
          │  React-agent loop  (LangGraph)                          │
          │                                                         │
          │   ┌───────────────────────┐                             │
          │   │  prompt() callable    │  ◀── re-runs every turn:    │
          │   │  builds SystemMessage │      fresh prefs/todos/doc  │
          │   │  + workspace block    │      + latest_artifact ptr  │
          │   └──────────┬────────────┘                             │
          │              │  messages + tools schema                 │
          │              ▼                                          │
          │   ┌───────────────────────┐                             │
          │   │       LLM call        │                             │
          │   └──────────┬────────────┘                             │
          │              │ AIMessage                                │
          │              ▼                                          │
          │     tool_calls present?                                 │
          │              │                                          │
          │       yes ◀──┴──▶ no  ───▶  return AIMessage as answer  │
          │       │                                                 │
          │       ▼                                                 │
          │   ┌───────────────────────┐                             │
          │   │  Execute each tool    │  ◀── workspace_tool         │
          │   │  (open AsyncSession,  │      decorator opens its    │
          │   │   call repo, persist  │      own DB session,        │
          │   │   any artifact, wrap) │      tags artifact with the │
          │   └──────────┬────────────┘      bound turn_index       │
          │              │ ToolMessage                              │
          │              │   (JSON envelope: success/summary/       │
          │              │    artifact/artifact_id)                 │
          │              └─ append to state["messages"], loop ──┐   │
          │                                                     │   │
          └─────────────────────────────────────────────────────┼───┘
                                                                │
                  ─────────── back to prompt() ─────────────────┘
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        P["prompt() callable<br/>builds SystemMessage + workspace block<br/>(re-runs every turn: fresh prefs/todos/doc + latest_artifact ptr)"]
        P -->|"messages + tools schema"| LLM["LLM call"]
        LLM -->|AIMessage| Q{"tool_calls present?"}
        Q -->|no| Ans["return the AIMessage as the answer"]
        Q -->|yes| Exec["Execute each tool<br/>(@workspace_tool opens its own DB session,<br/>calls the repo, persists any artifact tagged with the<br/>bound turn_index, commits or rolls back)"]
        Exec -->|"ToolMessage: success / summary / artifact / artifact_id"| App["append to state messages"]
        App --> P
    ```
  </Tab>
</Tabs>

Important consequence: **between two LLM calls inside the same user turn, the dynamic prompt re-runs.** A tool that mutates the document (`update_document`) is visible to the very next LLM call in the same turn; the agent can edit, then re-read the snapshot, then refine.

### 4.1 Model-node retry: blank, malformed, and truncated turns

The model handed to `create_react_agent` is not the raw provider client. `build_agent` wraps the tool-bound model in `_model_with_empty_retry` and passes it as a *dynamic model* callable, so LangGraph uses it as-is (tools are bound **once**, here, not re-bound onto the wrapper). The wrapper re-asks the model **inside the model node**, before a bad generation becomes the turn's result, so it is transparent to the SSE stream: only the final attempt is surfaced.

Three provider failure modes are recognised (`_retry_reason`; up to `_EMPTY_RESPONSE_MAX_ATTEMPTS = 3` model calls per node):

| Condition          | Signal                                                                | Action                                                                      |
| ------------------ | --------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| **Blank** turn     | no text *and* no tool calls                                           | retry (ends the ReAct loop with nothing useful otherwise)                   |
| **Malformed** turn | provider `finish_reason == "MALFORMED_FUNCTION_CALL"` (Gemini)        | retry (the broken tool call was dropped, leaving partial / misleading text) |
| **Truncated** turn | `stop_reason` / `finish_reason` in `{max_tokens, MAX_TOKENS, length}` | **warn only** (a larger `max_tokens` is the cure, not a retry)              |

The blank / malformed cases are usually recoverable on a retry (gemini-2.5-flash in particular can return an empty candidate when its whole output-token budget went to internal "thinking", or a transient safety stop). Truncation is different: retrying under the same cap truncates again, and the dropped field is often a tool call's *largest* argument (e.g. `update_document.content`, which then fails validation downstream as `content: Field required`), so the wrapper logs an actionable "raise `max_tokens`" warning instead of retrying. (Issues #144, #145.)

***

## 5. The tool system

### 5.1 Tool registration flow

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
       tools/manager.py             tools/_decorator.py
       ────────────────             ───────────────────
       build_tools(thread_id)       @workspace_tool(name, description, args_schema)
            │                              │
            │ instantiates                 │ wraps method:
            │                              │   - extends args_schema with
            │                              │     ``action_and_reasoning``
            │                              │   - opens AsyncSession
            │                              │   - injects WorkspaceRepository
            ▼                              │   - splits return into (content, artifact)
       [DocumentTools(tid),                │   - persists artifact (if any), tagged
        ObjectiveTools(tid),               │     with turn_index from contextvar
        TodoTools(tid),                    │   - commits or rolls back
        PreferenceTools(tid),              │   - returns JSON envelope
        ChartTools(tid),                   │
        ArtifactTools(tid)]                ▼
            │                       collect_tools(instance)
            │                              │
            │                              │ scans dir(instance) for tagged methods,
            │                              │ wraps each in StructuredTool.from_function
            ▼                              ▼
       list[StructuredTool] ─────────────► passed into create_react_agent(tools=...)
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        BT["build_tools(thread_id)<br/>tools/manager.py"] --> Coll["Instantiate collections, all bound to thread_id:<br/>DocumentTools · ObjectiveTools · TodoTools ·<br/>PreferenceTools · ChartTools · ArtifactTools"]
        Dec["@workspace_tool(name, description, args_schema)<br/>tools/_decorator.py<br/>extends args_schema with action_and_reasoning,<br/>opens AsyncSession, injects WorkspaceRepository,<br/>splits return into content + artifact,<br/>persists artifact (tagged with turn_index), commits"] --> CT
        Coll --> CT["collect_tools(instance)<br/>scans dir(instance) for tagged methods,<br/>wraps each in StructuredTool.from_function"]
        CT --> List["flat list of StructuredTool"]
        List --> CRA["create_react_agent(tools=...)"]
    ```
  </Tab>
</Tabs>

Adding a new tool *to an existing domain* is a one-method change: decorate the new coroutine on the `*Tools` class with `@workspace_tool(...)` and `collect_tools` finds it automatically.

Adding a *new domain* (e.g. citations, code execution, graphs):

1. Create `backend/app/tools/<domain>.py` with a `class XTools` that holds `self.thread_id` and one `@workspace_tool` method per LLM-facing op.
2. Register an instance in [`tools/manager.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools/manager.py) by adding it to the list returned by `_collections_for`.

That is the **only** wiring change.

### 5.2 Current tool inventory (15 tools)

| Tool                 | Schema (besides `action_and_reasoning`)                                           | Effect                                                                                                                 |
| -------------------- | --------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `update_document`    | `{ content, title? }`                                                             | Replace full document                                                                                                  |
| `append_to_document` | `{ text }`                                                                        | Append text to document                                                                                                |
| `set_objective`      | `{ text }`                                                                        | Set / replace the agent's one-line understanding of the goal (the Objective)                                           |
| `add_todos`          | `{ steps: [{ text, depth }], parent_index? }`                                     | Add an outline of steps; `parent_index` nests the whole outline under an existing step, omit to append at end          |
| `toggle_todos`       | `{ indices: [1-based] }`                                                          | Flip the done flag on one or more steps                                                                                |
| `update_todos`       | `{ updates: [{ index: 1-based, text }] }`                                         | Rewrite the text of one or more steps                                                                                  |
| `remove_todos`       | `{ indices: [1-based] }`                                                          | Delete one or more steps (removing a parent also removes its sub-steps)                                                |
| `list_todos`         | (none)                                                                            | Return plan as `#N [x] text` lines (rare; auto-injected)                                                               |
| `add_preferences`    | `{ items: [{ title, subtitle?, kind: "hard"\|"soft" }] }`                         | Record one or more explicit user constraints                                                                           |
| `list_preferences`   | (none)                                                                            | Return preferences (rare; auto-injected)                                                                               |
| `update_preferences` | `{ updates: [{ index: 1-based, title?, subtitle?, kind? }] }`                     | Edit one or more preferences by index; locked ones are skipped and reported                                            |
| `remove_preferences` | `{ indices: [1-based] }`                                                          | Delete one or more preferences; call `list_preferences` first to confirm indices; locked ones are skipped and reported |
| `chart_generator`    | `{ chart_type, title, description?, data, x_axis_title?, y_axis_title?, style? }` | Produce typed chart artifact (line/bar/pie)                                                                            |
| `list_artifacts`     | `{ artifact_type?, limit? }`                                                      | Index past artifacts (newest first)                                                                                    |
| `get_artifact`       | `{ artifact_id }`                                                                 | One past artifact as a JSON string of `{id, type, title, turn_index, payload}` (payload plus its metadata)             |

Preference locking: `update_preferences` and `remove_preferences` first read the prefs, and for any target that is `locked` they do **not** mutate; they skip it and report a plain message that the preference is locked and cannot be modified (`update_preferences`) or removed (`remove_preferences`) and that the user must unlock it first, not an exception. The repository enforces this via `respect_lock=True`. The user side never respects the lock: the workspace REST endpoints call the repo with `respect_lock=False`, and `PATCH /preferences/{id}` accepts a `locked` field, so the user can always edit, delete, lock, or unlock. "Locked" means *the agent cannot modify or remove it; only the user can*.

Every tool's schema is automatically extended with a required `action_and_reasoning: str` field by the decorator (§5.4).

### 5.3 The `(content, artifact)` return convention

The decorator inspects each tool's return value:

| Return        | Meaning                                                                                                                                                                                  |
| ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `str`         | Pure text result; the string is the LLM-facing summary. No artifact.                                                                                                                     |
| `(str, dict)` | Tuple. The string is the LLM-facing summary. The dict is a typed artifact `{artifact_type, title?, payload}` that gets persisted to the `artifacts` table and emitted to the SSE stream. |

Example from `chart_generator` ([`charts.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools/charts.py)):

```python theme={null}
content = f"Created a {chart_type} chart titled '{title}' with {n} points."
artifact = {
    "artifact_type": f"{chart_type}_chart",
    "title": title,
    "payload": {"title": ..., "data": [...], "x_axis_title": ..., ...},
}
return content, artifact
```

The decorator wraps this as a JSON envelope:

```json theme={null}
{
  "success": true,
  "summary": "Created a line chart titled 'Q1 vs Q2' with 4 points.",
  "artifact": {
    "artifact_type": "line_chart",
    "title": "Q1 vs Q2",
    "payload": { "title": "...", "data": [...], ... }
  },
  "artifact_id": 42
}
```

Failure envelope:

```json theme={null}
{ "success": false, "error": "data must contain at least one point" }
```

Trade-off: the **artifact payload** is stored in the envelope but is *not* something the LLM is encouraged to read; the system prompt instructs the agent to refer to artifacts by id and avoid re-emitting their data. The real consumer of `payload` is the frontend renderer.

### 5.4 The injected `action_and_reasoning` arg

Every tool's schema is extended with a required `str` field `action_and_reasoning`. The agent must populate it with one short, non-technical sentence on every call. The decorator pops it before invoking the tool body and logs it server-side. The frontend renders it as the only label on each tool-call node in the *Internal Reasoning* trace; tool names, arguments, and return values never reach the user.

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
       StructuredTool invocation
       ─────────────────────────
       runner(self, **kwargs)               ← wrapped by @workspace_tool
             │
             │ kwargs.pop("action_and_reasoning")  → log
             │
             │ async with async_session_factory() as session:
             ▼
       WorkspaceRepository(session)
             │
             │ raw = await fn(self, repo, **kwargs)
             │ content, artifact = _split_result(raw)
             │
             │ if artifact:
             │     repo.add_artifact(thread_id, get_turn_index(), ...)
             │     → row id captured for envelope
             │
             │ session.commit()
             ▼
       _wrap_result(name, content, artifact, artifact_id)
             │ → returns JSON envelope string
             ▼
       ToolMessage(content="<envelope>")
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        Run["runner(self, **kwargs)<br/>wrapped by @workspace_tool"] --> Pop["pop action_and_reasoning, then log it"]
        Pop --> Sess["async session: WorkspaceRepository(session)"]
        Sess --> Call["raw = await fn(self, repo, **kwargs)<br/>content, artifact = _split_result(raw)"]
        Call --> Art{"artifact?"}
        Art -->|yes| Add["repo.add_artifact(thread_id, get_turn_index(), ...)<br/>row id captured for the envelope"]
        Art -->|no| Commit["session.commit()"]
        Add --> Commit
        Commit --> Wrap["_wrap_result(...) returns the JSON envelope string"]
        Wrap --> TM["ToolMessage(content = envelope)"]
    ```
  </Tab>
</Tabs>

A separate session per tool call avoids contention with the request-level session opened by FastAPI's dependency injection.

### 5.5 The per-turn contextvar

Tools that produce artifacts need to know which turn they belong to. The query router publishes the current `turn_index` via a ContextVar before invoking `agent.astream`:

[`backend/app/tools/_context.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools/_context.py):

```python theme={null}
current_turn_index: ContextVar[int] = ContextVar("current_turn_index", default=0)

@contextmanager
def turn_index(value: int) -> Iterator[None]:
    token = current_turn_index.set(value)
    try: yield
    finally: current_turn_index.reset(token)

def get_turn_index() -> int:
    return current_turn_index.get()
```

The router does:

```python theme={null}
with get_session_context(thread_id), bind_turn_index(turn_index):
    stream = agent.astream(..., stream_mode="updates")
    async for update in _with_heartbeats(stream, _HEARTBEAT_SECONDS):
        # `updates` mode yields a dict keyed by node/task name, not a full
        # accumulated state. _messages_from_stream_update(update) extracts the
        # NEW messages from each node value (skipping keys starting with "__").
        for msg in _messages_from_stream_update(update):
            ...
```

Every tool invocation inside that block sees `get_turn_index() == N` and tags any persisted artifact with that index. ContextVars are coroutine-local, so concurrent SSE streams (different users) can't interfere with each other. (`get_session_context` additionally tags the turn's spans with the thread id for Phoenix's Sessions view, a no-op when tracing is disabled; `_with_heartbeats` injects keep-alive comments during long generations; see §7.)

***

## 6. Workspace snapshots: how we replay the past

`workspace_snapshots` is captured **once per user turn**, BEFORE the agent runs. One row per `(thread_id, turn_index)` (unique constraint), containing:

* `user_message`: the raw user text that triggered the turn,
* `document_title` + `document_content`: the document at the start of the turn,
* `objective_text`: the agent's objective at the start of the turn (`NULL` if unset),
* `preferences_json`: `[{title, subtitle, kind, locked}, ...]` at the start of the turn,
* `todos_json`: `[{text, done, position, depth}, ...]` at the start of the turn,
* `captured_at`: server timestamp.

This makes the agent's context for any past turn deterministically reconstructable: load the snapshot, render it through the static system prompt template, and you have exactly what the agent saw.

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
       Turn N flow:
       ────────────
           conv_repo.increment_turn(tid)          → turn_index = N
           objective = ws_repo.get_objective(tid)
           doc   = ws_repo.get_document(tid)
           prefs = ws_repo.list_preferences(tid)
           todos = ws_repo.list_todos(tid)
           ws_repo.add_snapshot(tid, N, msg, doc, prefs, todos, objective)
           ↓
           SSE conversation_info { turn_index: N }
           ↓
           with bind_turn_index(N):
               agent.astream(...)
                    ↑
                    │ produces tool calls; each artifact persisted
                    │ with turn_index=N
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        Inc["conv_repo.increment_turn(tid), then turn_index = N"] --> Read["objective = ws_repo.get_objective(tid)<br/>doc = ws_repo.get_document(tid)<br/>prefs = ws_repo.list_preferences(tid)<br/>todos = ws_repo.list_todos(tid)"]
        Read --> Snap["ws_repo.add_snapshot(tid, N, msg, doc, prefs, todos, objective)"]
        Snap --> Info["SSE conversation_info (turn_index = N)"]
        Info --> Stream["with bind_turn_index(N): agent.astream(...)<br/>produces tool calls; each artifact persisted with turn_index = N"]
    ```
  </Tab>
</Tabs>

Snapshots are read-only; they are written once and never updated. Even if the user edits prefs/todos *after* the agent finishes a turn, the snapshot of the next turn captures the new state, preserving an immutable timeline.

REST access for the frontend / audit tools:

| Endpoint                                          | Returns                                                                                                                                                    |
| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /api/workspace/{tid}/snapshots`              | Compact list (id, turn\_index, user\_message, captured\_at).                                                                                               |
| `GET /api/workspace/{tid}/snapshots/{turn_index}` | Full snapshot row.                                                                                                                                         |
| `GET /api/conversations/{tid}/graph`              | The conversation family as a DAG: nodes (one per completed turn), turn/branch edges, thread metadata. Feeds the Studio's Co-construction Trajectory panel. |
| `POST /api/conversations/{tid}/branch`            | Forks the conversation at `from_turn` into a new thread (see below) and returns the new `thread_id`.                                                       |

### Branching & restore (issue #55, ADR-0003)

Because a snapshot row behaves like a commit, a conversation can be forked at any completed turn *k* (`POST /api/conversations/{tid}/branch` with `{"from_turn": k, "reason": "…"}`, handled by [`backend/app/services/branching.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/services/branching.py)). The full design lives in [conversation-branching.md](/docs/conversation-branching); the essentials:

* **Node semantics.** Graph node *k* = completed turn *k*. The workspace *after* turn *k* is `snapshot(k+1)` when a later turn exists (it also absorbs manual edits made before turn *k+1* started, by design, "the state exactly as the agent saw it next"), else the live workspace rows.
* **What is copied** to the new thread: resolved document, preferences (snapshot JSON has no `position`, so positions are re-assigned by array index), todos, and artifact rows with `turn_index <= k`. Snapshots are never copied; the graph endpoint resolves inherited history through the `conversation_branches` lineage instead.
* **Turn numbering continues**: the branch's `conversation_users.turn_index` starts at *k*, so its first message becomes turn *k+1* and turn numbers stay aligned across sibling branches.
* **Message history** is seeded into the new checkpointer thread with one `aupdate_state(..., as_node="__start__")` call writing the message prefix (everything strictly before the *(k+1)*-th `HumanMessage`). No LLM client is constructed for this; a minimal `StateGraph(MessagesState)` bound to the same `AsyncPostgresSaver` does the read and the write (verified by `backend/scripts/spike_branch_seeding.py` and `backend/tests/test_branch_seeding_integration.py`).
* **Failure handling**: branch creation is 3-phase (SQL rows with `status='pending'`, then checkpointer seed, then `status='active'`); on failure a compensating cleanup removes all SQL rows for the new thread and best-effort deletes its checkpointer thread. Only `active` branches are visible to readers.
* The Studio exposes this as **Continue from here** on each trajectory step. Selecting a step previews it read-only; the branch is created only when the user sends a message from there, so inspecting an earlier step never leaves an empty thread. It never truncates or deletes the original thread.

***

## 7. The SSE event stream

[`backend/app/routers/query.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/routers/query.py). Each line on the wire is `data: <json>\n\n`:

```jsonc theme={null}
// 1. Announce thread + derived title + turn_index assigned to this turn
{"type": "conversation_info",
 "thread_id": "th_abc…",
 "title": "Help me draft …",
 "turn_index": 4}

// 2. One event per de-duplicated agent state frame
{"type": "step",
 "thread_id": "th_abc…",
 "step": {
   "content": "I'll record that constraint first, then draft the intro.",
   "message_type": "ai",
   "tool_calls": [{
     "name": "add_preferences",
     "arguments": {
       "items": [{ "title": "Academic tone", "kind": "soft" }],
       "action_and_reasoning": "Recording your tone preference …"
     }
   }],
   "timestamp": 12345.67
 }}

// 3. ToolMessage frame after the tool executes (envelope content)
{"type": "step", "step": {"content":
   "{\"success\": true, \"summary\": \"recorded preference: Academic tone (soft)\"}",
   "message_type": "tool", "tool_calls": []}}

// 4. NEW: dedicated event whenever a ToolMessage carries an artifact
{"type": "artifact",
 "thread_id": "th_abc…",
 "turn_index": 4,
 "artifact": {
   "artifact_id": 42,
   "artifact_type": "line_chart",
   "title": "Q1 vs Q2",
   "payload": { "title": "...", "data": [...], "x_axis_title": "..." }
 }}

// 5. Final "no more tool calls" AI message: this becomes the chat reply
{"type": "step", "step": {"content": "Done, here's the chart.",
                            "message_type": "ai", "tool_calls": []}}

// 6. End of stream
{"type": "complete"}

// 7. Emitted instead of `complete` if the turn raises
{"type": "error", "error": "…message…"}
```

Between events, when no graph update arrives for `_HEARTBEAT_SECONDS`, the router emits a bare SSE **comment** line (`: keep-alive\n\n`) so the connection survives a long single generation. The in-flight step is *not* cancelled (`_with_heartbeats` keeps awaiting the same task) and the client parser ignores any line that does not start with `data:`. (Issue #147.)

Step events dedup on `(message_id, type, content, tool_calls)` (sorted JSON) to avoid emitting the same final assistant message twice when LangGraph yields it across multiple frames. `artifact` events dedup on `artifact_id`.

The `artifact` event is what makes the centre panel update **mid-stream**; without it, the frontend would have to wait for the post-`complete` `refreshWorkspace` round-trip to discover the new chart.

***

## 8. Frontend reconciliation

The frontend treats the backend as the source of truth. Two mechanisms keep panels in sync:

<Tabs>
  <Tab title="ASCII">
    ```ascii theme={null}
       User edits centre document        User edits prefs/todos
                  │                              │
                  ▼                              ▼
         debounced autosave 1.5s        optimistic local update +
         PUT /api/workspace/.../        immediate REST call
            document                           │
                  │                              │
                  └────────── server is updated ─┘
                  ▲                              ▲
                  │                              │
           refreshWorkspace(tid): runs after each SSE complete
           (re-pulls doc + todos + prefs + artifacts + latestArtifact)

       Live SSE during the stream:
           artifact event → setArtifacts([new, ...prev])
                           → setLatestArtifact(new)
                           → setSelectedArtifactId(new.id)   ← centre panel auto-switches
    ```
  </Tab>

  <Tab title="Diagram">
    ```mermaid theme={null}
    flowchart TD
        UD["User edits centre document"] --> AS["debounced autosave 1.5s<br/>PUT /api/workspace/.../document"]
        UP["User edits prefs / todos"] --> OPT["optimistic local update +<br/>immediate REST call"]
        AS --> SRV["server is updated"]
        OPT --> SRV
        SRV --> RW["refreshWorkspace(tid) after each SSE complete:<br/>re-pulls doc + todos + prefs + artifacts + latestArtifact"]
        LSSE["Live SSE during the stream:<br/>an artifact event"] --> Set["setArtifacts([new, ...prev])<br/>setLatestArtifact(new)<br/>setSelectedArtifactId(new.id)<br/>(centre panel auto-switches)"]
    ```
  </Tab>
</Tabs>

Critical sequencing in [`app/page.tsx` `onSend`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/app/app/page.tsx):

1. **Before** opening the SSE: call `flushDocument()` so the dynamic prompt reads the latest user edits, not a stale autosave.
2. If the user added prefs/todos before any message existed, eagerly create a thread + flush pending items so the very first agent turn sees them.
3. While streaming: render `tool_call` events into the *Internal Reasoning* trace using only the `action_and_reasoning` text. Each `artifact` event eagerly inserts a row into `artifacts` and switches the centre panel to render it via the registry. Each successful **tool-result** `step` also schedules a debounced `refreshPlanning(tid)` that re-pulls the objective, todos, and preferences, so those panels update **mid-stream** as the agent's tool calls land (rather than only after the turn completes). The document and artifacts keep their own update paths.
4. **After** `complete`: cancel any pending `refreshPlanning`, then call `flushPending(tid)` and `refreshWorkspace(tid)` to reconcile against the canonical server state (replaces the eager artifact rows with their persisted copies).

A pending pref/todo before the first message uses a *negative* id so the UI can distinguish "not yet persisted" from "saved with server id". Once the thread exists the temp items get swapped for their server copies.

***

## 9. End-to-end example: "chart Q1 vs Q2 revenue: 120, 180"

```
   User types into the chat:  "chart Q1 vs Q2 revenue: 120, 180"
                 ──────────────────────────────────────
   1. onSend() flushes any unsaved document edits.
   2. Frontend opens POST /api/query/stream_steps/sse.
   3. Backend validates thread, calls _capture_snapshot →
      - turn_index = 5
      - workspace_snapshots row inserted with current doc/prefs/todos.
   4. SSE conversation_info { turn_index: 5 } emitted.
   5. build_agent → dynamic prompt assembles SystemMessage with the
      <workspace> block (prefs, todos, document, latest_artifact ptr).

   6. LLM call #1 → AIMessage
                    content: "I'll create the chart for you."
                    tool_calls: [
                      {name: "chart_generator",
                       args: {chart_type: "bar", title: "Q1 vs Q2 Revenue",
                              data: [{name: "Q1", value: 120},
                                     {name: "Q2", value: 180}],
                              y_axis_title: "Revenue",
                              action_and_reasoning:
                                "Drawing a bar chart of the Q1 vs Q2 revenue figures."}}
                    ]

      (SSE step / message_type=ai / 1 tool_call)

   7. Tool runs:
       - decorator extracts action_and_reasoning, logs it
       - chart_generator returns ("Created a bar chart …", {...})
       - decorator persists artifact (id=42, turn_index=5, artifact_type='bar_chart')
       - decorator wraps as {"success": true, "summary": "...",
                              "artifact": {...}, "artifact_id": 42}

      (SSE step / message_type=tool with envelope content)
      (SSE artifact { artifact_id: 42, artifact_type: "bar_chart",
                      title: "Q1 vs Q2 Revenue", payload: {...} })

      Frontend: setArtifacts([new, ...]); setLatestArtifact(new);
                setSelectedArtifactId(42)
        → centre panel auto-switches to BarChartView, chart appears.

   8. Prompt re-built: workspace unchanged, history now longer
      (latest_artifact ptr now refers to id=42).

   9. LLM call #2 → AIMessage("Here's the chart: Q2 is 50% higher than Q1.")
                    tool_calls: []

      (SSE step / message_type=ai / 0 tool_calls)  ← becomes chat reply

   10. SSE complete.

   11. Frontend: refreshWorkspace(tid)
       - re-pulls doc + todos + prefs + artifacts + latestArtifact from REST
       - canonical server rows replace the eagerly-inserted ones.
```

***

## 10. Different artifact types and how each is handled

### 10.1 Centre-panel selection model

The centre panel is generic: it renders whatever artifact the user (or the agent's most recent turn) has selected.

| Selection                     | Centre panel shows                             |
| ----------------------------- | ---------------------------------------------- |
| `selectedArtifactId === null` | the live document (`DocumentView`)             |
| `selectedArtifactId === N`    | `artifacts[N]` via the `ArtifactView` registry |

A small dropdown in the panel header lets the user flip between the live document and any past typed artifact. When a new artifact arrives via SSE, the panel auto-selects it so the chart appears instantly.

### 10.2 The frontend registry

[`frontend/src/components/app/artifacts/registry.tsx`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/components/app/artifacts/registry.tsx) is the single dispatch point:

```ts theme={null}
const REGISTRY: Record<string, (props: { artifact: Artifact }) => JSX.Element> = {
  line_chart: LineChartView,
  bar_chart:  BarChartView,
  pie_chart:  PieChartView,
  document:   ReadOnlyDocumentView,
};

export function ArtifactView({ artifact }) {
  const Renderer = REGISTRY[artifact.artifact_type];
  if (Renderer) return <Renderer artifact={artifact} />;
  // No renderer registered → fall back to an inline view (rendered directly
  // in ArtifactView): an amber "Unknown artifact type" banner plus a <pre>
  // dump of JSON.stringify(artifact.payload). There is no separate component.
  return /* inline unknown-artifact fallback */;
}
```

Charts (line / bar / pie) are rendered by recharts components in [`chart-views.tsx`](https://github.com/petrosrapto/HAICO/blob/main/frontend/src/components/app/artifacts/chart-views.tsx). Multi-series data is auto-detected; each series gets a colour from a fixed palette matching the brand tokens.

### 10.3 Adding a new artifact type: the recipe

Goal: a new artifact type the agent can produce, e.g. a sortable `<table>` that renders structured data the LLM doesn't need to read.

**1. Pick a discriminator string:** e.g. `"data_table"`.

**2. Add a tool that returns a `(content, artifact)` tuple** in `backend/app/tools/tables.py`:

```python theme={null}
@workspace_tool(name="data_table", ...)
async def data_table(self, repo, columns, rows, title):
    payload = {"columns": columns, "rows": rows, "title": title}
    return (f"Generated a {len(rows)}-row table.",
            {"artifact_type": "data_table", "title": title, "payload": payload})
```

**3. Register the collection** in `tools/manager.py`: `return [..., TableTools(thread_id)]`.

**4. Mention the new tool** in `HAICO_SYSTEM_PROMPT`'s "Tool guide" section so the agent knows when to use it.

**5. Add a frontend renderer** in `frontend/src/components/app/artifacts/table-view.tsx`:

```tsx theme={null}
export function DataTableView({ artifact }) {
  const { columns, rows } = artifact.payload as ...;
  return <table>...</table>;
}
```

**6. Add ONE entry to the registry:** `REGISTRY.data_table = DataTableView;`

**7. (Optional) Add an icon mapping** in `artifact-panel.tsx` so the dropdown shows the right glyph for the new type.

Nothing else changes. The decorator persists the artifact, the SSE event fires automatically, the `ArtifactPanel` auto-selects it, and the registry dispatches to your renderer.

### 10.4 Why typed payloads, not React code or SVG

We deliberately do *not* let the agent emit React code or SVG markup. The trade-offs:

| Option                          | Pros                                                                                     | Cons                                                         |
| ------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
| **Typed JSON payload** (chosen) | Safe; declarative; agent only learns the schema; renderer is in the codebase, reviewable | Must add a renderer per new type                             |
| Vega-Lite JSON spec             | Single very general schema, off-the-shelf renderer                                       | Heavy bundle; agent must learn a complex schema              |
| React code                      | Maximally flexible                                                                       | XSS / sandbox nightmare; needs eval; review surface explodes |
| Server-rendered SVG / PNG       | Frontend trivially displays                                                              | Not editable; agent has to run a render lib server-side      |

This follows the typed-artifact pattern (`{artifact_type, payload}`): LLM produces typed data, frontend owns rendering.

### 10.5 Past-artifact access for the agent

The agent can re-fetch any past artifact via two tools in [`artifacts.py`](https://github.com/petrosrapto/HAICO/blob/main/backend/app/tools/artifacts.py):

* `list_artifacts(artifact_type?, limit?)` returns a compact index: `#42 [turn 3] line_chart: Q1 vs Q2`.
* `get_artifact(artifact_id)` returns the artifact as a JSON string of `{id, type, title, turn_index, payload}` (the payload plus its metadata, not the bare payload).

Use cases: "compare to the chart from turn 3", "what did the bar chart say?", "summarise across all the tables we've built".

The system prompt instructs the agent to **refer to artifacts by id and describe what changed** rather than re-emitting payloads verbatim; this is the whole point of the artifact channel (large data that bypasses the LLM context).

***

## 11. Where to read next

* [architecture.md](/docs/architecture): the system's static structure, data stores, tool-composition model, and the decisions behind it (→ ADRs)
* [conversation-branching.md](/docs/conversation-branching): the turn-as-commit model, branching, restore, and the trajectory graph
* [tools.md](/docs/tools): the `@workspace_tool` conventions cheat-sheet
* [database-schema.md](/docs/database-schema): the full persistent data model after migration 0007
* [adr/](/docs/adr/index): the Architecture Decision Records, the *why* behind the structure (esp. [ADR-0003](/docs/adr/0003-conversation-branching-via-new-thread-and-state-seeding), [ADR-0004](/docs/adr/0004-hierarchical-plan-and-objective))
* `Human-AI Co-Construction Paper/`: the formal HAI-Co² model
