Skip to main content
HAI-Co² is artifact-centric: the thing the human and the agent build together is an artifact in the centre panel, and most of the system exists to produce, version, and render it. The panel shows one of two kinds of artifact at a time:
  • the live document (the default X̂ working object): free-form text the agent writes with update_document / append_to_document and the user edits directly. It lives in the documents table, one row per thread.
  • a typed artifact: structured, renderable data a tool emits (a chart today, a table or diagram tomorrow). Each one is an immutable row in the artifacts table, tagged with the turn that produced it, and rendered by a dedicated frontend component.
When a tool produces a typed artifact mid-turn, the centre panel auto-switches to render it; selecting the document again returns to the live working object. This page explains what typed artifacts are, why they are designed this way, and how one flows from the agent to the screen.

​
The typed-artifact pattern

A tool never emits markup, SVG, or React. It emits a typed JSON payload plus a short artifact_type discriminator, and the frontend owns the rendering. The pattern is described in charts.py and buys three things:
  • Safety. The agent cannot inject HTML, scripts, or styles, so there is no XSS or sandboxing surface. The renderer is trusted code in the repo, not model output.
  • A small agent surface. The agent learns a compact data schema (for a chart: chart_type, data, a few labels), not a rendering API. That keeps tool descriptions short and the model reliable.
  • Large data that bypasses the context window. An artifact payload can be large. It is persisted and streamed to the browser, but it is not injected back into the agent’s context. The agent refers to past artifacts by id (see What the agent sees), so a 500-row dataset never costs context tokens.

​
Anatomy of an artifact

A tool body signals an artifact by returning a (content, artifact) tuple instead of a plain string. The content is the short confirmation the agent reads; the artifact dict is everything the frontend needs:
Once persisted, it becomes a row in the artifacts table: A composite index ix_artifacts_thread_turn on (thread_id, turn_index) keeps per-turn lookups cheap. The payload is deliberately schemaless at the database level: a new artifact type needs no migration, only a tool that fills it and a renderer that reads it. The full schema and its place in the data model are in Database schema.

​
Lifecycle: produce, persist, stream, render

1

Produce (a tool)

A tool returns (content, artifact). The chart tool, for example, validates and normalises its data, then returns the confirmation string plus {artifact_type: "line_chart", title, payload}. See the Chart tool.
2

Persist (the decorator)

The @workspace_tool decorator splits the tuple, and if an artifact is present it calls repo.add_artifact(...) tagging the row with the current turn_index (read from a per-turn contextvar set just before the agent runs). The new row id is surfaced in the success envelope as artifact_id, so the agent can reference it later.
3

Stream (the SSE layer)

The query router (query.py) parses each tool message, and when one carries an artifact it emits a dedicated SSE artifact event, separate from the step and complete events. It fires for any artifact_type and is deduplicated by artifact_id, so the panel updates live, mid-turn, before the agent’s final reply.
4

Render (the frontend registry)

The browser dispatches on artifact_type through the REGISTRY map in registry.tsx to a React renderer (the chart views use recharts). Renderers validate the payload defensively, and an unregistered type falls back to a banner plus a pretty-printed JSON dump, so a bad or unknown payload never crashes the panel.
The tool result the agent reads back is a JSON envelope, never the payload:
On the browser side the artifact is typed as:

​
What the agent sees

To keep payloads out of the context window, the agent is given a pointer, not the data. Each turn, the workspace block injects only the latest artifact’s id, type, turn, and title:
When the agent actually needs the data (to compare against, summarise, or revise it) it calls the read-back tools:
  • list_artifacts returns a compact, newest-first index of past artifacts for the thread.
  • get_artifact returns one artifact’s full payload plus metadata, scoped to the current conversation.
Both are documented in Past-artifact tools, and the injection mechanics are in Agent core logic §10.

​
Built-in artifact types

The three chart types share one tool, chart_generator; the type string is built as f"{chart_type}_chart". The document renderer is registered so a document-shaped artifact would display, but no current tool produces one (the live working document lives in the documents table, not the artifacts table). Because artifact_type is a free-form string with no database allow-list, adding a type is purely additive.

​
Artifacts across branches and history

The turn_index tag is what lets artifacts participate in HAI-Co²’s versioning. When a conversation is branched at turn k, every artifact with turn_index <= k is copied onto the new thread (re-inserted with fresh ids, in ascending original-id order so latest-artifact lookups stay correct). Workspace snapshots are deliberately not copied (each one embeds a full document body, so duplicating them would be quadratic in storage); the pre-branch history is reconstructed lazily instead. The trade-off and the copy rules are detailed in Database schema.

​
Add a new artifact type

End to end it is three small edits: a tool that returns the artifact tuple (no extra backend wiring, the decorator and SSE layer handle persistence and streaming for any type), a React renderer, and one line registering it in REGISTRY. The full recipe with code is in Extending HAI-Co² §3.