Skip to main content

​
HAICO: Feature Catalog

A complete, end-to-end inventory of what the HAICO application does, from the browser UI down to the agent, tools, data model, and operations. HAICO is a cross-domain reference implementation of the Human-AI Co-Construction (HAI-Co²) framework (Dutta et al., 2025). It reframes generative AI as an equal partner that co-edits a persistent, structured workspace with a domain expert, rather than a one-shot autocompletion engine. The “²” marks its dual nature: both the solution (the working artifact) and the objective (the latent utility, encoded as preferences + a plan) are shaped together.
  • Live: haico.gr (production) · dev.haico.gr (development)
  • License: Apache-2.0
  • This document is grounded in the source tree and the GitHub release notes; it reflects the state through release v1.1.0 (production) / dev-v1.1.1 (dev).

​
Table of contents

  1. At a glance
  2. Technology stack
  3. Frontend features
  4. Backend features
  5. Operations & engineering features
  6. Version history (from release notes)

​
1. At a glance

The application is built around a persistent, per-thread shared workspace that both the human and the agent read and write. The agent never edits via opaque text generation; it mutates the workspace through typed tools, and the UI reflects those mutations live as the agent streams.

​
2. Technology stack

Frontend: frontend/
  • Next.js 14 (App Router) + React 18 + TypeScript
  • Tailwind CSS + @tailwindcss/typography, tailwind-merge, clsx
  • framer-motion (animation), lucide-react (icons)
  • @xyflow/react (React Flow, trajectory graph), recharts (charts)
  • react-markdown + remark-gfm (document rendering)
  • @react-oauth/google, react-google-recaptcha-v3 (auth)
  • axios + a thin fetch wrapper; Vitest + Testing Library for tests
Backend: backend/
  • FastAPI + Uvicorn (async), Pydantic v2 / pydantic-settings
  • SQLAlchemy 2.0 async (asyncpg) + Alembic migrations; PostgreSQL
  • LangGraph 1.2 ReAct agent + Postgres checkpointer (langgraph-checkpoint-postgres)
  • LangChain provider integrations (OpenAI, Anthropic, Google, Mistral, Cohere, AWS Bedrock)
  • PyJWT + bcrypt + google-auth (auth); aiosmtplib (email); httpx (CAPTCHA)
  • Arize Phoenix / OpenInference + OpenTelemetry, or LangSmith (tracing)
Infra: Docker Compose per environment, Nginx reverse proxy, GHCR container images, GitHub Actions CI/CD.

​
3. Frontend features

​
3.1 Public site & marketing pages

Reachable without an account. (frontend/src/app/)
  • Landing page (page.tsx): hero (“Build complex artifacts together”), the three core principles (equal partners, joint search, co-constructed objectives), four foundational characteristics, an applications showcase (scholarly writing / related-work generation, scientific visualization / chart generation, cybersecurity / CVE retrieval & triage), an architecture strip, and an “open framework” section (Apache-2.0, public repo, pluggable domains).
  • About page (about/page.tsx): defines HAI-Co² as a third mode between full automation and pure assistance, with a comparison table (vs. full automation, human-in-the-loop, RLHF), ethics-by-partnership, application domains, the academic reference (Dutta et al., 2025), and a collaborators/supervisors grid.
  • Privacy policy (privacy/page.tsx): a GDPR-aligned policy that frames HAI-Co² as a research study (not a commercial product) whose purpose is data collection: the legal basis (Art. 6(1)(a) consent) and data controller; what is collected (account info, co-construction sessions, feedback signals, activity logs); how it’s used (session continuity + scientific research + publication of anonymized data, both required to take part); third-party AI processing (session content is sent to LLM providers, possibly outside the EEA under appropriate safeguards); anonymization before publication under an open licence; a sensitive-data warning; retention windows; the full data-subject rights (access/rectification/erasure/restriction/objection/portability, withdrawal, and complaint to a supervisory authority); an optional email-updates opt-in; and cookie/local-storage disclosure.
  • Login / Register (login/page.tsx): a dual-mode tabbed form:
    • Register: username + email + password (6+ chars), protected by reCAPTCHA v3; success banner instructs the user to check email for verification.
    • Sign in: username or email + password, reCAPTCHA v3.
    • Continue with Google (OAuth) when configured.
    • Inline success/error banners; links to privacy & terms.
  • Email verification (verify-email/page.tsx): auto-verifies from a tokenized link with loading / success / error states, then routes back to login.
  • Profile (profile/page.tsx): shows username, role badge, auth provider, user ID, email; an admin-only link to the console; and a sign-out action.
  • Recording-consent gate (consent-gate.tsx): a one-time modal shown on first Studio entry that enrols the user in the HAI-Co² research study. It explains that HAI-Co² is a research study (not a commercial product), that taking part records their co-construction sessions and may publish their anonymized data for research (both required to participate), and that messages are processed by third-party AI providers; it warns against entering sensitive data. It offers one optional, unticked opt-in: email me about new releases & research updates. Choices are I agree and take part (records the recording + publication consent + the optional e-mail choice) or Decline and sign out. The Studio is blocked until consent is recorded; participation is voluntary.
  • Client-side session (lib/api.ts, which also exports the session helpers): JWT stored in localStorage; authentication, role, and expiry are decoded client-side (no round-trip), and unauthenticated users are redirected to login.

​
3.3 The Co-Construction Studio

The core application at /app (app/app/page.tsx): a four-panel, split-screen workspace with a live SSE chat. Components live in frontend/src/components/app/. Layout & live behavior
  • Four surfaces: Preferences (top-left), Planning = Objective + Plan (lower-left), Document/Artifact (centre), and Chat + Trajectory map (right).
  • Real-time streaming: assistant text types in, reasoning chips animate in sequence, and side panels refresh mid-stream as tool results land.
  • Per-thread workspace: switching conversations swaps the entire workspace.
  • Conversation history sidebar lists past sessions (branches nested under parents).
A. Preferences panel (preferences-panel.tsx)
  • Add via a modal (Title, optional Description, Hard/Soft toggle).
  • Hard (rose “H” badge) = must-hold; Soft (blue “S” badge) = guidance.
  • Per-preference lock (amber padlock): when locked, the agent may not modify or remove it (the user still can). (Release #86.)
  • Inline edit of title/description; click the badge to flip Hard/Soft; X to delete.
  • Drag-and-drop reordering, persisted server-side (debounced). (Release #88.)
B. Planning panel: Objective + Todos
  • Objective (preferences-panel.tsx): the agent’s evolving read of the user’s goal. Inline-editable (Cmd/Ctrl+Enter to save) with a history popover showing how it changed across the conversation. (Evolving objective, release #156.)
  • Todo plan (todo-panel.tsx): the agent’s nestable, dotted-numbered plan (1, 1.1, 1.1.1, 2…):
    • Leaf-only progress bar (doesn’t double-count parents).
    • Toggle done, inline-edit text, indent/outdent to nest, drag-to-reorder, add subtodos, delete; a markdown preview of the full checklist. (Hierarchical planning, release #156; reorder, release #88.)
C. Document / Artifact panel (document-panel.tsx, artifact-panel.tsx)
  • Markdown document editor with Preview/Edit toggle, auto-growing title, autosave (1.5s debounce) with a live save-status indicator (“All changes saved” / “Unsaved changes” / “Saving…”), and a live word count.
  • Artifact switcher dropdown: the live document plus all past typed artifacts (with type icons + turn index). Auto-switches to a freshly generated artifact mid-stream; the user can switch back to any earlier one.
  • Typed artifact rendering via a registry (artifacts/registry.tsx):
    • Charts (chart-views.tsx): line, bar, and pie, rendered with Recharts.
    • Documents (document-view.tsx).
    • New artifact types plug in by registering against an artifact_type discriminator.
D. Chat panel (chat-panel.tsx)
  • Streamed conversation: user messages and assistant replies, with a foldable thinking trace rendered as reasoning chips: internal thought → tool call → tool result (success/error), animated in playback after streaming completes.
  • Typewriter animation on fresh replies; instant render on history load.
  • Per-message actions: retry an assistant reply or edit & resend a user message; each forks a new branch (“fork, never destroy”). (Per-message actions, release #156.)
  • Composer: cycling sample-question placeholder, per-message model selector, Enter-to-send / Shift+Enter for newline, disabled while streaming.
  • Each user message is tagged with its trajectory step label and a “continue from here” affordance.

​
3.4 Conversation trajectory & branching

The co-construction trajectory map above the chat (components/app/graph/, hooks/use-conversation-graph.ts, lib/graph.ts). (Trajectory graph, release #120.)
  • Interactive DAG (React Flow): nodes are user turns, laid out by turn (x) and branch lane (y). The displayed path is drawn in red; off-path branches are muted.
  • Dotted step numbering with path-wide branch ordinals (1, 2, 3.1, 4.1.1…) so labels never collide; empty branches appear as stubs.
  • Node detail card (node-detail-card.tsx): click a node to see its mini transcript (user message, agent summary, reasoning-step count, artifact indicator) and a Continue from here button.
  • Branch edges (lane-edge.tsx) carry the optional fork reason.
  • Continue from here / restore: previews any past step read-only, restoring the full workspace (objective, preferences, plan, document, artifacts) as of that turn from the server snapshot. No branch is created until the user sends a message, at which point a new thread is forked; the original conversation stays intact.
  • An unsaved-edit guard warns before discarding manual edits made in preview mode.
  • Pan/zoom, drag-to-reposition nodes (layout remembered), and a collapse/expand toggle.

​
3.5 Model & provider selection

A per-message model picker in the chat composer (lib/providers.ts), grouped by provider. The lineups are curated and kept current across releases (#96–#111). Groups:
  • Anthropic (Claude Opus / Sonnet / Haiku families)
  • OpenAI (GPT and o-series reasoning models)
  • Google (Gemini + open Gemma)
  • Mistral (Mistral / Ministral / Magistral / Codestral / Devstral)
  • Cohere (Command family)
  • xAI (Grok), via OpenAI-compatible endpoint
  • DeepSeek, via OpenAI-compatible endpoint (refreshed to V4, release #111)
  • Alibaba / Qwen, via OpenAI-compatible endpoint
  • TogetherAI (Llama, Qwen, GPT-OSS, etc.), via OpenAI-compatible endpoint
The frontend sends {api, model_id, endpoint_url?}; the backend resolves credentials per provider. Default selection is a fast Gemini model.

​
3.6 Guided studio tour (onboarding)

A first-run guided walkthrough (studio-tour.tsx, studio-tour-demo.tsx). (Release #127.)
  • A multi-step spotlight tour over the real panels with a scripted demo session (typed sample questions, streamed replies, captured preferences, a drawn chart).
  • Auto-starts once per user (tracked in localStorage) after the consent gate; replayable from the ”?” tour button in the navbar.
  • The background Studio is made inert during the tour (blocks clicks/Tab focus).

​
3.7 Admin console

Role-gated console at /admin (admin/page.tsx; non-admins are redirected).
  • Paginated user table (username, email, auth provider, role, approval status, created date).
  • Click to toggle role (user ↔ admin) and toggle approval (Active ↔ Disabled).
  • Delete a user and bulk-delete selected users (with confirmation).
  • Recent activity log modal: last ~100 audit events (timestamp, user, action, IP, detail).
  • Feedback dashboard at /admin/feedback (admin/feedback/page.tsx): headline counts (likes, dislikes, reports, threads marked complete), the sentiment split per co-construction dimension, a 30-day trend, the report-category breakdown (recharts), and a filterable, paginated reports table. Branch-copied feedback is excluded from every figure.

​
3.8 Shared UI & layout

  • Navbar (layout/navbar.tsx): brand mark, institution logos, nav links (Home / Studio / About), the Studio tour button, and an authenticated user dropdown (Profile, Admin if applicable, Sign out).
  • Footer (layout/footer.tsx).
  • Confirm dialog (ui/confirm-dialog.tsx): styled modal for destructive actions, with optional type-to-confirm phrase, focus trap, and a destructive (red) variant. (Release #83.)
  • Info tip (ui/info-tip.tsx): hover ”?” tooltips that explain fields throughout the UI.
  • Brand components (components/brand/): brand mark, creator card, institution logos.
  • Auth providers wrapper (providers/auth-providers.tsx): Google OAuth + reCAPTCHA v3 context.

​
4. Backend features

FastAPI app (backend/app/main.py) exposing REST + an SSE streaming endpoint, an async PostgreSQL layer, and a LangGraph ReAct agent.

​
4.1 API surface

Routers under backend/app/routers/. All workspace and conversation endpoints enforce thread ownership. Auth: auth.py (/api/auth)
  • POST /register: local sign-up (reCAPTCHA-gated); sends a verification email.
  • POST /login: password login (username or email; reCAPTCHA-gated); requires a verified, non-disabled account; returns a JWT.
  • POST /google-auth: Google OAuth login/auto-registration (ID-token, with a userinfo fallback); links by google_id → email → auto-create.
  • GET /verify-email: verify via tokenized link (24h expiry).
  • POST /resend-verification: resend (enumeration-safe response).
  • GET /me: current user profile.
  • POST /consent: record session-recording consent (idempotent). (Release #87.)
Conversations: conversations.py (/api/conversations)
  • POST /: mint an empty thread; GET /: list (with branch info); DELETE /{id}: soft-delete (preserves checkpointer data).
  • GET /{id}/messages: full history reconstructed from the checkpointer.
  • POST /{id}/branch: fork at a past turn (copies workspace + seeds message prefix).
  • GET /{id}/graph: the conversation family as a DAG (nodes, edges, threads, active path).
Query / agent SSE: query.py (/api/query)
  • POST /stream_steps/sse: stream the agent turn as Server-Sent Events. Captures a workspace snapshot and bumps turn_index before the agent runs; emits conversation_info, step, artifact, complete, and error events with 10s keep-alive heartbeats; accepts per-request model config overrides.
Workspace: workspace.py (/api/workspace/{thread_id}/…)
  • Document: GET/PUT /document.
  • Objective: GET/PUT /objective, GET /objective/history.
  • Todos: list, add, toggle, edit, delete, PATCH /reorder, PATCH /{id}/reindent.
  • Preferences: list, add, edit (incl. locked), delete, PATCH /reorder.
  • Artifacts: list (filterable by type), latest, get by id.
  • Snapshots: list per-turn snapshots and fetch a full snapshot by turn_index.
Feedback: feedback.py (/api/feedback, ownership-checked)
  • PUT /: upsert one feedback row (message reaction/report, a co-construction dimension, or completion); DELETE /: toggle one row off.
  • GET /{thread_id}: all of the caller’s feedback for a thread (frontend hydration).
  • Anchored on (thread_id, turn_index) so feedback lines up with graph nodes and survives branching. See User feedback.
Admin: admin.py (/api/admin, admin-only)
  • List users (paginated), approve/disapprove, change role, delete, bulk-delete, and read the activity log. (Self-targeting guards prevent locking yourself out.)
  • Feedback dashboard data: GET /feedback/stats, GET /feedback/reports, the recent-feedback feed (GET /feedback), and GET /feedback/node (one user’s full submission for a turn); all count/show only genuine (origin='user') rows.
  • Observability + triage: GET /threads/{thread_id}/trace resolves and caches the conversation’s Phoenix trace deep-link; POST /feedback/reports/issue files a GitHub issue from a report with full context (report, trace, transcript), idempotently (the secret GITHUB_TOKEN comes from env, while the repo and default labels live in config.*.yaml under github:).
Health: health.py. GET /health (liveness), GET /ready (readiness).

​
4.2 The agent core (LangGraph ReAct)

backend/app/services/agent/builder.py. Deep dive: agent-core-logic.md.
  • ReAct agent (create_react_agent) with an AsyncPostgres checkpointer keyed by thread_id, so memory persists while the agent itself is rebuilt per request.
  • Dynamically rebuilt system prompt each turn: injects the current workspace as XML blocks: <objective>, <preferences> (grouped, lock-marked), <todos> (numbered outline), <document> (truncated, full body still tool-accessible), and a <latest_artifact> pointer. The prompt is recomputed, never persisted into the checkpointer.
  • Two-phase reasoning: a short plain-language action_and_reasoning on every tool call (surfaced as reasoning chips), then a concise final reply that summarizes workspace changes without narrating tools.
  • Behavioral contract baked into the prompt: set/refine the objective, plan with todos for multi-step work (and sweep them complete), persist explicit preferences (propose latent ones), respect locked preferences, retry-once on tool errors, ask one pointed question on ambiguity.
  • Model-node reliability wrapper: retries up to 3× on blank or malformed generations (e.g. Gemini MALFORMED_FUNCTION_CALL) and detects/logs token-limit truncation, transparent to the SSE stream.
  • Per-turn snapshotting & turn tagging: each turn binds a turn_index via a contextvar so produced artifacts are attributed to the turn.

​
4.3 Agent tools

Composed per-thread by build_tools(thread_id) (tools/manager.py); each *Tools collection is bound to the same thread and flattened into one tool list (≈15 tools). The @workspace_tool decorator (tools/_decorator.py) handles a fresh DB session per call, artifact persistence, and the JSON result envelope. Reference: tools.md.
  • Document (documents.py): update_document (full replace), append_to_document.
  • Objective (objective.py): set_objective.
  • Todos (todos.py): add_todos (batch, nestable), toggle_todos, update_todos, remove_todos (cascades subtree), list_todos.
  • Preferences (preferences.py): add_preferences (batch, hard/soft), update_preferences, remove_preferences (both skip locked items), list_preferences.
  • Charts (charts.py): chart_generator for line/bar/pie charts with optional style controls; emits a typed artifact ({artifact_type, payload}) that the frontend renders (the agent never emits SVG/JSX).
  • Artifacts (artifacts.py): list_artifacts, get_artifact (so the agent can reference/compare earlier outputs).

​
4.4 LLM provider integration

Pluggable factory get_llm(config) (services/llm/).
  • Six native integrations: OpenAI (openai.py), Anthropic (anthropic.py), Google (google.py), Mistral (mistral.py), Cohere (cohere.py), AWS Bedrock (bedrock.py, Converse API).
  • OpenAI-compatible endpoint routing: a single OpenAI client reaches DeepSeek, Alibaba DashScope, TogetherAI, xAI, and vLLM via endpoint_url, with per-endpoint API-key mapping.
  • Gemma compatibility shim: system messages merged into the first human message, tool-calling emulated via JSON-schema prompting/parsing.
  • Provider quirk handling: temperature stripped for reasoning models (o-series) and newer Claude Opus models that reject it.
  • Per-request override → YAML default → hard-coded default resolution; missing credentials surface the provider name in the error rather than crashing.
  • Structured output helper (structured_output.py).

​
4.5 Workspace persistence & data model

PostgreSQL via SQLAlchemy async (db/models.py, repositories in db/repositories/). Full schema: database-schema.md. Alongside these, LangGraph checkpointer tables store the full per-thread message history. Repositories encapsulate all access (UserRepository, ConversationRepository, WorkspaceRepository, BranchRepository, FeedbackRepository); migrations live under backend/alembic/.

​
4.6 Conversation branching service

services/branching.py. Reference: conversation-branching.md, ADR-0003.
  • Three-phase fork: (1) commit a pending branch row + copy workspace/artifacts as-of turn k; (2) seed the parent’s message-history prefix into the new checkpointer thread (aupdate_state); (3) flip the branch to active. On failure, a compensating cleanup removes the half-created thread so it never appears.
  • Graph assembly: builds the conversation family (nodes = turns, edges = turn / fork links), highlights the active path, includes empty-branch stubs, and bounds itself (node cap with truncation flag; ancestor-walk depth cap).
routers/auth.py, routers/deps.py.
  • Local auth: bcrypt-hashed passwords; login blocked until email is verified and while the account is disabled.
  • Google OAuth: ID-token verification with userinfo fallback; account linking and auto-provisioning; Google-verified email auto-marks the account verified.
  • JWT (HS256): claims carry user id, username, role; configurable expiry; dependency helpers get_current_user / require_user / require_approved.
  • Admin role gating on the entire admin router with self-lockout guards.
  • Recording-consent gate: agent access is gated on recording_consent_at. Because HAI-Co² is a research study, agreeing records BOTH recording_consent_at and research_publish_consent_at (recording + publication are required to take part), each idempotently, via POST /api/auth/consent; the optional body carries only the e-mail opt-in (email_updates_opt_in).
  • CORS restricted to configured origins.

​
4.8 Email & CAPTCHA services

  • Email (services/email.py): async SMTP (aiosmtplib) verification emails (plain-text + HTML, 24h token); failures are logged, not fatal (registration still succeeds; user can resend). Skipped if SMTP is unconfigured (dev mode).
  • CAPTCHA (services/captcha.py): reCAPTCHA v3 siteverify with score threshold + action matching on register/login/ google-auth/resend; supports a bypass token for CI; skipped if unconfigured.

​
4.9 Observability & tracing

services/observability.py. (Release #113.)
  • Selectable backends: none | langsmith | phoenix | both.
  • LangSmith via LangChain’s native tracing env vars.
  • Phoenix (self-hosted, Arize) auto-instrumented via OpenInference + OpenTelemetry; spans are tagged with session.id = thread_id so traces group by conversation.
  • Graceful degradation: missing packages / disabled providers are skipped silently.

​
4.10 Configuration

core/config.py: layered config from environment variables → .env → config.yaml → defaults. Covers runtime/logging, DB URL, CORS, JWT, Google OAuth client id, reCAPTCHA, SMTP, the frontend URL, every LLM provider key (including third-party OpenAI-compatible keys and AWS Bedrock), default LLM settings, and the observability provider selection.

​
5. Operations & engineering features

​
5.1 Deployment topology

Per-environment Docker Compose stacks (deployment/): local, dev, prod, each with its own config.*.yaml, env templates, Nginx vhost, and deploy/ cleanup scripts. Production is served at haico.gr, development at dev.haico.gr, behind an Nginx reverse proxy; container images are published to GHCR (haico-backend, haico-frontend).

​
5.2 CI/CD & release model

  • Two-branch, tag-driven model (ADR-0001): develop → dev channel, main → production; promotion fast-forwards develop → main and tags vX.Y.Z.
  • CI (.github/workflows/ci.yml) builds/tests and deploys the dev environment on dev tags.
  • Release Drafter auto-generates dev and prod release notes (release-drafter-dev.yml, release-drafter-prod.yml).
  • Supporting automations: promotion, dev-ready / approved notifications, labeler, stale-issue management, branch naming.

​
5.3 AI-assisted development pathway (Coco)

A provider-agnostic, label-gated AI-assisted development pathway (ADR-0002), built on GitHub Agentic Workflows, that can plan → implement → iterate → review pull requests using Claude, Codex, or Copilot engines (.github/workflows/ai-*.md). It runs alongside the traditional human pathway, selected per issue by label. Documented in ai-pathway.md, automations.md, and github-projects-copilot-automation.md. (Releases #56–#156.)

​
5.4 Testing

  • Backend: pytest suite (backend/tests/) with coverage, including branch-seeding integration tests pinned to exact LangGraph versions.
  • Frontend: Vitest + Testing Library (component and lib tests colocated under frontend/src/).
  • End-to-end: a pytest E2E suite (e2e/) covering auth, conversation, workspace journeys, the SSE stream, and the provider/model parsers.

​
6. Version history (from release notes)

The highest production release is v1.1.0; the development channel is at dev-v1.1.1. For the authoritative, auto-generated changelog see the GitHub Releases page.
Generated from a source-tree and release-notes review. Section anchors above link to the implementing files; the deep-dive docs (architecture.md, agent-core-logic.md, tools.md, database-schema.md, conversation-branching.md) carry the full detail.