Brain — AI agent desktop workbench

The problem

The first version rebuilt traces from the shape of chat messages after the fact. With sub-agents running in parallel it broke in specific ways: a parent message overwritten when an agent closed, a running spawn ending early, turn IDs that pointed at nothing, span end times that were wrong. Debugging an agent meant reading logs by hand.

What I changed

• An event contract: 13 event kinds (turns, messages, thinking, tool calls and six sub-agent events) and four end states: completed, failed, cancelled, orphaned. The structure is fixed at the moment it is collected, not guessed afterwards.
• One trace collector that runs beside the agent and never touches UI state. It sends only the observations that changed since the last upload.
• One shared Langfuse project, separated by user ID, instead of a project per user. The controller only authenticates, stamps the owner and forwards.
• A local SQLite outbox on Node's built-in node:sqlite in WAL mode. An upload that succeeds is deleted from it; one that fails is retried with backoff, and a restart picks up where it stopped. It is capped at 500 rows, 7 days and 8 MiB per entry. Observation IDs are stable hashes, so a retry never creates a duplicate.

Details that mattered

• Codex reports token counts as running totals, so each turn is the difference between two totals. Durations are the sum of turns, not the span from first to last.
• Personal views are always filtered to the owner. Reading someone else's trace needs a separate permission; a department admin sees only their department, and anything out of scope returns "not found".
• The bundler mistook node:sqlite for a third-party package, and the queue quietly fell back to memory only. I fixed the import and added a check on the packaged output.

The outbox narrows the window in which data can be lost. It does not promise that nothing is ever lost.

The portal has its own trace tree with personal, department and company views, and the usage dashboard reads the same events. In production.

The decision

The built-in browser was for people only. An agent could not act on the page the user had open, and could not use their logins. My first plan, on August 9, gave the agent a separate page of its own and no password store. Two days later I reversed it: one Browser Host, where the person and the agent share the same pages, tabs and session.

What keeps the person and the agent on the same page is not a component. It is one rule: one tab, one page instance.

How control works

• The model sees one tool, browser_js. It writes JavaScript against a browser facade of 20 actions, generated from a single catalog and shared by all three agent engines.
• Each session gets a short-lived capability and each turn a unique lease. Every tab has a serial command queue with hard cancel, and every request carries turn, page and control epochs, so after a navigation or a takeover a stale request cannot commit.
• Only one side works a tab at a time, and the person always wins. Stop blocks further actions; it does not pretend to undo a form that was already submitted.
• A blocklist beats the allowlist. Cookies, storage, clipboard and script execution are asked per site the first time: allow once, always for this site, or deny, and any of it can be revoked.

Visible and safe

• Before the agent acts, the target glows at its edges, a simulated cursor travels a curve in 400 to 1,200 ms, and short text is typed at 45 ms a character, so the person can follow what is happening.
• Passwords are encrypted with Electron safeStorage and never leave the main process in plain text; the agent needs per-site permission to use one. Passwords, cookies and history can be imported once from Chrome, Edge or Brave.
• The audit record is built so that secrets never enter it: "typed into a password field", not the text redacted later.
• Each session has its own browser instance, up to six kept alive and recycled least-recently-used. Background throttling is off, otherwise an agent working in a background session slows to a crawl.

The move from webview to WebContentsView is in the plan but not done; today it runs on webview, with a CDP proxy for Playwright.

Port first

I ported Open Design, an open-source AI design workbench, pinned at v0.14.2: the studio, the daemon, the shared packages and the assets, with a record of where each one came from. The whole-site iframe became React source wired in directly; iframes stay only for untrusted previews. Online deploy, public sharing and a separate memory were cut.

Then rebuild it

The port was a static asset browser with an export button, sitting next to a chat that did not know it existed. Context going in, results coming out and feedback coming back were all broken. From August 5 I rebuilt it as an extension (design systems, templates, preview, comments, versions, export) plus a plugin that carries the skills, reusing Brain's own projects, sessions, files and terminal.

• I added two capabilities to the extension platform. Preview bridges: an extension declares its own injected script, up to 256 KiB, pinned by SHA-256, at most 8 per extension. Composer drafts: a panel can write a draft into the input box but never send it, up to 8 KiB and 5 times in 10 seconds.
• I took out the design-specific code that had been wired into the core.
• State lives inside the project: one design.md that the agent reads, a state.json with optimistic locking by revision, and version snapshots with a SHA-256 for every file.

When the platform moved on

When the platform changed to its 1.6 API, desktop export did not come with it, so PDF and PPTX export are off for now. Source annotations that can no longer be aligned are dropped rather than guessed.

An element without an annotation just can't be edited by hand. That is far safer than a wrong annotation writing over the source.

It now has 333 design skills and 297 templates in a repository of its own. I also gave a 30-minute, 12-slide talk on it to the team. The conclusion: Open Design is not a new design model. It is a context system that makes a general coding agent follow a design process.

Office and Markdown

• Read-only preview for DOCX, XLSX, PPTX and PDF. Spreadsheets are parsed in a Web Worker and only the cells that exist are rendered; slides load lazily in a window. Files are capped at 20 MiB, and ZIP64, encrypted files, unsafe paths and fake extensions are refused. Selecting text starts an Ask AI that points back to the source.
• A Markdown editor with rich text, source and diff modes, on MDXEditor with a CodeMirror merge view. I fixed edits lost while a save was running, stale content after switching files, and outside changes overwriting open ones.

Windows and releases

• Desktop release management and auto-update across platforms, in May and June.
• A PowerShell guardian swallowed the local server's ready signal. I fixed it while keeping the job object and the exit code, and a half-started app no longer waits out the full 15-second timeout.
• A synchronous version check was blocked by Defender's first scan; it became an existence check, a cache and an asynchronous parse.
• A full disk wrote log text into the packaged package.json; the build now verifies it.
• Paths with Chinese characters: the server builds absolute paths, and the front end no longer joins them itself.

• The team's LLM gateway, in three generations: CLIProxyAPI in March, sub2api at the end of April, New API in June. I did the selection, containers, intranet deployment, keys, quota and usage. It grew from the 8 people on my team to 40+ users, then I handed it over.
• Usage metrics: a Go collector read session logs into PostgreSQL, keyed by session and turn so a second read never counts twice, with dbt models and a Lightdash dashboard on top. Once the gateway could supply 7 of the 9 fields itself, the collector was retired.
• Deployment: Docker Compose for the registry, PostgreSQL, Redis, MinIO and the gateway; kind for the controller and workspaces; Kustomize instead of Helm; init, build-and-push, deploy and smoke scripts.
• Documentation: a MkDocs Material site served by the Go controller itself, later rewritten as 30 pages, plus a skill that answers questions from it.

• Fix a structure where it is created. Traces became reliable once their shape was fixed at collection instead of reconstructed later.
• A rule holds better than a component. The shared browser works because one tab has one page instance, whatever renders it.
• Write down what isn't done. The comparison table in my talk stayed marked "to be filled" rather than carrying numbers I didn't have; the WebContentsView move is still a plan; Windows is still slow under heavy tool use.