Essay · Interfaces
From text to visual: agents should show, not tell
Coding agents now do an hour of work and report it in twelve paragraphs. We think the answer to that is not shorter text. It is a different medium: a place you can walk through, where the agent points at what it means. Vision is our open-source experiment with that idea, built on Claude Code.
- Text was the first interface, not the last. Chat came first because text is what models produce. Models now write programs, and a scene is just a program that draws.
- Structure is lost in prose. A plan, a pipeline or a diff has a shape. A paragraph flattens it, and the reader has to rebuild it in their head.
- Pointing beats describing. "The third check in the grading box" is a sentence. An agent's cursor on that check, with a dotted line to its note, is not.
- The conversation can live in the picture. Click any part to ask about it; the question arrives with that part attached.
The problem: answers got long
A year ago an assistant answered a question. Now an agent works for an hour, changes forty files, runs the tests and tells you about it. The telling is still a chat message. It has headings and bullets, and it is still a wall you skim.
The trouble is not length. It is that most of what an agent explains has a shape: steps that lead into each other, parts inside parts, something that goes round a loop and comes back. Prose is one-dimensional. Every shape has to be flattened into a sequence of sentences, and then unflattened again by the reader.
Why the medium is changing now
Three things changed at once.
Models can write the picture. A 3D scene is a description: these parts, here, labelled so, connected like this, seen in this order. That is structured text, which is exactly what models are good at producing. The model does not draw pixels. It writes the world, and a renderer draws it.
Models can check the picture. An agent can screenshot its own page in a headless browser, look at the image, see that a label is cramped or a part is hidden, and fix it. That loop is what makes generated visuals usable rather than decorative.
Rendering is free. A WebGL page in a panel beside the editor costs nothing to run. There is no service to call and no image model to pay for.
What an answer looks like in Vision
You ask a question in Claude Code. Claude writes a scene, builds it into a single page, captures every stop to check its own work, and puts it in a panel beside the chat. The world is drawn as halftone dots, light or dark.
Nothing plays by itself. You press → and ←, and Claude's mark walks to each part as you go. A stop about the whole rests in the corner; a stop about one thing points at that thing.
Click empty space and everything goes back to rest. Double-click it and a plain message box opens. While Claude is working, the world blurs and its mark turns in the middle. When it's done, the world comes back.
What we learned building it
Most of the work was not rendering. It was deciding what not to do.
- Real objects only when the thing is that object. Our first pass used a 3D printer for "build", a mailbox for "publish" and an alarm clock for "wake up". It looked busy and explained nothing. Now a laptop means you typing, a camera means screenshots, and everything else is drawn as what it is: a file with its real text, a page, a folder.
- Boxes are lazy, metaphors are worse. Seven labelled squares is a slide, not a world. Models are pulled from free libraries (Sketchfab via Objaverse) with their credits, but only when one fits.
- The reader drives. Autoplay was the first thing we removed. People read at different speeds, and they want to go back.
- The agent's words and its pointer are one thing. When the note and the cursor moved separately, it read as two characters. Glued together, it reads as someone showing you.
- Honest status matters. A spinner that keeps going after the agent stopped is worse than no spinner. The panel shows "working" only between your message and the agent's stop signal.
Limits
An experiment, not a product. Vision is slow next to text: a detailed world takes one to three minutes to write, build and check. It works in Claude Code with VS Code (Cursor and Windsurf should work; we haven't tested them), and needs Python, Node and Chrome. The model still sometimes picks a weak layout or a forced object, and a one-line answer should stay one line. We have not measured whether people understand faster this way; that is the next question.
Try it
In Claude Code:
/plugin marketplace add ZekeVoigt/andromeda-vision-public /plugin install vision@andromeda-vision
Then install the panel: download vision-0.3.0.vsix from the
latest release
and run Extensions: Install from VSIX…. Ask for something with a shape:
/vision how does a git rebase work
To make it the default, add a line to your CLAUDE.md saying that answers with
structure should use /vision. It is MIT licensed; models keep their own licenses and
are credited in each world.
Where this goes
We don't think text goes away. We think it becomes the floor rather than the ceiling: the agent picks the medium per answer, the way a good colleague decides between saying something, sketching it, and walking you over to the whiteboard. A yes is a word. A plan is steps. A system is a place.
The interesting part is the last one. Once the answer is a place, the conversation can happen inside it: you point, the agent points back, and the thing you're both talking about stays in view. That is closer to how people explain things to each other than any chat window has been.
If you try Vision, we'd like to hear where it helped and where it got in the way: open an issue.