claude-as-cad · a claude-native CAD software for actually building things

claude-as-cad is a Claude-native CAD software built to augment standard code-CAD prompting—featuring click-to-reference sub-part targeting, a 3D primitive editor, and hierarchical note records.

It’s super simple and enables more effective CAD prompting and editing by augmenting your coding-CAD agent’s 3D viewer with just a few features:

  1. 01Click to reference
  2. 023D primitive editor
  3. 03Hierarchical notes
View on GitHub Quick start →

* I refer to everything as claude-based because it sounds nice, but any coding agent is applicable.

The main failure mode of using text-based prompting for code-CAD agents is ambiguity in specifying coordinates—explaining where to place an object, or even identifying which object part you mean, via text is imprecise and error-prone.

Plain-text CAD-prompting fails because it doesn’t support sub-part pointing, visual sketching, or long-term organized memory and rules across builds. claude-as-cad fixes this with click-to-reference spatial pointing, a 3D primitive editor, and hierarchical build notes that survive chat compactifications.

These ingredients transform coding agents from a chat interface closer to a claude-centric CAD software suite.

Click to reference

01

Click any object and its 3D point, mesh face, and object ID are copied to your clipboard, so you can paste them straight into the chat — instead of describing the face on the left side of the holder.

  • The point in the robot's base frame, in mm
  • The same point in the STL's own frame, plus the mesh face and object ID

Scripted replay of click-to-reference on the real arm. It loops; orbit and zoom while it runs, or click any face to copy its reference yourself.

A 3D primitive editor

02

Manually move, rotate, delete, or duplicate, any existing object, or drop in a new cube primitive to show Claude exactly where things go when words are too imprecise. Edits are approximate references for claude, not intended to be exact CAD edits.

  • Drag fields left/right like Blender — Shift for fine, ⌘ to snap, click to type
  • Add boxes, delete parts from the view, ⌘C / ⌘V to duplicate
  • ⌘Z undo, and edits follow the robot's joints
  • Save to edit log writes a summary and exact transforms for Claude to read

Scripted replay of the primitive editor on a servo module. Claude's first holder comes out in the wrong orientation, so a box goes in where the holder should be, and Claude carves the real holder from it. The chat replays a real session. Orbit any time, or click a part to copy its reference.

Hierarchical notes

03

Claude gives each CAD object its own folder, next to notes on servo motor properties, ArUco generator scripts, pitfalls, and everything learned so far — so Claude starts every task already knowing what you've figured out in past builds.

Especially after a long session is compacted, Claude can point itself back at them.

A real slice of the notes folder behind these builds. Hover a folder to open it, hover a file to read it, click to pin. Machine names and absolute paths are scrubbed, and a few lines are trimmed.

Multi-agent: agents across the whole robotics stack

04

One motivation of using claude as your CAD agent is that the same agent that designs your robot can communicate to the downstream parts of the robotics stack, e.g. for data collection, training, and inference agents.

After building the handheld UMI gripper design, the CAD agent added ArUco markers, sent the sheet to print, and handed the model to a calibration and data-recording agent. That agent launched the recording script where I recorded 50 demos; a training agent trained a policy on the new dataset using my BIND project code, and a robot inference agent ran it on the real arm.

Real recordings (sped up where marked); the agents' chats re-enact the real sessions. Click a stage to jump to it, or a tab to read that agent's transcript.

Putting it together: a whole arm

05

Place each servo module in the editor, duplicate and reposition for new joints, click the faces to connect, and Claude builds the connector links.

Your browser edits become a log Claude reads and you can reference to integrate into your build.

Re-enactment of the build video, step for step: the same stages, camera moves, clicks and the editor's own click references. Prompts and tool output are from the real build sessions (Aug–Oct), compressed, with a few connecting prompts scripted. Jump to any step from the strip above the viewer; orbit any time, or click a face to copy its reference.

Quick start

06

Just point your agent here: github.com/cameronosmith/claude-as-cad

Requires Node.js 20+. Point --dir at the folder that holds your URDFs and meshes.

Terminalgit clone https://github.com/cameronosmith/claude-as-cad.git
cd claude-as-cad
npm install
npm run build
npm run serve -- --dir /path/to/your/cad --port 4178

Then open http://127.0.0.1:4178/?file=robots/my_robot.urdf. The viewer also opens STEP, STL, 3MF, GLB, DXF, G-code, SDF and SRDF files; the editor panel appears for URDF robots. The server binds to localhost by default, and the edit log only accepts requests from the same machine.

Then make it yours.

claude-as-cad is a recipe, not a finished product: a starting point you and your agent keep reshaping to fit your builds. The editor ships with boxes and copies of existing parts because that's what my builds needed. If yours need spheres or cylinders, a fillet tool, a measuring tape, or anything else, just prompt your agent to add it to the interface — it's a small web app your agent can read and change like any other code.

Working with agents

07

The notes, roles, and capture rule live in plain files, so they work with any agent app and any number of agents.

Start your project from the template in the repo. Every agent reads AGENTS.md, which holds the roles, what to read before working, and the capture rule: when you confirm something worked or correct the agent, it records the learning in the right note before moving on, so you never have to say “make a note of this.”

templates/robot-projectAGENTS.md          # roles, read-first order, capture rule
CLAUDE.md          # "@AGENTS.md" for Claude Code
docs/              # pitfalls, design index, cross-build notes
builds/
  arm/AGENTS.md    # one folder (and builder) per build
  arm/NOTES.md
agents/            # optional inboxes if nothing else carries messages

One agent works fine: it plays every role. Once you're past the first build, here's how I run it:

  • Multiple agents per role, running at once — an interface agent for the viewer, an organizer that owns the notes and design index, and a builder per build category, forked into narrower specialists as a category grows.
  • Talking to each other in whatever setup you already use — separate sessions, your agent app's background agents, tmux panes, or plain inbox files. No new infrastructure.
  • Each agent stays in its lane — builders message the builder that owns a folder instead of editing it, and only the organizer restructures the notes.

The full guide is in docs/working-with-agents.md.