# Camera Calibration Notes — April 19, 2026

## Panda Camera Calibration — Finally Working

Got it working after printing a new ArUco board and writing a better solver. Non-trivial process.

## Key Lessons

### Pose distribution matters
- The distribution of calibration poses exposed to the solver significantly affects precision
- Some runs were finicky/imprecise depending on which poses were used
- Need good coverage of rotations and translations, not just a few convenient poses

### The deeper issue: end-to-end differentiability of robot building

The camera calibration problem is fundamentally: **where is the robot base in the camera frame?** This is a simple geometric question. The simplest solution: install a marker at a known location on the robot base during manufacturing. But because we don't have end-to-end differentiability of the robot building process, we can't do this. We have to recover the transform post-build, which is inherently error-prone.

This is NOT an argument against camera calibration or toward black-box end-to-end learning. It's a more nuanced point:

**We need differentiability with respect to the robot hardware itself.** If we're designing and building robots (e.g., HANDA, custom 3D-printed arms), we should bake calibration into the design — known marker placements, precise mounting points, calibration-by-construction rather than calibration-by-recovery.

Off-the-shelf robot camera calibration is frustrating because it's solving a problem that shouldn't exist if the robot was designed with calibration in mind.

**+1 for building our own robots** — when you design the robot, you can ensure the base-to-camera transform is known by construction, not recovered post-hoc.

## What Worked
- New ArUco board (freshly printed, flat, good contrast)
- Better PnP solver with ambiguity resolution
- Careful pose distribution covering the workspace
- Multiple calibration runs, selecting the one with lowest reprojection error