/data2/cameron/miniconda3/envs/uva/lib/python3.10/site-packages/sympy/external/gmpy.py:139: UserWarning: gmpy2 version is too old to use (2.0.0 or newer required) gmpy = import_module('gmpy2', min_module_version=_GMPY2_MIN_VERSION, wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information. wandb: Currently logged in as: cameronsmithbusiness. Use `wandb login --relogin` to force relogin wandb: - Waiting for wandb.init()... wandb: \ Waiting for wandb.init()... wandb: Tracking run with wandb version 0.18.3 wandb: Run data is saved locally in /data/cameron/scratch_files/hand_vids/wandb/run-20260417_203736-k1zdbl74 wandb: Run `wandb offline` to turn off syncing. wandb: Syncing run hand_wrist_tracker wandb: ⭐️ View project at https://wandb.ai/cameronsmithbusiness/para_hand_tracking wandb: 🚀 View run at https://wandb.ai/cameronsmithbusiness/para_hand_tracking/runs/k1zdbl74 wandb: - 10.837 MB of 10.837 MB uploaded wandb: wandb: wandb: Run history: wandb: epoch ▁▁▁▂▂▂▂▃▃▃▃▄▄▄▄▅▅▅▅▆▆▆▆▇▇▇▇███ wandb: test/loss ▁▁▁▁▁▂▂▂▃▃▃▃▄▄▅▅▆▇▇▇▆▇▆█▇█▇█▆█ wandb: test/pixel_error ▃▁▁▃▂▄▄█▄▅▂▄▃█▄▅▄▄▅▄▄▄▁▄▃▆▇▄▇▇ wandb: train/loss ██▇▇▇▆▆▅▅▄▄▃▃▃▂▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁ wandb: train/pixel_error █▇▇▇▇▇▆▆▅▅▄▃▂▂▂▂▂▂▁▁▁▁▁▁▁▁▁▁▁▁ wandb: wandb: Run summary: wandb: epoch 30 wandb: test/loss 19.70548 wandb: test/pixel_error 104.46974 wandb: train/loss 0.55122 wandb: train/pixel_error 7.93322 wandb: wandb: 🚀 View run hand_wrist_tracker at: https://wandb.ai/cameronsmithbusiness/para_hand_tracking/runs/k1zdbl74 wandb: ⭐️ View project at: https://wandb.ai/cameronsmithbusiness/para_hand_tracking wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 22 other file(s) wandb: Find logs at: ./wandb/run-20260417_203736-k1zdbl74/logs HandKeypointDataset: 264 samples from 2 videos HandKeypointDataset: 4 samples from 1 videos Loading DINOv2 model... ✓ DINOv2 backbone is trainable ✓ DINO embedding dim: 384 ✓ Learnable start keypoint embedding (dim=384) ✓ Feature convs: 3× Conv2d(3×3) at pred_size=64 ✓ Volume head → (B, 4, 1, 64, 64) ✓ Gripper head → (B, 4, 2, 64, 64) [1×1 conv, CE 2-class] ✓ Rotation head → (B, 4, 3, 32, 64, 64) [1×1 conv, CE] Model params: 32,832,780 Epoch 1/30: train_loss=6.7441 train_px=86.0px | test_loss=6.6446 test_px=81.7px -> Saved best (test_loss=6.6446, test_px=81.7px) Epoch 2/30: train_loss=6.3159 train_px=79.3px | test_loss=6.3914 test_px=69.9px -> Saved best (test_loss=6.3914, test_px=69.9px) Epoch 3/30: train_loss=6.0412 train_px=76.6px | test_loss=6.7461 test_px=68.9px Epoch 4/30: train_loss=5.7994 train_px=77.1px | test_loss=6.6404 test_px=77.7px Epoch 5/30: train_loss=5.4823 train_px=73.9px | test_loss=7.2001 test_px=76.3px Epoch 6/30: train_loss=5.1309 train_px=71.9px | test_loss=7.7883 test_px=86.7px Epoch 7/30: train_loss=4.7152 train_px=68.3px | test_loss=8.2837 test_px=85.9px Epoch 8/30: train_loss=4.3633 train_px=64.2px | test_loss=8.3500 test_px=105.6px Epoch 9/30: train_loss=3.8333 train_px=54.2px | test_loss=9.6208 test_px=83.6px Epoch 10/30: train_loss=3.3573 train_px=47.3px | test_loss=9.6708 test_px=90.1px Epoch 11/30: train_loss=2.8807 train_px=39.9px | test_loss=10.9367 test_px=75.2px Epoch 12/30: train_loss=2.4490 train_px=31.3px | test_loss=11.1435 test_px=84.8px Epoch 13/30: train_loss=2.1433 train_px=24.1px | test_loss=11.8673 test_px=81.0px Epoch 14/30: train_loss=1.9226 train_px=22.8px | test_loss=11.7680 test_px=107.9px Epoch 15/30: train_loss=1.6401 train_px=16.6px | test_loss=14.8021 test_px=83.0px Epoch 16/30: train_loss=1.5069 train_px=15.2px | test_loss=14.1378 test_px=93.9px Epoch 17/30: train_loss=1.3830 train_px=14.1px | test_loss=16.2409 test_px=84.9px Epoch 18/30: train_loss=1.2517 train_px=13.6px | test_loss=17.5599 test_px=87.4px Epoch 19/30: train_loss=1.1577 train_px=11.3px | test_loss=17.2540 test_px=91.0px Epoch 20/30: train_loss=1.1284 train_px=10.3px | test_loss=17.6488 test_px=84.5px Epoch 21/30: train_loss=1.0116 train_px=9.8px | test_loss=16.8487 test_px=87.6px Epoch 22/30: train_loss=0.8965 train_px=9.6px | test_loss=17.2287 test_px=87.4px Epoch 23/30: train_loss=0.9571 train_px=12.6px | test_loss=16.8672 test_px=71.4px Epoch 24/30: train_loss=0.7893 train_px=9.5px | test_loss=19.7347 test_px=86.5px Epoch 25/30: train_loss=0.8424 train_px=11.5px | test_loss=18.0023 test_px=79.1px Epoch 26/30: train_loss=0.6761 train_px=9.2px | test_loss=19.8674 test_px=94.2px Epoch 27/30: train_loss=0.7698 train_px=8.9px | test_loss=18.3141 test_px=99.8px Epoch 28/30: train_loss=0.6872 train_px=9.6px | test_loss=18.9570 test_px=85.7px Epoch 29/30: train_loss=0.6564 train_px=9.0px | test_loss=16.6940 test_px=102.9px Epoch 30/30: train_loss=0.5512 train_px=7.9px | test_loss=19.7055 test_px=104.5px Done! Best test loss: 6.3914 Checkpoints: /data/cameron/scratch_files/hand_vids/checkpoints wandb run: https://wandb.ai/cameronsmithbusiness/para_hand_tracking/runs/k1zdbl74