MultiCamAssembly

MultiCamAssembly is a distributed software framework for multi-camera perception and coordination in automated crane-based timber construction. It orchestrates multiple, distributed assembly robots (cambots) with embedded vision that detect geometric features in prefabricated timber elements and transmit observations to a central coordinator for multi-view pose estimation. The fused pose estimate enables vision-guided crane motion during on-site assembly.

Structure

base/ — UDP server, device management, plotting tools, CLI helpers cambots/ — camera utilities, YOLO integration, motor control, and programs overlay_tracker/ — geometry, detections, tracking, rendering, and CLI ssh/ — SSH config (e.g., lights-off profile)

Project Structure

For a full folder tree with short descriptions for every file, see STRUCTURE.md

Key Workflows

Main App Workflow

Power on remote robots and connect to the local network Verify SSH config (e.g. ssh/config_lightsoff) matches the network Run base/programs/App.py

Guided steps:

  Device discovery and selection
  Remote code updates
  Component selection

Model Training Workflow

Create labeled data using Label Studio Update dataset paths in yolo/yolo_train.yaml Set hyperparameters and epochs in yolo/yolo_train.py Train model with yolo/yolo_train.py Copy trained weights to cambots/yolo/models/ Generate TensorRT engines on remote devices Validate inference on remote devices

Pushing and Pulling Workflow

Push code via deploy.bat (Windows) or deploy.sh (Linux) Command: ./deploy.sh lightsoff-j01 [config_file] Pushes cambots/ without deleting remote-only files Pull data via fetch.bat (Windows) or fetch.sh (Linux) Command: ./fetch.sh lightsoff-j01 [-d] Retrieves data from remote cambots/save/ to local save/

Camera Calibration Workflow

Required only if cameras change Measure checkerboard cell size and update square_size Run calibrate_camera.py on remote devices Capture 20 checkerboard images Copy camera_calibration_data.npz to each remote device

Logging Workflow

Enable logging in base/programs/App.py UDP messages logged as JSONL files in logs/ Used for post-processing and algorithm prototyping