Autonomous Hexapod Architecture: Dual-Pi Compute, 40 TOPS Vision NPU, and Evaluating Modern Edge LLMs

This hexapod has two kinds of work with very different timing requirements: periodic gait and servo control (updating 20 servo channels on a stable schedule) and asynchronous planning and perception (360° LIDAR SLAM, 40 TOPS neural vision, and language-driven tasks).

This is my working plan for upgrading a Freenove Big Hexapod into an autonomous robot that can run offline. I split gait control from the Pi 5 workloads, then work through sensing, model choice, power, and the order in which I would test the build.

1. Split gait control from the brain

On one SBC, a heavy SLAM loop or local LLM burst can delay gait software. A PCA9685 continues generating PWM after values are written, which stabilizes the pulse train, but the Linux command loop still has best-effort scheduling. Commands need timeouts, bounded rates, a watchdog, and a safe stop on missed updates.

I would isolate those workloads on two boards connected by a hardware UART bus:

┌────────────────────────────────────────────────────────┐
│               TOP: 360° LIDAR (LD19 / C1)               │  ← 2D Point Cloud for SLAM
├────────────────────────────────────────────────────────┤
│           MIDDLE: AI BRAIN BOX (Detachable)            │
│  Raspberry Pi 5 (16GB) ──[PCIe Gen2]── Hailo-10H NPU  │  ← 40 TOPS Vision + LLM Planner
│  Camera Module 3 (CSI) ─────────────── Active Cooler   │
├──────────────────────────┬─────────────────────────────┤
│                          │  UART (115200 8N1 + CRC8)    │
├──────────────────────────┴─────────────────────────────┤
│             BOTTOM: KINEMATIC BODY UNIT                │
│  Pi Zero 2W ──[I2C]── Robot Shield / PCA9685           │  ← Stable PWM; best-effort Linux command loop
│  4S pack → fused disconnect → dedicated regulators     │
│       ├─ rated ~6V high-current BEC → servo bus         │  ← Never connect 4S VBAT directly to MG996R
│       └─ separate regulated 5V rails → Pi 5 / Pi Zero   │
└────────────────────────────────────────────────────────┘

The PCA9685 supplies control pulses, not motor power. The servo bus needs a dedicated regulator or BEC whose input rating covers a fully charged 4S pack and whose output is set within the verified servo-voltage range. Grounds must share a deliberate reference, while high servo current should travel through fused distribution wiring rather than through either Raspberry Pi or the PCA9685 board.

2. Keep the Hailo-10H on vision

In this plan, the Hailo-10H NPU is reserved for vision inference rather than time-sharing with language generation:

• Vision stream: The Camera Module 3 feeds frames via CSI to rpicam-apps. Detection rate and latency must be measured with the exact compiled model, resolution, camera pipeline, and thermal state.
• CPU headroom: NPU offload leaves the Pi 5 CPU available for SLAM localization and language-model inference, though camera handling and orchestration still have CPU overhead.
• 360° mapping: The LD19 LIDAR sits on a 3D-printed 40mm pillar above the leg swing radius and feeds 2D scans to slam-toolbox.

3. Choosing an edge model

Earlier plans referenced models such as Gemma or BitNet. For a 16GB Raspberry Pi 5, I care more about bounded tool calls and usable planning than broad benchmark scores. Published or desktop measurements do not predict performance on this combined Pi, SLAM, and vision workload, so the candidates need to be tested on the assembled stack:

First model to evaluate: Qwen 2.5 (1.5B or 3B)

I would start with Qwen 2.5 1.5B / 3B-Instruct rather than the earlier Gemma checkpoints:

• Tool invocation: Qwen 2.5 was fine-tuned on structured data and tool schemas. I still need to validate syntax errors and unsafe commands at the UART boundary rather than assume perfect JSON.
• Memory headroom: A small Q4_K_M checkpoint may leave more room for SLAM maps, OpenCV buffers, and trajectory history, but peak resident memory must be measured while all services run.
• Response speed: Token rate alone is not the acceptance criterion. Measure end-to-end latency through parsing, validation, dispatch, and gait execution.

4. UART command dispatch and JSON

The Brain unit maps high-level function calls into compact, checksummed JSON packets dispatched over UART (/dev/ttyAMA0 at 115200 baud):

# Tool schema exposed to on-device Qwen 2.5 engine
tools = [
    {
        "name": "navigate_step",
        "description": "Execute directional hexapod locomotion",
        "parameters": {
            "type": "object",
            "properties": {
                "direction": {"type": "string", "enum": ["forward", "backward", "left", "right"]},
                "steps": {"type": "integer", "minimum": 1, "maximum": 50},
                "speed_profile": {"type": "string", "enum": ["creep", "normal", "fast"]}
            },
            "required": ["direction", "steps"]
        }
    },
    {
        "name": "set_posture",
        "description": "Adjust chassis height and body pitch/roll angles",
        "parameters": {
            "type": "object",
            "properties": {
                "height_mm": {"type": "number", "minimum": 40, "maximum": 120},
                "pitch_deg": {"type": "number", "minimum": -20, "maximum": 20},
                "roll_deg": {"type": "number", "minimum": -20, "maximum": 20}
            }
        }
    }
]

# Serial packet transmitted to Pi Zero 2W
# {"cmd": "walk", "dir": "forward", "steps": 10, "spd": 2, "crc": "a4"}

5. Power and thermal design

Running 20 servos alongside two SBCs and an accelerator requires separate, correctly rated power domains. The proposed 4S LiPo is a source rail only: at nominal voltage it is already far above the MG996R’s specified 4.8–6.6V operating range, and a fully charged pack is higher still. Never connect the servos directly to 4S VBAT.

I am not publishing a runtime estimate before measuring current at the battery during standing, gait start, steady walking, turns, blocked-joint faults, and simultaneous compute load. Pack usable energy, regulator efficiency, terrain, servo heating, and low-voltage cutoff all affect runtime. The BEC, wiring, fuses, and connectors must be selected from measured peaks and component datasheets rather than the earlier 38W assumption.

6. Build and validation order

1. Phase 1 — Bench power and actuator safety: Validate regulator output and polarity without servos attached; then test one servo, one fused bank, and finally the full distribution board with current logging, thermal checks, a hardware disconnect, and conservative current limits.
2. Phase 2 — Mechanical calibration: Build the stock chassis, calibrate joint centers, verify joint limits with the robot supported, and test the body controller at reduced speed. Add command expiry, a heartbeat watchdog, and a defined safe posture before untethered walking.
3. Phase 3 — Brain and vision stack: Assemble Pi 5, cooling, camera, and AI HAT+ 2; verify PCIe enumeration and record vision throughput, latency, temperature, and throttling with the exact compiled model and resolution.
4. Phase 4 — Edge-model evaluation: Run each candidate against the same bounded-action suite while SLAM and vision are active. Record end-to-end decision latency, schema failures, unsafe-command rejection, peak memory, and recovery behavior.
5. Phase 5 — SLAM and autonomous loop: Integrate LIDAR on the Brain unit, validate mapping while tethered, then test navigation in a controlled area with an operator-held disconnect. A battery monitor should trigger a tested stop or kneel sequence before the regulator reaches dropout; it is not a substitute for the pack’s protection and cutoff strategy.