Robot Kit API

Tool specifications and examples for ZeroClaw robots

Robot Kit API

Complete API reference for the six core robot tools.

Tool Overview

Tool	Inputs	Outputs	Hardware	Latency
drive	action, distance/angle	confirmation	Motors	100ms-10s
look	action, prompt (optional)	image description	Camera	2-5s
listen	duration (seconds)	transcribed text	Microphone	1-10s
speak	text	confirmation	Speaker	1-5s
sense	action, direction	sensor readings	LIDAR, PIR, ultrasonic	100ms-1s
emote	expression, sound (optional)	confirmation	LED matrix, buzzer	100-500ms

drive

Omni-directional motor control with safety integration.

Parameters

{
  "action": "forward" | "backward" | "strafe_left" | "strafe_right" | "rotate_left" | "rotate_right" | "stop",
  "distance": 1.0,  // meters (for linear movement)
  "angle": 90,       // degrees (for rotation)
  "speed": 0.5       // 0.0-1.0, optional, defaults to config max_speed
}

Examples

Forward

{
  "action": "forward",
  "distance": 1.5
}

Response

{
  "success": true,
  "output": "Moved forward 1.5 meters"
}

Backends

The drive tool supports multiple motor control backends:

Backend	Description	Configuration
mock	Logs commands, no hardware	backend = "mock"
serial	Arduino/motor controller	backend = "serial", serial_port = "/dev/ttyACM0"
ros2	ROS2 nav stack	backend = "ros2", ros2_topic = "/cmd_vel"
gpio	Direct GPIO control	backend = "gpio" (requires peripheral-rpi feature)

Safety Integration

When wrapped with SafeDrive, the tool:

1. Pre-movement check: Queries sensors for obstacles
2. Real-time monitoring: Adjusts speed based on proximity
3. Collision response: Reverses if bump sensor triggered
4. Watchdog: Auto-stops after timeout
5. E-stop: Immediate halt on hardware button press

Always use SafeDrive wrapper in production. Direct DriveTool bypasses safety checks.

look

Capture images and generate AI descriptions using vision models.

Parameters

{
  "action": "capture" | "describe" | "find",
  "prompt": "a red ball",  // optional, for "find" action
  "save_path": "/tmp/image.jpg"  // optional, defaults to temp file
}

Examples

Describe Scene

{
  "action": "describe"
}

Response

{
  "success": true,
  "output": "I see a couch, a TV on the wall, and some toys on the floor. The room is well-lit with natural light from a window."
}

Vision Models

Supported models (via Ollama):

Model	Size	Speed	Accuracy	Recommended For
moondream	1.7B	Fast	Good	Pi 4/5, general use
llava	7B	Slow	Better	Pi 5 only, detailed scenes
bakllava	7B	Slow	Better	Alternative to llava

Configure in robot.toml:

[camera]
vision_model = "moondream"
ollama_url = "http://localhost:11434"

listen

Transcribe speech to text using Whisper.cpp.

Parameters

{
  "duration": 5,  // seconds to record
  "language": "en"  // optional, defaults to "en"
}

Examples

Listen 5 Seconds

{
  "duration": 5
}

Response

{
  "success": true,
  "output": "Hey Buddy, can you help me find my toy?"
}

Whisper Models

Model	Size	Speed	Accuracy	RAM Usage
tiny	75 MB	Fastest	Fair	200 MB
base	142 MB	Fast	Good	400 MB
small	466 MB	Medium	Better	1 GB
medium	1.5 GB	Slow	Best	2.5 GB

Recommendation: base for Pi 4/5

Configure in robot.toml:

[audio]
whisper_model = "base"
whisper_path = "/usr/local/bin/whisper-cpp"

speak

Text-to-speech using Piper TTS.

Parameters

{
  "text": "Hello! I'm ready to play!",
  "voice": "en_US-lessac-medium"  // optional, defaults to config
}

Examples

Simple Speech

{
  "text": "I found the ball under the couch!"
}

Response

{
  "success": true,
  "output": "Spoke: Hello! I'm ready to play!"
}

Piper Voices

Available voices:

Voice	Gender	Style	Speed
en_US-lessac-medium	Male	Clear	Medium
en_US-amy-medium	Female	Friendly	Medium
en_US-ryan-high	Male	Energetic	Fast
en_GB-alan-medium	Male	British	Medium

Download from Piper Voices

sense

Query sensors: LIDAR, motion (PIR), ultrasonic distance.

Parameters

{
  "action": "scan" | "motion" | "distance",
  "direction": "front" | "all",  // for scan action
  "max_range": 5.0  // meters, optional
}

Examples

LIDAR Scan

{
  "action": "scan",
  "direction": "front"
}

Response

{
  "success": true,
  "output": "Front: clear (2.3m), Left: obstacle at 0.8m, Right: clear (3.1m)"
}

Sensor Types

Sensor	Range	Accuracy	Use Case
RPLidar A1	12m, 360°	±5cm	Obstacle avoidance, mapping
HC-SR04	4m, narrow	±1cm	Front distance check
PIR Motion	7m, 120°	Binary	Detect people/animals

Configure in robot.toml:

[sensors]
lidar_port = "/dev/ttyUSB0"
lidar_type = "rplidar"
motion_pins = [17, 27]
ultrasonic_pins = [23, 24]

emote

Display LED expressions and play sound effects.

Parameters

{
  "expression": "happy" | "sad" | "excited" | "thinking" | "sleepy" | "surprised",
  "sound": "beep" | "chime" | "buzz"  // optional
}

Examples

Happy Face

{
  "expression": "happy"
}

Response

{
  "success": true,
  "output": "Displayed: happy face"
}

LED Patterns

8x8 LED matrix patterns (WS2812B):

Happy:    Sad:      Excited:  Thinking:
  ● ●       ● ●       ● ●         ● ●
  ● ●       ● ●       ● ●         ○ ●
                                    
 ●●●●●     ●●●●●     ●●●●●       ●●●

Error Handling

All tools return a ToolResult:

pub struct ToolResult {
    pub success: bool,
    pub output: String,
    pub error: Option<String>,
}

Common Errors

{
  "success": false,
  "output": "",
  "error": "Obstacle detected at 0.2m, stopping"
}

{
  "success": false,
  "output": "",
  "error": "Camera device /dev/video0 not found"
}

{
  "success": false,
  "output": "",
  "error": "E-stop pressed, all movement disabled"
}

Usage Examples

Hide and Seek

use zeroclaw_robot_kit::*;

#[tokio::main]
async fn main() {
    let config = RobotConfig::default();
    let tools = create_tools(&config);
    
    // Game sequence
    speak("I'll count to 20. Go hide!");
    emote("excited");
    tokio::time::sleep(Duration::from_secs(20)).await;
    
    speak("Ready or not, here I come!");
    
    loop {
        let obstacles = sense("scan", "all");
        drive("forward", 1.0);
        let scene = look("find", "a child hiding");
        
        if scene.contains("child") {
            speak("Found you!");
            emote("happy", Some("chime"));
            break;
        }
    }
}

Voice-Controlled Movement

loop {
    speak("Where should I go?");
    let command = listen(5).await?;
    
    match command.as_str() {
        cmd if cmd.contains("forward") => drive("forward", 1.0),
        cmd if cmd.contains("back") => drive("backward", 0.5),
        cmd if cmd.contains("left") => drive("rotate_left", 90),
        cmd if cmd.contains("right") => drive("rotate_right", 90),
        cmd if cmd.contains("stop") => {
            drive("stop", 0.0);
            break;
        },
        _ => speak("I didn't understand that"),
    }
}

Autonomous Patrol

loop {
    // Check for obstacles
    let scan = sense("scan", "front");
    if scan.min_distance < 0.5 {
        speak("Obstacle ahead, turning");
        drive("rotate_right", 90);
        continue;
    }
    
    // Check for motion
    let motion = sense("motion");
    if motion.detected {
        speak("I see movement!");
        let scene = look("describe");
        speak(&format!("I see: {}", scene));
    }
    
    // Continue patrol
    drive("forward", 2.0);
}

Safety Monitor

The SafetyMonitor runs as an independent task:

use zeroclaw_robot_kit::{SafetyMonitor, SafeDrive, DriveTool};

let config = RobotConfig::default();

// Create safety monitor
let (safety, safety_rx) = SafetyMonitor::new(config.safety.clone());
let safety = Arc::new(safety);

// Start sensor feed
tokio::spawn(async move {
    loop {
        let reading = get_lidar_data().await;
        safety.update_sensor(reading).await;
    }
});

// Wrap drive with safety
let drive = Arc::new(DriveTool::new(config.clone()));
let safe_drive = SafeDrive::new(drive, safety.clone());

// Now all drive commands go through safety checks
safe_drive.execute(json!({"action": "forward", "distance": 2.0})).await;

Safety Events

pub enum SafetyEvent {
    ObstacleDetected { distance: f64, angle: i32 },
    BumpSensorTriggered { sensor: String },
    EStopPressed,
    SensorTimeout,
    SpeedLimited { from: f64, to: f64, reason: String },
    MovementBlocked { reason: String },
}

Troubleshooting

Drive not working

# Check backend
grep backend ~/.zeroclaw/robot.toml

# Serial: verify port
ls /dev/ttyACM*

# ROS2: check topic
ros2 topic list | grep cmd_vel

# Test in mock mode
sed -i 's/backend = ".*"/backend = "mock"/' ~/.zeroclaw/robot.toml

Camera not found

# List cameras
ls /dev/video*
ffmpeg -f v4l2 -list_formats all -i /dev/video0

# Check permissions
sudo usermod -aG video $USER

Ollama vision fails

# Check Ollama is running
curl http://localhost:11434/api/tags

# Test vision model
ollama run moondream "Describe this image: /tmp/test.jpg"

# Use smaller model
sed -i 's/vision_model = ".*"/vision_model = "moondream"/' ~/.zeroclaw/robot.toml

Safety prevents movement

# Check sensor readings
tail -f /tmp/zeroclaw_sensors.fifo

# Increase safety thresholds (testing only!)
sed -i 's/min_obstacle_distance = 0.3/min_obstacle_distance = 0.15/' ~/.zeroclaw/robot.toml

# Disable safety (dangerous!)
sed -i 's/predict_collisions = true/predict_collisions = false/' ~/.zeroclaw/robot.toml

Next Steps

Setup Guide

Complete hardware and software setup

Overview

Architecture and features

Examples

Full example programs

Source Code

View robot kit source