Rowve — Co-founder Demo Plan (Marco)

Horizon: ~2 days
Platform: Marco (SO-101 on differential-drive trike, Dora-first)
Goal: Show a live AR overlay demo that proves technical depth, product taste, and direct fit for a farm-weeding robot — while Rowve finishes first-pass co-founder calls.

Context

Shortly after you shipped your first ACT policy on SO-101, you connected on X with another builder who started his own SO-101 project. You encouraged him; you followed each other. He grew a large audience as a marketing-forward extrovert while you were quieter on the build side.

You showed him Marco a couple of months ago. He has since mounted his SO-101 on a tank-like base — very similar to Marco — aimed at weeding on farms. Improving agriculture with robots is a personal fantasy of yours, so you reached out about joining. He said he is looking for co-founders. The project is brand new; this week he is talking to many people.

Marco’s body and SO-101 are unchanged. As of 2026-08-19 the tank treads are gone: two large rear wheels on the same servos plus an unpowered pivoting caster. Same differential kinematics (in-place pivot still works); less slip so wheel odom can support mapping.

You pitched what you can bring:

Dora-first stack instead of ROS (lighter, AI-friendly dataflows — already Marco’s direction)
Phone AR overlays so an operator sees live what the robot plans to do and how it is labeling the world
Concrete weeding effector thinking: lasers, steam, snippers, and pulse electrocution (root damage, low energy) — with the shared conclusion that the winner must balance low energy with quick and effective kill

This README is the execution plan for the next couple of days: a demo strong enough that when he finishes those calls, you are not another “interested engineer” — you are the person who already showed the product vision on hardware.

Demo thesis (what he should walk away believing)

You already have a working Dora robot stack on a differential + SO-101 form factor like his (Marco is now a trike; Rowve’s weeder is tank-like).
You can make the robot’s intent visible and legible to a human via AR — critical for trust in field robotics.
You can wire perception → labeled world → planned motion → operator UI quickly.
Your ideas (AR, Dora, weeding effector tradeoffs, laser-as-proxy for “energy on target”) map directly onto Rowve’s farm use case.

Primary demo (must ship): Phone (or tablet) AR view of Marco planning a trip to a named place (“master bedroom”) and heading that way down the hallway — with the rear MARCO AprilTag nameplate (tag36h11 id 532) for AR pose.

Secondary demo (strongly desired): OAK-D object detection (shoe, chair, etc.) with AR cues (arrow + label from robot toward object; bounding boxes only if angles allow).

Stretch / top-of-funnel wow: Laser-on-target as a stop-and-align kill loop (LiteWeed), not “track while rolling” as v1. Detect a vegetation stand-in → halt → aim → timed beam → re-look. Creep-while-holding-beam is a bonus clip only. Prior art: docs/research/WEEDING_PRIOR_ART.md.

Architecture for the sprint (keep it shippable)

Prefer web AR over a native Android app for day 1–2 speed. You already have AR web-app experience to reuse.

┌─────────────────────┐     Dora graphs      ┌──────────────────────┐
│  Marco (Pi 5 LeBot) │◄────────────────────►│  Phone / laptop web  │
│  drive + odom       │   HTTPS / JSON       │  AR overlay UI       │
│  map + named places │                      │  (Three.js + WASM)   │
│  OAK-D + arbiter    │                      │  AprilTag 36h11      │
│  intent / plan      │                      │  location buttons    │
└─────────────────────┘                      └──────────────────────┘

Operator flow (MVP):

Point phone at the MARCO AprilTag nameplate on the rear plate → AR world locks to robot.
Tap a location button (e.g. Master Bedroom) → robot plans/drives toward that labeled map pose.
AR draws a path / hallway arrow showing intent (“heading to Master Bedroom”).
If OAK-D sees a chair/shoe, AR shows an arrow + label from the robot toward that detection (box optional).

Why buttons first: Named goals beat free-form speech for a reliable 2-day demo. LLM routing can return later.

Progress log (updated 2026-08-21)

AR overlay

Coded on the Orange Pi (2026-08-12), validated live on Marco (2026-08-13) with Tailscale HTTPS, then YOLO arrows + object avoid on hardware (2026-08-15).

Local preview: python3 robots/so101/scripts/serve_ar_local.py → http://127.0.0.1:8787/?demo=1
Next-dev prompt: HANDOFF_MAPPING.md. Locked cals: robots/so101/docs/DRIVE_CAL.md. Rowve narrative prompt: HANDOFF_ROWVE.md.

Live 2026-08-15 — object avoid

DRY_RUN=0 WALL_AVOID_ENABLE=1 MAX_PATH_M=3.00 on Marco. Graph detect=on, motors live. Path ~2.38 m then AVOID_PERSON / AVOID_BACKPACK (couch also in view): creep + yaw away, no ram. Operator: worked great. Sit-still: DRY_RUN=1 WALL_AVOID_ENABLE=0. Do not pkill -f "dora run" over SSH.

Session 2026-08-15 — AR HUD + camera tabs + arrow yaw

HUD is graph / tag / motion / Play / STOP plus AR / OAK-D / Wrist tabs. OAK JPEG from the wall-avoid DepthAI process; wrist JPEG via ffmpeg (not OpenCV V4L). Shoulder OAK points straight forward (camera_yaw_deg=0 in oakd_detect.json, shared with object-avoid). Hard-refresh the phone after graph restart.

Session 2026-08-19 — trike drivetrain (treads off)

Same Marco body + SO-101. Tank treads removed. Two large rear wheels on servos 7 / 8, unpowered front caster that pivots freely. Software still tank_*. Goal: less variable slip so odom is useful for mapping.

Physical wheel radius is essentially the same as the old tread sprocket → start wheel_radius_m=0.052235 (banded LOCKED) and run the same linear 2 m (--distance-m 2.0 --x 0.1875).
Pass 1: odom 2.0010 m, tape 208.2 cm, yaw -0.02°, L/R delta -0.2 mm. New radius 0.054349. No steer trim in code (yaw-hold was the old left-drift compensator).
Pass 2: --no-yaw-hold. Odom 2.0039 m, tape 201.7 cm, yaw +0.79°, body ~5 cm right. New radius 0.054704.
Pass 3: radius 0.054704, --no-yaw-hold. Odom 2.0001 m, tape 201.2 cm, yaw +0.81°. New radius 0.055029.
Pass 4: 360° at carpet 0.304899. Odom +360.12°, body ~495° (overshot 135°, no slip). New track 0.221819.
Pass 5: 360° at 0.221819. Odom +360.01°, body ~363°. New track 0.219992.
Pass 6: 360° nailed at 0.219992. LOCKED.
Pass 7: 20 cm/s. Odom 1.9970 m, tape 199.2 cm. Cruise LOCKED 0.20.
Pass 8: accel 0.32. Odom 1.9941 m, tape 199.8 cm (2 mm shy). Radius + cruise 0.20 + accel 0.32 locked. Track later recailed to 0.22612 (skid 360). Full table: robots/so101/docs/DRIVE_CAL.md. Next: HANDOFF_MAPPING.md.
Carpet profile is still the tread-era lock; do not use it until recaled.
dora-so101 left stopped after the distance script.

Session 2026-08-15 — banded treads LOCKED (before mapping; hardware retired 2026-08-19)

Aqua rubber band on each of 32 treads per side. Hard-floor default is this profile:

Bare-tread (superseded): radius 0.055721, track 0.422672
LOCKED: wheel_radius_m=0.052235, track_width_m=0.441318
Linear: --distance-m 2.0 --x 0.1875 (0.1875 is m/s). Pass 1 tape 188 cm → new radius. Pass 2 odom 2.0039 m, tape exact 200 cm.
Fast 360 skipped left reverse sprockets (~5×); track from 15° short estimate (not a clean spin). Do not repeat that pivot for cal.

Same day — YOLO avoid + Rowve video: heading-balance + object cone at 10 cm/s, then +30% (vx_mps=0.13, vx_creep_mps=0.104). Both worked great; operator captured a video. Speeds left at +30% in wall_avoid.json. MAX_PATH_M=3.00.

Session 2026-08-21 — skid/ICC lock, tag range vs stereo

Wheels: skid track_width_m=0.22612 LOCKED; one-wheel 0.26029; ICC close 1.021 @ 12 cm, 1.050 @ 24 cm (different radii need different scales). Table: docs/DRIVE_CAL.md.

AprilTag size-range is face-on only. Yaw-in-place vs sofa 101: at ~22° bearing, size-range dropped ~36 cm (AABB grew) while stereo-at-tag moved ~10 cm; PnP collapsed to 0.34 m. Face-on, stereo was ~38 cm farther than size-range. Tag-odom loop “30 cm closer” was that AABB lie; operator: robot was back at the start, ~18° CCW overshoot from --skid-extra-deg 8. Occupancy now inflates hits 20 cm and skips stamps when |image_bearing| > 15°.

Session 2026-08-17 — sofa/bathroom marker hallway

Graph tank_marker_hall_local.yml (no YOLO). Sofa 101 at 120 cm → 45 cm right arc → wall-avoid until bathroom 201 → turn at 110 cm, 90° left arc. Distance from tag size in the image (PnP once said 0.47 m across the room). Homing steers to center the tag in the OAK-D, not body-frame bearing_deg. Lost-tag search follows the last-seen side. Runner: DRY_RUN=0 IDLE_STOP=0 ./robots/so101/scripts/run_tank_marker_hall_local.sh.

Session 2026-08-17 — Stage 2 occupancy (wired)

Tag-anchored 2D map beside the hallway graph. First live paint used a tall 16–62% depth band (looked over the sofa; 3k wall cells). Mapping now uses a camera-height slice (46–54%, median per column), stamps only while a room tag is visible and nearly face-on (stamp_max_abs_bearing_deg 15), and inflates hits 20 cm. Node occupancy-mapper. GET /map + phone Map tab.

Done 2026-08-11

Wall avoid findings (2026-08-11): yaw-hold alone drifts ~6 cm/1.5 m. Early sticky side-strip auto flipped / over-rotated into the opposite wall. Fix: compare depth just left vs right of image center (estimate_heading_balance); yaw away from nearer side; CLEAR when open ahead and balanced. Hallway 1.80 m retest corrected L then R correctly. Gains: yaw_imbalance_kp=42, yaw_max_degps=40, front_clear_m=1.45, deadband=0.10. Dora dry-run publishes pose + status over HTTP.

Frozen configs: robots/so101/configs/tank_drive*.json, wall_avoid.json
Test scripts: robots/so101/scripts/test_tank_*.py, test_wall_avoid.py, run_tank_wall_avoid_local.sh

Lessons (carry into Dora `cmd_vel`)

Voltage: leader 5V (7.4V motors); follower/tank 12V.
Carpet ≠ hard floor. Default = hard floor. Carpet/high-grip: smaller radius + track. Hard-floor track on carpet overshot ~140° past 360° before carpet track lock. Recail both after the trike conversion.
Contact slip / surface dominates. Treads (even banded) made odom a short-horizon prior. Trike is the attempt to make wheel ticks trustworthy for mapping.
Tracks ≠ ideal wheels. ~2″ front→rear tread motion is gone; caster trail remains. Effective radius is still empirical per surface.
Pivots slide centimeters on hard floor with treads; map frames walk if only wheels integrated. Recail track_width on the trike after linear radius.
Rowve demo: wheel odom = short-horizon prior until trike recail is locked; AR/named places follow vision-corrected pose.
No short time-caps on distance runs; stop on odom (safety timeout only).
Stop dora-so101 teleop when doing wheel-only tests (waits for leader USB).
Depth ≠ wheel gap — calibrate with tape; don’t treat slant range as chassis clearance.
Heading = ahead L/R imbalance — sticky side strips caused same-direction over-rotate; CLEAR when open ahead and balanced.

Still open / NEXT

AprilTag mounted — tag36h11 id 532, 133.5 mm; mid-tread 6 cm outside black side edge; velocity 4 cm forward (configs/ar_marker.json)
Room tags ready to print — sofa 101, bathroom door 201 (assets/ar_marker/room_*_A4.pdf)
Named places — waypoint JSON + button go-to (drive once, save pose from /state)
Web AR client — speed + path + STOP in robots/so101/web-ar/
Apply to Marco — rsync + Tailscale Serve on LeBot + live lock id 532
OAK-D object detect arrows — YOLO nano in the wall-avoid graph; object-cone avoid live 2026-08-15
Stretch: laser-on-target
Row follow (next): bottles as crop rows — in-the-moment corridor, not occupancy. Prompt: HANDOFF_ROW_FOLLOW.md

Mapping: Paused 2026-08-22. Live hallway occupancy was not usable (sofa gap filled, hall heading lie, drifting bath tag). Write-up: robots/so101/docs/MAPPING_POSTMORTEM.md. Retry only after a 6-DoF IMU. Stereo stays for avoid / range-to-object.

See copy-paste prompt: HANDOFF_ROWVE.md. Wall avoid details: robots/so101/docs/WALL_AVOID.md.

Day 0 checklist (before coding hard)

Hardware and baseline — do this first so Day 1 is not blocked.

Power base + arm bus; confirm leader/follower serial by-id paths if teleop needed
Confirm tank drive motors addressable (IDs 7 / 8 on follower bus)
Wrist cam + OAK-D Lite on USB3; DepthAI opens on Pi
Print/affix the MARCO AprilTag nameplate on the rear white plate (robots/so101/assets/ar_marker/)
Phone and Pi on same Wi‑Fi; Pi IP 192.168.1.140 (ssh marco)
Dora on Pi smoke-checked (dora 0.5.0)

Known starting point from the archived Dora state:

Arm teleop + calibration graphs exist
OAK-D color/depth/RGBD Dora nodes + scripts exist
Perception LLM camera router exists (useful later; not required for location buttons)
Tank drive/odom scripts now exist; Dora-wrapped cmd_vel + mapping still the remaining gap vs old ROS scaffolding in robots/so101/docs/*_OLD_ROS.md

Day 1 — Drive, map, named places, intent stream

Morning: Tank drive + odometry in Dora

Outcome: Soft cmd_vel-style control and a believable odom frame for short indoor runs.

Differential-drive tank commands (left/right wheel goals) — prototype scripts on Pi
Odom distance from wheel feedback; tune signs + wheel radius on open floor
Yaw-hold from wheel-odom heading error (critical — equal cmds alone drift)
Tune track_width via pivot test (hard floor + carpet profiles locked)
Wrap into Dora node + dead-man / stop on lost heartbeat
Manual drive smoke test: forward / stop (pivot next)

Success: From SSH or a tiny local UI, Marco drives and pose roughly matches taped floor marks over a few meters.

Afternoon: Mapping + labeled places + “go to bedroom”

Outcome: A map (even crude) with named poses and a go-to that looks intentional in the hallway.

Pragmatic options (pick the fastest that works):

Pose graph of waypoints (fastest): drive once, save named poses (master_bedroom, hallway, living_room) in a JSON map; go-to = path of waypoints + simple controller.
Lightweight occupancy / SLAM if time allows after waypoints work.

Record / store map frame + named places
Implement go_to_place(name) → publishes plan state: {goal, path_hints, status}
Expose plan + pose over HTTP or WebSocket for the AR client (JSON is enough)
Location buttons in a minimal web page (even without AR yet): tap → robot moves

Success: Tap Master Bedroom → Marco starts down the hallway toward that place; phone/laptop shows live status text at minimum.

End of Day 1 gate

If drive + named go-to works, you have a co-founder story even before AR polish. Do not skip this gate for prettier overlays.

Day 2 — AR overlay + perception cues (+ laser stretch)

Morning: Web AR lock + intent overlay

Outcome: Phone camera + marker → AR content anchored to Marco showing planned motion.

Stand up a small web app (vanilla JS + Three.js + AprilTag WASM in robots/so101/web-ar/; no MindAR)
Track tag36h11 id 532 (worker + robot_T_tag); preview mode without camera
Subscribe to Pi pose via GET /state (poll; WS later)
Draw: left/right tread speed labels + 2.5 s instantaneous path (straight/arc/pivot)
Draw: Goal label (“Master Bedroom”) + named-place path
Keep UI sparse: brand/demo clarity over dashboard clutter

Success: Standing behind Marco in the hallway, AR clearly shows it is planning / heading to the master bedroom.

Midday: OAK-D detection → AR cues

Outcome: Live “I see a chair that way” without requiring perfect 3D boxes.

Run object detection on OAK-D stream (on-device YOLO/DepthAI model or Pi-side detector — whichever installs cleanest today)
Emit detections: {label, confidence, bearing_or_pixel, optional depth}
AR: arrow from robot toward detection + label (“chair”, “shoe”)
Skip tight 3D boxes unless depth + extrinsic math is already behaving; arrows are enough for the pitch

Success: Point phone at robot; when OAK-D sees a chair, AR shows a labeled arrow in that direction. Live 2026-08-15; yaw retuned same day.

Afternoon stretch: Laser “energy on target” proxy

Maps to farm weeding (LiteWeed stop-and-align, not continuous tracking). Full write-up: docs/research/WEEDING_PRIOR_ART.md.

Co-mount a small laser pointer with the targeting camera (wrist preferred; safe class; eye safety)
Digital on/off from Dora (GPIO / relay / USB switch)
Indoor “weed” = green blob (ExHSV/ExG) or a green stand-in on the floor — not a COCO chair
Closed loop: detect → halt base → aim to blob centroid → laser on for dwell f(blob area) → re-look
Bonus clip only: creep while holding the beam (harder; not v1 success)

Success: Video of stop → aim → timed beam on the stand-in; AR showing candidate + dwell. Bonus: short creep clip.

Weeding effector notes (for conversation, not Day 1–2 build)

Shared product question with Rowve: best weed kill that is low energy and quick/effective.

Read first: docs/research/WEEDING_PRIOR_ART.md — OpenWeedLocator (colour + YOLO → zone actuate) and LiteWeed (~$500, 4 W 450 nm, stop-and-align, 30–60 s dwell).

Option	Pros	Cons / risks
Laser	Precise, no soil contact, demoable; LiteWeed shows 4 W / stop / ~30–60 s kills early pigweed/purslane/nutsedge at ~96%	Throughput (seconds per weed), eye safety, overburn to crop/soil, regulations
Steam	Non-chemical	Energy/water, slow, bulky
Snippers / mechanical	Familiar, physical certainty	Complexity, wear, miss roots, soil disturbance
Pulse electrocution	Low energy story; can damage roots	Safety, soil moisture variability, compliance
OWL-style spot spray	Cheapest detect→GPIO loop, proven in fallow	Still herbicide; not our chemical-free story

Demo strategy: Indoor proxy is stop-and-align + dwell from blob size (LiteWeed control law), with AR showing the candidate and planned seconds. Creep-while-firing is optional wow, not the success bar. Colour ExHSV/ExG on a down-looking camera (wrist) is the weed stand-in — not COCO chairs. Pulse / mechanical stay in the talk track because LiteWeed’s dwell is too slow for high-density hectares/hour.

Perception after the hallway demo: vegetation gate (OWL) → optional crop-mask hybrid in-row → tiny plant detector only with our camera height. Forward OAK-D stays drive/avoid; weeding looks at the ground.

What to show him (suggested pitch flow)

Form factor match: “Same differential + SO-101 family as your weeder — here’s Marco live (trike wheels now; same kinematics).”
Dora-first: Show one graph / dataflow; contrast with heavy ROS bringup for iteration speed.
AR trust layer: Phone AR — named goal, hallway intent, detection arrows.
Farm bridge: “Same stack: map rows → vegetation/weed labels → AR veto → stop-and-align energy on target.” OWL already does cheap detect→actuate; LiteWeed already does $500 laser stop-and-burn; we add trust + manipulator + Dora.
Laser clip (if ready): Stop, aim, dwell from size; path to pulse / mechanical if throughput needs it.
Ask: Co-founder seat focused on robot stack, perception, and operator UX (AR).

Repo touchpoints (start here)

Piece	Location
Dora SO-101 package	`robots/so101/`
Teleop graph	`robots/so101/graphs/mono_teleop_real_so101.yml`
AprilTag nameplate (print)	`robots/so101/assets/ar_marker/`
Room tags (sofa 101, bath 201)	`robots/so101/assets/ar_marker/room_*_A4.pdf` + `configs/ar_room_markers.json`
AR operator UI	`robots/so101/web-ar/` + `docs/AR_OPERATOR.md`
Tank/trike forward / distance / rotate tests	`robots/so101/scripts/test_tank_*.py`
OAK-D / wrist / YOLO graphs + scripts	`robots/so101/graphs/`, `robots/so101/scripts/`
Perception LLM router (optional later)	`robots/so101/graphs/perception_llm_router_local.yml`
Old ROS tank / Nav2 notes (reference only)	`robots/so101/docs/NEXT_STEPS_OLD_ROS.md`
Weeding prior art (OWL + LiteWeed)	`docs/research/WEEDING_PRIOR_ART.md`
Env	Pi: `~/Marco-Dora/dora-lerobot/.venv` + LeRobot `~/LeRobot/le-robot/.venv` for bus tests

Likely new work for this sprint (to create as we go):

Named-place map store + go-to planner/controller
Phone HTTPS (Tailscale Serve on Marco) for live camera lock
Optional laser GPIO node + stop-and-align detect graph (ExHSV blob, not COCO)

Risks and how we de-risk in 48 hours

Risk	Mitigation
Tank odom not in Dora yet	Waypoint map + short hallway; don’t need perfect Nav2. Trike recail is to make that odom less slippy.
AR pose noisy	Large marker, good lighting, stand close behind robot
Full 3D boxes hard	Arrows + labels only
Detection flaky	Few indoor classes; high confidence threshold; staged props
Scope creep	Ship named go-to + AR path first; laser is stretch
Wrong network / latency	Same Wi‑Fi, LAN only, no cloud for the live demo

Definition of done (co-founder demo)

Must have

Marco drives under Dora with usable short-range odom
At least one named place (master_bedroom) and button/go-to that starts hallway motion
Web AR locked to robot marker showing goal + direction/path intent live

Should have

OAK-D detections with AR arrow + label for ≥1 object class

Nice to have

Laser on/off from software (co-mounted with targeting cam)
Stop-and-align dwell on a green stand-in; optional creep clip

When those boxes are checked, the conversation is no longer “I have ideas” — it is “here is the product layer your weeder will need, running on a twin of your robot.”

FilesExpand file tree

README-Rowve.md

Latest commit

History