feat: IP Webcam phone camera support (BACKLOG 6B.1)

- perception/camera.py: Camera(source) now accepts int (USB) or str (URL).
  normalize_camera_source() turns bare host:port into http://host:port/video.
  HTTP streams get CAP_PROP_BUFFERSIZE=1 to avoid frame lag.
- config.py: CAMERA_SOURCE reads DECRAS_CAMERA env (digits -> int, else URL).
- server.py + scripts/calibrate.py: pass CAMERA_SOURCE to Camera().
- .mcp.json: DECRAS_CAMERA=192.168.129.1:8080 for the MCP subprocess.
- tests/test_perception.py: 5 new normalization tests.
- Live-validated end-to-end: 1920x1080 BGR frame from phone.
- BACKLOG + PROJECT_STATUS updated.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: Alex-Hache

.mcp.json

@@ -6,7 +6,8 @@
       "cwd": "/home/alexandre/IA/DecRAS",
       "env": {
         "SIMULATE": "false",
-        "LEROBOT_FOLLOWER_PORT": "/dev/decras_follower"
+        "LEROBOT_FOLLOWER_PORT": "/dev/decras_follower",
+        "DECRAS_CAMERA": "192.168.129.1:8080"
       }
     }
   }

BACKLOG.md

@@ -106,14 +106,9 @@
 
 ### 6B.1 Camera setup
 
-- [ ] **Install IP Webcam on phone** — Android app, connect to same WiFi as robot PC. Note stream URL (`http://<phone-ip>:8080/video`).
-  - *Done when*: Stream URL opens in browser with live video
-  - *Time*: 10 min
+- [x] **Install IP Webcam on phone** — DONE (2026-04-19): stream at `http://192.168.129.1:8080/video`, verified returning 1920×1080 BGR frames.
 
-- [ ] **Wire phone camera into perception pipeline** — Update `perception/camera.py` to accept an IP Webcam URL as input source (alongside USB webcam). Scene graph builder should work unchanged.
-  - *Done when*: `observe()` MCP tool returns scene graph with detected objects via phone camera
-  - *Time*: 30 min
-  - *Tag*: `claude-code`
+- [x] **Wire phone camera into perception pipeline** — DONE (2026-04-19): `Camera(source)` accepts int (USB) or string URL; bare `host:port` normalized to `http://host:port/video`. `CAMERA_SOURCE` in `config.py` reads `DECRAS_CAMERA` env var (digits → int, else URL). `.mcp.json` sets `DECRAS_CAMERA=192.168.129.1:8080`. Live test: `Camera(CAMERA_SOURCE).capture()` returns a real frame from the phone; detector runs without error.
 
 ### 6B.2 LLM loop end-to-end
 

PROJECT_STATUS.md

@@ -34,7 +34,7 @@ mcp_server/
   robot/kinematics.py    — placo FK/IK engine (PyBullet DIRECT headless client)
   sim/base.py            — Abstract SimEnvironment (designed for Isaac Sim swap)
   sim/pybullet_env.py    — PyBullet backend
-  perception/camera.py   — Webcam capture (optional, degrades gracefully)
+  perception/camera.py   — Webcam capture (USB int device or IP Webcam URL via CAMERA_SOURCE / DECRAS_CAMERA env; degrades to simulated frames when SIMULATE=true)
   perception/detector.py — Color-based object detection
   perception/scene_graph.py — Scene graph builder
   episode.py             — Episode recorder
@@ -97,7 +97,7 @@ datasets/
 - **placo FK/IK `.pos` suffix bug fixed (April 2026)** — `joints_to_cartesian()` and `cartesian_to_joints()` silently ignored input joint angles due to key name mismatch (`.pos` suffix from LeRobot). Fixed via `_normalize_joint_dict()`. FK/IK now produces correct results, validated with 10cm XY square on hardware. Remaining concern: IK still changes wrist_flex during EE-space moves → Z drop when moving in X.
 - **`move_cartesian_delta` rewritten (April 2026)** — was reading mid-motion `get_observation()` between sub-steps and using a fast inner interpolation (200ms) the servo couldn't track. Both bugs caused 30-50% reach. New implementation plans waypoints upfront from FK seed, chains IK seeds, uses convergence-based wait + active-hold. Reach now 80-95% on horizontal/down moves.
 - **+Z up moves limited to 25-50% reach** — Feetech servo torque insufficient to lift arm against gravity to IK target. Tracked in BACKLOG (Z-up gravity compensation task).
-- Camera/perception pipeline not tested on real hardware yet
+- Camera pipeline live-tested with IP Webcam on phone (April 2026): 1920×1080 frames from `http://192.168.129.1:8080/video`. Detector runs end-to-end. Real-scene detector tuning (HSV ranges for actual lab objects, not demo red cup / blue plate) not yet done.
 - Servo convergence under gravity load requires active hold loops (repeated send_joint_positions)
 - Only 2 ports: follower on `/dev/ttyACM0`, leader on `/dev/ttyACM1`
 
@@ -172,7 +172,7 @@ All 8 motion primitives hardware-validated (March 2026):
 - **Test Zero**: replay segmenter v2 output on hardware — critical gate for Path B
 
 **Phase 6B — Full Loop (NEXT)**:
-- Camera setup (IP Webcam on phone) + wire into perception pipeline
+- ~~Camera setup (IP Webcam on phone) + wire into perception pipeline~~ — DONE (phone MJPEG stream, `DECRAS_CAMERA` env var, live-tested)
 - LLM observe-think-act loop end-to-end on real hardware
 - Analyze LLM failure modes
 - First RAG experiment: inject demo sequence as few-shot example

mcp_server/config.py

@@ -21,6 +21,18 @@
 # Camera-to-robot affine transform (populate during calibration)
 CAMERA_TO_ROBOT_MATRIX = None  # 3x3 affine, set by calibration script
 
+# Camera source: int for USB device index, str for IP Webcam URL.
+# Override via env var DECRAS_CAMERA, e.g. DECRAS_CAMERA=192.168.129.1:8080
+#   — bare host:port is normalized to http://host:port/video by Camera().
+#   — integers are parsed as USB device indices.
+_camera_env = os.environ.get("DECRAS_CAMERA", "").strip()
+if _camera_env == "":
+    CAMERA_SOURCE: int | str = 0
+elif _camera_env.isdigit():
+    CAMERA_SOURCE = int(_camera_env)
+else:
+    CAMERA_SOURCE = _camera_env
+
 # Robot workspace limits (meters, in robot frame)
 WORKSPACE = {
     "x_min": 0.0,

mcp_server/perception/camera.py

@@ -1,4 +1,10 @@
-"""Webcam capture and frame buffering."""
+"""Webcam capture and frame buffering.
+
+``Camera`` accepts either:
+  - int   — OpenCV USB device index (``Camera(0)``)
+  - str   — an IP Webcam MJPEG stream URL (``Camera("http://192.168.129.1:8080/video")``)
+            Bare ``host:port`` is accepted and normalized to ``http://host:port/video``.
+"""
 
 import logging
 import numpy as np
@@ -13,9 +19,31 @@
     logger.warning("opencv-python not installed — camera will use simulated frames")
 
 
+def normalize_camera_source(source: int | str) -> int | str:
+    """Pass ints through; coerce string URLs into a form OpenCV can open.
+
+    Rules for strings:
+      - If it already has a scheme (``http://``/``https://``/``rtsp://``), leave it.
+      - Otherwise prepend ``http://``.
+      - If no path is provided (just ``host:port``), append ``/video`` (IP Webcam default MJPEG endpoint).
+    """
+    if isinstance(source, int):
+        return source
+    s = str(source).strip()
+    if not s:
+        raise ValueError("Camera source string is empty")
+    lower = s.lower()
+    if not (lower.startswith("http://") or lower.startswith("https://") or lower.startswith("rtsp://")):
+        s = "http://" + s
+    scheme, _, rest = s.partition("://")
+    if "/" not in rest:
+        s = f"{scheme}://{rest}/video"
+    return s
+
+
 class Camera:
-    def __init__(self, device_id: int = 0):
-        self._device_id = device_id
+    def __init__(self, source: int | str = 0):
+        self._source = normalize_camera_source(source)
         self._cap = None
         self._last_frame: np.ndarray | None = None
 
@@ -25,10 +53,13 @@ def __init__(self, device_id: int = 0):
             logger.info("Camera running in SIMULATE mode")
 
     def _open(self):
-        self._cap = cv2.VideoCapture(self._device_id)
+        self._cap = cv2.VideoCapture(self._source)
         if not self._cap.isOpened():
-            raise RuntimeError(f"Cannot open camera device {self._device_id}")
-        logger.info("Camera opened on device %d", self._device_id)
+            raise RuntimeError(f"Cannot open camera source {self._source!r}")
+        if isinstance(self._source, str):
+            # Minimize latency on HTTP streams — we always want the newest frame, not a backlog.
+            self._cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)
+        logger.info("Camera opened on source %r", self._source)
 
     def capture(self) -> np.ndarray:
         """Capture a single frame. Returns BGR numpy array."""

mcp_server/server.py

@@ -63,8 +63,9 @@ def _get_camera():
         return None
     try:
         from mcp_server.perception.camera import Camera
-        camera = Camera()
-        logger.info("Camera opened")
+        from mcp_server.config import CAMERA_SOURCE
+        camera = Camera(CAMERA_SOURCE)
+        logger.info("Camera opened (source=%r)", CAMERA_SOURCE)
     except Exception as e:
         logger.warning("Camera not available: %s", e)
     return camera

scripts/calibrate.py

@@ -44,9 +44,10 @@ def collect_points_interactive() -> tuple[np.ndarray, np.ndarray]:
 
     Returns (pixel_points, robot_points) as Nx2 and Nx2 arrays.
     """
+    from mcp_server.config import CAMERA_SOURCE
     from mcp_server.perception.camera import Camera
 
-    camera = Camera()
+    camera = Camera(CAMERA_SOURCE)
     pixel_points = []
     robot_points = []
     click_pos = [None]

tests/test_perception.py

@@ -1,10 +1,36 @@
 """Tests for the perception pipeline (camera, detector, scene graph)."""
 
-from mcp_server.perception.camera import Camera
+import pytest
+
+from mcp_server.perception.camera import Camera, normalize_camera_source
 from mcp_server.perception.detector import detect_objects
 from mcp_server.perception.scene_graph import build_scene_graph
 
 
+def test_normalize_camera_source_passes_int():
+    assert normalize_camera_source(0) == 0
+    assert normalize_camera_source(2) == 2
+
+
+def test_normalize_camera_source_bare_host_port():
+    assert normalize_camera_source("192.168.129.1:8080") == "http://192.168.129.1:8080/video"
+
+
+def test_normalize_camera_source_preserves_scheme():
+    assert normalize_camera_source("http://cam/video") == "http://cam/video"
+    assert normalize_camera_source("https://cam:443/stream") == "https://cam:443/stream"
+    assert normalize_camera_source("rtsp://cam/live") == "rtsp://cam/live"
+
+
+def test_normalize_camera_source_appends_video_path():
+    assert normalize_camera_source("http://cam:8080") == "http://cam:8080/video"
+
+
+def test_normalize_camera_source_rejects_empty():
+    with pytest.raises(ValueError):
+        normalize_camera_source("")
+
+
 def test_camera_capture():
     camera = Camera()
     frame = camera.capture()