AI Hide and Seek Arena

A full working hide-and-seek game where multiple AI hiders and seekers play on a grid with a live web UI.

Features

• Multi-agent game: configurable number of hiders and seekers
• Live 3D browser UI (custom WebGL-style renderer, no external CDN)
• Controls, scoreboard, and events feed
• Obstacles and round-based gameplay
• Sealed room generation with hiders spawning inside and seekers outside
• Pushable obstacles for both teams (used to block/unblock room entry)
• Seekers have directional vision (FOV + range + line-of-sight checks)
• Seekers pursue in straighter lines and keep short-term memory of last seen hider positions
• Q-learning brains for all agents
• Recent-move replay memory so behavior improves over time
• WebSocket streaming for real-time updates
• Optional PyBullet 3D physics sandbox API (/api/physics/*) for next-stage RL work
• Web UI auto-switches to physics spectator mode when physics state is available
• PPO self-play training CLI with checkpoints and policy-vs-policy evaluation matches

Quick Start

1. Create and activate a virtual environment (optional but recommended).
2. Install dependencies:

pip install -r requirements.txt

3. Start the server:

uvicorn backend.main:app --reload --port 8010

4. Open:

http://127.0.0.1:8010

Note: the 3D frontend is fully local and does not require CDN script imports.

Alternative:

python main.py

main.py defaults to port 8010 and supports override via env var:

$env:PORT=9000; python main.py

Controls

• Start / Pause: run or pause training
• Next Round: reset positions/obstacles but keep learned Q-tables
• Full Reset: clear scores and learned Q-tables
• Speed: adjust simulation tick rate
• Apply Config: change grid size, number of agents, obstacles, and round length

Physics Sandbox (Step 1)

This repo now includes a separate PyBullet world with:

• Arena boundary walls
• A sealed room with a blocked doorway
• Hiders spawned inside the room and seekers outside
• Movable boxes and ramps as physical objects
• Movable wall segments as tactical blockers
• Agent bodies simulated in 3D physics
• Current UI can visualize this physics state directly as a spectator view

API endpoints:

• GET /api/physics/state
• POST /api/physics/step with { "steps": 60, "autopilot": true }
  • You can also pass manual action maps:
  • { "steps": 1, "autopilot": false, "actions": { "S1": { "move": [1.0, 0.0] } } }
• Supported action keys per agent: move, turn, yaw, grab, release, push
• Autopilot now uses doorway interactions (grab/release/push) to contest room entry with props/ramps/walls
• POST /api/physics/reset with { "seed": 123 }

If available=false, inspect diagnostic in the response. It now returns the actual Python import error.

PPO Self-Play (Step 3 Start)

Training CLI:

python -m backend.ppo_self_play train --role seeker --timesteps 300000 --envs 4
python -m backend.ppo_self_play train --role hider --timesteps 300000 --envs 4

League mode (alternates seeker/hider automatically):

python -m backend.ppo_self_play league --cycles 4 --timesteps-per-cycle 200000 --envs 4

Evaluation matches (checkpoint vs checkpoint, or model vs scripted):

python -m backend.ppo_self_play eval --seeker-model artifacts/selfplay/seeker/seeker_latest.zip --hider-model artifacts/selfplay/hider/hider_latest.zip --episodes 20

Artifacts:

• Checkpoints: artifacts/selfplay/<role>/checkpoints/*.zip
• Latest model: artifacts/selfplay/<role>/<role>_latest.zip
• Eval logs: artifacts/selfplay/<role>/eval_matches.jsonl

Physics action keys used by training/evaluation:

• move: [x, z]
• turn: scalar yaw-rate control
• yaw: explicit heading override
• grab: boolean
• release: boolean
• push: scalar force factor

Roadmap Status

• Add real physics sim (PyBullet) and object bodies (boxes/ramps/walls): Completed
• Keep your current web UI as spectator mode fed from physics state: Completed
• Replace grid actions with physics actions (move/turn/grab/release/push): Completed
• Add PPO self-play training loop with checkpoints and evaluation matches: Started (baseline CLI + checkpoints + eval matches added)

Run standalone demo loop:

python -m backend.physics_demo --steps 1800

GUI mode (if supported by your machine):

python -m backend.physics_demo --gui --steps 1800

How learning works

Each agent uses:

• Q-learning update each step
• Epsilon-greedy exploration/exploitation
• Replay from recent memory buffer (replay_recent) to reinforce useful recent transitions

This gives agents the ability to improve from recent moves while still exploring.