PhysiLens

A visual physics engine for extracting real-world scientific measurements from a static 2D image.

PhysiLens lets you upload a photograph, calibrate it against a reference object of known size, isolate a subject with AI-powered segmentation, and receive a scientific report covering that subject's real-world dimensions, distance, and — where motion is visible — kinematics.

---

How it works

1. Spatial calibration — Draw a line over a reference object of known real-world height (a person, a doorway, a tire) and enter its length. PhysiLens derives a meters-per-pixel scale factor from that single reference.
2. AI perception — Segment Anything (SAM) isolates your chosen subject with a single click. Depth-Anything-V2 estimates relative scene depth for Z-axis context.
3. Deterministic physics — Every physical quantity in the final report (height, width, surface area, volume, speed, mass, kinetic energy) is computed with closed-form mathematics and least-squares regression — never guessed by a model. See engine/core/physics.py for the full derivations.
4. Scientific report — Results are compiled into a shareable PDF report, generated entirely client-side.

Architecture

PhysiLens/
├── engine/                 FastAPI backend — image processing & physics
│   ├── main.py             API routes, session handling, validation
│   ├── schemas.py          Pydantic request/response contracts
│   ├── config.py           Environment-driven settings
│   ├── core/
│   │   ├── physics.py      Pure deterministic math (no AI)
│   │   └── analyzer.py     SAM + Depth-Anything-V2 wrappers (lazy-loaded)
│   └── utils/
│       └── report_gen.py   Report assembly / confidence scoring
│
└── interface/               Next.js 16 frontend
    ├── app/                 App Router pages & layout
    ├── components/
    │   ├── AnalysisDashboard.tsx   Workflow orchestration
    │   ├── CanvasOverlay.tsx       Interactive measurement canvas
    │   ├── MetricsPanel.tsx        Live data readout
    │   ├── ReportModal.tsx         Report preview
    │   └── ExportButton.tsx        PDF export trigger
    └── lib/
        ├── utils.ts         API client, formatters
        ├── types.ts         Shared TypeScript contracts
        └── export-pdf.ts    jsPDF report generation

Design principle: the AI models (SAM, Depth-Anything-V2) are used strictly for perception — "where is the subject" and "what's the relative depth." Every physical number reported to the user is then derived by explicit, auditable formulas in physics.py, never inferred directly by a model. This keeps results reproducible and explainable.

Prerequisites

• Backend: Python 3.11+, a CUDA-capable GPU recommended (CPU inference works but is slow for SAM/Depth-Anything)
• Frontend: Node.js 20+, pnpm 9+

Getting started

1. Backend (engine)

cd engine
python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env            # then edit values as needed

Download a SAM checkpoint (the vit_b variant is the smallest/fastest) from the official repository and place it at the path referenced by PHYSILENS_SAM_CHECKPOINT in your .env (default: engine/weights/sam_vit_b_01ec64.pth).

uvicorn main:app --reload --port 8000

The API will be available at http://localhost:8000, with interactive docs at /docs.

2. Frontend (interface)

cd interface
pnpm install
cp .env.example .env.local      # then edit NEXT_PUBLIC_ENGINE_API_URL if needed
pnpm dev

The app will be available at http://localhost:3000.

Model loading & first-request latency

SAM and Depth-Anything-V2 are lazy-loaded: they're only loaded into memory the first time /api/segment or /api/report is called, not at server startup. This keeps the server lightweight to start, but means the first subject-selection click or report generation after a fresh server start can take anywhere from several seconds to over a minute (model weights + inference on CPU are slow). The frontend surfaces this explicitly with a "loading model, first run only" message so it isn't mistaken for the click doing nothing.

Two ways to avoid this surprising a user in production:

1. Call /api/warmup right after starting the server (e.g. in a deploy hook, docker-compose healthcheck script, or CI smoke test):

   curl -X POST http://localhost:8000/api/warmup

   This forces both models to load immediately and returns their status, so you control when the cold-load cost happens.

2. Set PHYSILENS_WARMUP_ON_STARTUP=true in your .env to load both models automatically during server startup, before the app starts accepting requests. This delays server readiness by the full model-load time — make sure your container orchestrator's startup/readiness probes account for that (a generous start_period in the Dockerfile HEALTHCHECK, or an equivalent readiness delay in Kubernetes).

Security notes

• CORS on the backend is restricted to an explicit origin allow-list (PHYSILENS_CORS_ORIGINS_RAW) — never a wildcard.
• Uploaded images are validated for content-type, byte size, and pixel dimensions before any processing occurs.
• Analysis sessions live in memory with a configurable TTL (PHYSILENS_SESSION_TTL_SECONDS) and are never persisted to disk.
• No secrets are hardcoded anywhere in this repository; all configuration is environment-driven. Copy the provided .env.example files and fill in your own values locally.
• The frontend sets X-Frame-Options, X-Content-Type-Options, and a restrictive Permissions-Policy on every response.

Before deploying, review engine/config.py and set PHYSILENS_ENABLE_DOCS=false and PHYSILENS_DEBUG=false in production environments.

Scaling notes

The in-memory SessionStore in engine/main.py is intentionally minimal (a get/set/update interface) so it can be swapped for a Redis-backed implementation in horizontally-scaled deployments without touching route logic.

Limitations

PhysiLens produces estimates, not certified measurements. Accuracy depends entirely on:

• The precision of the user-drawn calibration line and the accuracy of the entered reference length,
• Monocular depth estimation being inherently relative (absolute distance requires an explicit reference anchor),
• Published average densities/masses being used as fallbacks when an object's specific material properties are unknown.

Every generated report includes an explicit methodology and disclaimers section.

License

Released under the MIT License.

Contributing

Issues and pull requests are welcome. Please ensure any contribution touching engine/core/physics.py includes the underlying derivation or citation for the formula used — this module intentionally has no "black box" math.

---

Created and maintained by BZDevelopments (@BZDevelopments) — BzDev.