perf(inference): optimize video pipeline metrics (#16)

Add warm-up-aware latency reporting, decode and write measurements, complete source-to-output metrics, running-total FPS calculation, tests, and documentation.

Author: NafizNoyon

README.md

@@ -43,6 +43,7 @@ performance, and deployment-oriented engineering.
 - Controlled PyTorch-vs-ONNX Runtime raw-backend benchmarking
 - Deterministic severity-based low-light, blur, and noise evaluation
 - Config-driven ONNX Runtime video and camera inference
+- Warm-up-aware end-to-end video pipeline profiling
 
 ## Dataset Ingestion
 
@@ -203,6 +204,18 @@ must not be treated as complete real-road application performance.
 See [Config-Driven ONNX Video Inference](docs/video_inference.md)
 for commands, architecture, smoke-test evidence, and limitations.
 
+## Video Pipeline Optimization
+
+A controlled thread-count audit retained eight ONNX Runtime threads
+and identified inference as the dominant component.
+
+The pipeline now excludes five warm-up frames, measures decode and
+video-writing latency separately, reports complete source-to-output
+throughput, and calculates running FPS using running totals.
+
+See [End-to-End Video Pipeline Optimization](docs/video_optimization.md)
+for the audit, implementation, validation, and limitations.
+
 ## Benchmarking
 
 The project includes a reproducible repeated-run inference benchmark.
@@ -236,11 +249,12 @@ Completed milestones:
 12. Controlled PyTorch-vs-ONNX Runtime CPU benchmarking
 13. Deterministic severity-based robustness evaluation
 14. Config-driven ONNX Runtime video inference
+15. Warm-up-aware end-to-end video pipeline optimization
 
 Current milestone:
 
 ```text
-video inference completed -> end-to-end optimization -> Docker packaging
+end-to-end optimization completed -> real-world validation -> Docker packaging
 ```
 
 ## Planned Technology Stack

configs/video_inference.yaml

@@ -17,6 +17,7 @@ runtime:
   iou: 0.60
   max_detections: 300
   max_frames: 0
+  warmup_frames: 5
 
 output:
   save_video: true

docs/video_inference.md

@@ -55,6 +55,7 @@ edge-traffic-video --config configs/video_inference.yaml --source input.mp4 --ou
 | Confidence threshold | 0.20 |
 | NMS IoU threshold | 0.60 |
 | Maximum detections | 300 |
+| Warm-up frames | 5 |
 | Default source | Camera index 0 |
 | Codec | mp4v |
 | Display window | Disabled |
@@ -145,6 +146,16 @@ The JSON report includes:
 - Camera capture, video writer, and display windows are released.
 - A zero-frame source is treated as a runtime failure.
 
+## Optimization
+
+The pipeline now excludes configurable warm-up frames, records
+decode and video-writing latency separately, reports complete
+source-to-output latency, and calculates running FPS using
+constant-time running totals.
+
+See [End-to-End Video Pipeline Optimization](video_optimization.md)
+for the thread audit, validation, and remaining limitations.
+
 ## Limitations
 
 - The controlled smoke source repeated one validation image.

docs/video_optimization.md

@@ -0,0 +1,58 @@
+# End-to-End Video Pipeline Optimization
+
+## Audit evidence
+
+A controlled 640 x 360 repeated-frame audit tested ONNX Runtime
+thread counts 1, 2, 4, 8, and 16. Each setting used 10 warm-up
+runs and 50 measured runs.
+
+| Threads | Total mean | P95 | FPS | Bottleneck |
+|---:|---:|---:|---:|---|
+| 1 | 25.117 ms | 28.115 ms | 39.81 | Inference |
+| 2 | 17.231 ms | 23.765 ms | 58.03 | Inference |
+| 4 | 13.267 ms | 15.894 ms | 75.37 | Inference |
+| 8 | 11.942 ms | 12.429 ms | 83.74 | Inference |
+| 16 | 33.209 ms | 42.960 ms | 30.11 | Postprocessing |
+
+Eight threads remain configured because they produced the best
+controlled complete-pipeline mean latency.
+
+## Implemented changes
+
+- Add five configurable warm-up frames.
+- Exclude warm-up frames from reported statistics.
+- Measure OpenCV frame-decoding latency.
+- Measure annotated-video writing latency.
+- Preserve processing-only latency.
+- Add complete source-to-output latency.
+- Replace latency-history copying with running totals.
+- Report processed, warm-up, and measured frame counts.
+- Fall back to all frames when a source is shorter than warm-up.
+
+## Controlled validation
+
+| Measurement | Result |
+|---|---:|
+| Frames processed | 40 |
+| Frames measured | 35 |
+| Total detections | 73 |
+| Mean decode latency | 0.2895 ms |
+| Mean preprocessing latency | 0.7911 ms |
+| Mean inference latency | 8.2303 ms |
+| Mean postprocessing latency | 1.3188 ms |
+| Mean rendering latency | 0.2413 ms |
+| Mean video-writing latency | 1.1832 ms |
+| Mean processing latency | 11.7716 ms |
+| Processing P95 | 12.7646 ms |
+| Processing throughput | 84.95 FPS |
+| Mean source-to-output latency | 12.0628 ms |
+| Source-to-output P95 | 13.0193 ms |
+| Source-to-output throughput | 82.90 FPS |
+| Annotated frames decoded | 40 |
+
+## Limitations
+
+- The source repeats one validation image.
+- The measurements describe this CPU, resolution, codec, and environment.
+- VideoWriter timing may not include final buffered disk flush.
+- Real-road, webcam, 720p, 1080p, and long-run testing remain separate.