We need to benchmark the performance of all supported models in our library across different hardware platforms and publish the results in the project documentation.
The goal is to establish a consistent CPU and NVIDIA baseline and compare it against Intel hardware to understand model performance, identify hardware-specific bottlenecks, and provide users with representative performance expectations.
Target Hardware
Intel PTL CPU (Ultra 3 series) - CPU baseline
NVIDIA RTX 3090 Ti — GPU baseline
Intel iGPU PTL
Intel Arc B70
Scope
Benchmark all currently supported models for the relevant tasks, using a consistent benchmarking methodology across all devices.
The benchmark should cover all Geti supported tasks:
Object Detection (OD)
Instance Segmentation (IS)
Classification (CLS)
Metrics
At minimum, collect:
Inference latency (OpenVINO where applicable including int8 results)
Throughput (FPS / images per second) (OpenVINO where applicable including int8 results)
Batch size
Input resolution
Training iteration time on 1 "general" dataset like COCO
Model name and version
Hardware device
Software/runtime versions
We need to benchmark the performance of all supported models in our library across different hardware platforms and publish the results in the project documentation.
The goal is to establish a consistent CPU and NVIDIA baseline and compare it against Intel hardware to understand model performance, identify hardware-specific bottlenecks, and provide users with representative performance expectations.
Target Hardware
Intel PTL CPU (Ultra 3 series) - CPU baseline
NVIDIA RTX 3090 Ti — GPU baseline
Intel iGPU PTL
Intel Arc B70
Scope
Benchmark all currently supported models for the relevant tasks, using a consistent benchmarking methodology across all devices.
The benchmark should cover all Geti supported tasks:
Object Detection (OD)
Instance Segmentation (IS)
Classification (CLS)
Metrics
At minimum, collect:
Inference latency (OpenVINO where applicable including int8 results)
Throughput (FPS / images per second) (OpenVINO where applicable including int8 results)
Batch size
Input resolution
Training iteration time on 1 "general" dataset like COCO
Model name and version
Hardware device
Software/runtime versions