add diversity-aware active review prioritization

Author: jiapengLi11

.github/workflows/ci.yml

@@ -45,6 +45,11 @@ jobs:
           python examples/create_near_duplicate_fixture.py --output /tmp/ylr-near-duplicates
           yolo-label-recovery cluster /tmp/ylr-near-duplicates --output-dir /tmp/ylr-near-duplicate-output --redact-paths
           python -c "import json; d=json.load(open('/tmp/ylr-near-duplicate-output/near_duplicate_summary.json')); assert d['totals']['images_discovered']==11 and d['totals']['fingerprint_failures']==1 and d['totals']['clusters']==3 and d['totals']['clustered_images']==7 and d['totals']['cross_split_clusters']==2"
+      - name: Run active review prioritization demo
+        run: |
+          python examples/create_prioritization_fixture.py --output-dir /tmp/ylr-priority-fixture
+          yolo-label-recovery prioritize /tmp/ylr-priority-fixture/candidates_review.csv /tmp/ylr-priority-fixture/dataset --output-dir /tmp/ylr-priority-output --budget 12 --redact-paths
+          python -c "import json; d=json.load(open('/tmp/ylr-priority-output/prioritization_summary.json')); assert d['totals']['candidate_rows']==36 and d['totals']['selected_images']==12 and d['totals']['selected_classes']==6 and len({r['candidate_classes'] for r in d['queue'][:6]})==6"
       - name: Build wheel
         run: python -m pip wheel --no-deps --wheel-dir dist .
       - name: Reject private artifacts

.gitignore

@@ -47,3 +47,4 @@ Desktop.ini
 !examples/consensus/*.csv
 !examples/consensus/output/*.csv
 !examples/near_duplicates/output/*.csv
+!examples/prioritization/output/*.csv

CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.8.0
+
+- Added the CPU-only, read-only `prioritize` command for image-level active human-review queues.
+- Added normalized confidence entropy, dynamically decayed class rarity and greedy dHash/aHash visual diversity.
+- Added safe resolution of absolute and relative candidate image paths with dataset-boundary enforcement.
+- Added ranked image/candidate exports, full-pool evidence, failure isolation and a self-contained HTML report.
+- Added a reproducible imbalanced six-class fixture, static screenshot, bilingual documentation and CI coverage.
+
 ## 0.7.0
 
 - Added the model-free, read-only `cluster` command for perceptual near-duplicate review grouping.

README.md

@@ -53,6 +53,12 @@ The public fixture contains `96` primary candidates across six classes. Of `72`
 
 The public fixture groups resize, JPEG recompression and brightness variants without merging black and white low-texture frames. It finds `3` groups containing `7` images, reduces first-pass review to `3` representatives and flags `2` groups crossing dataset splits.
 
+### Diversity-aware active review queue
+
+![Pre-generated active review prioritization report](docs/assets/prioritization-preview.png)
+
+The public fixture contains `36` imbalanced REVIEW images. A budget of `12` covers all `6` classes, with one class represented in each of the first six positions. Dynamic rarity prevents small classes from being ignored while perceptual diversity suppresses repeated frames.
+
 ## Why this project exists
 
 Multi-class datasets often contain combined scenes such as `person + helmet + smoking` or `person + slipper`. If the original annotation process focused on one target at a time, valid objects from other classes can be missing. Training a new multi-class model on incomplete labels can make the model learn the wrong supervision signal.
@@ -93,6 +99,7 @@ flowchart TD
 - Audited candidate decisions can calibrate class-specific AUTO policies using a Wilson precision lower bound and REVIEW policies using positive recall.
 - Independent Teacher candidate streams can gate AUTO decisions with one-to-one spatial agreement without loading two models together.
 - Perceptual hashes, a BK-tree and conservative visual guards group repeated review work and expose near-duplicate split leakage.
+- Image-level active review combines confidence entropy, dynamically decayed class rarity and greedy perceptual diversity.
 - Every scan records a local manifest with parameters, image inventory, package versions, CUDA and GPU metadata.
 
 ## One-minute public demo
@@ -159,6 +166,15 @@ yolo-label-recovery cluster D:\data\mining-safety `
   --redact-paths
 ```
 
+Build a limited-budget, diversity-aware human review queue:
+
+```powershell
+yolo-label-recovery prioritize D:\runs\candidates_review.csv D:\data\mining-safety `
+  --output-dir D:\runs\priority-review `
+  --budget 500 `
+  --redact-paths
+```
+
 For GPU-assisted label recovery, first install the CUDA-compatible PyTorch build required by the target GPU, then install the inference extra:
 
 ```powershell
@@ -277,6 +293,8 @@ See:
 - [Cross-Teacher consensus (Simplified Chinese)](docs/CONSENSUS.zh-CN.md)
 - [Perceptual near-duplicate grouping](docs/NEAR_DUPLICATES.md)
 - [Perceptual near-duplicate grouping (Simplified Chinese)](docs/NEAR_DUPLICATES.zh-CN.md)
+- [Active review prioritization](docs/ACTIVE_REVIEW.md)
+- [Active review prioritization (Simplified Chinese)](docs/ACTIVE_REVIEW.zh-CN.md)
 - [Interview presentation](docs/INTERVIEW_STORY.md)
 - [Portfolio and interview guide](docs/PORTFOLIO_GUIDE.md)
 - [Portfolio and interview guide (Simplified Chinese)](docs/PORTFOLIO_GUIDE.zh-CN.md)

README.zh-CN.md

@@ -53,6 +53,12 @@
 
 公开样例能够聚合缩放、JPEG 重压缩和亮度变化图片，同时不会错误合并纯黑与纯白低纹理帧。结果包含 `3` 组、共 `7` 张图片，只需优先审核 `3` 张代表图，并发现 `2` 组跨数据划分近重复。
 
+### 多样性感知主动审核队列
+
+![预生成主动审核优先级报告](docs/assets/prioritization-preview.png)
+
+公开样例包含 `36` 张类别不均衡的 REVIEW 图片。预算为 `12` 时覆盖全部 `6` 类，前六个位置每类各占一个。动态稀缺度避免小类被忽略，感知多样性则抑制重复连续帧。
+
 ## 为什么需要这个项目
 
 多类别数据集经常包含 `person + helmet + smoking`、`person + slipper` 等联合场景。如果原始标注工作每次只关注一个目标，图中其他类别的有效目标就可能漏标。使用不完整标签训练多类别模型时，这些目标会被当作背景，从而向模型传递错误监督信号。
@@ -93,6 +99,7 @@ flowchart TD
 - 使用人工审核候选校准分类别策略：AUTO 采用 Wilson 精度置信下限，REVIEW 采用正样本召回约束。
 - 使用独立 Teacher 候选流进行一对一空间一致性门控，无需同时加载两个模型。
 - 使用感知哈希、BK-tree 和保守视觉约束压缩重复审核工作，并发现跨划分近重复泄漏。
+- 图片级主动审核联合置信度熵、动态衰减类别稀缺度和贪心感知多样性。
 - 每次扫描生成 manifest，记录参数、图片清单、依赖版本、CUDA 和 GPU 信息。
 
 ## 一分钟公开演示
@@ -159,6 +166,15 @@ yolo-label-recovery cluster D:\data\mining-safety `
   --redact-paths
 ```
 
+生成有限预算、多样性感知的人工审核队列：
+
+```powershell
+yolo-label-recovery prioritize D:\runs\candidates_review.csv D:\data\mining-safety `
+  --output-dir D:\runs\priority-review `
+  --budget 500 `
+  --redact-paths
+```
+
 如需使用 GPU 自动补标，请先安装与目标 GPU/CUDA 兼容的 PyTorch，再安装推理依赖：
 
 ```powershell
@@ -277,6 +293,8 @@ out-root/
 - [跨 Teacher 一致性门控（英文）](docs/CONSENSUS.md)
 - [感知近重复聚类（中文）](docs/NEAR_DUPLICATES.zh-CN.md)
 - [感知近重复聚类（英文）](docs/NEAR_DUPLICATES.md)
+- [主动审核优先级（中文）](docs/ACTIVE_REVIEW.zh-CN.md)
+- [主动审核优先级（英文）](docs/ACTIVE_REVIEW.md)
 - [面试项目讲解](docs/INTERVIEW_STORY.md)
 - [作品集与面试指南（中文）](docs/PORTFOLIO_GUIDE.zh-CN.md)
 - [作品集与面试指南（英文）](docs/PORTFOLIO_GUIDE.md)

docs/ACTIVE_REVIEW.md

@@ -0,0 +1,69 @@
+# Active review prioritization
+
+[English](ACTIVE_REVIEW.md) | [简体中文](ACTIVE_REVIEW.zh-CN.md)
+
+The `prioritize` command converts a large candidate CSV into a deterministic, image-level human-review queue. It balances three signals instead of sorting by confidence alone:
+
+- **Uncertainty:** normalized Bernoulli entropy of detector confidence, highest near `0.5`.
+- **Rarity:** inverse square-root class frequency, dynamically decayed as that class receives review slots.
+- **Visual diversity:** greedy minimum dHash/aHash distance from the already selected images.
+
+The command is CPU-only and read-only. It never changes source candidates, images or labels.
+
+## Usage
+
+```powershell
+yolo-label-recovery prioritize D:\runs\candidates_review.csv D:\data\mining-safety `
+  --output-dir D:\runs\priority-review `
+  --budget 500 `
+  --workers 4 `
+  --uncertainty-weight 0.45 `
+  --rarity-weight 0.20 `
+  --diversity-weight 0.35 `
+  --redact-paths
+```
+
+The weights must be non-negative and sum to `1`. REVIEW is the only default mode; add `--modes review auto` only when AUTO spot checks intentionally share the same queue.
+
+## Image-level acquisition
+
+Human review normally opens an image, not an isolated prediction row. The command therefore groups every filtered candidate with the same `split + image`, keeps all associated classes/boxes together and computes image-level uncertainty and rarity. Candidate paths may be absolute, dataset-relative or split-relative, but they must resolve inside the dataset root.
+
+For every queue position, the acquisition function is:
+
+```text
+priority = 0.45 * uncertainty
+         + 0.20 * dynamic_class_rarity
+         + 0.35 * min_visual_distance_to_selected_set
+```
+
+After an image is selected, class coverage counts and every remaining image's minimum visual distance are updated. This greedy process is deterministic and requires approximately `O(N * B)` compact-hash comparisons for `N` review images and budget `B`; full-resolution pixels are not retained in memory.
+
+## Outputs
+
+| File | Purpose |
+|---|---|
+| `review_queue.csv` | One ranked row per selected image with score components |
+| `review_queue_candidates.csv` | Original candidate rows for selected images with priority rank |
+| `review_pool.csv` | All readable review images and selected status |
+| `image_failures.csv` | Missing, escaping or corrupt image evidence |
+| `prioritization_summary.json` | Policy, totals, distributions and queue data |
+| `prioritization_report.html` | Self-contained report for delivery and portfolio use |
+
+## Statistical boundary
+
+This queue is intentionally biased toward uncertain, rare and diverse examples. It improves issue discovery per reviewer-hour, but **must not** be used to estimate unbiased precision, recall or defect prevalence. Use a separate random or stratified-random audit sample for metrics; use this active queue for remediation.
+
+Detector confidence is also not a calibrated probability by default. Entropy is an acquisition heuristic here, not proof that a prediction is incorrect. Calibration and human verdicts remain separate stages.
+
+## Public fixture
+
+```powershell
+python examples\create_prioritization_fixture.py --output-dir .priority-fixture
+yolo-label-recovery prioritize .priority-fixture\candidates_review.csv .priority-fixture\dataset `
+  --output-dir .priority-output `
+  --budget 12 `
+  --redact-paths
+```
+
+The fixture contains `36` review images with intentionally imbalanced classes. The queue selects `12`, covers all `6` classes, and gives each class one position within the first six selections before assigning additional slots.

docs/ACTIVE_REVIEW.zh-CN.md

@@ -0,0 +1,69 @@
+# 主动审核优先级队列
+
+[English](ACTIVE_REVIEW.md) | [简体中文](ACTIVE_REVIEW.zh-CN.md)
+
+`prioritize` 命令将大量候选 CSV 转换为确定性的图片级人工审核队列。它不是简单按置信度排序，而是联合三个信号：
+
+- **不确定性：** 检测置信度的归一化伯努利熵，在 `0.5` 附近最高。
+- **类别稀缺度：** 类别频率的平方根倒数，并随着该类别已经获得审核席位而动态衰减。
+- **视觉多样性：** 当前图片到已选集合的最小 dHash/aHash 感知距离。
+
+该命令仅使用 CPU 且严格只读，不会修改源候选、图片或标签。
+
+## 使用方法
+
+```powershell
+yolo-label-recovery prioritize D:\runs\candidates_review.csv D:\data\mining-safety `
+  --output-dir D:\runs\priority-review `
+  --budget 500 `
+  --workers 4 `
+  --uncertainty-weight 0.45 `
+  --rarity-weight 0.20 `
+  --diversity-weight 0.35 `
+  --redact-paths
+```
+
+三个权重必须非负且总和为 `1`。默认只处理 REVIEW；只有明确需要在同一队列抽查 AUTO 时，才使用 `--modes review auto`。
+
+## 图片级获取策略
+
+人工通常打开整张图片审核，而不是孤立查看一行预测。因此工具会按 `split + image` 聚合同一图片的所有候选，保留关联类别和框，并计算图片级不确定性与稀缺度。候选路径可以是绝对路径、数据集相对路径或划分内相对路径，但解析结果必须位于数据集根目录内。
+
+每个队列位置使用以下获取函数：
+
+```text
+优先级 = 0.45 * 不确定性
+       + 0.20 * 动态类别稀缺度
+       + 0.35 * 到已选集合的最小视觉距离
+```
+
+每选中一张图片，就更新类别覆盖数量和所有剩余图片的最小视觉距离。该贪心过程完全确定；对于 `N` 张审核图片和预算 `B`，大约需要 `O(N * B)` 次紧凑哈希比较，不会在内存中保留全分辨率像素。
+
+## 输出文件
+
+| 文件 | 用途 |
+|---|---|
+| `review_queue.csv` | 每张选中图片一行，包含排名和各评分分量 |
+| `review_queue_candidates.csv` | 选中图片对应的原始候选行及优先级 |
+| `review_pool.csv` | 全部可读审核图片及是否入选 |
+| `image_failures.csv` | 缺失、越界或损坏图片证据 |
+| `prioritization_summary.json` | 策略、总量、分布与队列数据 |
+| `prioritization_report.html` | 可交付、可用于作品集的自包含报告 |
+
+## 统计边界
+
+该队列有意偏向不确定、稀缺和多样的样本，可提高每小时发现问题的效率，但**不能**用于估计无偏精度、召回率或缺陷比例。模型指标必须使用单独的随机或分层随机审核样本；主动队列用于发现与修复问题。
+
+检测置信度默认也不是经过校准的概率。这里的熵只是获取启发式，不代表预测一定错误。阈值校准和人工结论仍是独立阶段。
+
+## 公开样例
+
+```powershell
+python examples\create_prioritization_fixture.py --output-dir .priority-fixture
+yolo-label-recovery prioritize .priority-fixture\candidates_review.csv .priority-fixture\dataset `
+  --output-dir .priority-output `
+  --budget 12 `
+  --redact-paths
+```
+
+公开样例包含 `36` 张类别不均衡的审核图片，最终选择 `12` 张并覆盖全部 `6` 类；前六个位置每类各获得一个席位，之后才继续分配额外审核预算。

docs/ARCHITECTURE.md

@@ -62,3 +62,9 @@ flowchart LR
 ```
 
 The calibration sample is evidence for the routing policy, not a replacement for an independent model test set. A policy should be recalibrated when the Teacher model, target camera domain, operating conditions or annotation rules change.
+
+## Active review acquisition
+
+The optional `prioritize` stage consumes candidate CSV evidence after scanning. It groups boxes by image, resolves each image inside the dataset boundary and ranks a fixed review budget using confidence entropy, dynamically decayed class rarity and minimum perceptual distance to the already selected set.
+
+This queue is intentionally biased toward informative cases. It belongs on the remediation path, not the model-metric path: precision and recall estimates still require an independent random or stratified-random audit sample.

docs/INTERVIEW_STORY.md

@@ -20,6 +20,7 @@ Long scans are checkpointed after every committed batch. Resume validates a run
 - Atomic checkpoint/resume avoids restarting completed teacher/split work.
 - The HTML report turns audit artifacts into a reviewable delivery and GitHub demo.
 - Perceptual near-duplicate grouping uses compact hashes and BK-tree radius search to reduce repeated review without loading all pixels or comparing every pair.
+- Active review ranking uses confidence entropy, diminishing class-rarity rewards and greedy visual diversity, while keeping metric estimation on a separate unbiased sample.
 - `doctor` and `manifest.json` make environment differences visible instead of leaving CUDA and dependency drift implicit.
 
 ## Honest limitation

docs/PORTFOLIO_GUIDE.md

@@ -23,6 +23,7 @@ The difficult part was operational reliability on a long `K x N` scan. I added t
 | Evaluation and policy | `calibrate` command | Turning reviewed outcomes into precision/recall-constrained class policies |
 | Ensemble policy | `consensus` command | Independent evidence, one-to-one matching and coverage/risk tradeoffs |
 | Scalable similarity search | `cluster` command | Perceptual hashes, BK-tree radius search and conservative collision guards |
+| Active learning | `prioritize` command | Uncertainty, dynamic class balance, visual diversity and biased-sampling boundaries |
 | Software quality | package, CLI, tests and CI | Public fixture, privacy checks and release build |
 | Communication | HTML reports and architecture docs | Turning model work into reviewable project evidence |
 
@@ -35,7 +36,8 @@ The difficult part was operational reliability on a long `K x N` scan. I added t
 5. Open `examples/calibration/output/calibration.html` to explain why each class receives a different threshold.
 6. Open `examples/consensus/output/consensus.html` to show unsupported AUTO candidates being downgraded.
 7. Open `examples/near_duplicates/output/near_duplicate_report.html` to show review compression and split leakage.
-8. Run `yolo-label-recovery doctor` to show environment diagnostics.
+8. Open `examples/prioritization/output/prioritization_report.html` to show a limited-budget six-class review queue.
+9. Run `yolo-label-recovery doctor` to show environment diagnostics.
 
 This demonstration works without a GPU or private model weights. A full teacher scan remains an optional second demonstration when suitable public weights and data are available.
 
@@ -73,6 +75,10 @@ No. It makes the prediction stronger evidence. The project retains REVIEW routin
 
 Brute force grows as `O(N²)`. The `cluster` command stores compact fingerprints and uses a BK-tree for Hamming-radius candidate search, then applies aHash, aspect-ratio and low-texture luminance safeguards. It still treats every group as review evidence rather than an automatic deletion decision.
 
+**Can the active review queue be used to report model accuracy?**
+
+No. It deliberately oversamples uncertain, rare and visually diverse cases, so its acceptance rate is biased. It is a remediation queue for finding useful failures per reviewer-hour. Accuracy still requires a separate random or stratified-random audit sample.
+
 ## Claims to avoid
 
 - Do not claim the teachers eliminate all missing labels.