Skip to content

AC: respect calculate_average in multi_accuracy and f1-score - #4064

Open
syahra712 wants to merge 1 commit into
openvinotoolkit:masterfrom
syahra712:fix-multilabel-calculate-average
Open

syahra712 wants to merge 1 commit into
openvinotoolkit:masterfrom
syahra712:fix-multilabel-calculate-average

Conversation

@syahra712

Copy link
Copy Markdown

Summary

The multi_accuracy and f1-score metrics in multilabel_recognition.py mishandle calculate_average: False.

  • f1-score builds its per-class values from MultiLabelPrecision and MultiLabelRecall, which receive the same config. When calculate_average is False, those two return only the per-class values. F1Score.evaluate still does precisions[:-1] / recalls[:-1], so the last class is silently dropped. With 3 labels, the metric returns 2 values while meta['names'] lists 3.
  • multi_accuracy ignores the option and always appends the average. meta['names'] has no average entry in that case, so the values and names don't match.

The fix slices the precision/recall results to the number of labels in f1-score, and returns only the per-class values from multi_accuracy when the average is disabled. This is how multi_precision and multi_recall already behave.

Tests

New tests/test_multilabel_metrics.py covers both metrics with and without the average, on a 3-label example with hand-computed values. On master, the two calculate_average: False tests fail: f1-score returns [1.0, 0.667] and multi_accuracy returns [1.0, 0.5, 0.5, 0.667]. With this change all 4 pass. The changed module adds no new pylint findings (.pylintrc, max line 120).

With calculate_average: False, f1-score still dropped the last element of
the precision/recall results as if it were the average, so the last class
was missing from the output. multi_accuracy ignored the option and always
appended the average, which did not match its meta names.
@workflow-lab

Copy link
Copy Markdown

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants