about dice evaluation

[hu2274898]

Thank you for your response.☺️ I have an additional question regarding the computation of the Dice score.

I first convert the discrete segmentation of the template into a one-hot encoding. This one-hot segmentation is then warped using the predicted deformation field via trilinear interpolation. After warping, I apply an argmax over the channel dimension to convert the warped one-hot map back into a discrete label segmentation. Finally, I compute the Dice score between this discrete warped template segmentation and the discrete ground-truth segmentation of the target image.

Concretely, the evaluation code looks like this:

dice_metric = DiceMetric(include_background=False, reduction="mean", get_not_nans=False)
svf_warped_atlas_segs = bilinear(atlas_seg, pos_deform_field)
(svf_warped_atlas_segs and inf_src_segs are one-hot segmentation maps)
dice_metric(
y_pred=svf_warped_atlas_segs.argmax(dim=1, keepdim=True),
y=inf_src_segs.argmax(dim=1, keepdim=True),
)
dice_avg = dice_metric.aggregate().item()
dice_metric.reset()

I also noticed that this “argmax + hard Dice” approach tends to give higher Dice scores than using a soft Dice computed directly on the continuous warped one-hot maps. I would like to confirm whether this evaluation protocol is consistent with yours, and whether you also evaluate Dice on discrete labels obtained after argmax.😊

[neel-dey]

Hi Xiaotian,

Given the discrete integer-valued atlas segmentation, I just warped it with nearest neighbors interpolation.

What you're describing (converting to one-hot, linearly warping, and then discretizing) is typical if you're also estimating segmentation labels alongside the atlases -- maybe you're thinking of that instead? We didn't estimate atlas segmentations in our paper and just got the atlas segmentations by majority voting (Predict-HD) or by DrawEM (dHCP).

If you are predicting segmentation labels as well, then "argmax + hard Dice" would be the way to do it :)

[hu2274898]

Hi! Yes, you’re absolutely right😃 — I am indeed trying to estimate both the segmentation labels and the atlas at the same time. I also noticed that you obtained the segmentation maps using majority voting (which I guess means assigning each voxel the label that appears most frequently across subjects, right?).
But I’m still a bit confused: if you only warped the atlas segmentation using nearest-neighbor interpolation, how exactly did you compute the Dice score? 🥲
Your explanation really means a lot to me as a beginner. Thank you so much!😊

[neel-dey]

(which I guess means assigning each voxel the label that appears most frequently across subjects, right?).

Warp all the segmentations corresponding to the training images to the atlas space. Then, for each voxel, assign it the label that most frequently appears at that voxel. For conditional templates, this gets a bit nuanced; see Appendix B.1.1 for details.

BTW, you can now probably just use synthseg from freesurfer (which came out in the time since that paper) to segment your templates.

But I’m still a bit confused: if you only warped the atlas segmentation using nearest-neighbor interpolation, how exactly did you compute the Dice score? 🥲

Just regular hard dice (implemented via sklearn's f1_score function with 'macro' averaging and excluding the background label). As you know, soft dice is just a differentiable approximation of Dice. For eval, we don't need differentiability, so we use the exact "hard" Dice.