NTXent loss when more than 2 positive samples?

[jeanfelixM]

I need clarification about how the loss is computed in the case we have more than 2 positive samples ? I imagine that the reducer will somehow mean every possible loss, but not sure how it works.

For example, if I have some features F in a tensor size (latent_size,6) and the labels for the pairs are labels = (1,1,1,2,2,2), it means each "1" is positive to the other and each 1,2 are negative pairs, and each "2" are positives with one another.
so NTXentLoss(temperature=0.20)(F,labels) will compute the loss for every possible set of positive pairs and average it ? something like
loss for (0,1) + loss for (0,2) + (1,2) (with 3,4,5 as the negatives)
+
(3,4) + (3,5) + (4,5) (with 0,1,2 as the negatives)

[KevinMusgrave]

I'm busy right now but I'll try to get back to you later today. In the meantime, let me know if the section "How exactly is the NTXentLoss computed?" helps in the docs: https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#ntxentloss

[KevinMusgrave]

It'll be the average of:

• loss for (0,1), with 3,4,5 as negatives
• loss for (0,2), with 3,4,5 as negatives
• loss for (1,0), with 3,4,5 as negatives
• loss for (1,2), with 3,4,5 as negatives
• loss for (2,0), with 3,4,5 as negatives
• loss for (2,1), with 3,4,5 as negatives
• loss for (3,4), with 0,1,2 as negatives
• loss for (3,5), with 0,1,2 as negatives
• loss for (4,3), with 0,1,2 as negatives
• loss for (4,5), with 0,1,2 as negatives
• loss for (5,3), with 0,1,2 as negatives
• loss for (5,4), with 0,1,2 as negatives

[jeanfelixM]

Ok it is clear now, thanks a lot!