Utility for finding the best threshold?

[ir2718]

Hi,

I would like to use an ONNX optimized version of a model trained using your library. However, for the use case of one-shot classification I need to determine a threshold for classification. Are there any utility functions which can calculate the optimal threshold based on a validation set, and a metric that I want to optimize for?

[KevinMusgrave]

No there isn't a utility function for that. I guess you mean optimal distance for some k-nn based classification?

[ir2718]

Yes, that's exactly what I mean. Ideally, it would take in a model, distance, splits to evaluate, and metric to optimize for. Would you be open to PRs for this? Ofcourse, if you think this would be a good addition to the library.

[KevinMusgrave]

A PR would be great, assuming there isn't already a library that does this. There are probably other people looking for something similar, so I think it would be a good addition.