Hi, I have been trying to finetune VGGT on my custom (object-centric) dataset, and the finetuned model generally predicts poses and depth quite well, but it seems to prefer predicting very low confidence (barely above than 1.0) for views where the object is far away. Currently I am freezing the depth and conf heads and only finetuning parts of the backbone to preserve the quality of the dense predictions.
Since the conf-weighted depth loss uses absolute depth error, it seems like farther regions will naturally receive lower confidence when the (relative) error is comparable, simply because their absolute depth errors are larger.
Is this interpretation correct? If so, do you recommend any way to counteract this effect during finetuning? For example, would you suggest disabling the confidence-weighted term, using a relative/log-depth loss, or something like that? Thank you!
Hi, I have been trying to finetune VGGT on my custom (object-centric) dataset, and the finetuned model generally predicts poses and depth quite well, but it seems to prefer predicting very low confidence (barely above than 1.0) for views where the object is far away. Currently I am freezing the depth and conf heads and only finetuning parts of the backbone to preserve the quality of the dense predictions.
Since the conf-weighted depth loss uses absolute depth error, it seems like farther regions will naturally receive lower confidence when the (relative) error is comparable, simply because their absolute depth errors are larger.
Is this interpretation correct? If so, do you recommend any way to counteract this effect during finetuning? For example, would you suggest disabling the confidence-weighted term, using a relative/log-depth loss, or something like that? Thank you!