Hi, thank you for releasing this implementation.
While reading init_embeddings in
models/mmseg/models/sam/image_encoder.py, I noticed a potential issue related to tensor layout and reshaping.
If the input x has shape (N, H, W, C) (NHWC), then:
x.permute(0, 3, 1, 2) is not assigned back to x, so it is only used to infer (N, C, H, W) numerically.
The subsequent reshape(N, C, H*W) is therefore applied to the original NHWC tensor.
In this case, the reshape is performed under NHWC memory order, which breaks the intended semantic that each token corresponds to one spatial location (h, w) with a C-dimensional feature vector.
As a result, the final (N, H*W, C) tensor does not represent “one pixel → one token” in the usual ViT/SAM sense, even though the operation is numerically valid and does not raise an error.
Hi, thank you for releasing this implementation.
While reading init_embeddings in
models/mmseg/models/sam/image_encoder.py, I noticed a potential issue related to tensor layout and reshaping.
If the input x has shape (N, H, W, C) (NHWC), then:
x.permute(0, 3, 1, 2) is not assigned back to x, so it is only used to infer (N, C, H, W) numerically.
The subsequent reshape(N, C, H*W) is therefore applied to the original NHWC tensor.
In this case, the reshape is performed under NHWC memory order, which breaks the intended semantic that each token corresponds to one spatial location (h, w) with a C-dimensional feature vector.
As a result, the final (N, H*W, C) tensor does not represent “one pixel → one token” in the usual ViT/SAM sense, even though the operation is numerically valid and does not raise an error.