metric in object detection

[TLDX-XIONG]

Have you validated the model performance on object detection datasets such as COCO? I tested siglip2-base-patch16-512 and fg-clip2-base and found that the metrics of fg-clip2 are lower than those of siglip2.

[binwang777]

Hello, could you share the scores from your testing? Here are the results I just tested for siglip2-base-patch16-512 on COCO. Since siglip2 cannot output semantically aligned dense features, I used a fair evaluation strategy: first cropping the image from the original based on the boxes, then resizing it to 512 for testing.

I also tested the performance of the FG-CLIP 2 base version on COCO. Our method supports dense feature output, and on the text side we use box_head. The results are as follows:

[TLDX-XIONG]

Oh, you might have misunderstood my meaning. I'm referring to its use in downstream object detection tasks (using mAP as the metric), rather than the BBox Classification already shown in the paper.

[binwang777]

Oh, you might have misunderstood my meaning. I'm referring to its use in downstream object detection tasks (using mAP as the metric), rather than the BBox Classification already shown in the paper.

I understand what you mean. We have not tried experiments where we replace the backbone in traditional detection frameworks. We only experimented with OVD and open segmentation.
However, based on my experience, I think that when training detection with FG-CLIP2, it’s better to avoid using too small a resolution. Specifically, before feeding images into the image_processor, you should resize them based on the short side to 512 or to whatever resolution you need.