Evaluation

[ahyunSeo]

Hello,

I'm trying to reproduce Instance segmentation in VOC2012 or COCO.
Do you have any plan for releasing evaluation code for Instance and Semantic Segmentation?
I want to know how to post-process the result after meanshift training.
All I see is train and visualization scripts throughout this repo.
Also could you provide your result from the released version of your code?
I saw somewhere that it is different from the ones on the paper.

Thank you and Happy new year!

Best regards,
Ahyun

[aimerykong]

Hi, Ahyun,

Thanks for your interest! I'm not going to release the post-processing&evaluation code including ranking. Basically what I did is to collect morphological features of the proposal candidate, like ratio of width to height, ratio of object area to the outer square area, calibrated absolute height, etc. This part is ad-hoc and not a novel component of our work. But if you want to rank and evaluation, I'd suggest turning to other rankers like the part trained with Faster RCNN or mask-RCNN. These are better scoring functions for ranking the proposals. Hope this helps.

Regards,
Shu

[ahyunSeo]

@aimerykong

Thank you for your reply.
I'm sorry to bother you but I want to clarify some points. I'll really appreciate if you correct me.

This is what I got:

1. Train instance embedding network, use the final result after the Meanshift loop as instance proposals(clustering results).
2. Train deeplab v3 with pairwise inst embedding loss and the original k-way cross-entropy loss at the same time for semantic segmentation.
3. Transfer the result of 2 to 1.
4. Rescore and ranking between proposals

Could you tell me more about part 3 and 4? Sorry that I'm not familiar with semantic segmentation task.

For part 3, in the paper it says

To label detected instances with semantic labels, we use the semantic segmentation model described above to generate labels and then use a simple voting strategy to transfer these predictions to the instance proposals.

Did you mean you collect the class prediction of all the pixels inside the instance proposal and then choose the major one?

Best regards,
Ahyun

[aimerykong]

Hi, Ahyun,

Yes, you are right. I build a voting method to combine predicted class labels and the instance proposals. The idea is basically about counting the major class labels within a proposal.

As for re-ranking, as described before, I combine morphological features of the proposal mask and train a regressor to IoU (or classification). But I'd suggest you use other trained rankers like MaskRCNN or FasterRCNN, as these are better at this task.

Regards,
Shu