Solving Instance Detection from an Open-World Perspective

Qianqian Shen¹ · Yunhan Zhao² · Nahyun Kwon³ · Jeeeun Kim³ · Yanan Li⁴ · Shu Kong^5,6

¹Zhejiang University ²UC Irvine ³Texas A&M University ⁴Zhejiang Lab ⁵University of Macau ⁶Institute of Collaborative

Movitated by the InsDet's open-world nature, we exploit diverse open data and foundation models to solve InsDet in the open world (IDOW). To better adapt FM for instance-level feature matching, we introduce distractor sampling to sample patches of random background images as universal negative data to all object instances, and novel-view synthesisgenerate more visual references not only training but for testing. Our IDOW outperforms prior works by >10 AP in both conventional and novel instance detection settings.

Code

The project is built on detectron2, SAM, GroundingDINO, and DINOv2.

The Jupyter notebooks files demonstrate our IDOW on HR-InsDet dataset and RoboTools dataset.

Citation

If you find our project useful, please consider citing:

@inproceedings{shen2025solving,
  title={Solving Instance Detection from an Open-World Perspective},
  author={Shen, Qianqian and Zhao, Yunhan and Kwon, Nahyun and Kim, Jeeeun and Li, Yanan and Kong, Shu},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2025}