|
1 | | -<div align="center"> |
2 | | - <img src="resources/ai4rs-logo.png" width="800"/> |
3 | | -</div> |
| 1 | +# Real-Time Oriented Object Detection Transformer in Remote Sensing Images (TGRS 2026) |
4 | 2 |
|
| 3 | +[IEEE TGRS Link](https://ieeexplore.ieee.org/document/11424629) |
5 | 4 |
|
6 | 5 |
|
7 | | -<div align="center"> |
8 | 6 |
|
9 | | -[📘使用文档](https://mmrotate.readthedocs.io/zh_CN/1.x/) | |
10 | | -[🛠️安装教程](https://mmrotate.readthedocs.io/zh_CN/1.x/get_started.html) | |
11 | | -[👀模型库](https://mmrotate.readthedocs.io/zh_CN/1.x/model_zoo.html) | |
| 7 | +## Abstract |
12 | 8 |
|
| 9 | +Recent real-time detection transformers have gained popularity due to their simplicity and efficiency. However, these detectors do not explicitly model object rotation, especially in remote sensing imagery where objects appear at arbitrary angles, leading to challenges in angle representation, matching cost, and training stability. In this paper, **we propose a real-time oriented object detection transformer, the first real-time end-to-end oriented object detector to the best of our knowledge**, that addresses the above issues. Specifically, angle distribution refinement is proposed to reformulate angle regression as an iterative refinement of probability distributions, thereby capturing the uncertainty of object rotation and providing a more fine-grained angle representation. Then, we incorporate a Chamfer distance cost into bipartite matching, measuring box distance via vertex sets, enabling more accurate geometric alignment and eliminating ambiguous matches. Moreover, we propose oriented contrastive denoising to stabilize training and analyze four noise modes. We observe that a ground truth can be assigned to different index queries across different decoder layers, and analyze this issue using the proposed instability metric. We design a series of model variants and experiments to validate the proposed method. |
13 | 10 |
|
14 | | -[📘Documentation](https://mmrotate.readthedocs.io/en/1.x/) | |
15 | | -[🛠️Installation](https://mmrotate.readthedocs.io/en/1.x/get_started.html) | |
16 | | -[👀Model Zoo](https://mmrotate.readthedocs.io/en/1.x/model_zoo.html) |
| 11 | +Code is available at https://github.com/wokaikaixinxin/ai4rs |
17 | 12 |
|
18 | | -</div> |
19 | | - |
20 | | - |
21 | | - |
22 | | - |
23 | | -## Introduction |
24 | | - |
25 | | - |
26 | | -We hope to integrate remote sensing related work based on **MMLab**, especially **MMDetection** and **MMRotate**. |
27 | | - |
28 | | - |
29 | | -## Model Zoo |
30 | | - |
31 | | -<details open> |
32 | | -<summary><b>Real Time </b></summary> |
33 | | - |
34 | | -| | | | | |
35 | | -| :---: | :---: | :---: | :---: | |
36 | | -|[Rotated YOLOX (arXiv 2021)](./projects/rotated_yolox/README.md) |[RTMDet (arXiv 2022)](configs/rotated_rtmdet/README.md) | [Rotated YOLOMS (TPAMI 2025)](./projects/rotated_yoloms/README.md) | [RTDETR (CVPR' 2024)](./projects/rtdetr/README.md) | |
37 | | - |
38 | | -</details> |
39 | | - |
40 | | - |
41 | | -<details open> |
42 | | -<summary><b>Oriented Object Detection - Architecture </b></summary> |
43 | | - |
44 | | -| | | | | |
45 | | -| :---: | :---: | :---: | :---: | |
46 | | -| [Rotated RetinaNet-OBB/HBB<br>(ICCV'2017)](configs/rotated_retinanet/README.md) | [Rotated FasterRCNN-OBB<br>(TPAMI'2017)](configs/rotated_faster_rcnn/README.md) | [Rotated RepPoints-OBB<br>(ICCV'2019)](configs/rotated_reppoints/README.md) | [Rotated FCOS<br>(ICCV'2019)](configs/rotated_fcos/README.md) | |
47 | | -| [RoI Transformer<br>(CVPR'2019)](configs/roi_trans/README.md) | [Gliding Vertex<br>(TPAMI'2020)](configs/gliding_vertex/README.md) | [Rotated ATSS-OBB<br>(CVPR'2020)](configs/rotated_atss/README.md) | [R<sup>3</sup>Det<br>(AAAI'2021)](configs/r3det/README.md) | |
48 | | - | [S<sup>2</sup>A-Net<br>(TGRS'2021)](configs/s2anet/README.md) | [ReDet<br>(CVPR'2021)](configs/redet/README.md) | [Beyond Bounding-Box<br>(CVPR'2021)](configs/cfa/README.md) | [Oriented R-CNN<br>(ICCV'2021)](configs/oriented_rcnn/README.md) | |
49 | | -| [Rotated YOLOX <br>(arXiv 2021)](./projects/rotated_yolox/README.md) | [Rotated Deformable DETR <br> (ICLR'2021)](./projects/rotated_deformable_detr/README.md) |[SASM<br>(AAAI'2022)](configs/sasm_reppoints/README.md) | [Oriented RepPoints<br>(CVPR'2022)](configs/oriented_reppoints/README.md) | |
50 | | -| [RTMDet<br>(arXiv 2022)](configs/rotated_rtmdet/README.md) |[Rotated DiffusionDet<br>(ICCV'2023)](./projects/rotated_DiffusionDet/README.md) | [OrientedFormer<br>(TGRS' 2024)](projects/OrientedFormer/README.md)| [RTDETR<br> (CVPR' 2024)](./projects/rtdetr/README.md) | |
51 | | -|[ReDiffDet base<br>(CVPR'2025)](./projects/GSDet_baseline/README_ReDiffDet_baseline.md)| [GSDet base<br>(IJCAI'2025)](./projects/GSDet_baseline/README_GSDet_baseline.md)| [Rotated YOLOMS<br>(TPAMI 2025)](./projects/rotated_yoloms/README.md) | [MessDet<br>(ICCV'2025)](./projects/MessDet/README.md) | |
52 | | -</details> |
53 | | - |
54 | | - |
55 | | -<details open> |
56 | | -<summary><b>Oriented Object Detection - Loss</b></summary> |
57 | | - |
58 | | -| | | | | |
59 | | -| :---: | :---: | :---: | :---: | |
60 | | -| [GWD<br>(ICML'2021)](configs/gwd/README.md) | [KLD<br>(NeurIPS'2021)](configs/kld/README.md) | [KFIoU<br>(ICLR'2023)](configs/kfiou/README.md) | | |
61 | | -</details> |
62 | | - |
63 | | -<details open> |
64 | | -<summary><b>Oriented Object Detection - Coder</b></summary> |
65 | | - |
66 | | -| | | | | | |
67 | | -| :---: | :---: | :---: | :---: | :---: | |
68 | | -| [CSL<br>(ECCV'2020)](configs/csl/README.md) | [Oriented R-CNN<br>(ICCV'2021)](configs/oriented_rcnn/README.md) | [PSC<br>(CVPR'2023)](configs/psc/README.md) | [ACM<br>(CVPR'2024)](./projects/ACM/README.md) |[GauCho<br>(CVPR'2025)](projects/GauCho/README.md) | |
69 | | -</details> |
70 | | - |
71 | | - |
72 | | -<details open> |
73 | | -<summary><b>Oriented Object Detection - Backbone</b></summary> |
74 | | - |
75 | | -| | | | | | |
76 | | -| :---: | :---: | :---: | :---: | :---: | |
77 | | -| [ConvNeXt<br>(CVPR'2022)](./configs/convnext/README.md)| [LSKNet<br>(ICCV'2023)](projects/LSKNet/README.md) | [ARC<br>(ICCV'2023)](./projects/ARC/README.md) | [PKINet<br>(CVPR'2024)](./projects/PKINet/README.md) | [SARDet 100K<br>(Nips'2024)](./projects/SARDet_100K/README.md) | |
78 | | -| [GRA<br>(ECCV'2024)](./projects/GRA/README.md) |[Strip R-CNN<br>(Arxiv'2025)](./projects/Strip_RCNN/README.md) | [LEGNet<br>(ICCVW'2025)](./projects/LEGNet/README.md) | [LWGANet<br>(Arxiv'2025)](./projects/LWGANet/README.md) | | |
79 | | -</details> |
80 | | - |
81 | | - |
82 | | -<details open> |
83 | | -<summary><b>Oriented Object Detection - Weakly Supervise</b></summary> |
84 | | - |
85 | | -| | | | | | |
86 | | -| :---: | :---: | :---: | :---: | :---: | |
87 | | -| [H2RBox<br>(ICLR'2023)](configs/h2rbox/README.md) | [H2RBox-v2<br>(Nips'2023)](configs/h2rbox_v2/README.md) | [Point2Rbox<br>(CVPR'2024)](./projects/Point2Rbox/README.md) | [Point2Rbox-v2<br>(CVPR'2025)](./projects/Point2Rbox_v2/README.md)| [WhollyWOOD<br>(TPAMI'2025)](./projects/WhollyWOOD/README.md) | |
88 | | -</details> |
89 | | - |
90 | | - |
91 | | -<details open> |
92 | | -<summary><b>Oriented Object Detection - Semi Supervise</b></summary> |
93 | | - |
94 | | -Coming soon |
95 | | -</details> |
96 | | - |
97 | | - |
98 | | -<details open> |
99 | | -<summary><b>SAR</b></summary> |
100 | | - |
101 | | -| | | | | |
102 | | -| :---: | :---: | :---: | :---: | |
103 | | -| [SARDet 100K<br>(Nips'2024)](./projects/SARDet_100K/README.md) | [RSAR <br> (CVPR'2025)](./tools/data/rsar/README.md) | | | |
104 | | - |
105 | | -<details open> |
106 | | -<summary><b>SAM</b></summary> |
107 | | - |
108 | | -| | | | | |
109 | | -| :---: | :---: | :---: | :---: | |
110 | | -| [MMRotate SAM](./projects/mmrotate-sam/README.md) | | | | |
111 | | - |
112 | | -## Installation |
113 | | - |
114 | | - |
115 | | -<!--请参考[快速入门文档](https://mmrotate.readthedocs.io/zh_CN/1.x/get_started.html)进行安装。--> |
116 | | -<!--Please read the [GET STARTED](https://mmrotate.readthedocs.io/zh_CN/1.x/get_started.html) for installation.--> |
117 | | - |
118 | | - |
119 | | -To support H2rbox_v2, point2rbox, and mamba, we use **pytorch-2.x** |
120 | | - |
121 | | - |
122 | | -<!--**第一步：** 安装Anaconda 或 Miniconda--> |
123 | | - |
124 | | -**Step 1:** Install Anaconda or Miniconda |
125 | | - |
126 | | -<!--**第二步：** 创建一个虚拟环境并且切换至该虚拟环境中--> |
127 | | - |
128 | | -**Step 2:** Create a virtual environment |
129 | | - |
130 | | -``` |
131 | | -conda create --name ai4rs python=3.10 -y |
132 | | -conda activate ai4rs |
133 | | -``` |
134 | | - |
135 | | -<!--**第三步：** 根据 [Pytorch的官方说明](https://pytorch.org/get-started/previous-versions/) 安装Pytorch, 例如：--> |
136 | | - |
137 | | -**Step 3:** Install Pytorch according to [official instructions](https://pytorch.org/get-started/previous-versions/). For example: |
138 | | - |
139 | | -``` |
140 | | -conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia |
141 | | -``` |
142 | | - |
143 | | -Verify whether pytorch supports cuda |
144 | | - |
145 | | -``` |
146 | | -python -c "import torch; print(torch.cuda.is_available())" |
147 | | -``` |
148 | | - |
149 | | - |
150 | | - |
151 | | -<!--**第四步：** 安装 MMEngine 和 MMCV, 并且我们建议使用 MIM 来完成安装--> |
152 | | - |
153 | | -**Step 4:** Install MMEngine and MMCV, and we recommend using MIM to complete the installation |
154 | | - |
155 | | - |
156 | | -``` |
157 | | -pip install -U openmim -i https://pypi.tuna.tsinghua.edu.cn/simple |
158 | | -mim install mmengine -i https://pypi.tuna.tsinghua.edu.cn/simple |
159 | | -mim install "mmcv>2.0.0rc4, <2.2.0" -i https://pypi.tuna.tsinghua.edu.cn/simple |
160 | | -``` |
161 | | - |
162 | | -<!--**第五步：** 安装 MMDetection--> |
163 | | - |
164 | | -**Step 5:** Install MMDetection |
165 | | - |
166 | | -``` |
167 | | -mim install 'mmdet>3.0.0rc6, <3.4.0' -i https://pypi.tuna.tsinghua.edu.cn/simple |
168 | | -``` |
169 | | - |
170 | | -<!--**第六步：** 安装 ai4rs--> |
171 | | - |
172 | | -**Step 6:** Install ai4rs |
173 | | - |
174 | | -``` |
175 | | -git clone https://github.com/wokaikaixinxin/ai4rs.git |
176 | | -cd ai4rs |
177 | | -pip install -v -e . -i https://pypi.tuna.tsinghua.edu.cn/simple |
178 | | -# "-v" means verbose, or more output |
179 | | -# "-e" means installing a project in editable mode, |
180 | | -# thus any local modifications made to the code will take effect without reinstallation. |
181 | | -``` |
182 | | - |
183 | | -**Step 7:** Version of NumPy |
184 | | - |
185 | | -If the NumPy version is incompatible, downgrade the NumPy version to 1.x. |
186 | | - |
187 | | -``` |
188 | | -A module that was compiled using NumPy 1.x cannot be run in |
189 | | -NumPy 2.0.1 as it may crash. To support both 1.x and 2.x |
190 | | -versions of NumPy, modules must be compiled with NumPy 2.0. |
191 | | -Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. |
192 | | -
|
193 | | -If you are a user of the module, the easiest solution will be to |
194 | | -downgrade to 'numpy<2' or try to upgrade the affected module. |
195 | | -We expect that some modules will need time to support NumPy 2. |
196 | | -``` |
197 | | - |
198 | | -``` |
199 | | -pip install "numpy<2" -i https://pypi.tuna.tsinghua.edu.cn/simple |
200 | | -``` |
201 | | - |
202 | | - |
203 | | -## Data Preparation |
204 | | - |
205 | | - |
206 | | - |
207 | | -Please refer to [data_preparation.md](tools/data/README.md) to prepare the data |
208 | | - |
209 | | - |
210 | | -| | | | | |
211 | | -| :---: | :---: | :---: | :---: | |
212 | | -| [DOTA](./tools/data/dota/README.md) | [DIOR](./tools/data/dior/README.md) | [SSDD](./tools/data/ssdd/README.md) | [HRSC](./tools/data/hrsc/README.md) | |
213 | | -| [HRSID](./tools/data/hrsid/README.md) | [SRSDD](./tools/data/srsdd/README.md) | [RSDD](./tools/data/rsdd/README.md) | [ICDAR2015](./tools/data/icdar2015/README.md) | |
214 | | -| [SARDet 100K](./tools/data/sardet_100k/README.md) | [RSAR](./tools/data/rsar/README.md) | [FAIR1M](./tools/data/fair/README.md) | | |
215 | | - |
216 | | - |
217 | | - |
218 | | -## Train |
219 | | - |
220 | | -**Single-node single-GPU** |
221 | | -``` |
222 | | -python tools/train.py config_path |
223 | | -``` |
224 | | -For example: |
225 | | -``` |
226 | | -python tools/train.py projects/GSDet_baseline/configs/GSDet_r50_b900_h2h4_h2r1_r2r1_2x_dior.py |
227 | | -``` |
228 | | - |
229 | | -**Single-node multi-GPU** |
230 | | -``` |
231 | | -bash tools/dist_train.sh config_path num_gpus |
232 | | -``` |
233 | | -For example: |
234 | | -``` |
235 | | -bash tools/dist_train.sh projects/GSDet_baseline/configs/GSDet_r50_b900_h2h4_h2r1_r2r1_2x_dior.py 2 |
236 | | -``` |
237 | | - |
238 | | -## Test |
239 | | - |
240 | | -**Single-node single-GPU** |
241 | | -``` |
242 | | -python tools/test.py config_path checkpoint_path |
243 | | -``` |
244 | | -For example: |
245 | | -``` |
246 | | -python tools/test.py configs/h2rbox_v2/h2rbox_v2-le90_r50_fpn-1x_dota.py work_dirs/h2rbox_v2-le90_r50_fpn-1x_dota-fa5ad1d2.pth |
247 | | -``` |
248 | | - |
249 | | -**Single-node multi-GPU** |
250 | | -``` |
251 | | -bash tools/dist_test.sh config_path checkpoint_path num_gpus |
252 | | -``` |
253 | | -For example: |
254 | | -``` |
255 | | -bash tools/dist_test.sh configs/h2rbox_v2/h2rbox_v2-le90_r50_fpn-1x_dota.py work_dirs/h2rbox_v2-le90_r50_fpn-1x_dota-fa5ad1d2.pth 2 |
256 | | -``` |
257 | | - |
258 | | -## Getting Started |
259 | | - |
260 | | -<!--请阅读[概述](https://mmrotate.readthedocs.io/zh_CN/1.x/get_started.html)对 Openmmlab 进行初步的了解。--> |
261 | | - |
262 | | -Please see [Overview](https://mmrotate.readthedocs.io/en/1.x/overview.html) for the general introduction of Openmmlab. |
263 | | - |
264 | | -<!--为了帮助用户更进一步了解 Openmmlab，我们准备了用户指南和进阶指南，请阅读我们的[文档](https://mmrotate.readthedocs.io/zh_CN/1.x/)：--> |
265 | | - |
266 | | -For detailed user guides and advanced guides, please refer to our [documentation](https://mmrotate.readthedocs.io/en/1.x/): |
267 | | - |
268 | | - |
269 | | -## FAQ |
270 | | - |
271 | | -<!--请参考 [FAQ](docs/en/notes/faq.md) 了解其他用户的常见问题。--> |
272 | | - |
273 | | -Please refer to [FAQ](https://github.com/open-mmlab/mmrotate/blob/1.x/docs/en/notes/faq.md) for frequently asked questions. |
274 | | - |
275 | | - |
276 | | - |
277 | | - |
278 | | -## Acknowledgement |
279 | | - |
280 | | -[OpenMMLab](https://openmmlab.com) |
281 | | - |
282 | | -[OpenMMLab platform](https://platform.openmmlab.com) |
283 | | - |
284 | | -[MMDetection](https://github.com/open-mmlab/mmdetection) |
285 | | - |
286 | | -[MMRotate](https://github.com/open-mmlab/MMRotate) |
287 | | - |
288 | | -## Citation |
289 | | - |
290 | | -<!--如果你在研究中使用了本项目的代码或者性能基准，请参考如下 bibtex 引用 ai4rs--> |
291 | | - |
292 | | -If you use this toolbox or benchmark in your research, please cite this project ai4rs |
| 13 | +# Bibtex |
293 | 14 |
|
294 | 15 | ```bibtex |
295 | | -
|
296 | | -``` |
297 | | - |
298 | | - |
299 | | - |
| 16 | +@ARTICLE{11424629, |
| 17 | + author={Ding, Zeyu and Zhou, Yong and Zhao, Jiaqi and Du, Wen-Liang and Li, Xixi and Yao, Rui and Saddik, Abdulmotaleb El}, |
| 18 | + journal={IEEE Transactions on Geoscience and Remote Sensing}, |
| 19 | + title={Real-Time Oriented Object Detection Transformer in Remote Sensing Images}, |
| 20 | + year={2026}, |
| 21 | + volume={}, |
| 22 | + number={}, |
| 23 | + pages={1-1}, |
| 24 | + keywords={Real-time systems;Transformers;Detectors;Remote sensing;Costs;Training;Accuracy;YOLO;Uncertainty;Noise reduction;Oriented object detection;detection transformer;real-time detector;remote sensing}, |
| 25 | + doi={10.1109/TGRS.2026.3671683}} |
| 26 | +``` |
0 commit comments