LBurst Learned Burst Feature Finder
- We introduce a learning-based robotic burst feature finder, LBurst, a joint feature detector and descriptor that finds learned features with well defined scale within a low-light robotic burst
- We enable robots to detect noise-tolerant features in low light by employing consistent motion within bursts and leveraging uncorrelated noise between them
- We demonstrate overall improved 3D reconstruction from drone images in low-light (millilux) conditions, outperforming single image-based feature extractors and robotic burst feature finder in low-SNR scenes.
Publications
• A. Ravendran, M. Bryson, and D. G. Dansereau, “LBurst: Learning-based robotic burst feature extraction for 3D reconstruction in low light,” 2024. Preprint available here.
Downloads
Gallery
(click to enlarge)
During inference, we only pass a single burst to find multi-scale features.
We compute ground truth by operating R2D2 on original high SNR images (black) and demonstrate improved overall matching performance.
Our method converges for 60% of diverse viewpoint scenes, outperforming R2D2 (12.5%) with accurate pose estimates on the generated subset of HPatches robotic burst dataset with varying viewpoints.
By selecting features with top-scored confidence for both detection and descriptor, we find high quality true features and avoid spurious features for reconstruction.
Our feature finder avoids most of the spurious features and regions with low contrast, successfully locating high quality true features within the burst.
State-of-the-art feature extractors, both classical and learning-based methods demonstrate poor performance on captured single images.
We operate SIFT, SuperPoint and R2D2 on burst-merged images which benefit from improved SNR. We also directly employ robotic bursts with BuFF and our learning-based feature finder.
Overall, our proposed approach, LBurst reconstructs the scene with all input images, with improved 3D points and complete structure.
Learned burst features, LBurst, also show improved performance compared to physics-based BuFF feature finder.
Citing
@article{ravendran2024lburst, title={{LBurst}: Learning-based Robotic Burst Feature Extraction for {3D} Reconstruction in Low Light}, author={Ravendran, Ahalya and Bryson, Mitch and Dansereau, Donald G.}, journal={arXiv preprint arXiv:2410.23522}, year={2024} }