Pubs Projects Tools Join Team About Home

LBurst Learned Burst Feature Finder

Feature matching at night: A commercial drone (top-left) captures imagery at millilux conditions at night that is too noisy for conventional 3D reconstruction in low light (top-right). This is because of the high rate of spurious features offered by conventional features like scale invariant feature transform (SIFT) (blue) and less reliable matches offered by learned features like repeatable and reliable detector and descriptor (R2D2) (red) on noisy images. The proposed LBurst yields higher-quality feature matches from reliable regions of the images (yellow) for reconstruction.
  • We introduce a learning-based robotic burst feature finder, LBurst, a joint feature detector and descriptor that finds learned features with well defined scale within a low-light robotic burst
  • We enable robots to detect noise-tolerant features in low light by employing consistent motion within bursts and leveraging uncorrelated noise between them
  • We demonstrate overall improved 3D reconstruction from drone images in low-light (millilux) conditions, outperforming single image-based feature extractors and robotic burst feature finder in low-SNR scenes.
This work aims to identify learning-based features in low-light robotic bursts that enhance low-light reconstruction, benefiting a wide range of drone applications, including nighttime drone delivery and understanding nocturnal behaviors of animals.

Publications

•  A. Ravendran, M. Bryson, and D. G. Dansereau, “LBurst: Learning-based robotic burst feature extraction for 3D reconstruction in low light,” 2024. Preprint available here.

Themes

Downloads

The code is on GitHub here.

The drone imagery dataset used for validation is available here (11 GB download).

We capture robotic bursts along drone trajectories using DJI Phantom Pro 4 and DJI Mini Pro 3, each contains 15 bursts of 7 images each.

Citing

If you find this work useful please cite
@article{ravendran2024lburst,
  title={{LBurst}: Learning-based Robotic Burst Feature Extraction for {3D} Reconstruction in Low Light},
  author={Ravendran, Ahalya and Bryson, Mitch and Dansereau, Donald G.},
  journal={arXiv preprint arXiv:2410.23522},
  year={2024}
}