Pubs Projects Tools Join Team About Home

Projects: Novel Cameras

Segment Anything in Light Fields for Real-Time Applications via Constrained Prompting

  • An effective light field segmentation method
  • Combines epipolar constraints with the rich semantics learned by the pretrained SAM 2 foundation model for cross-view mask matching
  • Produces results of the same quality as the Segment Anything 2 (SAM 2) video tracking baseline, while being 7 times faster
  • Can be inferenced in real-time for autonomous driving problems such as object pose tracking

TaCOS: Task-Specific Camera Optimization with Simulation

  • An end-to-end camera design method that co-designs cameras with perception tasks
  • We combine derivative-free and gradient-based optimizers and support continuous, discrete, and categorical parameters
  • A camera simulation including virtual environments and a physics-based noise model
  • Key step in simplifying the process of designing cameras for robots

Inherently Privacy-Preserving Robotic Vision

  • Robotic vision without capturing images or allowing image reconstruction
  • We propose guidelines and demonstrate localisation in simulation
  • Call to action for advancing inherently private vision systems
  • Try the reconstruction challenge!

NOCaL: Calibration-Free Odometry

  • Automatically interpreting new cameras by jointly learning novel view sythesis, odometry, and a camera model
  • A hypernetwork allows training with a wealth of existing cameras and datasets
  • A semi-supervised light field network adapts to newly introduced cameras
  • This work is a key step to automated integration of emerging camera technologies

Learning to See with Sparse Light Field Video Cameras

  • Unsupervised learning of odometry and depth from sparse 4D light fields
  • Encoding sparse LFs for consumption by 2D CNNs for odometry and shape estimation
  • Toward unsupervised interpretation of general LF cameras and new imaging devices

Light Stage Object Classifier

  • Fast classification of visually similar objects using multiplexed illumination
  • Using light stage capture and rendering to drive optimization of multiplexing codes
  • Outperforms naive and conventional multiplexing patterns in accuracy and speed

Wide-FOV Monocentric LF Camera

  • 138° 72 Mpix LF panoramas through a single, spherical lens
  • Spherical parameterization compatible with planar light fields
  • World's first single-lens wide-FOV LF camera

Spinning Omnidirectional Stereo Camera

Mirrored light field video camera adapter

  • 3D printed base + laser-cut acrylic mirrors
  • Creates a virtual array of cameras that measures a light field
  • Optimization scheme finds best mirror array for a specific application
  • D. Tsai, D. G. Dansereau, S. Martin, and P. Corke, “Mirrored light field video camera adapter,” Queensland University of Technology, Dec. 2016. Available here.

Exploiting parallax in panoramic capture to construct light fields

  • We show that parallax, usually considered a nuisance, can be exploited to build light fields
  • We turn an Ocular Robotics camera pointing system into an adaptive light field camera
  • We demonstrate light field capture, refocusing and low-light image enhancement
  • Best paper award, ACRA 2014
  • D. G. Dansereau, D. Wood, S. Montabone, and S. B. Williams, “Exploiting parallax in panoramic capture to construct light fields,” in Australasian Conference on Robotics and Automation (ACRA), 2014. Available here.