☎

Calibration between the IMU and camera is essential, but there are multiple methods to achieve this. A few references are listed.

Toby and I found several articles on visual-inertial calibration. Horn is the „original one”, apparently coming up with the need for this system. The work by Lobo and Dias extends this, but needs to place one of two sensors at the center of a turntable. Hol et al. ‚08 proposes a method where this extra tool is not needed. A less dense description can be found in Hol.

Today I also found an article of Hol et al. ‚06 about sensor fusion for AR, between inertial and visual sensors. They fuse the readings with an EKF. It is unclear if they include the accelerometer and gyroscope readings in the state vector, so they can predict them. In their conclusion section, they suggest for further investigation in ‚design of accurate self-calibration methods, including uncertainty measures for the computer vision measurements’. As my work won’t be really original (it’s mainly a different localization module for the camera), I might have to look more into calibration?

Hol et al. ‚06 also uses FREE-D, ‚a conventional AR-tracking system requiring a heavy infrastructure (lots of markers on the ceiling)’ which can be used as a ground truth system. I haven’t investigated this too much yet, but it sounds like it might be outdated (1997) by now.

References

  • Berthold K.P. Horn. Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A, 4(4):629–642, 1987. [ bib | DOI ]
    
@article{horn1987closed,
  title = {Closed-form solution of absolute orientation using unit quaternions},
  author = {Horn, Berthold K.P.},
  journal = {Journal of the Optical Society of America A},
  volume = {4},
  number = {4},
  pages = {629--642},
  year = {1987},
  publisher = {Optical Society of America},
  doi = {10.1364/JOSAA.4.000629}
}
  • G.A. Thomas, J. Jin, T. Niblett, and C. Urquhart. A versatile camera position measurement system for virtual reality TV production. IET Conference Proceedings, pages 284–289(5), January 1997. [ bib | DOI ]
    
@article{thomas1997free-d,
  title = {A versatile camera position measurement system for virtual reality {TV} production},
  author = {Thomas, G.A. and Jin, J. and Niblett, T. and Urquhart, C.},
  journal = {IET Conference Proceedings},
  year = {1997},
  month = {January},
  pages = {284-289(5)},
  publisher = {Institution of Engineering and Technology},
  language = {English},
  doi = {10.1049/cp:19971284}
}
  • J.D. Hol, T.B. Schon, F. Gustafsson, and P.J. Slycke. Sensor fusion for augmented reality. In Information Fusion, 2006 9th International Conference on, pages 1–6, July 2006. [ bib | DOI ]
    
@inproceedings{hol2006sensor,
  author = {Hol, J.D. and Schon, T.B. and Gustafsson, F. and Slycke, P.J.},
  booktitle = {Information Fusion, 2006 9th International Conference on},
  title = {Sensor Fusion for Augmented Reality},
  year = {2006},
  month = jul,
  pages = {1-6},
  keywords = {Kalman filters;augmented reality;computer vision;motion estimation;sensor fusion;AR;Kalman filtering;augmented reality;camera;computer vision;motion estimation;nonlinear estimation problem;orientation estimation;position estimation;sensor fusion;Acoustic sensors;Application software;Augmented reality;Cameras;Computer vision;Delay;Layout;Magnetic sensors;Sensor fusion;TV;Augmented Reality;Computer Vision;Inertial Navigation;Kalman Filter;Sensor fusion},
  doi = {10.1109/ICIF.2006.301604}
}
  • Jorge Lobo and Jorge Dias. Relative pose calibration between visual and inertial sensors. The International Journal of Robotics Research, 26(6):561–575, June 2007. [ bib | .pdf | DOI ]
    
@article{lobo2007relative,
  title = {Relative pose calibration between visual and inertial sensors},
  author = {Lobo, Jorge and Dias, Jorge},
  journal = {The International Journal of Robotics Research},
  volume = {26},
  number = {6},
  pages = {561--575},
  month = jun,
  year = {2007},
  publisher = {SAGE Publications},
  url = {http://ijr.sagepub.com/content/26/6/561.full.pdf},
  doi = {10.1177/0278364907079276}
}
  • Jeroen D. Hol, Thomas B. Schön, and Fredrik Gustafsson. A new algorithm for calibrating a combined camera and imu sensor unit. In Proceedings of the 10th International Conference on Control, Automation, Robotics and Vision, pages 1857–1862. IEEE, August 2008. [ bib | .pdf | DOI ]
    
@inproceedings{hol2008new,
  title = {A new algorithm for calibrating a combined camera and IMU sensor unit},
  author = {Hol, Jeroen D. and Sch\"on, Thomas B. and Gustafsson, Fredrik},
  booktitle = {Proceedings of the 10th International Conference on Control, Automation, Robotics and Vision},
  pages = {1857--1862},
  year = {2008},
  day = {25},
  month = aug,
  organization = {IEEE},
  url = {http://www.diva-portal.org/smash/get/diva2:316995/FULLTEXT01.pdf},
  doi = {10.1109/ICARCV.2008.4795810}
}
  • Jeroen D. Hol. Sensor fusion and calibration of inertial sensors, vision, ultrawideband and GPS. PhD thesis, Linköping University, Institute of Technology, 2011. [ bib | .pdf ]
    
@phdthesis{hol2011sensor,
  title = {Sensor fusion and calibration of inertial sensors, vision, ultrawideband and GPS},
  author = {Hol, Jeroen D.},
  year = {2011},
  school = {Link{\"o}ping University, Institute of Technology},
  url = {http://user.it.uu.se/~thosc112/team/hol2011.pdf}
}