☎

Thesis outline and dataset requirements

After speaking with Arnoud, I’m restarting again. Focus for today: thesis outline, and getting the requirements for the dataset on paper.

Last year, I suddenly had to stop due to health issues. Things are going better now again, and I’ve got a good support system behind me to start working again.

Thesis outline

The main issue with the outline is how to present the dataset and algorithm, two connected-but-separate parts of work. Two structures are in my head:

Outline 1 Outline 2
  • Title page
  • Acknowledgements
  • Abstract
  1. Introduction
    • Casual introduction to tracking, localization and mapping
    • Casual introduction to the problem
    • Examples of the problem
    • State research questions and hypotheses
    1. Thesis outline
      • The "In section X, topic Y will be discussed"-talk
  2. Background
    1. Simultaneous localization and mapping
      • Formal introduction to SLAM
      • Formal introduction to the problem?
      • Introduce old SLAMs (refs?), sensor-fused localization [1, 2], PTAM
      • Introduce ORBSLAM, VI-ORBSLAM, ORBSLAM2
    2. Available datasets
      • Discuss datasets of (VI-)ORBSLAM(2) and other cited papers
      • Show measured quantities
    3. Calibration
      • Casual introduction to calibration
      • Formal introduction to calibration
      • Investigate different "calibration modalities"
  3. Method
    • Thorough explanation of different techniques used in (VI-)ORBSLAM(2)
    • Explain how merging them would increase metrics
  1. Datasets
    • Show that only EuRoC MAV is applicable
    • Motivate that only 1 dataset is not enough
    1. EuRoC MAV Visual Inertial Dataset
    2. Industrial Robot Arm Dataset
      1. Method/Design: Hardware/software architecture
        • Sensors used
        • Ground truth/robot arm
        • Computing/recording device
        • Clock synchronisation
      2. Method/Design: Dataset features
        • Number of sequences
        • Calibration methods used
        • Visual features: fractal ArUco markers [1, 2, 3]
        • Path features: number of loops/intersections with own path
        • Motion features: max Δacceleration, noisiness
        • More?
      3. Results/Realisation: analysis of created dataset
      4. Discussion: dataset
  2. Visual-Inertial ORBSLAM2
    1. Algorithm: discuss how VI-ORBSLAM and ORBSLAM2 are merged into this new work
    2. Experiment: test ORBSLAM, VI-ORBSLAM, ORBSLAM2, and own algorithm on above datasets
    3. Results: analysis of algorithms performance on datasets
    4. Discussion: algorithms
  1. Implementation: discuss how VI-ORBSLAM and ORBSLAM2 are merged into this new work
  2. Experiment
    1. Datasets
      • Show that only EuRoC MAV is applicable
      • Motivate that only 1 dataset is not enough
      1. EuRoC MAV Visual Inertial Dataset
      2. Industrial Robot Arm Dataset
        1. Method/Design: Hardware/software architecture
          • Sensors used
          • Ground truth/robot arm
          • Computing/recording device
          • Clock synchronisation
        2. Method/Design: Dataset features
          • Number of sequences
          • Calibration methods used
          • Visual features: fractal ArUco markers [1, 2, 3]
          • Path features: number of loops/intersections with own path
          • Motion features: max Δacceleration, noisiness
          • More?
    2. Algorithms
      • Motivate why (ORBSLAM,) VI-ORBSLAM, ORBSLAM2 and own algorithm need to be tested to give a proper comparison of own algorithm and dataset
  3. Results
    1. Dataset
      • Does created dataset tick all boxes from experiment design?
      • Any important differences between the two IMU-camera modules?
    2. Algorithms
      • Compare results from (ORBSLAM,) VI-ORBSLAM, ORBSLAM2 and own algorithm
  4. Discussion
    1. Dataset
    2. Algorithms
  1. Conclusion
    1. Dataset
    2. SLAM method
    3. Future work
  1. Appendices...
  • Bibliography

In Outline 1, the projects of dataset creation and performing experiments on the dataset are viewed as separate. A motivation for this might be that this might leave space to analyse the two projects as independent from each other as possible. Outline 2 presents the dataset creation project as „experiment design” for the algorithm evaluation. The experimental setup will be evaluated together with the outcomes. In this matter, the link between the projects is demonstrated stronger, but might come over as a bit more messy.

This should be discussed with Arnoud.

Creating the dataset

Last year’s stop was health related. While still recovering, I am looking into how to continue. The main issue right now is pain when having an „active posture” for more than (roughly, on a good day) 2 hours. This will decrease over time, but I cannot just be idle in the meantime. So, I’m looking into how to create the right circumstances that I can make this dataset, without physically harming myself.

Lab assistant

Arnoud suggested to look for a lab assistant for doing the physical work. This person should do the physical tasks, while I can remain seated for most of the time. This does mean that my lab time will be even more precious, and should be prepared even better beforehand, as costs will increase when working longer than expected (for instance, when fixing bugs). One of the possible sources of getting a lab assistant, would be to contact student initiatives like the Dutch Nao Team, the students association of Information Sciences at the UvA, via, or students from the mechatronics and robotics programme at the HvA where I work.

The lab assistant should be paid for their labor. I have sent a request for support to UWV, the national institute responsible for implementation of laws regarding disability&disease and work&education, for financial support. They are known to be slow in reply, so I have also contacted the student counsellor (studentendecaan). I expect a soon reply from the counsellors, based on prior communications.

References

  • Gabriele Bleser and Didier Stricker. Advanced tracking through efficient image processing and visual-inertial sensor fusion. In Virtual Reality Conference, pages 137–144. IEEE, 2008. [ bib | DOI ]
    
@inproceedings{bleser2008advanced,
  author = {Bleser, Gabriele and Stricker, Didier},
  booktitle = {Virtual Reality Conference},
  title = {Advanced tracking through efficient image processing and visual-inertial sensor fusion},
  year = {2008},
  pages = {137--144},
  organization = {IEEE},
  doi = {10.1109/VR.2008.4480765}
}
  • Gabriele Bleser. Towards visual-inertial SLAM for mobile augmented reality. PhD thesis, Universität Kaiserslautern, 2009. [ bib ]
    
@phdthesis{bleser2009towards,
  title = {Towards visual-inertial {SLAM} for mobile augmented reality},
  author = {Bleser, Gabriele},
  year = {2009},
  school = {Universit\"{a}t Kaiserslautern},
  url = {https://www.xsens.com/images/stories/PDF/BleserPhD2009.pdf}
}
  • Raúl Mur-Artal, José M. Martínez Montiel, and Juan Domingo Tardós. ORB-SLAM: A versatile and accurate monocular SLAM system. IEEE Transactions on Robotics, 31(5):1147–1163, October 2015. [ bib | DOI ]
    
@article{murartal2015orbslam,
  author = {Mur-Artal, Raúl and Montiel, José M. Martínez and Tardós, Juan Domingo},
  journal = {IEEE Transactions on Robotics},
  title = { {ORB-SLAM}: A Versatile and Accurate Monocular {SLAM} System},
  year = {2015},
  month = oct,
  volume = {31},
  number = {5},
  pages = {1147-1163},
  keywords = {SLAM (robots);ORB-SLAM system;feature-based monocular simultaneous localization and mapping system;survival of the fittest strategy;Simultaneous localization and mapping;Cameras;Optimization;Feature extraction;Visualization;Real-time systems;Computational modeling;Lifelong mapping;localization;monocular vision;recognition;simultaneous localization and mapping (SLAM);Lifelong mapping;localization;monocular vision;recognition;simultaneous localization and mapping (SLAM)},
  doi = {10.1109/TRO.2015.2463671},
  issn = {1552-3098}
}
  • Sergio Garrido-Jurado, Rafael Muñoz-Salinas, Francisco Madrid-Cuevas, and Rafael Medina-Carnicer. Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recognition, 51, October 2015. [ bib | DOI ]
    
@article{garrido2015generation,
  author = {Garrido-Jurado, Sergio and Muñoz-Salinas, Rafael and Madrid-Cuevas, Francisco and Medina-Carnicer, Rafael},
  year = {2015},
  month = oct,
  pages = {},
  title = {Generation of fiducial marker dictionaries using Mixed Integer Linear Programming},
  volume = {51},
  journal = {Pattern Recognition},
  doi = {10.1016/j.patcog.2015.09.023}
}
  • Raúl Mur-Artal and Juan Domingo Tardós. Visual-inertial monocular SLAM with map reuse. IEEE Robotics and Automation Letters, 2(2):796–803, April 2017. [ bib | DOI ]
    
@article{murartal2017visual,
  author = {Mur-Artal, Raúl and Tardós, Juan Domingo},
  journal = {IEEE Robotics and Automation Letters},
  title = {Visual-Inertial Monocular {SLAM} With Map Reuse},
  year = {2017},
  month = apr,
  volume = {2},
  number = {2},
  pages = {796-803},
  keywords = {cameras;robot vision;SLAM (robots);visual-inertial monocular SLAM;map reuse;visual-inertial odometry;sensor incremental motion;trajectory estimation;visual-inertial simultaneous localization;zero-drift localization;camera configuration;IMU initialization method;gyroscope;accelerometer;micro-aerial vehicle public dataset;monocular camera;Optimization;Cameras;Gravity;Accelerometers;Simultaneous localization and mapping;Tracking loops;Sensor fusion;SLAM;visual-based navigation},
  doi = {10.1109/LRA.2017.2653359},
  issn = {2377-3766}
}
  • Raúl Mur-Artal and Juan Domingo Tardós. ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Transactions on Robotics, 33(5):1255–1262, October 2017. [ bib | DOI ]
    
@article{murartal2017orbslam2,
  title = { {ORB-SLAM2}: An Open-Source {SLAM} System for Monocular, Stereo, and {RGB-D} Cameras},
  author = {Mur-Artal, Raúl and Tardós, Juan Domingo},
  journal = {IEEE Transactions on Robotics},
  year = {2017},
  month = oct,
  volume = {33},
  number = {5},
  pages = {1255-1262},
  keywords = {cameras;distance measurement;Kalman filters;mobile robots;motion estimation;path planning;robot vision;SLAM (robots);ORB-SLAM;open-source SLAM system;lightweight localization mode;map points;zero-drift localization;SLAM community;monocular cameras;stereo cameras;simultaneous localization and mapping system;RGB-D cameras;Simultaneous localization and mapping;Cameras;Optimization;Feature extraction;Tracking loops;Trajectory;Localization;mapping;RGB-D;simultaneous localization and mapping (SLAM);stereo},
  doi = {10.1109/TRO.2017.2705103},
  issn = {1552-3098}
}
  • Francisco Romero-Ramirez, Rafael Muñoz-Salinas, and Rafael Medina-Carnicer. Speeded up detection of squared fiducial markers. Image and Vision Computing, 76, June 2018. [ bib | DOI ]
    
@article{romero2018speeded,
  author = {Romero-Ramirez, Francisco and Muñoz-Salinas, Rafael and Medina-Carnicer, Rafael},
  year = {2018},
  month = jun,
  pages = {},
  title = {Speeded Up Detection of Squared Fiducial Markers},
  volume = {76},
  journal = {Image and Vision Computing},
  doi = {10.1016/j.imavis.2018.05.004}
}
  • Francisco Romero-Ramirez, Rafael Muñoz-Salinas, and Rafael Medina-Carnicer. Fractal markers: a new approach for long-range marker pose estimation under occlusion. IEEE Access, PP:1–1, November 2019. [ bib | DOI ]
    
@article{romero2019fractal,
  author = {Romero-Ramirez, Francisco and Muñoz-Salinas, Rafael and Medina-Carnicer, Rafael},
  year = {2019},
  month = nov,
  pages = {1-1},
  title = {Fractal Markers: a new approach for long-range marker pose estimation under occlusion},
  volume = {PP},
  journal = {IEEE Access},
  doi = {10.1109/ACCESS.2019.2951204}
}