New mission: replace the camera’s orientation with information of the IMU. More detailed problem description:
Problem description
Given is a vision-inertial tracking system with the following four frames of reference:
- IMU: \(\cal{I}\)
- Camera: \(\cal{C}\)
- Marker: \(\cal{M}\)
- World: \(\cal{W}\)
Each frame \(\cal{F}\) has a pose (rotation/orientation and translation/position) \(T_{\cal{G} \cal{F}} = R_{\cal{G} \cal{F}} \circ \vec{t}_{\cal{G} \cal{F}}\) with respect to the other frames \(\cal{G}\). \(T_\cal{F} = R_\cal{F} \circ \vec{t}_\cal{F}\) denote the frame’s absolute pose (or, shorthand for \(T_{\cal{W} \cal{F}} = R_{\cal{W} \cal{F}} \circ \vec{t}_{\cal{W} \cal{F}}\)).
\(T\), \(R\) and \(\vec{t}\) can be represented as 4×4 matrices, but I will try to keep the discussion representation independent.
See below image for their relationships:
R_i
+-----+ T_c_i +--------+
| IMU |<-------| camera |
+-----+ +--------+
|
| T_c_m
|
v
+--------+ +-------+
| marker | | world |
+--------+ +-------+
\(T_{\cal{C} \cal{I}}\) is fixed, because frames \(c\) and \(i\) are attached to the same rigid body.
\(R_\cal{I}\) is observed by the IMU.
\(T_{\cal{C} \cal{M}}\) is observed by the camera.
Assume that the perception of \(T_{\cal{C} \cal{M}}\) is noisy, especially \(R_{\cal{C} \cal{M}}\). This implies that \(R_\cal{C}\) is noisy as well.
Find a method to update \(R_\cal{C}\) from the IMU’s data \(R_\cal{I}\). Try to keep \(\vec{t}_\cal{C}\) constant.