Multistate Constrained Invariant Kalman Filter for Rolling Shutter Camera and Imu Calibration

In this paper, we propose a novel algorithm for joint calibration of a rolling shutter camera and inertial sensors. Specifically, the proposed method performs online calibration to estimate the camera intrinsics, the readout time, the extrinsics and the time offset between the camera and the IMU, as well as IMU's bias.

By employing a multi-state constrained invariant Kalman filter, the proposed method processes the measurements of IMU and rolling shutter camera in a decoupled way, which greatly reduces the computational complexity of Jacobian matrices and further improves the robustness and accuracy. Experiments on public datasets demonstrate that the proposed approach can accurately calibrate the considered parameters and perform better than state-of-the-art methods in terms of accuracy and robustness.