Book chapter · Conference paper
Video Frame Interpolation via Cyclic Fine-Tuning and Asymmetric Reverse Flow
The objective in video frame interpolation is to predict additional in-between frames in a video while retaining natural motion and good visual quality. In this work, we use a convolutional neural network (CNN) that takes two frames as input and predicts two optical flows with pixelwise weights. The flows are from an unknown in-between frame to the input frames.
The input frames are warped with the predicted flows, multiplied by the predicted weights, and added to form the in-between frame. We also propose a new strategy to improve the performance of video frame interpolation models: we reconstruct the original frames using the learned model by reusing the predicted frames as input for the model.
This is used during inference to fine-tune the model so that it predicts the best possible frames. Our model outperforms the publicly available state-of-the-art methods on multiple datasets.
Language: | English |
---|---|
Publisher: | Springer |
Year: | 2019 |
Pages: | 311-323 |
Proceedings: | 2019 Scandinavian Conference on Image Analysis |
Series: | Lecture Notes in Computer Science |
Journal subtitle: | 21st Scandinavian Conference, Scia 2019, Norrköping, Sweden, June 11–13, 2019, Proceedings |
ISBN: | 3030202046 , 3030202054 , 9783030202040 and 9783030202057 |
ISSN: | 16113349 and 03029743 |
Types: | Book chapter and Conference paper |
DOI: | 10.1007/978-3-030-20205-7_26 |
ORCIDs: | Hannemose, Morten , Jensen, Janus Nørtoft , Einarsson, Gudmundur , Dahl, Anders Bjorholm , Frisvad, Jeppe Revall , 0000-0002-8307-7411 , 0000-0002-7765-1747 , 0000-0002-6096-3648 and 0000-0002-5698-5983 |