ORCID
https://orcid.org/0000-0002-5784-5093
Date of Award
Winter 12-2021
Degree Name
Master of Science (MS)
Degree Type
Dissertation
Abstract
Video frame interpolation aims to synthesis a non-exists intermediate frame guided by two successive frames. Recently, some work shows excellent results in learning continuous representation of temporally-varying 3D objects with neural field (NF), which could be used for interpolating the original video. However, these methods require several videos from different viewing angles, the information of camera poses, learning for each specific scene, and achieving sub-optimal results for video frame interpolation. To this end, we propose a new learning neural field representation-based model, Neural Video Representation (NeVR) to learn a continuous representation of videos for high-quality video interpolation. Unlike the traditional video interpolation algorithm, which directly synthesis the whole intermediate frame, our model aims to map the temporal-spatial coordinates of the queried pixels to the corresponding pixel value of the interpolated frame. Additionally, NeVR takes a latent feature code associated with queried pixels as input to enhance the image quality. That feature code contains the information of local implicit features and bilateral motion of the input frames and is obtained by a jointly trained encoder. Our experiments show that the proposed algorithm outperforms the state-of-the-art methods in video frame interpolation on several benchmark datasets.
Language
English (en)
Chair
Ulugbek Kamilov
Committee Members
Joseph A. O’Sullivan, Umberto Villa