Robust Multi-View Video Synopsis and Panoramic Navigation
出版項
2018
說明
1 online resource (170 pages)
文字
text
無媒介
computer
成冊
online resource
附註
Source: Dissertations Abstracts International, Volume: 80-08, Section: B
Publisher info.: Dissertation/Thesis
Thesis (Ph.D.)--The Chinese University of Hong Kong (Hong Kong), 2018
Includes bibliographical references
With the rapid growth of the multi-view videos, effective and efficient organization and browsing of such contents bring about research challenges as well as opportunities. It is becoming an increasingly important issue to provide users with immersive and efficient multi-view video browsing experience. Panoramic display and video synopsis are promising methods for solving this issue. However, existing large-view panoramic display methods are usually based on computational intensive 3D reconstruction, and multiple synopsis outputs are difficult for users to comprehend. To tackle these problems, in this thesis, we focus our research on robust and efficient approaches in street view interpolation, dynamic video stitching for panoramic display, and compact multi-view video synopsis, which can be widely applied for multi-view surveillance videos, large-scale displays, and mobile cameras applications. Wide-baseline street view interpolation is useful but very challenging. Existing approaches either rely on heavyweight 3D reconstruction or computationally intensive deep networks. We present a lightweight and efficient approach that utilizes homography computing and refining operators to estimate piecewise smooth homographies between input views. We combine homography fitting and homography propagation based on reliable and unreliable superpixel discrimination. Our framework dramatically increases the accuracy and robustness of the estimated homographies. We integrate the concepts of homography and mesh warping and propose a novel homography-constrained warping formulation which enforces smoothness between neighboring homographies by utilizing the first-order continuity of the warped mesh. This further eliminates small artifacts of overlapping, stretching, etc. The proposed method improves the state of the art and demonstrates that homography computation suffices for multi-view interpolation. Our experiments on city and rural datasets validate the efficiency and effectiveness of our approach. Video stitching produces a panoramic video with a large field of viewing, which can essentially enhance the immersive viewing experience. However, videos captured by hand-held mobile cameras usually contain heavy shakiness and large parallax, which are challenging to stitch. We propose a novel approach of video stitching and stabilization for videos captured by mobile devices. The main component of our method is a unified video stitching and stabilization optimization that computes stitching and stabilization simultaneously, we can obtain the best stitching and stabilization results relative to each other without any bias to one of them. To make the optimization robust, we propose a method to identify the background of input videos. This allows us to apply our optimization in background regions only, which is the key to handle large parallax problem. We further propose a method to distinguish between right and false matches and encapsulate the false match elimination scheme and our optimization into a loop, to prevent the optimization from being affected by bad feature matches. Experiments on a diverse of examples show that our results are much better than (challenging cases) or at least on par with (simple cases) the results of previous approaches. Multi-view video synopsis aims at generating a brief representative video from multiple long inputs. We propose to solve the problem by joint object-shifting and camera view-switching. We synchronize the input videos and group the same object in different videos together. Then we shift the groups of objects along the time axis to obtain multiple synopsis videos with condensed activities. After that, our system automatically selects the most appropriate frame from one of the synopsis videos at each time, and all the selected frames make up our final synopsis result. We construct a simultaneous object-shifting and view-switching optimization framework to obtain the best object shifting result and view selection result with respect to each other. We also propose an alternative optimization strategy composed of graph cuts and dynamic programming to solve the unified optimization. Our experiments demonstrate the effectiveness and user convenience of our multi-video synopsis method. In the future work, we will work on more complex videos stitching to deal with large close-up moving objects, which occupy most of the video frame and interfere with the accurate estimating of camera path. We will support better user interface to control the multi-view synopsis browsing, for example, the users can generate a video synopsis with a different preference for different classes of objects
Electronic reproduction. Ann Arbor, Mich. : ProQuest, 2020
