Structure-aware Video Style Transfer with Map art

Thi-Ngoc-Hanh Le, Ya-Hsuan Chen, and Tong-Yee Lee, Senior Member, IEEE

National Cheng-Kung University

Home Video Results Image Results Comparisons Demo video Failure result Source code

Abstract

Changing the style of an image/video while preserving its content is a crucial criterion to access a new neural style transfer algorithm. However, it is very challenging to transfer a new map art style to a certain video in which \say{content} comprises a map background and animation objects. In this paper, we present a novel comprehensive system that solves the problems in transferring map art style in such video. Our system takes as input an arbitrary video, a map image, and an off-the-shelf map art image. It then generates an artistic video without damaging the functionality of the map and the consistency in details. To solve this challenge, we propose a novel network, Map Art Video Network (MAViNet), the tailored objective functions, and a rich training set with rich animation contents and different map structures. We have evaluated our method on various challenging cases and many comparisons with those of the related works. Our method substantially outperforms state-of-the-art methods in terms of visual quality and meets the mentioned criteria in this research domain.

Keywords -- style transfer video, coherence, map art, CNN, MAViNet

BibTeX

@article{lestructure, 

                      title={Structure-aware Video Style Transfer with Map Art}, 

                      author={Le, Thi-Ngoc-Hanh and Chen, Ya-Hsuan and Lee, Tong-Yee}, 

                      journal={ACM Transactions on Multimedia Computing, Communications and Applications}, 

                      publisher={ACM New York, NY}

                    }

Grant

This work was supported in part by the National Science and Technology Council, (under Nos. 111-2221-E-006-112-MY3 and 110-2221-E-006-135-MY3) Taiwan, Republic of China.