tony_fig_award_whiteProf. Tong-Yee (Tony) Lee is a Chair Professor in the Computer Science and Information Engineering Department of National Cheng-Kung University, and the head of the Computer Graphics Group/Visual System Laboratory. His research interests are in multimedia retargeting, computer graphics, visualization, and medical applications. His email: tonylee@mail.ncku.edu.tw and the research summary of his major publications is listed below. (2019/01/23)

 

 

 

Multimedia Retargeting

In this research, we have studied different methods/optimization to solve multimedia retargeting problems from image/video to stereo images/video. The image is divided into either uniformed grids or irregular patches. For video resizing, we consider either in-between frame-coherence or continuous volumetric space coherence. In addition, we also consider summarizing objects (i.e. delete similar objects) to resize images. To determine similar objects’ importance, we measure and predict visual importance of similar objects.

cropwarp_teaser.bmp

(1)   Fan Tang, Weiming Dong, Yiping Meng,Chongyang Ma, Fuzhang Wu, Xinrui Li, Tong-Yee Lee " Image Retargetability," IEEE Transactions on Multimedia (TMM)(major revision),2018/11

(2)   Yan Kong, Weiming Dong, Xing Mei, Chongyang Ma, Tong-Yee Lee, Siwei Lyu, Feiyue Huang, and Xiaopeng Zhang, "Measuring and Predicting Visual Importance of Similar Objects," IEEE Transactions on Visualization and Computer Graphics (TVCG), No. 12 - Dec., vol. 22, 2016, pp. 2564-2578, (SCI/EI)

(3)   Weiming Dong, Fuzhang Wu, Yan Kong, Xing Mei, Tong-Yee Lee, Xiaopeng Zhang ,"Image Retargeting by Content-Aware Synthesis," IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 22, No. 2, 2016, pp.1088~1101 (SCI/EI)

(4)   Weiming Dong, Ning Zhou, Tong-Yee Lee, Fuzhang Wu, Yan Kong and Xiaopeng Zhang, “Summarization-Based Image Resizing by Intelligent Object Carving,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 20, No. 1, Jan. Issue, 2014, pp. 111-124 (SCI/EI)

(5)   Shih-Syun Lin, Chao-hung Lin Yu-Hsuan Kuo; Tong-Yee Lee, “Consistent Volumetric Warping Using Floating Boundaries for Stereoscopic Video Retargeting,” IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), vol. 5, No. 26, 2016, pp. 801-813 (SCI/EI)

(6)   Shih-Syun Lin, Chao-Hung Lin, Shu-Huai Chang, Tong-Yee Lee, “Object-Coherence Warping for Stereoscopic Image Retargeting,” IEEE Transactions on Circuits and Systems for Video Technology (TCSVT),Vol. 24, Issue No. 5, May 2014, pp. 759-768 (SCI/EI)

(7)   Shih-Syun Lin, Chao-Hung Lin, I-Cheng Yeh, Shu-Huai Chang, Chih-Kuo Yeh, Tong-Yee Lee, “Content-aware Video Retargeting Using Object-preserving Warping,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 19, No. 10, Oct. 2013, pp. 1677-1688. (SCI/EI)

(8)   Shih-Syun Lin, I-Cheng Yeh, Chao-Hung Lin, Tong-Yee Lee, “Patch-based Image Warping for Content-Aware Retargeting,” IEEE Transactions on Multimedia (TMM), Vol. 15, No. 2, 2013, Feb., pp. 359 -368 (SCI/EI)

(9)   YuShuen Wang, Jen-Hung Hsiao, Olga Sorkine, Tong-Yee Lee, “Scalable and Coherent Video  Resizing with Per-Frame Optimization,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH 2011), Vol. 30, No.4, Aug. 2011 (SCI/EI)

(10)   Huisi Wu, Yushuen Wang, Kun-Chuan Feng, Tien-Tsin Wong, Tong-Yee Lee, Pheng-Ann Heng, “Resizing by Summarization”, Vol. 29, No. 6, Article No. 159, ACM Transactions on Graphics (TOG) (also in Proceedings of SIGGRAPH Asia 2010), Seoul, 2010 (SCI/EI)

(11)   YuShuen Wang, Hui-Chih Lin, Olga Sorkine, Tong-Yee Lee, “Motion-based Video Retargeting with Optimized Crop-and-Warp,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH 2010), Vol. 29, No.3, Article No. 90, pp. 1-9, July 2010 (SCI/EI)

(12)   YuShuen Wang, Hongbo Fu, Olga Sorkine, Tong-Yee Lee, and Hans-Peter Seidel, “Motion-Aware Temporal Coherence for Video Resizing,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH Asia 2009), Vol. 28, No.5, Article: 127, Dec. 2009, (SCI/EI)

(13)   Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, Tong-Yee Lee, “Optimized Scale-and-Stretch for Image Resizing,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH Asia 2008) Vol. 27, No.5, Dec. 2008 (SCI/EI)

 

Surface Modeling/Texture Mapping/Geometrric Steganography/AGIM

In this research, we have studied different methods/optimization to solve/reconstruct surface modeling problems such as skeleton extraction, geometric blending for stylized models, 3D surface fitting/morphing using templates and high-relief surface reconstruction from a single image. In addition, we also solve constrained texture mapping problems on 3D models and discuss how to hide multiple layers from 7-13 layers steganography information on 3D models and also how to store 3D geometrical information on 2D adaptive geometry image (AGIM) files.

 

http://graphics.csie.ncku.edu.tw/Tony/record_new.files/image013.gif

(1)   Yunfei Fu, Hongchuan Yu, Chih-Kuo Yeh, Jianjun Zhang, Tong-Yee Lee "High Relief from Brush Painting" IEEE Transactions on Visualization and Computer Graphics (TVCG) (accepted) 2018/06(SCI/EI)

(2)   Yi-Jheng Huang, Wen-Chieh Lin, I-Cheng Yeh, Tong-Yee Lee "Geometric and Textural Blending Approach to Design Stylized Models," IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 24, no. 2, pp. pp. 1114-1126, Feb., 2018 (SCI/EI)

(3)   Chih-Kuo Yeh, Shi-Yang Huang, Pradeep Kumar Jayaraman, Chi-Wing Fu, Tong-Yee Lee "Interactive High-Relief Reconstruction for Organic and Double-sided Objects from a Photo," IEEE Transactions on Visualization and Computer Graphics (TVCG),  No. 7, July, Vol. 23, 2017, pp. 1796-1808 (SCI/EI)

(4)   Hongchuan YU, Tong-Yee Lee, I-Cheng Yeh, Xiaosong Yang, Wenxi Li and Jian J. Zhang, “RBF based Reparameterization Method for Constrained Texture Mapping,” IEEE Transactions on Visualization and Computer Graphics (TVCG), July 2012 (vol. 18 no. 7) pp. 1115-1124 (SCI/EI)

(5)   I-Cheng Yeh, Chao-Hung Lin, Olga Sorkine, Tong-Yee Lee, “Template-based 3D Model Fitting Using Dual-domain Relaxation,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol., 17, No. 8, pp. 1178-1190, 2011 (SCI/EI)

(6)   Min-Wen Chao, Chao-Hung Lin, Cheng-Wei Yu and Tong-Yee Lee, “A High Capacity 3D Steganography Algorithm,” IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 15 no. 2,pp. 274-284, 2009.

(7)   Yu-Shuen Wang, Tong-Yee Lee, “Curve Skeleton Extraction Using Non-Linear Least Squares Optimization,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 14, No. 4. 2008, pp. 926-936

(8)   Oscar Kin-Chung Au, Chiew-Lan Tai, Hung-Kuo Chu, Daniel Cohen-Or, Tong-Yee Lee, “SkeletonExtraction by Mesh Contraction,” ACM Transaction on Graphics (TOG) (also in Proceedings of ACM SIGGRAPH 2008) Vol. 27, No.3, Article: 44, August 2008 (SCI/EI)

(9)   Zhi-Yuan Yao, Tong-Yee Lee, “Adaptive Geometry Image,” IEEE Transactions on Visualization and Computer Graphics (TVCG), July/Aug, Vol. 14, No. 4, 2008, pp. 948-960 (SCI/EI)

(10)   Tong-Yee Lee, Shao-Wei Yen I-Cheng Yeh, “Texture Mapping with Hard Constraints Using Warping Scheme,” IEEE Transactions on Visualization and Computer Graphics (TVCG), March/April, Vol. 14,No. 2, pp. 382-395,2008 (SCI/EI)

 

Non-photo-realistic rendering (NPR) & Illusion

In this research, we have developed stylized NPR, art-related research such as Escher Arts and Camouflage Images, and illusion related methods/application such as Self-Animating Images. In addition, some of our approaches can be potentially applied to create high-quality aesthetic QR code and still and animating “Captcha” images/video such as “Emerging Images”. We also parallelize sort-last polygon rendering on 2D mesh multi-computers in my earlier rendering research. Recently, we also decompose intrinsic images using illumination optimization from a single image and also create depth of field rendering effect using multilayer-neighborhood optimization.

http://graphics.csie.ncku.edu.tw/SAI/index.files/image002.jpg

(1)   Benxuan Zhang, Bin Sheng, Ping Li, and Tong-Yee Lee "Depth of Field Rendering Using Multilayer-Neighborhood Optimization" IEEE Transactions on Visualization and Computer Graphics (TVCG) (accepted) 2019/01 (SCI/EI)

(2)   Bin Sheng, Ping Li, Yuxi Jin, Ping Tan, and Tong-Yee Lee, "Intrinsic Images Decomposition Using Illumination Optimization from a Single Image" IEEE Transactions on Visualization and Computer Graphics (TVCG) (accepted) (SCI/EI)

(3)   Shih-Syun Lin, Charles C. Morace, Chao-Hung Lin, Li-Fong Hsu, Tong-Yee Lee "Generation of Escher Arts with Dual Perception" IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 24, no. 2, pp. 1103-1113, Feb.,2018, (SCI/EI)

(4)   Ying-Miao Kuo, Hung-Kuo Chu, Ming-Te Chi, Ruen-rone Lee, Tong-Yee Lee, "Ambiguous Figure-Ground Images," IEEE Transactions on Visualization and Computer Graphics (TVCG),  Issue No. 05 - May, vol. 23, 2017, pp. 1534-1545, 2017 (SCI/EI)

(5)   Shih-Syun Lin, Min-Chun Hu, Chien-Han Lee, Tong-Yee Lee "Efficient QR Code Beautification with High Quality Visual Content," IEEE Transactions on Multimedia (TMM), Sept., 1515-1524, 2015 (SCI/EI)

(6)   Zhi-Yuan Yao, Ming-Te Chi, Tong-Yee Lee, Tao Ju, “Region-Based Line Field Design Using Harmonic Functions,” Vol. 18, No. 6, June Issue, IEEE Transactions on Visualization and Computer Graphics (TVCG), pp. 902-913, 2012 (SCI/EI)

(7)   Hung-Kuo Chu, Wei-Hsin Hsu, Niloy J. Mitra, Daniel Cohen-Or, Tien-Tsin Wong, Tong-Yee Lee, “Camouflage Images,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH 2010), Vol. 29, No.3, Article No. 51, pp. 1-8, July 2010 (SCI/EI)

(8)   Niloy J. Mitra, Hung-Kuo Chu, Tong-Yee Lee, Lior Wolf, Hezy Yeshurun, Daniel Cohen-Or, “Emerging Images, ” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH Asia 2009), Vol. 28, No.5, Dec. 2009, (SCI/EI)

(9)   Ming-Te Chi, Tong-Yee Lee, Yingge Qu, Tien-Tsin Wong, “Self-Animating Images: Illusory Motion Using Repeated Asymmetric Patterns,” ACM Transaction on Graphics (TOG) (also in Proceedings of ACM SIGGRAPH 2008) Vol. 27, No.3, August 2008 (SCI/EI)

(10)   Chung-Ren Yan, Ming-Te Chi, Tong-Yee Lee, Wen-Chieh Lin, ”Stylized Rendering Using Samples of a Painted Image,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 14, No. 2, pp. 468-480, 2008(SCI/EI)

(11)   Ming-Te Chi, Tong-Yee Lee, "Stylized and Abstract Painterly Rendering System Using a Multi-Scale Segmented Sphere Hierarchy", VOL. 12, NO. 1, JANUARY/FEBRUARY, pp. 61-72, 2006, IEEE Transactions on Visualization and Computer Graphics (TVCG) (SCI/EI).

(12)   Ping-Hsien Lin, Tong-Yee Lee, “Camera- Sampling Field and Its Applications,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 10, No. 3, May/June, 2004, pp. 241-251. (SCI/E).

(13)   Tong-Yee Lee, C.S Raghavendra, J.B. Nicholas, ``Image Composition Schemes for Sort-Last Polygon Rendering on 2-D Mesh Multicomputers,'' Volume 2, Number 3 of the IEEE Transactions on Visualization and Computer Graphics (TVCG), pp. 202-217, 1996. (SCI/EI)

 

Motion Overview + Animation Generation

We have developed 3D camera systems to generate good views (single v.s. multiple camera paths) of 3D animation (single character v.s. multiple characters), designed good searching method for 3D animation modelling and some methods to generate 2.5D cartoon hair animation and propose double-sided 2.5D carton animation, and also proposed physics-based 3D biped character animation. In addition, we also proposed methods to stabilize video sequence, i.e., avoid shaking effect.

Motion Overview of Human Actions

(1)   Chih-Kuo Yeh, Pradeep Kumar Jayaraman, Xiaopei Liu, Chi-Wing Fu, Tong-Yee Lee "2.5D Cartoon Hair Modeling and Manipulation" IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 21, No.1, 2015, pp. 304-314 (SCI/EI)

(2)   YuShuen Wang, Feng Liu, Pu-Sheng Hsu, Tong-Yee Lee, “Spatially and Temporally Optimized Video Stabilization,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 19, No. 8, pp.1354-61, 2013. (SCI/EI)

(3)   Chih-Kuo Yeh, Peng Song, Peng-Yen Lin, Chi-Wing Fu, Chao-Hung Lin, TongYee Lee, “Double-sided 2.5D Graphics,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 19, No. 2, pp. 223-235, 2013 (SCI/EI)

(4)   I-Cheng Yeh, Wen-Chieh Lin, Tong-Yee Lee, Hsin-Ju Han, Jehee Lee, Manmyung Kim, “Social-Event-Driven Camera Control for Multi-Character Animations,” IEEE Transactions on Visualization and Computer Graphics (TVCG) 2012, Sept Issue. (SCI/EI)

(5)   Min-Wen Chao, Chao-Hung Lin, Jackie Assa, Tong-Yee Lee, "Human Motion Retrieval from Hand-drawn Sketch ," IEEE Transactions on Visualization and Computer Graphics (TVCG)Vol. 18, No. 5, May Issue, pp. 729-740, 2012. (SCI/EI)

(6)   Yao-Yang Tsai, Wen-Chieh Lin, Kuangyou B. Cheng, Jehee Lee, Tong-Yee Lee, “Real-time Physics-based 3D Biped Character Animation Using an Inverted Pendulum Model ” IEEE Transaction Visualization and Computer Graphics (TVCG)March/April Issue, Vol. 16, No. 2, pp. 325-337,2010 (SCI/EI)

(7)   Jackie Assa, Daniel Cohen-Or, I-Cheng Yeh, Tong-Yee Lee, “Motion Overview of Human Actions,” ACM Transaction on Graphics (TOG) (also in Proceedings of SIGGRAPH Asia 2008) Vol. 27, No.5, Dec. 2008 (SCI/EI)

 

Focus+Context/Word Cloud/Map Art Visualization

We have designed visualization techniques for focus+context, word cloud and map art visualization. In addition, we also extend our method to handle multi-focus application to visualize coherent time-varying graphs.

http://graphics.csie.ncku.edu.tw/Tony/record_new.files/image038.gif

(1)   Ming-Te Chi, Shih-Syun Lin, Shiang-Yi Chen, Chao-Hung Lin, Tong-Yee Lee, "Morphable Word Clouds for Time-varying Text Data Visualization," IEEE Transactions on Visualization and Computer Graphics (TVCG), No.12 - Dec. ,vol.21, pp: 1415-1426, 2015 (SCI/EI) (also presented at ACM SIGGRAPH Asia, Kobe, Japan, Nov. 2-5. 2015) (SCI/EI)

(2)   Shih-Syun Lin ,Chao-Hung Lin, Yan-Jhang Hu, Tong-Yee Lee, “Drawing Road Networks with Mental Maps,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 20, No.9, pp. 1241-1252, Sept. Issue, 2014. (SCI/EI) (also presented at ACM SIGGRAPH Asia 2014, Shenzhen, China. Dec. 2014) (SCI/EI)

(3)   Kun-Chuan Feng, Chaoli Wang, Han-Wei Shen, Tong-Yee Lee, “Coherent Time-Varying Graph Drawing with Multi-Focus+Context Interaction,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 18, No. 8, August 2012, pp. 1330-1342(SCI/EI)

(4)   YuShuen Wang, Chaoli Wang, Tong-Yee Lee, Kwan-Liu Ma, “Feature-Preserving Volume Data Reduction and Focus+Context Visualization,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 17, No.2, Feb. Issue, pp. 171-181, 2011 (SCI/EI)

(5)   Yu-Shuen Wang, Tong-Yee Lee, Chiew-Lan Tai, “Focus+Context Visualization with Distortion Minimization,” IEEE Transactions on Visualization and Computer Graphics (TVCG) (also in Proceedings of IEEE Visualization), Volume 14, Number 6, November, 2008 (SCI/EI)

 

Morphing, Shape Interpolation and its Medical Applications

We design 3D metamorphosis/shape interpolation techniques for solving different conditions, fixed and different poses and also try to generate smooth deformation with different surface simplification. In addition, we apply morphing technique to 2D/3D medical image interpolation and 3D surgical simulation.  

livingstone_elephant_resize.jpg

(1)             Hung-Kuo Chu, Tong-Yee Lee, “Multi-resolution Mean Shift Clustering Algorithm for Shape Interpolation,” IEEE Transactions on Visualization and Computer Graphics (TVCG), vol. 15, No. 5, pp. 853-866, 2009(SCI/EI)

(2)             Tong-Yee Lee, Chao-Hung Lin, Yu-Shuen Wang, Tai-Guang Chen," Animation Key-frame Extraction and Simplification Using Deformation Analysis," IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), April, Vol. 18, No. 4, 2008, pp. 478-486  (SCI/EI)

(3)             Tong-Yee Lee, Chung-Ren Yen, Ming-Te Chi, “Stylized Rendering for Anatomic Visualization,” Special Issue - Anatomic Rendering and VisualizationIEEE Computing in Science & Engineering, vol. 9, no. 1, pp. 13-19, Jan/Feb, 2007 (SCI/EI)

(4)             Chao-Hung Lin, Tong-Yee Lee, “Metamorphosis of 3D Polyhedral Models Using Progressive Connectivity Transformations,” IEEE Transactions on Visualization and Computer Graphics (TVCG), Jan./Feb. Issue, Vol. 11, No.1, pp. 2-12, 2005(SCI/EI)

(5)             Tong-Yee Lee, Po-Hua Huang, “Fast and Intuitive Metamorphosis of 3D Polyhedra Models Using SMCC Mesh Merging Scheme, “ IEEE Transactions on Visualization and Computer Graphics (TVCG), Vol. 9, No. 1, Jan-March, 2003, pp 85-98(SCI/EI)

(6)             Tong-Yee Lee, Chao-Hung Lin, “Feature-guided Shape-based Image Interpolation”, IEEE Transactions on Medical Imaging (IEEE TMI), Dec. 2002 Vol. 21, No. 12, pp. 1479-1489. (SCI/EI)

(7)             Tong-Yee Lee, Wen-Hsui Wang," Morphology-based Three-dimensional Interpolation," IEEE Transactions on Medical Imaging (IEEE TMI) Vol. 19, Issue 7, pp. 711-721, 2000 (SCI/EI)

(8)             Tong-Yee Lee, Chao-Hung Lin, Han-Ying Lin, "Computer-aided Prototype System for Nose Surgery (Rhinoplasty)," IEEE Transactions on Information Technology in Biomedicine (IEEE TITB), Vol. 5, Issue 4, pp. 271-278, 2001(SCI/EI)

(9)             Tong-Yee Lee, Yung-Nein Sun, Y. C. Lin, Leewen Lin, C.N. Lee, " Three Dimensional Facial Model Reconstruction & Plastic Surgery Simulation," IEEE Transactions on Information Technology in Biomedicine (IEEE TITB), Vol. 3, No. 3, pp. 214-220, 1999. (SCI/EI)

(10)         Tong-Yee Lee, Ping-Hsien Lin, Chao-Hung Lin, Yung-Nein Sun, Xi-Zhang Lin, "Interactive 3D Virtual Colonoscopy System," IEEE Transactions on Information Technology in Biomedicine (IEEE TITB), Vol 03, No. 02, pp. 139-150, 1999. (SCI/EI)