Chung-Yi Weng

About Me

Hi, I'm Chung-Yi, a PhD student from GRAIL of CSE, UW, working with Prof. Brian Curless and Prof. Ira Kemelmacher in pursuing the beauty of computer science. Before going to UW, I was Technical Lead in Magic Technology Team of CyberLink, working on computer vision, computational photography and machine learning. The joyful work experience leads me back to academic world to further explore these super exciting subjects.

I received my bachelor and master degree from CSIE, NTU, Taiwan, advised by Prof. Ja-Ling Wu, who inspired me very much in multimedia and social network analysis.

I love coding, thinking, dreaming, and admire for the power of imagination. I believe the best way to uncover the secrets behind enigmatic phenomenon is through simple intuition and creative imagination!

Work Experience

CyberLink, one of the biggest software company in Taiwan, offers the most easy-to-use multimedia editing and playback software to worldwide users. Here I served as Technical Lead and was very fortunate to cooperate with a group of brilliant guys, developing 13 projects and filing 9 patents related to computer vision and machine learning. The research experience was so wonderful that I decided to study abroad with the dream of being one of the best scientists of these fields in the world!

Research

RoleNet
RoleNet: Movie Analysis from the Perspective of Social Networks
Chung-Yi Weng, Wei-Ta Chu, Ja-Ling Wu
IEEE TRANSACTIONS ON MULTIMEDIA 2009 - Master Theis
Inspired by idea of social network analysis, we propose a novel way to analyze movie videos from the perspective of social networks. The relationship between characters in a movie is elaborately described as a network, called RoleNet. Based on RoletNet, further network analysis is performed to extract semantic information in the movie, including leading roles, macro/micro community structures, and hidden story lines and story units.
Paper
Tiling Slideshow
Tiling Slideshow
Jun-Cheng Chen, Wei-Ta Chu, Jin-Hau Kuo, Chung-Yi Weng, Ja-Ling Wu
ACM MULTIMEDIA 2006 - Best Paper Award
Tiling Slideshow is a brave new photo displaying method to arrange photos in a tile-like manner, coordinating with the pace of background music. Photo clustering is applied based on the relationship between photos; music beat detection is perfomed in order to trigger the progress of slideshow; photo importance is computed to help ROI determination. Finally, the layout organization is formulated as a constrianed optimization problem to make sure the most satisfied composition results could be produced.
Paper Project Website

Projects

become brad pitt
Become Brad Pitt
Final Project of Computer Vision (UW CSE 576)
In this project, we designed a real-time face reenactment system which enables users to control celebrities' head motion and facial expression by moving their own head. We also implemented a transition effect which smoothly morphs one face to another when the user changes to control another celebrity. Our system involves implementing a high-speed tracking module, a puppetry module to control facial expression and a animation module to morph the faces. Experiments show that our system can reenact the head motion and facial expression of target celebrities (e.q. Brad Pitt) very well in real time.
Report Demo Video
hyperlapse
Hyperlapse Video Creation
Final Project of Computer Graphics (UW CSE 557)
Given a video (usually taken with hand-held devices) and a specified target speed, we want to create a hyperlapse video which "speeds up" the video to the target speed. The result should be robust to camera motions and meet the target speed.
Project Website
Magic Selection
Magic Selection
Industry Project in CyberLink
Magic Selection is an intelligent object seletion tool, offering an effective way to assist users to complete the difficult task. Through dragging a brush like cursor, users specify parts of desired object region, and the system would automatically exapnd the selection region to the most possible object region, improving laborious work of the notorious task.
Smart Lasso
Smart Lasso
Industry Project in CyberLink
Smart Lasso is another easy-operating object selection tool. Users only need to move a cursor along the object boundary roughly, and the system would automatically adjust the drawn boundary line to fit the most possible object boundary, making the difficult task become easy to operate.
Soft Matting
Soft Matting
Industry Project in CyberLink
Soft Matting has an interactive interface to help users perform image matting and is especially useful to refine selected object boundary, which usually has hair-like structure or is relatively transparent. The desired modified region is labelled roughly by users, triggering the system to perform image matting. Alpha value of each pixel, which is important information if we edit or apply effect on selected object, in the labeled region would be estimated, improving the selection result significantly.
Perfect Group Shot
Perfect Group Shot
Industry Project in CyberLink
Perfect Group Shot helps people take the greatest group photo. By capturing photos continuously, the system gets several facial expressions of each person in these photos. Next, it would select the best facial expression (ex. smile, no blink) for each person and seamlessly composite these faces into the final image. Manual mode, where users could select the desired facial expression of each person, is also supported.
Sequence Shot
Sequence Shot
Industry Project in CyberLink
Sequence Shot provides an effective way to create time-lapse like effect in a image. Several objects in different images would be seamlessly composited into single image to produce continuous and dynamic illusion. The adopted kernel algorithm behind the technique is similar with Perfect Group Shot.
Panorama
Panorama
Industry Project in CyberLink
Panorama is a technique to stitch several images into one panorama image, assisting people to get more wide-angle view in capturing the world. In our implementation, the system only uses pixel data as reference information and is robust to panning, zooming, and moving object. Furthermore, the system allows input image sequence is out of order because the order would be corrected during the analysis.
Holmes
Holmes ‐ (Object Tracking by Rectangle or Point)
Industry Project in CyberLink
Holmes could track objects in videos, where two types of input object representation, either rectangle or point, are allowed. With Holmes, users don‘t need to edit a video frame by frame when adding materials, like dialogue boxes, on a target object, because the object would be tracked automatically and the added materials would be adjusted accordingly.
Cupid
Cupid ‐ (Object Tracking by Object Boundary)
Industry Project in CyberLink
Cupid, a more powerful object tracking engine, estimates precise object boundary during the tracking. Initially, users select the target object region through Magic Selection (a selection technique mentioned before), and then the system would track corresponding object region in the following frames. An object mask is generated during the tracking, making more wide varieties of video editing possible, like applying filters or changing color on the specified region without altering other background objects.
Robust Face Detection
Robust Face Detection
Industry Project in CyberLink
Triggered by wide varieties of applications, we decided to develop a face detector on our own, which should be rapid and robust to various rotating angles (including rotation-in-plane, and rotation-off-plane). In our implementation, hierarchical concept is imposed on the structure of conventional training algorithm to accelerate the processing, and a sparse and high dimensional feature descriptor is used in the kernel classfier to increase discrimination power. The evaluation results shows the detector could compete with Apple iPhoto and Google Picasa in both quality and speed.
Facial Landmark Localization
Facial Landmark Localization
Industry Project in CyberLink
The technique of facial landmark localization is developed because of its usefulness in recognizing faces and adjusting facial features on a face. Based on idea of machine learning, an optimal iteration process is trained by selecting the most discriminative features determined by large amounts of faces tagged with ground truth, making the localization is rapid and robust to various environments.
Face Login
Face Login
Industry Project in CyberLink
Face Login applies face recognition to helping people login a system without keying in passwords. It determines validness of login users by recognizing their faces based on pre-trained faces given by the users. The recognition process is quick, precise, and robust to various lighting conditions.
FaceME
FaceME ‐ (Face Clustering/Recognition in Photos/Videos)
Industry Project in CyberLink
FaceME analyzes photos/videos to group similar faces into clusters, assisting people to effectively tag face names and search photos/videos where user-specified people appear. Suggested name is offered by recognizing untagged faces in the group with tagged faces to help users accelerate the tagging process, making the notorious task could be completed with just a few clicks.
Chaplin
Chaplin ‐ (Hand Tracking Based Button Control System)
Industry Project in CyberLink
Chaplin is a hand tracking based button control system, using user‘s hand as the medium of remote control. Waving hand is handshaking event of the system. Once waving hand is detected, the system would display a control panel including multiple control buttons on the screen, tracking the user‘s hand to determine which button is pressed. The system only uses received pixel data from traditional webcams, where no depth information is acquired.

Patents

Patent No. Country Title
US 8,121,358 US C.-Y. Weng, W.-T. Tsai, and C.-M. Lee, "Method of Grouping Images by Faces"
US 8,531,478 US C.-M. Lee, C.-Y. Weng, "Method of Browsing Photos based on People"
US 8,649,602 US S.-M. Tang, C.-Y. Weng, and J.-H. Huang, "Systems and Methods for Tagging Photos"
US 8,693,739 US C.-Y. Weng, S.-M. Tang, and H.-C. Huang , "Systems and Methods for Performing Facial Detection"
US 8,761,519 US M.-H. Chang, C.-Y. Weng, "System and method for Selecting an Object Boundary in an Image"
US 8,769,409 US C.-Y. Weng, H.-C. Huang , "Systems and Methods for Improving Object Detection"
US 8,867,789 US H.-W. Hsiao, C.-Y. Weng, "Systems and Methods for Tracking an Object in a Video"
US 9,336,583 US H.-C. Huang, H.-W. Hsiao, C.-Y. Weng, and C.-D. Chung, "Systems and Methods for Image Editing "
JP 4,925,370 JP C.-Y. Weng, W.-T. Tsai, and C.-M. Lee, "Method of Grouping Images by Faces"