Monthly Archives: April 2017

Congratulations to Yuzhuo Ren for passing her defense on April 26, 2017. Her Ph.D. thesis is entitled “Machine Learning Techniques for Outdoor and Indoor Layout Estimation”.

Abstract of thesis:

In my dissertation, I study three research problems: 1) Outdoor geometric labeling, and 2) Indoor layout estimation and 3) 3D object detection.

A novel method that extracts global attributes from outdoor images to facilitate geometric layout labeling is proposed. The proposed Global-attributes Assisted Labeling (GAL) system exploits both local features and global attributes. The performance of the proposed GAL system is demonstrated and benchmarked with several state-of-the-art algorithms against a popular outdoor scene layout dataset.

Existing solutions to indoor layout estimation largely rely on hand-craft features and vanishing lines. They often fail in highly cluttered indoor scenes. The proposed coarse-to-fine indoor layout estimation (CFILE) method consists of two stages: 1) coarse layout estimation; and 2) fine layout localization. In the first stage, we adopt a fully convolutional neural network (FCN) to obtain a coarse-scale room layout estimate that is close to the ground truth globally. In the second stage, we formulate an optimization framework that enforces several constraints such as layout contour straightness, surface smoothness and geometric constraints for layout detail refinement. The proposed CFILE system offers the state-of-the-art performance on two common benchmark datasets.

Given a RGB-D image, we examine the 3D object detection problem with an objective to produce a bounding box around the object and classify its category. This is a challenging problem due to high intra-class variance, illumination change, background clutter and occlusion. Here, we propose a novel solution that integrates the context information together to provide a robust 3D object detection solution. Extensive experiments are conducted to demonstrate that the proposed Context-3D method achieves the [...]

Professor C.-C. Jay Kuo visited the Biomedical Engineering Department in the James  H. Clark Center of the Stanford University on April 17 to attend a research project meeting. Professor Tsung Hsiai of UCLA, Professor Alison Marsden of Stanford and Professor Jay Kuo have a joint NIH project on “Sheer Stress and Light-Sheets to Study Cardiac Trabeculation”. In this project, the team developed a Super Resolution Light-Sheet Microscopy (SRLSM) technology to advance the field of mechanotransduction and cardiac development. The role of MCL in this project is to incorporate computational algorithm to synchronize the cardiac cycle with the SRLSM-captured images for reconstruction of 4-D (3D + time) simulation. Two MCL PhD students have contributed to this project. They are Hao Xu and Ruiyuan Lin. Hao already graduated in 2017 January and now is working at Google. The MCL team provides the expertise to capture the beating hearts by period determination, relative shift determination, absolute shift determination, and post-processing.

Professor Kuo was very impressed by Stanford’s beautiful campus. He visited the campus before about 30 years ago. He hopes to have more opportunities to visit the Silicon Valley to meet MCL alumni.

When watching images/videos on a TV, we often have many questions about the image/video. What is the name of the beautiful places? What is the name of the actors? Which store sell the actor’s car at big discounts? Imagine one day we have a smart TV which can interactively answer your questions, and recommend relevant shopping/travel advertisements. We will enjoy more convenience and have more funs on watching TV.

MCL members, Bing Li, Zhehang Ding and Yuhang Su are collaborating with Samsung Company on Interactive Advisement for Smart TV. At the first year, we focus on automatic image/video caption. Image/video caption is to describe an image/video by a sentence instead of detecting objects.

Currently, we propose three pipelines for this project. The first pipeline is general image caption. The second and third pipeline are respectively place aware caption and face aware caption, such that our system can achieve better performance in vertical industrials such as travel, entertainment, sport and etc. For general image caption, we develop a detection method which achieves 84% mAP. For place-ware annotation, since no image datasets is for world-wide famous places, we collect images from 118 famous places in 21 countries to construct a landmark dataset. For face aware annotation, we construct a celebrity dataset, and face detection and face recognition method based on CNN.

In our future work, we will put more efforts into video caption.

Spotting text in a natural scene image is a challenging task. It involves text localization in the image and text recognition given these localized text image patches. To tackle this problem, traditional optical character recognition (OCR) techniques – which are designed specifically for black and white text contents – give way to more sophisticated methods like neural networks.

Yuanhang Su, one MCL member, is now collaborating with Inha University, Korean Airline and Pratt & Whitney institute for collaborative engineering (PWICE) to build a text spotting system. Our lab has developed a comprehensive text spotting system that can localize and recognize text in natural scene images by using combined convolutional neural network (CNN) and recurrent neural network (RNN) architecture. Our system is able to deal with English and Korean text contents.