• Permalink Gallery

    Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal

Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal

Mozhdeh Rouhsedaghat received her bachelor’s degree from the EE dept. of Sharif University of Technology. She is currently a Ph.D. student in Media Communications Lab at the University of Southern California, under the supervision of Prof. C.-C. Jay Kuo. Her research interests include computer vision and deep learning. She was a research intern at PayPal during the summer. Here is a short interview with Mozhdeh.

1. How does the study in USC and MCL help you?

During my Ph.D. studies at USC and MCL, I achieved a solid understanding of deep learning and machine learning and strengthened my research skills. So I was able to explore a research area during my internship and achieve great results. At MCL lab, we write weekly reports and hold seminars which helped me improve my writing and presentation skills as well.

2. How was it like working at PayPal?

This year because of the global pandemic, all the interns worked remotely. So PayPal provided the required equipment for all the interns and the University Program Team at PayPal tried to make the whole experience more interesting and exciting. I had daily meetings with my mentor and weekly meetings with my manager. Overall, I was very satisfied with the whole experience.

3. Do you have any suggestions for current graduate students?

When you want to apply for a position make sure that the mentioned responsibilities match your goals. For example, Ph.D. students usually prefer a research position. My second advice is to apply early for the internship positions as most of the positions are offered 5-7 months prior to their start date.

By |September 20th, 2020|News|Comments Off on Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal|
  • Permalink Gallery

    Congratulations to Yeji Shen for His Summer Internship at Facebook

Congratulations to Yeji Shen for His Summer Internship at Facebook

Yeji Shen is a PhD candidate in Multimedia Communication Lab (MCL) in USC, supervised by Prof. C.-C. Jay Kuo. He received his Bachelor’s degree in Computer Science from Peking University, Beijing, China in June 2016. Since August 2016, he has been pursuing his PhD degree in MCL. His research interests include Machine Learning, Computer Vision and Artificial Intelligence. During this summer, he did an internship at Facebook. Here is a short interview with Yeji.

1. How does the study in USC and MCL help you?

First of all, in MCL, I learned to have a reasonable understanding of the research topics that I’ve been focusing on, like active learning, 3D vision and some semi-supervised learning. Such understanding is pretty helpful and valuable for both job interviews and the actual working experience. Second, I got to have a reasonable level of presentation skills, which I believe is very important in the future career. Third, a tough mind. Life is challenging. Only those with a tough mind can get through all those difficulties and obtain happiness.

2. How was it like working at Facebook?

The internship this year was a remote one. Different from normal working style, interns needed to work at home with the equipment sent by the company. (Of course, I need to mail them back.) Compared to a normal internship, the main pros are: 1) No need to physically move to the bay area. And thus the fee for house rent was saved. 2) Commuting time was saved. However, it is also clear that some cons are: 1) Harder to communicate. 2) Less interaction with team members. 3) It just didn’t feel good when the remote working style lasts for too long. Still, the overall feeling was not bad.

3. [...]

By |September 13th, 2020|News|Comments Off on Congratulations to Yeji Shen for His Summer Internship at Facebook|
  • Permalink Gallery

    Congratulations to Kaitai Zhang for His Summer Internship at Facebook

Congratulations to Kaitai Zhang for His Summer Internship at Facebook

Kaitai Zhang is currently a fourth year Ph.D. candidate at Multimedia Communication Lab. His research mainly focus on computer vision, machine learning and deep learning. Kaitai received an internship offer and spent the past summer at Facebook. Here is a short interview with Kaitai.


1. How does the study in USC and MCL help you? (technically and psychologically)

I believe my research experience at MCL and my education background from USC are the foundation on which I could get the internship opportunity. From the technical side, all my machine learning-related projects from MCL help a lot to get hiring manger’s attention during the interview process(This is especially important if you want to get into a very popular team). From the psychological side, I found the industrial project I worked on are even more beneficial to me that I expected. It is more like an opportunity to get exposed to real-world problem from industry and learn how things work in companies, which could make our students more well-prepared for the internship.

Beside the above two aspects, I also want to mention another advantage for students from MCL, which is the extraordinary reputation and wide alumni network of our lab. More than one engineers talked to me about our alumni and their awesome works at Facebook.


2. How was it like working at Facebook?

The internship at Facebook was like an amazing journey. Here I will focus on one thing that impressed me most. It is the move-fast working style at Facebook. People at Facebook are moving fast on all aspects. They are very energetic and acute. There is a daily sync meeting and also few ad-hoc meetings to discuss things efficiently. People like asking others for help and also like helping others, and this is how they unblock themselves when meet [...]

By |September 6th, 2020|News|Comments Off on Congratulations to Kaitai Zhang for His Summer Internship at Facebook|
  • Permalink Gallery

    Congratulations to Jiali Duan for His Summer Internship at Amazon

Congratulations to Jiali Duan for His Summer Internship at Amazon

Jiali Duan is a PhD candidate supervised by Prof. C-.C. Jay Kuo at MCL. He interned at Amazon A9 during Summer. Here is a short interview with Jiali.


1. How does the study in USC and MCL help you? (technically and psychologically)

First, MCL helped me lay a solid foundation for research and communication. Academic and technical exchange of

ideas happen almost daily for applied scientist intern at Amazon, making it a necessity to communicate concisely and logically. Thankfully, I got trained for this aspect at MCL by sticking to weekly report and regular personal meetings. Second, the ability to prototype new ideas is intensely tested during internship. Original research here at MCL prepare us well for this aspect.


2. How was it like working at Amazon A9?

I had two internship experience at A9, a physical one in last year and a remote one this year. Generally, the physical one is much better. Last year, I was working at A9 Palo Alto. The company is located at University St, within walking distance to StandFord. Due to its location, there’re many nice restaurants and the neighborhood is very safe. The internship program was also very kind to provide free baseball/football match tickets, Santa Cruz tour and Seattle headquarter tour.


3. Do you have any suggestions for current graduate students?

I know that some companies provide more software development positions than research positions and some allow limited number of interview trials from the same person. So, be ready when you try. In terms of suggestions, be prepared for questions that reach beyond your scope of knowledge, which may require certain amount of improvision.

By |August 31st, 2020|News|Comments Off on Congratulations to Jiali Duan for His Summer Internship at Amazon|

Congratulations to Bin Wang for His Summer Internship at JD

Bin Wang received his B.Eng. from University of Electronic Science and Technology of China in June, 2017. Since July 2017, He joined Media Communication Lab (MCL) at University of Southern California (USC) as a Ph.D. student, supervised by Prof. C.-C. Jay Kuo. His research interest includes natural language processing and machine learning.

1. How does the study in USC and MCL help you? (technically and psychologically)    

The research topics I have been working on for the last two years helped a lot to build a solid understanding of natural language understanding and machine learning field. Particularly the experience with representation learning and graph learning projects are really helpful and allows me to behave well in interviews and also get started quickly when doing intern projects. Additionally, our weekly report and presentation training really sharpened my writing and oral presentation skills, which is at least as important as the coding/implementation ability in a long run.

2. How was it like working at JD AI-Research?

Because of global pandemic, the year of 2020 is quite different for everyone. Instead of heading to Mountain View, all interns are working remotely. A more flexible working style is allowed. Here, we have daily meetings to get sync with supervisor and mentor. Each week, we also have to submit the weekly report for summarization and planning. At AI-Research group, the working style is very close to a university lab and the goal is also for publishing at high-tier conference in the AI field.

3. Do you have any suggestions for current graduate students? (e.g. interview strategy and preparation, etc.)       

Usually evaluation protocol varies with different companies and groups. Gathering information for your interested positions is extremely important and MCL alumnus can be [...]

By |August 23rd, 2020|News|Comments Off on Congratulations to Bin Wang for His Summer Internship at JD|

MCL Research on Face Gender Classification

Face attributes classification is an important topic in biometrics. The ancillary information of faces such as gender, age and ethnicity is referred to as soft biometrics in forensics. The face gender classification problem has been extensively studied for more than two decades. Before the resurgence of deep neural networks (DNNs) around 7-8 years ago, the problem was treated using the standard pattern recognition paradigm. It consists of two cascaded modules: 1) unsupervised feature extraction and 2) supervised classification via common machine learning tools such as support vector machine (SVM) and random forest (RF) classifiers.

We have seen a fast progress on this topic due to the application of deep learning (DL) technology in recent years. Cloud-based face verification, recognition and attributes classification technologies have become mature, and they have been used in many real world biometric systems. Convolution neural networks (CNNs) offer high performance accuracy. Yet, they rely on large learning models consisting of several hundreds of thousands or even millions of model parameters. The superior performance is contributed by factors such as higher input image resolutions, more and more training images and abundant computational/memory resources.

Edge/mobile computing in a resource-constrained environment cannot meet the above-mentioned conditions. The technology of our interest finds applications in rescue missions and/or field operational settings in remote locations. The accompanying face inference tasks are expected to execute inside a poor computing and communication infrastructure. It is essential to have a smaller learning model size, lower training and inference complexity, and lower input image resolution. The last requirement arises from the need to image individuals at farther standoff distances, which results in faces with fewer pixels.

In this research, MCL worked closely with ARL researchers in developing a new interpretable non-parametric machine [...]

By |August 17th, 2020|News|Comments Off on MCL Research on Face Gender Classification|

MCL Received Sponsored Research from Facebook

MCL received a grant from Facebook recently for joint research on next generation video coding technologies. Through this support, MCL researchers will collaborate with Facebook researchers to conduct video coding research in the next 2 years.

With the development of camera and sensor technologies, high resolution images and videos have become ubiquitous in daily life. Demands on fast transmission and efficiency store high quality images and videos increases dramatically. Problem on how to transmit and store media data efficiently have been widely discussed. Online high-resolution video meeting and live broadcasting also raise the pressure on fast encoding and decoding under the limitation of current bandwidth.

Numerous codecs have been developed during past 20 years including the well know H.264, MPEG-4 and latest H.265/HEVC. They are widely used in our daily life. H26x and MPEG-x standards are well supported in both software and hardware. There are many encoder and decoder chip sets available commercially (for example chips from System on Chip Technologies Inc.) which can speed up the process and be configured based on user specifications. While for the royal free codecs like AV1, it has higher complexity and not widely supported by the hardware chips which hinder its being widely used.

Channel wise Saab transformation has been proved to have advantages in exploring the spatial correlation with small model size. MCL researchers will use a block hierarchy transformation based on the channel-wise Saab transform framework to achieve lower encoding and decoding complexity while preserve the rate-distortion performance.

— by Dr. C.-C. Jay Kuo

By |August 10th, 2020|News|Comments Off on MCL Received Sponsored Research from Facebook|
  • Permalink Gallery

    MCL Received Sponsored Research from Army Research Laboratory

MCL Received Sponsored Research from Army Research Laboratory

MCL received a grant from the US Army Research Laboratory (ARL) recently for joint research on theory and applications of artificial intelligence (AI) and machine learning (ML) technologies. Through this support, MCL researchers will collaborate with ARL researchers to conduct fundamental and applied research in the next 4 years.

The fundamental research targets at explainable neural networks. Cybenko and Hornik et al. proved that the multi-layer perceptron (MLP) is a universal approximator in late 80s, which appears to be the most important theoretic result even up to now. However, for a given dataset, they do not provide a constructive procedure to build the MLP. We will investigate a systematic way to specify an MLP architecture and determine its model parameters. Resource-rich and resource-scarce networks refer to those have abundant and fewer model parameters, respectively. The objective of the stress test study is to understand the behavior of the transition of a network from a resource-rich one to a resource-scarce one. We would like to understand the behavior transition in several areas, including model sensitivity to different weight initializations, classification accuracy, overfitting, etc.

The applied research targets at spatial-temporal attention and semantic scene understanding via successive subspace learning (SSL). To extract key spatial-temporal information from visual data, facilitates video processing and understanding down-stream tasks. For example, an image contains one or several objects. Object detection and recognition is critical to image understanding. Also, motion provides cues for object tracking in video understanding. The sponsored research will also focus on two issues on multi-modality data, say, those obtained by RGB, depth and infrared sensors: 1) representation of multi-domain data and 2) understanding of multi-domain data.


—- by Dr. C.-C. Jay Kuo

By |August 2nd, 2020|News|Comments Off on MCL Received Sponsored Research from Army Research Laboratory|

MCL Research on Image Super-resolution

Image super-resolution (SR) is a classic problem in computer vision (CV), which aims at recovering a high-resolution image from a low-resolution image. As a type of supervised generative problem, image SR attracts wide attention due to its strong connection with other CV topics, such as object recognition, object alignment, texture synthesis and so on. Besides, it has extensive applications in real world, for example, medical diagnosis, remote sensing, biometric information identification, etc.

For the state-of-the-art approaches for SR, typically there are two mainstreams: 1) example-based learning methods, and 2) Deep Learning (CNN-based) methods. Example-based methods either exploit external low-high resolution exemplar pairs [1], or learn internal similarity of the same image with different resolution scales [2]. In order to tackle model overfitting and generativity, some dictionary strategies are normally applied for encoding (e.g. Sparse coding, SC). However, features used in example-based methods are usually traditional gradient-related or just handcraft, which may affect model efficiency. While CNN-based SR methods (e.g. SRCNN [3]) does not really distinguish between feature extraction and decision making. Lots of basic CNN models/blocks are applied to SR problem, e.g. GAN, residual learning, attention network, and provide superior SR results. Nevertheless, the non-explainable process and exhaustive training cost are serious drawbacks of CNN-based methods.

By taking advantage of reasonable feature extraction [4], we utilize spatial-spectral compatible features to express exemplar pairs. In addition, we formulate a Successive-Subspace-Learning-based (SSL-based) method to partition data into subspaces by feature statistics, and apply regression in each subspace for better local approximation. Moreover, some adaptation is also manipulated for better data fitting. In the future, we aim at providing such a SSL-based explainable method with high efficiency for SR problem.

— By Wei Wang



[1] Timofte, Radu, Vincent De Smet, and [...]

By |April 6th, 2020|Computer Vision and Scene Analysis, News, Research|Comments Off on MCL Research on Image Super-resolution|

MCL Research on Statistics-based Attention

Object detection and recognition has always been one of the key challenges in the field of computer vision for finding and identifying objects in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes and scales or even when they are translated or rotated. Many approaches to the task have been implemented over multiple decades, including handcrafted features, machine learning algorithms and deep learning.

Recently breakthrough results have been made via Deep Learning with loads of labelled data for supervised training, while Deep Learning is notorious for lacking in scalability and interpretability. With the advantage of recent proposed scalable and interpretable Pixelhop model [1] and pixelhop++ [2], a new object detection pipeline can be proposed via Object Proposal-> Feature Extraction using SSL ->  Classification, thus a machine learning based Statistic-based attention is the key to generate object proposals.

Apart from Deep Learning, object proposal via visual saliency on single images such as DRFI [3] can be a good start for a machine learning based object proposal. To further take advantage of the statistics from training data, we formulate the weakly supervised object proposal problem into object search with features capable of matching, such as SURF [4]. In the future, we aim to improve these results by further exploration with a query-retrieval based saliency proposal method along with adapted bag of word features.


-By Hongyu Fu

[1] Yueru Chen and C-C Jay Kuo, “Pixelhop: A successive subspace learning (ssl) method for object recognition,” Journal of Visual Communication and Image Representation, p. 102749, 2020.

[2]Yueru Chen , Mozhdeh Rouhsedaghat , Suya You, Raghuveer Rao and C.-C. Jay Kuo, [...]

By |March 22nd, 2020|News, Research|Comments Off on MCL Research on Statistics-based Attention|