weiwang

MCL Research on SSL-based Object Tracking

Object tracking is a fundamental computer vision problem that finds a wide range of applications, such as video surveillance, smart traffic system, autonomous driving cars and so on. Nowadays, most state-of-the-art object trackers adopt deep neural networks for high tracking performance at the expense of huge computational resources and heavy memory use. Here, we seek a more lightweight solution that requires fewer resources for training and inference and has a much smaller model size, thus making real-time tracking possible on small devices such as mobile phone and autonomous drones.

The proposed object tracker is built upon the PixelHop framework so that it is called OTHop (Object Tracking PixelHop). The term “hop” denotes the neighborhood of a pixel. OTHop conducts spectral analysis using Saab transform on neighborhoods of various sizes centered on a pixel through a sequence of cascaded dimension reduction units, which naturally forms a multi-resolution feature extraction scheme, thus helping capture unusual patterns that we should pay more attention to during tracking. Then we adopt the XGBoost classifier as the binary predictor to differentiate foreground pixels and background pixels. The classifier is pre-trained on some offline dataset and then updated online using either the initial frame or preceding frames with Saab coefficients as the input. Base on the classification results we derive the object bounding box.

To sum up, OTHop has the following main steps:

Extract joint spatial-spectral features based on the PixelHop framework;
Predict the probability of a spatial region, which can be of various sizes, of being a foreground object or a background region with a trained XGBoost binary classifier;
Fuse results obtained at different hops in Steps 2 to obtain the ultimate object bounding boxes.

The tracker is tested on the [...]

By |February 22nd, 2021|News|Comments Off on MCL Research on SSL-based Object Tracking|

MCL Research on SSL-based Object Proposal

Object proposal algorithms are needed to find bounding boxes for salient class-agnostic objects. It is an important pre-processing step for object detection in the wild. While most state-of-the-art object detection methods adopt an end-to-end deep neural networks, we aim at an independent object proposal unit that has low complexity and high performance. The proposed light-weight object proposal can be combined with any classification process to reduce model and computation complexity.

Our current method is built upon the PixelHop framework, it is called OPPHop (Object Proposal PixelHop). The term “hop” denotes the neighborhood of a pixel. OPPHop conducts spectral analysis on neighborhoods of different sizes centered on a pixel through a sequence of cascaded dimension reduction units. The neighborhoods of an object contain salient contours and, as a result, they have distinctive spectral signatures at a certain scale that matches the object size. The distinctive regions can be predicted based on supervised learning with Saab coefficients as the input.

 

— by Hongyu Fu

By |February 15th, 2021|News|Comments Off on MCL Research on SSL-based Object Proposal|

MCL Research on Fake Video Detection

As the number of Deepfake video contents grows rapidly, automatic Deepfake detection has received a lot of attention in the community of digital forensics. Deepfake videos can be potentially harmful to society, from non-consensual explicit content creation to forged media by foreign adversaries used in disinformation campaigns.

A light-weight high-performance Deepfake detection method, called DefakeHop, is proposed in this work. State-of-the-art Deepfake detection methods are built upon deep neural networks. DefakeHop extracts features automatically using the successive subspace learning (SSL) principle from various parts of face images. DefakeHop consists of three main modules: 1) PixelHop++, 2) feature distillation and 3) ensemble classification. To derive the rich feature representation of faces, DefakeHop extracts features using PixelHop++ units from various parts of face images. The theory of PixelHop++ have been developed by Kuo et al. using SSL. PixelHop++ has been recently used for feature learning from low-resolution face images but, to the best of our knowledge, this is the first time that it is used for feature learning from patches extracted from high-resolution color face images. Since features extracted by PixelHop++ are still not concise enough for classification, we also propose an effective feature distillation module to further reduce the feature dimension and derive a more concise description of the face. Our feature distillation module uses spatial dimension reduction to remove spatial correlation in a face and a soft classifier to include semantic meaning for each channel. Using this module the feature dimension is significantly reduced and only the most important information is kept. With a small model size of 42,845 parameters, DefakeHop achieves state-of-the-art performance with the area under the ROC curve (AUC) of 100%, 94.95%, and 90.56% on UADFV, Celeb-DF v1 and Celeb-DF v2 datasets, [...]

By |February 8th, 2021|News|Comments Off on MCL Research on Fake Video Detection|
  • Permalink Gallery

    Professor Kuo Received Technology and Engineering Emmy Award

Professor Kuo Received Technology and Engineering Emmy Award

National Awards Committee’s Technology & Engineering Achievement Committee of the National Academy of Television Arts and Sciences (NATAS) made the announcement on the recipients of the 2020 Technology and Engineering Emmy Award on January 25, 2021. MCL Director, Professor C.-C. Jay Kuo, was one of the recipients for his work on “Development of Perceptual Metrics for Video Encoding Optimization.” Professor Kuo had a brief interview on this prestigious recognition.

Q: You are honored for your work in Development of Perceptual Metrics for Video Encoding Optimization. Can you explain in very basic terms, what exactly this technology is and what it is used for?

A: This Technology and Engineering Emmy award is the outcome of my research collaboration with Netflix. We developed a new video quality assessment method called VMAF (Video Multimethod Assessment Fusion). VMAF is used by Netflix not only for video quality assessment but also for video encoding optimization. VMAF contributes to high quality streaming video from Netflix as well as other video streaming service providers.

Q: In just one or two lines, can you share why it’s so important?

A: Netflix makes VMAF an open-source tool to maximize its impact. It is the de facto standard in video quality assessment for premium video content in video streaming industry.

Q: If possible, can you share some well-known shows/movies/streaming services that use this technology?

A: VMAF-optimized encodes have covered the majority of Netflix’s streaming hours today, and VMAF has been used as a quality monitoring tool for almost all of Netflix’s streaming hours. Outside of Netflix, many companies use VMAF as well. The list includes as Twitch, Hostar, Crunchyroll, Tencent Cloud, Billibilli, among others.

By |February 1st, 2021|News|Comments Off on Professor Kuo Received Technology and Engineering Emmy Award|

Welcome MCL New Member Jiahui Zhang

We have a new member, Jiahui Zhang, joining MCL in Spring 2021. Here is a short interview with Jiazhi with our great welcome.

1. Could you briefly introduce yourself and your research interests?

My name is Jiahui Zhang, I am a second-year master student in the Department of Electronic Engineering in USC. I got my bachelor degree from Beijing University of Technology. I am a sports fan. In my spare time, I like playing sports and watching sports games. I also like traveling to view good scenery. My research interests include deep learning, computer vision especially representation learning.

2. What is your impression about MCL and USC?

USC is a great school that could provide student a enjoyable environment on living, communicating and studying.

MCL is a wonderful lab filled with a number of intelligent researchers. Everyone is an expert in their research field. Besides, people in MCL lab from Professor Kuo to every lab member are very kind and friendly. People help each other on living, studying, and researching, which build a warm environment in the lab.

3. What is your future expectation and plan in MCL?

MCL has many great and excellent researchers, and I want to study and make friends with them. For academic, I want to accomplish some projects to accumulate my research experiences and make contribution to the lab.

By |January 24th, 2021|News|Comments Off on Welcome MCL New Member Jiahui Zhang|
  • Permalink Gallery

    Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal

Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal

Mozhdeh Rouhsedaghat received her bachelor’s degree from the EE dept. of Sharif University of Technology. She is currently a Ph.D. student in Media Communications Lab at the University of Southern California, under the supervision of Prof. C.-C. Jay Kuo. Her research interests include computer vision and deep learning. She was a research intern at PayPal during the summer. Here is a short interview with Mozhdeh.

1. How does the study in USC and MCL help you?

During my Ph.D. studies at USC and MCL, I achieved a solid understanding of deep learning and machine learning and strengthened my research skills. So I was able to explore a research area during my internship and achieve great results. At MCL lab, we write weekly reports and hold seminars which helped me improve my writing and presentation skills as well.

2. How was it like working at PayPal?

This year because of the global pandemic, all the interns worked remotely. So PayPal provided the required equipment for all the interns and the University Program Team at PayPal tried to make the whole experience more interesting and exciting. I had daily meetings with my mentor and weekly meetings with my manager. Overall, I was very satisfied with the whole experience.

3. Do you have any suggestions for current graduate students?

When you want to apply for a position make sure that the mentioned responsibilities match your goals. For example, Ph.D. students usually prefer a research position. My second advice is to apply early for the internship positions as most of the positions are offered 5-7 months prior to their start date.

By |September 20th, 2020|News|Comments Off on Congratulations to Mozhdeh Rouhsedaghat for Her Summer Internship at PayPal|
  • Permalink Gallery

    Congratulations to Yeji Shen for His Summer Internship at Facebook

Congratulations to Yeji Shen for His Summer Internship at Facebook

Yeji Shen is a PhD candidate in Multimedia Communication Lab (MCL) in USC, supervised by Prof. C.-C. Jay Kuo. He received his Bachelor’s degree in Computer Science from Peking University, Beijing, China in June 2016. Since August 2016, he has been pursuing his PhD degree in MCL. His research interests include Machine Learning, Computer Vision and Artificial Intelligence. During this summer, he did an internship at Facebook. Here is a short interview with Yeji.

1. How does the study in USC and MCL help you?

First of all, in MCL, I learned to have a reasonable understanding of the research topics that I’ve been focusing on, like active learning, 3D vision and some semi-supervised learning. Such understanding is pretty helpful and valuable for both job interviews and the actual working experience. Second, I got to have a reasonable level of presentation skills, which I believe is very important in the future career. Third, a tough mind. Life is challenging. Only those with a tough mind can get through all those difficulties and obtain happiness.

2. How was it like working at Facebook?

The internship this year was a remote one. Different from normal working style, interns needed to work at home with the equipment sent by the company. (Of course, I need to mail them back.) Compared to a normal internship, the main pros are: 1) No need to physically move to the bay area. And thus the fee for house rent was saved. 2) Commuting time was saved. However, it is also clear that some cons are: 1) Harder to communicate. 2) Less interaction with team members. 3) It just didn’t feel good when the remote working style lasts for too long. Still, the overall feeling was not bad.

3. [...]

By |September 13th, 2020|News|Comments Off on Congratulations to Yeji Shen for His Summer Internship at Facebook|
  • Permalink Gallery

    Congratulations to Kaitai Zhang for His Summer Internship at Facebook

Congratulations to Kaitai Zhang for His Summer Internship at Facebook

Kaitai Zhang is currently a fourth year Ph.D. candidate at Multimedia Communication Lab. His research mainly focus on computer vision, machine learning and deep learning. Kaitai received an internship offer and spent the past summer at Facebook. Here is a short interview with Kaitai.

 

1. How does the study in USC and MCL help you? (technically and psychologically)

I believe my research experience at MCL and my education background from USC are the foundation on which I could get the internship opportunity. From the technical side, all my machine learning-related projects from MCL help a lot to get hiring manger’s attention during the interview process(This is especially important if you want to get into a very popular team). From the psychological side, I found the industrial project I worked on are even more beneficial to me that I expected. It is more like an opportunity to get exposed to real-world problem from industry and learn how things work in companies, which could make our students more well-prepared for the internship.

Beside the above two aspects, I also want to mention another advantage for students from MCL, which is the extraordinary reputation and wide alumni network of our lab. More than one engineers talked to me about our alumni and their awesome works at Facebook.

 

2. How was it like working at Facebook?

The internship at Facebook was like an amazing journey. Here I will focus on one thing that impressed me most. It is the move-fast working style at Facebook. People at Facebook are moving fast on all aspects. They are very energetic and acute. There is a daily sync meeting and also few ad-hoc meetings to discuss things efficiently. People like asking others for help and also like helping others, and this is how they unblock themselves when meet [...]

By |September 6th, 2020|News|Comments Off on Congratulations to Kaitai Zhang for His Summer Internship at Facebook|
  • Permalink Gallery

    Congratulations to Jiali Duan for His Summer Internship at Amazon

Congratulations to Jiali Duan for His Summer Internship at Amazon

Jiali Duan is a PhD candidate supervised by Prof. C-.C. Jay Kuo at MCL. He interned at Amazon A9 during Summer. Here is a short interview with Jiali.

 

1. How does the study in USC and MCL help you? (technically and psychologically)

First, MCL helped me lay a solid foundation for research and communication. Academic and technical exchange of

ideas happen almost daily for applied scientist intern at Amazon, making it a necessity to communicate concisely and logically. Thankfully, I got trained for this aspect at MCL by sticking to weekly report and regular personal meetings. Second, the ability to prototype new ideas is intensely tested during internship. Original research here at MCL prepare us well for this aspect.

 

2. How was it like working at Amazon A9?

I had two internship experience at A9, a physical one in last year and a remote one this year. Generally, the physical one is much better. Last year, I was working at A9 Palo Alto. The company is located at University St, within walking distance to StandFord. Due to its location, there’re many nice restaurants and the neighborhood is very safe. The internship program was also very kind to provide free baseball/football match tickets, Santa Cruz tour and Seattle headquarter tour.

 

3. Do you have any suggestions for current graduate students?

I know that some companies provide more software development positions than research positions and some allow limited number of interview trials from the same person. So, be ready when you try. In terms of suggestions, be prepared for questions that reach beyond your scope of knowledge, which may require certain amount of improvision.

By |August 31st, 2020|News|Comments Off on Congratulations to Jiali Duan for His Summer Internship at Amazon|

Congratulations to Bin Wang for His Summer Internship at JD

Bin Wang received his B.Eng. from University of Electronic Science and Technology of China in June, 2017. Since July 2017, He joined Media Communication Lab (MCL) at University of Southern California (USC) as a Ph.D. student, supervised by Prof. C.-C. Jay Kuo. His research interest includes natural language processing and machine learning.

1. How does the study in USC and MCL help you? (technically and psychologically)    

The research topics I have been working on for the last two years helped a lot to build a solid understanding of natural language understanding and machine learning field. Particularly the experience with representation learning and graph learning projects are really helpful and allows me to behave well in interviews and also get started quickly when doing intern projects. Additionally, our weekly report and presentation training really sharpened my writing and oral presentation skills, which is at least as important as the coding/implementation ability in a long run.

2. How was it like working at JD AI-Research?

Because of global pandemic, the year of 2020 is quite different for everyone. Instead of heading to Mountain View, all interns are working remotely. A more flexible working style is allowed. Here, we have daily meetings to get sync with supervisor and mentor. Each week, we also have to submit the weekly report for summarization and planning. At AI-Research group, the working style is very close to a university lab and the goal is also for publishing at high-tier conference in the AI field.

3. Do you have any suggestions for current graduate students? (e.g. interview strategy and preparation, etc.)       

Usually evaluation protocol varies with different companies and groups. Gathering information for your interested positions is extremely important and MCL alumnus can be [...]

By |August 23rd, 2020|News|Comments Off on Congratulations to Bin Wang for His Summer Internship at JD|