weiwang

With the development of camera and sensor technologies, high-resolution images and videos have become ubiquitous in daily life. Demands on fast transmission and efficiency store high-quality images and videos increase dramatically. Problem on how to transmit and store media data efficiently have been widely discussed. Online high-resolution video meeting/live broadcasting also raise the pressure on fast encoding and decoding under the limitation of current bandwidth.

Numerous codecs have been developed during the past 20 years including the well know H.264, MPEG-4 and the latest H.265/HEVC which are widely used in our daily life. H26x and MPEG-x standards are well supported in both software and hardware. There are many encoder and decoder chipsets available commercially (for example chips from System on Chip Technologies Inc.) which can speed up the process and be configured based on user specifications. While the royal free codecs like AV1, it has higher complexity and not widely supported by the hardware chips which hinder it’s being widely used.

Previous frameworks used one layer transformation to perform the energy compaction task. We propose to use the multi-hop transform to perform this task which is supposed to have better energy compaction result. Presently, we focus on image compression (intra coding in video compression) to increase the performance while lower the complexity. In this project, we are trying to develop some low-complexity compression tools which can achieve comparable performance against the current standard.

Object tracking is a fundamental computer vision problem that finds a wide range of applications, such as video surveillance, smart traffic system, autonomous driving cars and so on. Nowadays, most state-of-the-art object trackers adopt deep neural networks for high tracking performance at the expense of huge computational resources and heavy memory use. Here, we seek a more lightweight solution that requires fewer resources for training and inference and has a much smaller model size, thus making real-time tracking possible on small devices such as mobile phone and autonomous drones.

The proposed object tracker is built upon the PixelHop framework so that it is called OTHop (Object Tracking PixelHop). The term “hop” denotes the neighborhood of a pixel. OTHop conducts spectral analysis using Saab transform on neighborhoods of various sizes centered on a pixel through a sequence of cascaded dimension reduction units, which naturally forms a multi-resolution feature extraction scheme, thus helping capture unusual patterns that we should pay more attention to during tracking. Then we adopt the XGBoost classifier as the binary predictor to differentiate foreground pixels and background pixels. The classifier is pre-trained on some offline dataset and then updated online using either the initial frame or preceding frames with Saab coefficients as the input. Base on the classification results we derive the object bounding box.

To sum up, OTHop has the following main steps:

Extract joint spatial-spectral features based on the PixelHop framework;
Predict the probability of a spatial region, which can be of various sizes, of being a foreground object or a background region with a trained XGBoost binary classifier;
Fuse results obtained at different hops in Steps 2 to obtain the ultimate object bounding boxes.

The tracker is tested on the [...]

As the number of Deepfake video contents grows rapidly, automatic Deepfake detection has received a lot of attention in the community of digital forensics. Deepfake videos can be potentially harmful to society, from non-consensual explicit content creation to forged media by foreign adversaries used in disinformation campaigns.

A light-weight high-performance Deepfake detection method, called DefakeHop, is proposed in this work. State-of-the-art Deepfake detection methods are built upon deep neural networks. DefakeHop extracts features automatically using the successive subspace learning (SSL) principle from various parts of face images. DefakeHop consists of three main modules: 1) PixelHop++, 2) feature distillation and 3) ensemble classification. To derive the rich feature representation of faces, DefakeHop extracts features using PixelHop++ units from various parts of face images. The theory of PixelHop++ have been developed by Kuo et al. using SSL. PixelHop++ has been recently used for feature learning from low-resolution face images but, to the best of our knowledge, this is the first time that it is used for feature learning from patches extracted from high-resolution color face images. Since features extracted by PixelHop++ are still not concise enough for classification, we also propose an effective feature distillation module to further reduce the feature dimension and derive a more concise description of the face. Our feature distillation module uses spatial dimension reduction to remove spatial correlation in a face and a soft classifier to include semantic meaning for each channel. Using this module the feature dimension is significantly reduced and only the most important information is kept. With a small model size of 42,845 parameters, DefakeHop achieves state-of-the-art performance with the area under the ROC curve (AUC) of 100%, 94.95%, and 90.56% on UADFV, Celeb-DF v1 and Celeb-DF v2 datasets, [...]

We have a new member, Jiahui Zhang, joining MCL in Spring 2021. Here is a short interview with Jiazhi with our great welcome.

1. Could you briefly introduce yourself and your research interests?

My name is Jiahui Zhang, I am a second-year master student in the Department of Electronic Engineering in USC. I got my bachelor degree from Beijing University of Technology. I am a sports fan. In my spare time, I like playing sports and watching sports games. I also like traveling to view good scenery. My research interests include deep learning, computer vision especially representation learning.

2. What is your impression about MCL and USC?

USC is a great school that could provide student a enjoyable environment on living, communicating and studying.

MCL is a wonderful lab filled with a number of intelligent researchers. Everyone is an expert in their research field. Besides, people in MCL lab from Professor Kuo to every lab member are very kind and friendly. People help each other on living, studying, and researching, which build a warm environment in the lab.

3. What is your future expectation and plan in MCL?

MCL has many great and excellent researchers, and I want to study and make friends with them. For academic, I want to accomplish some projects to accumulate my research experiences and make contribution to the lab.

Yeji Shen is a PhD candidate in Multimedia Communication Lab (MCL) in USC, supervised by Prof. C.-C. Jay Kuo. He received his Bachelor’s degree in Computer Science from Peking University, Beijing, China in June 2016. Since August 2016, he has been pursuing his PhD degree in MCL. His research interests include Machine Learning, Computer Vision and Artificial Intelligence. During this summer, he did an internship at Facebook. Here is a short interview with Yeji.

1. How does the study in USC and MCL help you?

First of all, in MCL, I learned to have a reasonable understanding of the research topics that I’ve been focusing on, like active learning, 3D vision and some semi-supervised learning. Such understanding is pretty helpful and valuable for both job interviews and the actual working experience. Second, I got to have a reasonable level of presentation skills, which I believe is very important in the future career. Third, a tough mind. Life is challenging. Only those with a tough mind can get through all those difficulties and obtain happiness.

2. How was it like working at Facebook?

The internship this year was a remote one. Different from normal working style, interns needed to work at home with the equipment sent by the company. (Of course, I need to mail them back.) Compared to a normal internship, the main pros are: 1) No need to physically move to the bay area. And thus the fee for house rent was saved. 2) Commuting time was saved. However, it is also clear that some cons are: 1) Harder to communicate. 2) Less interaction with team members. 3) It just didn’t feel good when the remote working style lasts for too long. Still, the overall feeling was not bad.

3. [...]