News

In Fall 2021, we have a new MCL member, Qingyang Zhou, joining our big family. Here is a short interview with Qingyang with our great welcome.

1. Could you briefly introduce yourself and your research interests?

Hello, I am Qingyang Zhou, a new member of MCL in 21fall. I received my Bachelor’s and Master’s degrees both in Shanghai, China. My research interest is multimedia signal processing, especially video coding and point cloud compression. I am passionate about these research areas and I have been working on different video coding standards for over 3 years before coming to USC. I am very happy to share with you my experience in this field.

2. Could you briefly introduce yourself and your research interests?

MCL is filled with productive and professional researchers. The lab offers us many different research directions from multimedia signal processing to computer vision. Professor Kuo is creative, hardworking, and full of insightful thoughts. I believe I would receive excellent academic training and become a good researcher at MCL. USC is also an excellent place with both a beautiful campus and strong academic background. I am very happy that I could continue my research at USC.

3. What is your future expectation and plan in MCL?

At MCL, I will continue to do research on multimedia-related areas. I have been doing this for years and there are still many questions waiting for me to solve. For the next few months, my attention will be mainly focused on Point Cloud Compression. For the next several years, I would like to get my academic skills improved and make good preparations before entering the industry or academia.

An image generative model learns the distribution of image samples from a certain domain and then generates new images that follow the learned distribution.  The design of image generative models involves analysis and generation two pipelines.  The former analyzes properties of training image samples while the latter generates new images after the training is completed. Only the generation unit is used for image generation in inference.  There is a resurge of interests in generative models due to the amazing performance achieved by deep-learning-based (DL-based) methods in general and generative adversarial networks (GANs) in particular.  Yet, DL-based methods attempt to solve a nonconvex optimization problem which is difficult to explain. GAN’s training may suffer from gradient vanishing, convergence difficulty and mode collapse. Furthermore, its implementation demands higher computational resources due to large model sizes.

The design of GANs demands that distributions of training and generated images be indistinguishable, which is implicitly achieved by training a generator/discriminator pair through end-to-end optimization of a cost function.  In contrast with the GAN approach, we propose a novel and explainable image generation method with explicit sample distribution modeling in this work.  For image analysis, we construct fine-to-coarse spatial-spectral subspaces using the PixelHop++ architecture and obtain sample distributions in each subspace. For image generation, we reverse the process by generating samples in the coarsest subspace and add more details to these samples gradually. Our solution, called GenHop (an acronym of Generative Pixelhop), offers an unconditional generative model. Based on MNIST and Fashion-MNIST two datasets, GenHop can generate visually pleasant images whose FID scores are comparable with those of DL-based generative models.

In image forensics, steganography and steganalysis are like the two ends of the same coin. image steganography is a technique to conceal secret messages in the images by slightly modifying the pixel values. Corresponding to image steganography, steganalysis is the process to reveal the presence of the hidden message in images. Recently, steganalysis are focusing on defending content-adaptive steganographic schemes, for example WOW, HILL and S-UNIWARD, etc. Fig.1 [1] illustrates the modifications of cover image from different steganographic method.  Content-adaptive steganography is lean to do modifications on complex texture regions, which makes embedding traces less detectable for steganalyzers.

Traditionally, hand crafted features together with machine learning classifiers have good performance on steganalysis, such as Spatial Rich Model and its variants. After the emerging of neural networks, different CNN architectures are utilized in the steganalysis literature. Because of the important property that CNNs are able to extract complex statistical dependencies from high dimensional input and learn hierarchical representations, CNN-based features usually achieve better performance than traditional hand-crafted features. However, CNN based models are suffering from long training time, large model size and enormous consumption of computation resources.

We would like to utilize green learning methodology in steganalysis field, by incorporating Saab transform as feature extraction module in the future. Saab transform has shown its capability of extracting the high frequency representations in a feedforward way and preserving the light-weighted model size at the same time.

References:

[1] Tang, Weixuan, et al. “Adaptive steganalysis based on embedding probabilities of pixels.” IEEE Transactions on Information Forensics and Security 11.4 (2015): 734-745.

[2] C.-C. J. Kuo and Y. Chen, “On data-driven saak transform,” Journal of Visual Communication and Image Representation, vol. 50, pp. 237–246, 2018.

[3] C.-C. J. Kuo, M. Zhang, S. Li, J. [...]

Object detection is one of the most essential and challenging tasks in computer vision, while most state-of-the-art object detection methods adopt an end-to-end deep neural network, we aim at an interpretable framework that has low complexity, high efficiency in training, and high performance. The method is built upon the PixelHop framework, as shown in fig 1. The term “hop” denotes the neighborhood of a pixel. Pixelhop conducts spectral analysis on neighborhoods of different sizes centered on a pixel through a sequence of cascaded dimension reduction units. The neighborhoods of an object contain representative patterns of the objects such as salient contours and, as a result, they have distinctive spectral signatures at a certain scale that matches the object size, thus bounding boxes and  class labels can be predicted based on supervised learning with Saab coefficients in proper hops as the representations.

Our method takes YOLO’s problem formulation as reference and ensembles three major modules to finish the object detection task. As shown in fig.1, by proper settings of Pixelhop, we divide all the objects into three different scales, i.e. large(as shown blue), medium (as shown in green), and small (as shown in red), and have hops with proper receptive field (RF) responsible for proposing corresponding anchor boxes for different scales (as shown comparing with the “cat” example). With the Saab coefficients at each hop, we propose anchor boxes at each spatial location, and for each anchor box we train module 1 to predict its confidence score, module 2 to predict its class label, module 3 to predict its box regression. Eventually for each image our model will first propose potential boxes and use non max suppression based on confidence score to keep the best proposed [...]

Automatic   synthesis   of   visually   pleasant   texture   that resembles  exemplary  texture  finds  applications  in  computer  graphics.  Texture  synthesis  has  been  studied  for several  decades  since  it  is  also  of  theoretical  interest  in texture analysis and modeling. Texture can be synthesized pixel-by-pixel or patch-by-patch based on an exemplary  pattern.  For  the  pixel-based  synthesis,  a  pixel conditioned on its squared neighbor was synthesized using the  conditional  probability  and  estimated  by  a  statistical method. Generally,  patch-based  texture  synthesis yields  higher  quality  than  pixel-based  texture  synthesis. Yet, searching the whole image for patch-based synthesis is  extremely  slow.  To  speed  up  the  process,  small patches of the exemplary texture can be stitched together to  form  a  larger  region. Although  these  methods can produce texture of higher quality, the diversity of produced textures is limited. Besides texture synthesis in the spatial domain, texture images from the spatial domain can be transformed to the spectral domain with certain filters (or kernels), thus exploiting the statistical correlation of filter responses for texture synthesis. Commonly used kernels include the Gabor filters and the steerable pyramid filter banks.

We  have  witnessed  amazing  quality  improvement  of synthesized  texture  over  the  last  five  to  six  years  due to  the  resurgence  of  neural  networks.  Texture  synthesis based on deep learning (DL), such as Convolutional Neural Networks  (CNNs)  and  Generative  Adversarial  Networks(GANs), yield visually pleasing results. DL-based methods learn transform kernels from numerous training data through end-to-end optimization. However, these methods have two main shortcomings: 1) a lack of mathematical  transparency  and  2)  a  higher  training  and  inference complexity. To address these drawbacks, we investigate a non-parametric and interpretable texture synthesis method, called NITES [1].

NITES  consists  of  three  steps.  First,  it  analyzes  the exemplary texture to obtain its joint spatial-spectral [...]