News

  • Permalink Gallery

    Toward Interpretable, Reliable, and Sustainable AI – the 2nd AI Wave

Toward Interpretable, Reliable, and Sustainable AI – the 2nd AI Wave

Rapid advances in artificial intelligence (AI) in the last decade have been primarily attributed to the applications of deep learning (DL) technologies. DL technologies have been widely applied to speech, audio, image, video, computer graphics, 3D models, and chatbots (e.g., ChatGPTs). These advances are viewed as the first AI wave. There are concerns with the first AI wave. DL solutions are a black box (i.e., not interpretable) and vulnerable to adversarial attacks (i.e., unreliable). Besides, the high carbon footprint yielded by large DL networks is a threat to our environment (i.e., not sustainable).

Many researchers are looking for an alternative solution that is interpretable, reliable, and sustainable. This is expected to be the second AI wave. To this end, MCL members have conducted research on green learning (GL) since 2015. Since GL was inspired by DL, the two share some similarities. As time goes by, the two become more and more different. Low carbon footprints, small model sizes, low computational complexity, and mathematical transparency characterize GL. It offers energy-effective solutions in cloud centers and mobile/edge devices. It has three main modules: 1) unsupervised representation learning, 2) supervised feature learning, and 3) decision learning. GL has been successfully applied to a few applications.

The fundamental ideas of GL solution, its demonstrated examples, and its technical outlook were detailed in a recent overview paper by Kuo and Madni, “Green learning: Introduction, examples, and outlook.” Journal of Visual Communication and Image Representation (2022): 103685. We expect to see more GL solutions and applications emerging. All MCL members will be fully devoted to this field in the next decade.

By |March 19th, 2023|News|Comments Off on Toward Interpretable, Reliable, and Sustainable AI – the 2nd AI Wave|

MCL Genealogical Ancestry Series: Johann Friedrich Pfaff

Zhiruo Zhou studied Johann Friedrich Pfaff and shared her study with MCL members in the pre-seminar sharing on Mar 6th, 2023. Johann Friedrich Pfaff was born 22 December 1765 in Stuttgart, Württemberg (now Germany), and died 21 April 1825 in Halle, Saxony (now Germany). He was an influential German mathematician and well known for his work on systems of partial differential equations.

Johann Friedrich was the second son in the family and had six brothers. Although he was probably the one that gained the most fame among his siblings, his brothers were also talented in various fields. His youngest brother Johann Wilhelm Pfaff was a mathematician, and his second youngest brother Christoph Heinrich Pfaff had experiences working on chemistry, medicine, pharmacy and electricity on animals.

Johann Friedrich was born in a family with the tradition of working for the government and therefore took the school named Hohe Karlsschule to receive training for government officials’ sons when he was nine years old. He spent around eleven years there and left when he had completed studies in law which was supposed to be a fitting subject for a civil servant. Driven by the interest in mathematics and encouragement from academy, Johann Friedrich decided to move toward scientific topics and studied mathematics and physics at the University of Göttingen for two years. Then he moved to Berlin to study astronomy and wrote his first paper on astronomy there.

In 1788, Johann Friedrich accepted the election as a professor of mathematics at the University of Helmstedt and stay there until 1810. He spent a lot of efforts on teaching and increasing the number of students of mathematics. He met the student Gauss there and had a good relationship with Gauss. However, [...]

By |March 12th, 2023|News|Comments Off on MCL Genealogical Ancestry Series: Johann Friedrich Pfaff|
  • Permalink Gallery

    MCL Research on Mask-Guided Image Synthesis Presented at AAAI-23

MCL Research on Mask-Guided Image Synthesis Presented at AAAI-23

Dr. Rouhsedaghat, a MCL alumna graduated last Summer, recently presented a work[1] on image synthesis related to her PhD thesis in AAAI-23. Here is the presentation summary from Dr. Rouhsedaghat:

We offer a method for one-shot mask-guided image synthesis that allows controlling manipulations of a single image by inverting a quasi-robust classifier equipped with strong regularizers. Our proposed method, entitled MAGIC , leverages structured gradients from a pre-trained quasi-robust classifier to better preserve the input semantics while preserving its classification accuracy, thereby guaranteeing credibility in the synthesis. Unlike current methods that use complex primitives to supervise the process or use attention maps as a weak supervisory signal, MAGIC aggregates gradients over the input, driven by a guide binary mask that enforces a strong, spatial prior. MAGIC implements a series of manipulations with a single framework achieving shape and location control, intense non-rigid shape deformations, and copy/move operations in the presence of repeating objects and gives users firm control over the synthesis by requiring to simply specify binary guide masks. Our study and findings are supported by various qualitative comparisons with the state-of-the-art on the same images sampled from ImageNet and quantitative analysis using machine perception along with a user survey of 100+ participants that endorse our synthesis quality.

[1]Rouhsedaghat, Mozhdeh, et al. “MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-Robust Classifier.” arXiv preprint arXiv:2209.11549 (2022).

By |March 5th, 2023|News, Research|Comments Off on MCL Research on Mask-Guided Image Synthesis Presented at AAAI-23|

MCL Research on Advanced Knowledge Graph Embedding

Knowledge graph embedding has been intensively studied in recent years. TransE, RotatE, and PairRE are the most representative and effective among distance-based KG embeddings that make use of simple translation, rotation, and scaling operations respectively to represent relations between entities. These models use simple geometric transformations yet achieve relatively good performance in link prediction. However, it is unclear how to leverage the strength of these individual operations and produce a better embedding model. It turns out that we can treat each operation as a basic building block, from which we can build the most effective KG embedding scoring functions. The composition of geometric operators is a well-established tool for image manipulation. It is natural to apply the same strategy to combine translation, rotation, and scaling operators in the 3D space. Our previous work CompoundE has demonstrated that the composition of affine operations can be an effective strategy in designing KG embedding. We are interested to know whether we can achieve even better results by cascading 3D operators.

We propose to extend our previous work by including more affine operations other than only translation, rotation, and scaling in our framework, and moving from 2D transformations to 3D transformations. In addition, we propose to enhance our CompoundE by addressing two critical issues. First, although Compounding operations lead to plenty of model variants, it is still unclear how to systematically search for a scoring function that performs the best for an individual dataset. We propose to use beam search to gradually build more complex scoring functions from simple but effective variants. To speed up the model tuning during the search, we use the pre-trained entity embedding from base models as initialization. Second, although ensemble learning is a popular [...]

By |February 26th, 2023|News|Comments Off on MCL Research on Advanced Knowledge Graph Embedding|

Celebration of SIPI 50th Anniversary

The SIPI 50th Anniversary Celebration was held on the USC Campus on Saturday, February 18th. The USC Signal and Image Processing Institute (SIPI) was one of the first research organizations in the world dedicated to image processing. Image processing work began at USC in 1962, and the Institute was founded in 1972 by William K. Pratt and Harry C. Andrews with support from the Defense Advanced Research Projects Agency (DARPA). SIPI has graduated around 500 PhDs over the last 50 years. As part of the SIPI 50th Anniversary celebration program, the inaugural SIPI Distinguished Alumni Awards have been established to honor USC SIPI graduates who have made professional and technical contributions that bring extraordinary distinction to themselves, the Institute, and the University.

Among the eight awardees, there are three MCL alumnus Ioannis Katsavounidis, Shan Liu and Chao Yang. Dr. Ioannis Katsavounidis won the senior academia award for outstanding contributions to objective video quality metrics and perceptual optimization of streaming videos. Dr. Shan Liu won the mid-career industry award for contributions to and leadership in multimedia technologies, standardization, and applications. Dr. Chao Yang won the junior industry award for contributions to computer vision, machine learning, and artificial intelligence in research publications and in commercial products and services. Congratulations to them!

Many MCL alumnus and current lab members joined the celebration and re-union. We had a great time together refreshing our memories on the past time and bridging different generations which composed of an important part of the MCL family. Happy 50th anniversary to SIPI and look forward to the brighter future for everyone!

 

Some information credit to:

https://minghsiehece.usc.edu/groups-and-institutes/sipi/sipi-50th-anniversary/

By |February 19th, 2023|News|Comments Off on Celebration of SIPI 50th Anniversary|
  • Permalink Gallery

    MCL Genealogical Ancestry Series: Karl Christian von Langsdorf

MCL Genealogical Ancestry Series: Karl Christian von Langsdorf

Hongyu Fu studied Karl Christian von Langsdorf and shared his study with MCL members in the pre-seminar sharing on February 6th, 2023. Karl Christian von Langsdorf, also known as Carl Christian von Langsdorff, was born 18 May 1757 in Nauheim, and died 10 June 1834 in Heidelberg. He was a German Mathematician, Geologist, Natural scientist and Saltwork Engineer. He is mostly well-known for his specialist in the area of saltworks engineering, and his subject to the 15-year-old Georg Ohm to a thorough examination of his knowledge of mathematics.

Karl Christian von Langsdorf’s father was Georg Melchior Langsdorff, who worked as saltworks archivist and master of the bursary in Nauheim, German. He finished his Gymnasium (secondary school) in Idstein 1773, and went to University of Göttingen to study with Abraham Gotthelf Kästner concerning Philosophy Law Mathematics in 1774, then he went to University of Giessen during 1776 – 1777. He finished his doctorate in 1781 in Erfurt, and then decided against academic career due to his health reasons.

Karl Christian von Langsdorf explored his early career in saltworks. Before his graduation, he  interned at the saltworks in Salzhausen, and devoted himself in Nidda, Hesse to the study of saltworks. After his graduation, he pursued a career in administration and became Rentmeister and land judge in Mülheim an der Ruhr. In 1784, he was active as a saltworks inspector in Gerabronn.

The academic career of Karl Christian von Langsdorf begins in 1798, he was given a full professorship in mechanical engineering at Erlangen, where he subjected the 15-year-old Georg Ohm to a thorough examination of his knowledge of mathematics. During 1803 – 1806, he taught mathematics and technology at Vilnius University in Russia, and returned with the aristocratic descriptor [...]

By |February 12th, 2023|News|Comments Off on MCL Genealogical Ancestry Series: Karl Christian von Langsdorf|

MCL Research on Knowledge Graph Entity Typing Prediction

Knowledge graph entity typing (KGET) is a task to predict the missing entity types in the knowledge graphs (KG). Previously, KG embedding (KGE) methods tried to solve the KGET task by introducing an auxiliary relation “hasType” to model the relationship between entities and their corresponding types. However, a single auxiliary relation has limited expressiveness for the diverse patterns between entities and types.

In this work, we try to assign more auxiliary relations based on the “context” of the types to improve the expressiveness of KGE methods. The context of a type is defined as a collection of attributes of the corresponding entities. As such, the neighborhood information is implicitly encoded when the auxiliary relations are introduced. Similar types might share the same auxiliary relation to model their relationship with entities. Fig. 1 shows an example of using multiple auxiliary relations to model the typing relationship for different entity types. From the figure, it’s intuitive to use different auxiliary relations to model typing relationships for “administrative district” and “person” since these two types are largely different from each other. In addition, for “writer” and “soccer player”, different auxiliary relations should also be considered since they shouldn’t be embedded closely to each other in the embedding space. However, some types, such as “writer and “lecturer”, co-occur with each other often so they can adopt the same auxiliary relation to model the relationships with entities.

In addition, we propose an iterative training scheme, named KGE-iter, to train KGE models for the KGET task. Fig. 2 is an illustration of the proposed iterative training scheme. The entity embeddings are first initialized by training with only factual triples. Then, typing information is used to fine-tune the entity embeddings. Two training stages [...]

By |February 5th, 2023|News|Comments Off on MCL Research on Knowledge Graph Entity Typing Prediction|

MCL Research on SO(3)-Invariant Point Cloud Classification

Usually, the early learning-based point cloud classification methods were developed under the assumption that all point clouds in the dataset are well aligned with the canonical axes. In such scenarios, the 3D Cartesian point coordinates were to learn features. As a consequence, when input point clouds were not aligned, the classification performance dropped significantly. The same assumption holds true in the PointHop and PointHop++ methods proposed by MCL.

In our work SO(3)-Invariant PointHop (or S3I-PointHop in short), we analyze the reason for failure of PointHop due to pose variations, and solve the problem by replacing its pose dependent modules with rotation invariant counterparts. Furthermore, we significantly simplify the PointHop pipeline by using only one single hop along with multiple spatial aggregation techniques. We begin by aligning the point cloud to its three principal axes. This offers a coarse alignment and comes with several ambiguities such as due to eigen vector sign and object asymmetries. The feature extraction process consists of constructing local and global point features. The geometric features are derived from distances and angles in a local point cloud neighborhood. Similarly, the covariance features are found by performing eigen decomposition of the local covariance matrix. The geometric and covariance features form the set of local features. The global features comprise of omni-directional octant features of points in the 3D space similar to PointHop. Later, the Saab transform is conducted.

For aggregating the local and global point features into a global shape feature, conical and spherical aggregation is proposed. For conical aggregation, along each positive and negative principal axes, cones with tip at the origin and at unit distance along the axis are constructed. Then, only the features of points lying inside the respective cones are [...]

By |January 29th, 2023|News|Comments Off on MCL Research on SO(3)-Invariant Point Cloud Classification|

Professor Kuo Being Elevated to 2022 ACM Fellow

ACM named 57 of its members ACM Fellows for wide-ranging and fundamental contributions in a wide range of disciplines related to computing in a press release issued on January 18, 2023.

MCL Director, Professor C.-C. Jay Kuo, was among the 57 Fellows of class 2022 for his contributions to technologies, applications, and mentorship in visual computing. Professor Kuo has been an influential and long-term leader in video technologies and applications for 30+ years, enduringly impacting both the academic and industry realms. He has made significant contributions to visual computing through industrial collaboration, standardization activities, and training of next-generation leaders. His lab at USC has contributed to collaborative sponsored projects from 70+ companies. He and his students have made key contributions adopted by international image and video coding standards. He has been granted 30 US patents. His video technologies have impacted people’s daily life from capturing or watching video with smartphones to viewing high-quality streamed video on large screens.

The ACM Fellows program recognizes the top 1% of ACM Members for their outstanding accomplishments in computing and information technology and excellent service to ACM and the larger computing community. Fellows are nominated by their peers and reviewed by a distinguished selection committee. Professor Kuo will go to San Francisco to attend the Fellow induction ceremony on June 10, 2023.

By |January 22nd, 2023|News|Comments Off on Professor Kuo Being Elevated to 2022 ACM Fellow|

MCL Research on Low-light Video Enhancement

Videos captured under low light conditions are often noisy and of poor visibility. Low-light video enhancement aims to improve viewers’ experience by increasing brightness, suppressing noise, and amplifying detailed texture.  The performance of computer vision tasks such as object tracking and face recognition can be severely affected under low-light noisy environments.  Hence, low-light video enhancement is needed to ensure the robustness of computer vision systems. Besides, the technology is highly demanded in consumer electronics such as video capturing by smart phones.

A self-supervised adaptive low-light video enhancement (SALVE) method is proposed in this work. SALVE first conducts an effective Retinex-based low-light image enhancement on a few key frames of an input low-light video. Next, it learns mappings from the low- to enhanced-light frames via Ridge regression.  Finally, it uses these mappings to enhance the remaining frames in the input video. SALVE is a hybrid method that combines components from a traditional Retinex-based image enhancement method and a learning-based method. The former component leads to a robust solution which is easily adaptive to new real-world environments. The latter component offers a fast, computationally inexpensive and temporally consistent solution. We conduct extensive experiments to show the superior performance of SALVE. Our user study shows that 87% of participants prefer SALVE over prior work.

First figure shows an overview of the proposed SALVE method.  For intra-coded frames (I frames), it estimates an illumination component and a reflectance component using the NATLE method.  For inter-coded frames (P/B frames), it predicts these components using a ridge regression learned from the last raw and enhanced I frame pairs.

Second figure shows a quantitative comparison table between our low-light video enhancement method and prior work. To further demonstrate the effectiveness of our method, we [...]

By |January 15th, 2023|News|Comments Off on MCL Research on Low-light Video Enhancement|