News

Sentence similarity evaluation has a wide range of applications in natural language processing, such as semantic similarity computation, text generation evaluation, and information retrieval. As one of the word-alignment-based methods, Word Mover’s Distance (WMD)[1] formulates text similarity evaluation as a minimum-cost flow problem. It finds the most efficient way to align the information between text sequences through a flow network defined by word-level similarities. By assigning flows to individual words, WMD computes text dissimilarity as the minimum cost of moving words’ flows from one sentence to another based on pre-trained word embeddings.

However, a naive WMD method does not perform well on sentence similarity evaluation for several reasons.
– First, WMD assigns word flow based on words’ frequency in a sentence. This frequency-based word weighting scheme is weak in capturing word importance when considering the statistics of the whole corpus.
– Second, the distance between words solely depends on the embedding of isolated words without considering the contextual and structural information of input sentences. Since the meaning of a sentence depends on individual words as well as their interaction, simply considering the alignment between individual words is deficient in evaluating sentence similarity.

MCL proposed a new syntax-aware word flow calculation method, Syntax-aware Word Mover’s Distance (SynWMD)[2], for sentence similarity evaluation.
– Words are first represented as a weighted graph based on the co-occurrence statistics obtained by dependency parsing trees. Then, a PageRank-based algorithm is used to infer word importance.
– The word distance model in WMD is enhanced by the context extracted from dependency parse trees, which is illustrated in Figure 1. The contextual information of words and structural information of sentences are explicitly modeled as additional subtree embeddings.
– As shown in Table 1, we [...]

Knowledge Graph encodes human-readable information and knowledge in graph format. Triples, denoted by (h,r,t), are basic elements of a KG, where h and t are head and tail entities while r is the relation connecting them. Both manual effort by domain experts and automated information extraction algorithms have contributed to the creation of many existing Knowledge Graphs today. However, given the limited information accessible to each individual and the limitation of algorithms, it is nearly impossible for a Knowledge Graph to perfectly capture every single piece of facts about the world. As such, Knowledge Graphs are often incomplete and many researchers have developed different algorithms to predict missing facts in Knowledge Graphs. Knowledge Graph Embedding models were first proposed to mainly solve the Knowledge Graph Completion problem. Besides, embedding models can also be useful in solving many downstream tasks such as entity classification and entity alignment.

MCL has been recently working on Effective Knowledge Graph Embedding. Translation, rotation, and scaling are three commonly used geometric manipulation operations in image processing. Besides, some of them are successfully used in developing effective knowledge graph embedding (KGE) models such as TransE and RotatE. Inspired by the synergy, we propose a new KGE model by leveraging all three operations. Since translation, rotation, and scaling operations are cascaded to form a compound one, the new model is named CompoundE. By casting CompoundE in the framework of group theory, we show that quite a few distanced-based KGE models are special cases of CompoundE. CompoundE extends the simple distance-based scoring functions to relation-dependent compound operations on head and/or tail entities. To demonstrate the effectiveness of CompoundE, we conduct experiments on three popular knowledge graph completion datasets. Experimental results show that CompoundE consistently [...]

Supervised learning is the main stream in pattern recognition, computer vision and natural language processing nowadays due to the great success of deep learning. On one hand, the performance of a learning system should improve as the number of training samples increases. On the other hand, some learning systems may benefit more than others from a large number of training samples. For example, deep neural networks (DNNs) often work better than classical learning systems that contain feature extraction and classification two stages. How the quantity of labeled samples affects the performance of learning systems is an important question in the data-driven era.

In fact, humans can learn effectively in a weakly supervised setting. In contrast, deep learning networks often need more labeled data to achieve good performance. What makes weak supervision and strong supervision different? There is little study on the design of supervision-scalable leaning systems. Is it possible to design a supervision-scalable learning system? Recently, MCL researchers attempt to shed light on these questions by choosing the object recognition problem as an illustrative example [1]. The design of two learning systems are presented that demonstrate an excellent scalable performance with respect to various supervision degrees. The first one adopts the classical histogram of oriented gradients (HOG) features, while the second one named improved PixelHop (IPHop) uses successive-subspace-learning (SSL) features [2]. The scalable learning system consists of three modules: representation learning, feature learning, and decision learning. In the second and the third modules, different designs are proposed to be adaptive to different supervision levels. Specifically, variance thresholding based feature selection and kNN classifier are used when the training size is small, while when the training size becomes larger, Discriminant Feature Test (DFT) [...]

The problem of facial expression recognition (FER) attempts to understand human emotion through facial image analysis. The technique can be applied to driver status monitoring, affective computing, and serious games. Solutions to FER can be categorized into two types: conventional methods and deep-learning-based (DL-based) methods. While conventional methods use hand-extracted features, DL-based methods conduct end-to-end optimization of certain networks whose performance highly depends on training data, the network architecture and the cost function. DL-based methods have become popular in recent years because of their higher performance. Yet, they demand a large model size. Although there has been research on reducing the number of parameters of DL models, it does not solve the computational complexity problem completely.

In this research, we are interested in a lightweight FER solution and name it ExpressionHop. ExpressionHop has low computational and memory complexity so that it is most suitable for mobile and edge computing environments. As shown in Figure 1, ExpressionHop consists of four modules: 1) cropping patches out based on facial landmarks, 2) applying filter banks to each patch to generate a rich set of joint spatial-spectral features, 3) conducting the discriminant feature test (DFT) to select features of higher discriminant power, and 4) performing the final classification task with a classifier. We conduct performance benchmarking on ExpressionHop, traditional and deep learning methods on several commonly used FER datasets such as JAFFE, CK+ and KDEF. Experimental results in table 1 show that ExpressionHop achieves comparable or better classification accuracy. Yet, its model size only has 30K parameters, which is significantly lower than those of deep learning methods.

As to the future research directions, there are several extensions to be pursued. First, it is desired to extend ExpressionHop to non-frontal [...]

MCL Director, Professor C.-C. Jay Kuo, was elected as one of 19 Academia Sinica’s 33rd Academicians. The news was announced on July 7, 2022. Professor Kuo was cited for his contributions to the fields of “multimedia computing” and “data science and engineering”.

Academia Sinica means ’Chinese Academy’. Its Chinese name is 中央硏究院. It was founded in 1928 in Nanjing and relocated to in Nangang, Taipei, in 1949. It is the national academy of the Republic of China (Taiwan). Academia Sinica supports research activities in a wide variety of disciplines, ranging from mathematical & physical sciences, to life sciences, and to humanities and social sciences. As an educational institute, it provides PhD training and scholarship through its English-language Taiwan International Graduate Program.

Professor Kuo acknowledged MCL alumni for his achievements, saying that “This honor is not only a recognition of me but also the outstanding performance of MCL alumni all over the world.” Furthermore, Professor Kuo was thankful to the strong support of the University of Southern California, the Viterbi School of Engineering, and the Ming-Hsieh Department of Electrical and Computer Engineering in the last three decades. He said, “Without the strongest support of the university, school and department, this would not happen at all.”