MCL Research on Explainable Deep Learning
The deep learning technologies such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have great impacts on modern machine learning due to their impressive performance in many application fields that involve learning, modeling, and processing of complex sensing data. Yet, the working principle of deep learning remains mysterious. Furthermore, it has several well-known weaknesses: 1) vulnerability to adversarial attacks, 2) demanding heavy supervision, 3) generalizability from one domain to the other. Professor Kuo and his PhD students at Media Communications Lab (MCL) have been working on explainable deep learning since 2014 and published a sequence of pioneering papers on this topic.
Explanation of nonlinear activation, convolutional filters and discriminability of trained features of CNNs [1]-[3]. The role of CNN’s nonlinear activation function is well explained in [1] at the first time. That is, the nonlinear activation operation is used to resolve the sign confusion problem due to the cascade of convolutional operations in multiple layers. This work received the 2018 best paper award from the Journal of Visual Communication and Image Representation. The convolutional filters is viewed as a rectified correlations on a sphere (RECOS) and CNN’s operation is interpreted as a multi-layer RECOS transform in [2]. The discriminability of trained features of a CNN at different convolution layers is analyzed using two quantitative metrics in [3] – the Gaussian confusion measure (GCM) and the cluster purity measure (CPM), The analysis is validated by experimental results.
Saak transform and its application to adversarial attacks [4]-[5]. Being inspired by deep learning, we develop a new mathematical transform called the Saak (Subspace approximation with augmented kernels) transform in [4]. The Saak and inverse Saak transforms provide signal analysis and synthesis tools, respectively. CNNs are known to [...]