News

Congratulations to Xinyu Wang for passing her defense! Xinyu’s thesis is titled “ Towards Efficient Visual Perception: From Feature Learning to Visual Reasoning.” Here is a brief summary of her thesis:

Visual perception serves as a fundamental component of modern computer vision, enabling the interpretation of large-scale image and video data. As visual data continues to grow in complexity, there is an increasing demand for efficient and scalable frameworks that bridge low-level representations and high-level understanding. This dissertation addresses this challenge by exploring a unified trajectory of efficient visual perception, evolving from feature learning to spatio-temporal modeling and ultimately to scene-conditioned visual reasoning. This dissertation first introduces a statistics-based feature generation framework for image classification, built upon the Least-squares Normal Transform, which reformulates classification as a regression problem for efficient feature learning. It generates discriminative and complementary features, boosting decision learning and training convergence with low computational overhead. It then investigates a particularly challenging visual task, video camouflaged object detection. The proposed GreenVCOD is a lightweight framework that captures temporal context through a Temporal Neighborhood prediction cube, enabling implicit motion modeling without additional computational cost. Building upon this, IDM-VCOD introduces a dual-motion design that combines implicit semantic refinement with explicit motion alignment, along with a selective activation mechanism to balance accuracy and efficiency. Finally, this dissertation shifts toward a reasoning-based paradigm for visual perception. Camouflaged object detection is reformulated as a scene-conditioned pattern-deviation reasoning problem. By leveraging background-aware retrieval and prototype-based reasoning, it identifies subtle deviations without relying on pixel-level supervision. Overall, the proposed methods demonstrate that lightweight design, combined with structured statistical modeling and scene-conditioned reasoning, can effectively address challenging visual perception tasks without relying on heavy supervision or large-scale model training.

We would like to congratulate Kevin Yang for receiving his Ph.D. degree during the Viterbi Hooding Ceremony held on May 13, 2026, at the Bovard Auditorium. Here is a brief sharing of his Ph.D. experience at MCL:

My Ph.D. journey at the University of Southern California has been both challenging and rewarding. My dissertation, titled “Interpretable and Efficient Multimodal Data Interplay: Algorithms and Applications,” focuses on developing machine learning methods that better connect and understand information across different modalities, such as text and images. Through this research, I explored ways to improve the interpretability and efficiency of multimodal systems while designing algorithms that better align with human perception and understanding. This experience strengthened my passion for artificial intelligence and its potential to create meaningful real-world impact.

I am deeply grateful to my advisor, Professor C.-C. Jay Kuo, for his continuous guidance, encouragement, and support throughout my Ph.D. journey. His mentorship has profoundly influenced both my research perspective and personal growth. I would also like to sincerely thank my labmates for their collaboration, friendship, and inspiring discussions along the way. The supportive environment in our lab made challenging moments manageable and accomplishments even more meaningful. I am excited to continue this journey by joining LinkedIn as an AI Engineer in June 2026, where I hope to contribute to the development of impactful AI systems and applications. I will always cherish the experiences, lessons, and relationships built during my Ph.D. years.

Our recent work explores how multi-modality memory can improve medical agents for visual question answering. By storing and retrieving useful information from medical images, patient queries, and clinical reports, the agent can better understand complex cases and provide more accurate and context-aware responses. This direction shows the potential of memory-enhanced medical AI systems for supporting medical VQA tasks.

Understanding animal motion behavior is important for understanding how the brain organizes memory, decision-making, and goal-directed action. In our research, we study mouse navigation behavior using the Morris Water Maze (MWM), a widely used behavioral paradigm for investigating spatial learning and memory in rodents. While conventional measures such as escape latency and path length provide useful summaries of performance, they often do not fully capture the rich and dynamic nature of movement during navigation.

Our research focuses on understanding mouse motion behavior at a finer temporal scale. Instead of treating each trial as a single behavioral unit, we examine how navigation strategies evolve within a trial, since mice may shift among exploration, wall-following, scanning, circling, and more direct platform-oriented movement over time. These within-trial changes can offer deeper insight into learning processes, behavioral flexibility, and group differences that may be overlooked by aggregate metrics alone.

To support this goal, we develop an interpretable and lightweight computational framework for analyzing tracked trajectories from behavioral videos. Our approach analyzes motion continuously over time and identifies sub-trajectory-level navigational states. The pipeline first corrects for tracking irregularities through uniform resampling and smoothing, then derives geometry- and kinematics-based descriptors such as curvature, displacement, turning behavior, and target alignment. These features are mapped to human-readable behavioral categories through a hierarchical rule-based inference process, followed by temporal refinement to reduce fragmented or implausible label switching.

This work emphasizes interpretability and practical usability in neuroscience research. By providing point-wise annotations and visually verifiable outputs, the framework enables behavioral phenotyping and hypothesis-driven analysis of strategy transitions and learning dynamics.

MCL members, Jintang Xue and Kevin Yang, presented their papers at the Winter Conference on Applications of Computer Vision (WACV) 2026, Tucson, AZ, USA.

The title of Jintang et al.’s paper is “Descrip3D: Enhancing Large Language Model-based 3D Scene Understanding with Object-Level Text Descriptions”. Here is a brief summary:

“Understanding 3D scenes goes beyond simply recognizing objects; it requires reasoning about the spatial and semantic relationships between them. Current 3D scene-language models often struggle with this relational understanding, particularly when visual embeddings alone do not adequately convey the roles and interactions of objects. In this paper, we introduce Descrip3D, a novel and powerful framework that explicitly encodes the relationships between objects using natural language. Unlike previous methods that rely only on 2D and 3D embeddings, Descrip3D enhances each object with a textual description that captures both its intrinsic attributes and contextual relationships. These relational cues are incorporated into the model through a dual-level integration: embedding fusion and prompt-level injection. This allows for unified reasoning across various tasks such as grounding, captioning, and question answering, all without the need for task-specific heads or additional supervision. When evaluated on five benchmark datasets, including ScanRefer, Multi3DRefer, ScanQA, SQA3D, and Scan2Cap, Descrip3D consistently outperforms strong baseline models, demonstrating the effectiveness of language-guided relational representation for understanding complex indoor scenes.”

Kevin’s paper is entitled “SVD-Det: A Lightweight Framework for Video Forgery Detection Using Semanticand Visual Defect Cues”, co-authored with Tianyu Zhang, Feng Qian, Bing Yan, and C.-C. Jay Kuo. The summary goes as follows:

“With the rapid proliferation of AI-generated content (AIGC) on multimedia platforms, efficient and reliable video forgery detection has become increasingly important. Existing approaches often rely on either visual artifacts or semantic inconsistencies, but suffer from high computational costs, [...]