News

  • Permalink Gallery

    MCL Research on 3D Perception with Large Foundational Models

MCL Research on 3D Perception with Large Foundational Models

Understanding and retrieving information in 3D scenes poses a significant challenge in artificial intelligence (AI) and machine learning (ML), particularly in grasping complex spatial relationships and detailed properties of objects in 3D spaces. Multiple tasks are suggested to assess 3D understanding, such as 3D object retrieval, 3D captioning, 3D question answering, 3D vision grounding, etc.

Existing methods can be roughly divided into two categories. The first category utilizes large 2D foundational models for feature extraction and maps 2D pixel-wise features to 3D point-wise features for 3D tasks. For example, the 3D-CLR model [1] extracts 2D features from multiview images with the CLIP-LSeg model [2] and maps the 2D features to 3D points in a reconstructed neural radiance field compact representation. The reasoning process is performed via a set of neural reasoning operators. The 3D-LLM model [3] utilizes 2D vision-language models (VLM) as the backbone. It extracts 2D features with the ConceptFusion model [4] and maps them to 3D points. Then, the 3D information is injected into a large language model to generate text outputs.

Another group of methods directly handles 3D point clouds with a 3D encoder and tries to align the extracted 3D features with the features from other modalities. This group of methods may require the training of a 3D encoder and may need many computational resources. For example, the Uni3D [5] leverages a unified vanilla transformer structurally equivalent to a 2D Vision Transformer (ViT) as the backbone to extract 3D features. Downstream tasks can be achieved after feature alignment among different modalities. It is also possible to leverage pre-trained 3D encoders. Point-SAM [6] utilizes the point cloud encoder from the Uni3D to transform the input point cloud into embeddings. It starts by sampling [...]

By |July 14th, 2024|News|Comments Off on MCL Research on 3D Perception with Large Foundational Models|

MCL Research on Prostate MRI Image Segmentation

Magnetic resonance imaging (MRI) is a good way to detect clinically significant prostate cancer and guide biopsies, due to the superior resolution and contrast of imaging, without harming the human body. Based on prostate MRI, prostate segmentation is a process to localize prostate boundaries for radiotherapy and automate the calculation of the prostate volume. Automatic prostate segmentation is an important step in computer-aided diagnosis of prostate cancer and treatment planning [1].

It is very hard to collect and obtain large annotated datasets for AI In Healthcare. We worked with USC Keck Medical School on this project, and they provided us with a large medical dataset, which was very precious and helpful. In addition to this dataset, we also used some public datasets like ISBI-2013 [2] and PROMISE-12[3] to analyze and evaluate our Green U-Shaped Learning (GUSL) methodology.

Our Green U-Shaped Learning (GUSL) framework is a feed-forward encoder-decoder system based on successive subspace learning (SSL), and it consists of two modules: 1) encoder: fine to coarse unsupervised representation learning with cascaded VoxelHop units, and 2) decoder: coarse to fine segmentation prediction with voxel-wise regression and local error correction. Our model is lightweight and totally transparent while keeping comparable performance.

We have done 5 cross-validations for the dataset from USC Keck Medical School. For T2-cube MRIs, the Dice Similarity Coefficient (DSC) of the prostate segmentation was over 93%. The USC Keck Medical School doctors were very satisfied with these impressive results. In the next step, we will apply our GUSL model to some public datasets and then compare and analyze the performance of our method with some state-of-the-art Deep Learning methods. In the future, we aim to develop methods for segmenting other organs like cardiac. I hope our methods [...]

By |July 7th, 2024|News|Comments Off on MCL Research on Prostate MRI Image Segmentation|

Professor Kuo Met MCL Alumni in Thailand

Professor C.-C. Jay Kuo, Director of MCL, visited Bangkok, Thailand, from June 22-26 to reunite with MCL alumni on his Asian trip. There are three MCL alumni in Thailand. They are:• Junavit Chalidabhongse (Lecturer, Faculty of Law, Thammasat University)• Wuttipong Kumwilaisak (Professor, King Mongkut’s University of Technology)• Tanaphol Thaipanich (CEO, Push Media Co., Ltd.)It has been 15 years since Professor Kuo’s last visit to Bangkok, and he received warm hospitality. Professor Kuo was proud of the outstanding performance of MCL alumni in both academia and industry.The ECTI Association in Thailand and the IEEE Thailand Section invited Professor Kuo to deliver a “Recent Developments and Outlook in Green AI/ML” seminar at the Faculty of Engineering, Chulalongkorn University, on June 24. The seminar was well attended. Professor Kuo had several sightseeing tours in Bangkok and enjoyed his spare time in Thailand.

By |June 30th, 2024|News|Comments Off on Professor Kuo Met MCL Alumni in Thailand|

Professor Kuo visited Singapore 

Professor C.-C. Jay Kuo, Director of MCL, visited Singapore from June 17-23 and attended two events in his Asia trip.For the first event, Professor Kuo gave a seminar titled “Demystify Artificial Intelligence and Technology Outlook” at the Nanyang Technological University on June 20 (Wednesday). The abstract of his talk is given below.“The term “Artificial Intelligence (AI)” was coined in 1956. Although the field evolved slowly in the first 55 years, we have witnessed rapid advances in AI in the last decade (e.g., the ChatGPT service). Questions are raised about AI’s role in human society and civilization, e.g., whether AI will replace human intelligence (HI), how HI and AI complement each other, etc. I will shed light on them. Then, I will comment on the limitations of today’s deep-learning-based (DL) AI models. DL-based models are neither interpretable nor sustainable. An alternative methodology is desired. To this end, I have investigated a new statistically based AI/ML framework called “Green Learning” (GL). GL significantly reduces the model size and complexity of DL models while yielding competitive performance and allowing mathematical transparency. GL adopts the feedforward one-pass training pipeline, so all intermediate results are explainable. Finally, I will address the AI role in human society and issues with today’s chatbots, such as bias and fairness.”Professor Kuo’s talk was hosted by Professor Woon Seng Gan at NTU. He also had a reunion with Professor Jiaying Liu of Peking University. Professor Liu was a visiting PhD student at MCL from August 2007 to August 2008.For the 2nd activity, Professor Kuo delivered a keynote speech at the Workshop titled “Emerging Trends and Innovations in Machine Learning and AI” on June 21 (Friday) organized by APSIPA Singapore Chapter and quite a few [...]

By |June 23rd, 2024|News|Comments Off on Professor Kuo visited Singapore |

Professor Kuo met MCL Alumni in Taiwan

Professor C.-C. Jay Kuo, Director of MCL, attended the Picture Coding Symposium (PCS) in Taichung, Taiwan, from June 12-14, 2024. Professor Kuo was invited to be a panelist for a panel program. The discussion topic was “Learned Image and Video Coding: Hype or Hope?” (see https://2024.picturecodingsymposium.org/panel/). All six panelists were optimistic about researching and developing learned image and video coding technologies for various reasons. Professor Kuo emphasized the differences between the classical and the modern learned coding methodologies. The former considers intra-content redundancy removal, while the latter examines inter-content redundancy removal. The former has been researched for four decades and reached its maturity. It isn’t easy to push further. The latter does have more opportunities. On the other hand, Professor Kuo was concerned about the high complexity and black-box nature of neural network codecs. He suggested an alternative non-neural-network-based approach to implement learned image and video codecs.After PCS, Professor Kuo went to Taipei for a reunion luncheon with MCL alums on June 15 (Saturday). Professor Kuo said, “It was a relaxing time during a busy week. Seeing our alums doing well in their careers and families was great.”

By |June 16th, 2024|News|Comments Off on Professor Kuo met MCL Alumni in Taiwan|

MCL Research on Nuclei Segmentation for Histological Images

Nuclei segmentation is a fundamental task required to analyze the underlying nuclei structure of an organ of interest. Cancer starts from the cells, and understanding the nuclei shapes, sizes and distribution can provide cues on whether or not a patient has cancer. Further analysis can also help in cancer grading and prognosis. However, studying whole slide images of biopsied tissues requires a large amount of time and effort. 

Such monotonous and laborious tasks can be simplified by using AI and ML, and can perhaps improve the accuracy of the detections as well. Some challenges in the nuclei segmentation task include inherent staining variations in the WSI, a wide variety of shapes and sizes in nuclei, and irregular boundaries which make it difficult to track the actual contours. Most of the current research in this area involves deep learning based architectures like the U-Net, R-CNN, and even Vision Transformer. These methods require a large number of training samples and high complexity to achieve generalization among the variations inherent in nuclei from different organs.

We propose to use a light-weight, interpretable, and simple Green Learning based approach to perform Nuclei Segmentation. Prior work on highly effective Unsupervised Nuclei Instance Segmentation (HUNIS) [1] forms the first stage of our current approach. To further improve HUNIS results, we now focus on the regions where HUNIS requires the help of labels. We divide our current task into two stages: (i) to identify those areas where we need help and (ii) to correct those areas towards their actual class. With the help of Saab Transform, our main task now is to perform feature engineering to identify the ideal features to implement the above two stages. 

References:

[1] V. Magoulianitis, Y. Yang, and C.-C. J. [...]

By |June 9th, 2024|News|Comments Off on MCL Research on Nuclei Segmentation for Histological Images|

MCL Research on Seismic Data Processing

Seismic data processing involves detecting earthquake signals and picking seismic phases from the diverse types of signals recorded by seismographs. During a seismic event, energy radiates from the focus (or hypocenter) as waves travel in all directions. These waves are categorized into body waves and surface waves. Understanding and accurately detecting these waves are crucial for rapid response and seismic hazard assessment.

Types of Seismic Waves

Body Waves: These waves travel through the Earth’s interior and are divided into two types:

P Waves (Primary Waves): The fastest waves, P waves are the first to be recorded on seismographs. They can travel through both solid and liquid media, causing the ground to move forward and backward.

S Waves (Secondary Waves): Following the P waves, S waves travel more slowly and cause a swinging motion that moves the ground up and down. S waves only travel through solid materials.

Surface Waves: Generated when body waves reach the Earth’s surface, surface waves spread out over the Earth’s surface. They are typically more destructive and damaging than body waves due to their larger amplitude and longer duration.

Importance of Seismic Phase Picking

Modern seismic networks continuously generate vast amounts of data. Manual analysis of this data is impractical due to the need for rapid response. Additionally, seismic data often contain significant noise and ambiguous signals, complicating interpretation. Accurate and efficient detection and phase picking are vital for reliable seismic event characterization, crucial for understanding seismic hazards and responding to potentially damaging earthquakes.

Deep Learning Approaches

Several deep learning (DL) models have been developed for seismic phase picking, including:

Generalized Phase Detector [1]

PhaseNet [2]

Earthquake Transformer [3]

These models achieve high accuracy in P-phase picking and can accurately identify the arrival of S waves, even when overlapped with the coda of [...]

By |June 2nd, 2024|News|Comments Off on MCL Research on Seismic Data Processing|

MCL Research on Point Cloud Surface Reconstruction

Surface reconstruction from point cloud scans plays a pivotal role in 3D vision and graphics, with diverse applications in areas such as AR/VR games, heritage preservation, and indoor/outdoor scene reconstruction. This task is inherently challenging due to the ill-posed nature of reconstructing continuous surfaces from discrete points. Furthermore, real-world point cloud scans introduce several obstacles, such as varying densities, sensor noise, and missing parts. These properties make the problem a long-standing one, continually driving researchers to seek more effective solutions.

Early research focused on constructing watertight objects using combinatorial methods [1][2], which inferred the connectivity between points directly. The mainstream of surface reconstruction adopts an implicit surface approach [3][4], where the surface is represented as an unknown continuous function solved by associated partial differential equations (PDEs). Although these methods offer good quality, they are constrained to predicting watertight objects and cannot handle highly distorted LiDAR scenes. Recently, indoor/outdoor scene reconstruction has gained more attention, with deep learning (DL) models [5] demonstrating success in solving this problem based on a supervised learning framework.

Despite their high reconstruction quality, DL models face challenges in generalizability and complexity. In scenarios such as point cloud compression, quality assessment, and dynamic point cloud processing, there is a growing need for low-complexity, low-latency surface reconstruction methods. However, existing DL-based methods often sacrifice simplicity for high reconstruction quality, leaving a gap for low-complexity solutions. We aim to develop an unsupervised few-shot learning method to achieve reconstruction for scenes with low complexity.

Building on our previous unsupervised framework (GPSR), we propose an enhanced version to handle non-watertight indoor/outdoor scenes, named Green Point Cloud Surface Reconstruction++ (GPSR++). The main idea involves building an unsigned distance field (UDF) through approximated heat diffusion and optimizing the surface using [...]

By |May 26th, 2024|News|Comments Off on MCL Research on Point Cloud Surface Reconstruction|

Congratulations to Chengwei Wei for Passing His Defense

Congratulations to Chengwei Wei for passing his defense. Chengwei’s thesis is titled “Syntax-aware natural language processing techniques and their applications.” His Dissertation Committee includes Jay Kuo (Chair), Antonio Ortega, and Swabha Swayamdipta (Outside Member). The Committee members were impressed by the wide range of topics conducted in Chengwei’s Ph.D. research.  Many thanks to our lab members for participating in his rehearsal and providing valuable feedback. The MCL News team invited Chengwei for a short talk on his thesis and PhD experience. Here is the summary. We thank Chengwei for his kind sharing and wish him all the best on his next journey. A high-level abstract of Chengwei’s thesis is given below:

”Syntax in language processing controls the structure of textual data, playing a crucial role in textual data understanding and generation. For example, syntax in natural language sentences governs the relationships between words, which is crucial for grasping the sentence’s overall meaning. In this thesis, we focus on two primary objectives: 1) Develop efficient methods for constructing syntactic structures. 2) Investigate the significance of syntax and integrate syntax-aware techniques into various Natural Language Processing (NLP) applications, spanning from word-level, sentence-level, document-level, and structured-data-level tasks.”

Chengwei shared his Ph.D. experience at MCL as follows :

I would first like to express my gratitude to Prof. Kuo for his guidance, patience, and unwavering support throughout this journey. His passion for research has been truly inspiring, leading me to join the lab in the summer of 2019 and driving me to pursue excellence in my academic endeavors. I am also thankful for his visionary insights into future research directions, exemplified by our collaborative effort on a survey paper on language models in the summer of 2022. This collaboration proved prescient, [...]

By |May 19th, 2024|News|Comments Off on Congratulations to Chengwei Wei for Passing His Defense|
  • Permalink Gallery

    Congratulations on MCL Members Attending Ph.D. Hooding Ceremony

Congratulations on MCL Members Attending Ph.D. Hooding Ceremony

Three MCL members attended the Viterbi PhD hooding ceremony on Wednesday, May 8, 2024, in the Bovard Auditorium. They were Zhanxuan Mei, Chengwei Wei and Wei Wang. Congratulations to them for their accomplishments in completing their PhD program at USC!

Zhanxuan Mei received his Bachelor’s degree in Electrical Engineering from Beijing Institute of Technology, China, in June 2018. He joined the Media Communication Lab in the summer of 2020. His research interests include image processing and video processing.

Chengwei Wei received his bachelor’s degree at Central South University, China in Jun 2018. He joined the Media Communications Lab in the summer of 2019. His research interests include signal processing, natural language processing, and machine learning. His thesis is titled “Syntax-aware Natural Language Processing Techniques and Their Applications”. 

Wei Wang received her bachelors in Applied Physics from Northeastern University (CN), and her MS degree in Materials Engineering from the University of Southern California in 2014 and 2016, respectively.. Her research interests include deep learning and image processing.

Congratulations to them all! Let us wish them all the best in the future!

By |May 12th, 2024|News|Comments Off on Congratulations on MCL Members Attending Ph.D. Hooding Ceremony|