News

  • Permalink Gallery

    MCL Research on Green Saliency-guided Blind Image Quality Assessment (GSBIQA)

MCL Research on Green Saliency-guided Blind Image Quality Assessment (GSBIQA)

Objective image quality assessment (IQA) is pivotal in various multimedia applications. It can be categorized into three distinct types: Full-Reference IQA (FR-IQA), Reduced-Reference IQA (RR-IQA), and No-Reference IQA (NR-IQA). FR-IQA directly compares a distorted image against a reference or original image to assess quality. RR-IQA, on the other hand, uses partial information from the reference images to evaluate the quality of the target images. NR-IQA, also known as blind image quality assessment (BIQA), becomes essential in scenarios where reference images are unavailable, such as at the receiver’s end or for user-generated content on social media. The demand for BIQA has surged with the increasing popularity of such platforms. BIQA is an essential task that estimates the perceptual quality of images without reference. This field is increasingly relevant due to the rise in user-generated content and mobile applications where reference images are typically unavailable.

The challenge in BIQA lies in the diversity of content and the presence of mixed distortion types. While many BIQA methods employ deep neural networks (DNNs) and incorporate saliency detectors to enhance performance, their large model sizes limit deployment on resource-constrained devices.

To address this challenge, we introduce a novel and non-deep-learning BIQA method with a lightweight saliency detection module, called Green Saliency-guided Blind Image Quality Assessment (GSBIQA). It is characterized by its minimal model size, reduced computational demands, and robust performance. The lightweight saliency detector in GSBIQA facilitates data cropping and decision ensemble and generates useful features in BIQA that emulate the attention mechanism. The GSBIQA method is structured around five key processes: 1) green saliency detection, 2) saliency-guided data cropping, 3) Green BIQA feature extraction, 4) local patch prediction, and 5) saliency-guided decision ensemble. Experimental results show that the performance of [...]

By |August 11th, 2024|News|Comments Off on MCL Research on Green Saliency-guided Blind Image Quality Assessment (GSBIQA)|

MCL Research on Prostate Lesion Detection from MRI Images

Research in healthcare systems has been mainly focused on automating several tasks in the clinical pipeline, aiming at enhancing or expediting physician’s diagnosis. Prostate Cancer (PCa) is widely known as one of the most frequently occurring cancer in diagnosis in men. If early diagnosed, the mortality rate is almost zero. Yet, should it goes under the radar -in the metastasis stage- that rate plummets at 31% [1]. For diagnosis, after a high level of Prostatic Specific Antigen (PSA), patients are recommended to undergo an MRI screening. Those patients with suspiciously looking lesions on the prostate gland will eventually undergo biopsy. The histology gives the definite answer whether a patient suffers from cancer. Nevertheless, it is observed in real practice that urologists tend to over-diagnose patients with csPCa, thus increasing the number of unnecessary biopsies. As such, this increases the diagnostic costs and patient discomfort.

Computer vision empowered with AI has shown promising results in the last decade. Computer-Aided Diagnosis (CAD) tools benefit from AI’s rapid evolution and many works have been proposed to automatically perform lesion detection and segmentation. Even though the Deep Learning (DL) paradigm is ubiquitous in modern AI, medical applications require more transparency behind feature extraction and thereby DL is often deemed as a “black-box” from physicians. Our proposed pipeline, PCa-RadHop [2], employs a novel and linear module for data-driven feature extraction and hence the decision making becomes more interpretable. PCa-RadHop receives three different modalities as input from the MRI-scanner (i.e. T2w, ADC, DWI), pertinent to PCa diagnosis. It consists of two stages. The first stage calculates a probability map about csPCa presence in a voxel-wise manner, while the second stage is meant to reduce the false positive rate on that heatmap [...]

By |August 4th, 2024|News|Comments Off on MCL Research on Prostate Lesion Detection from MRI Images|

MCL Research on Green Image Super-resolution

Single image super-resolution (SISR) is an intensively studied topic in image processing. It aims at recovering a high-resolution (HR) image from its low-resolution (LR) counterpart. SISR finds wide real-world applications such as remote sensing, medical imaging, and biometric identification. Besides, it attracts attention due to its connection with other tasks (e.g., image registration, compression, and synthesis). To deal with such ill-posed problem, we recently proposed two methods, LSR[1] and LSR++[2], by providing reasonable performance and effectively reduced complexity.

LSR consists of three cascaded modules:

Unsupervised Representation Learning by creating a pool of rich and diversified representations in the neighborhood of a target pixel.

Supervised Feature Learning by Relative Feature Test (RFT [3]) to select a subset from the representation pool that is most relevant to the underlying super-resolution task automatically, and

Supervised Decision Learning by predicting the residual of the target pixel based on the selected features through regression via classical machine learning, and effectively fusioning the predictions for more stable results.

LSR++ is promoted based on LSR, with emphasis on sample alignment, a more promising sample preparation process which is suitable for all patch-based computer vision problems. As illustrated in Fig 1, based on gradient histograms of patches along the eight reference directions (Fig.1.a), patch alignment utilizes patch rotations and flipping to meet the standard templates of gradient histograms, where D_max is the direction with the largest cumulative gradient magnitude, and D_max_orth_b and D_max_orth_s refer to the orthogonal directions to D_max with big and small cumulative gradient magnitude, respectively. By modifying the set of (D_max, D_max_orth_b, and D_max_orth_s) of a patch, patch alignment can regularize the edge pattern with the patch by directions perpendicular the edge (D_max) and directions along the edge (D_max_orth_b, D_max_orth_s). The process of patch [...]

By |July 28th, 2024|News|Comments Off on MCL Research on Green Image Super-resolution|

Professor Kuo Attended ICME in Niagara Falls, Canada

Professor C.-C. Jay Kuo, Director of MCL, attended the IEEE Conference on Multimedia Exposition (ICME) held in Niagara Falls, Canada, from July 15-19, 2024. Professor Kuo had dual roles in this conference as a Panel co-Chair and a keynote speaker. Professor Kuo gave his keynote on 7/18 (Thursday) on “Toward Interpretable and Sustainable AI via Green Learning.” Besides, Dr. Kuo and Dr. Zicheng Liu of AMD organized a panel as summarized below.

Panel Title: Generative AI – Opportunities, Challenges, and Open Questions

Panel Background: Generative AI has received a lot of attention due to the tremendous success of ChapGPT. Large foundation models have been trained, leading to various demos and potential applications such as text-to-image and text-to-video cross-domain generations. Resources have been invested in building massive computational and storage infrastructures. Furthermore, data collection and cleaning are essential to high system performance. In the face of these rapid developments, this panel will discuss opportunities, challenges, and open questions associated with generative AI.

Four panelists were invited:

Rogerio Feris, IBM Research

Lijuan Wang, Microsoft Research

Jiebo Luo, University of Rochester

Junsong Yuan, State University of New York at Buffalo

Q&A topics:

Today’s generative AI is tilted more toward “engineering” than “science.” Will this be a concern in the long run?

What are the major shortcomings of the current large foundation models?

How vital are “data collection and cleaning” tasks in generative AI? How do large companies carry out such tasks? Will we run out of data? If so, how soon?

Will “copyright,” “plagiarism,” and “hallucination” be issues? How can we address them? How can we trust the answers?

What roles can small AI companies and academia with limited resources play?

What is the future R&D direction of generative AI? What will be the next big breakthroughs?

Dr. Kuo also had a [...]

By |July 21st, 2024|News|Comments Off on Professor Kuo Attended ICME in Niagara Falls, Canada|
  • Permalink Gallery

    MCL Research on 3D Perception with Large Foundational Models

MCL Research on 3D Perception with Large Foundational Models

Understanding and retrieving information in 3D scenes poses a significant challenge in artificial intelligence (AI) and machine learning (ML), particularly in grasping complex spatial relationships and detailed properties of objects in 3D spaces. Multiple tasks are suggested to assess 3D understanding, such as 3D object retrieval, 3D captioning, 3D question answering, 3D vision grounding, etc.

Existing methods can be roughly divided into two categories. The first category utilizes large 2D foundational models for feature extraction and maps 2D pixel-wise features to 3D point-wise features for 3D tasks. For example, the 3D-CLR model [1] extracts 2D features from multiview images with the CLIP-LSeg model [2] and maps the 2D features to 3D points in a reconstructed neural radiance field compact representation. The reasoning process is performed via a set of neural reasoning operators. The 3D-LLM model [3] utilizes 2D vision-language models (VLM) as the backbone. It extracts 2D features with the ConceptFusion model [4] and maps them to 3D points. Then, the 3D information is injected into a large language model to generate text outputs.

Another group of methods directly handles 3D point clouds with a 3D encoder and tries to align the extracted 3D features with the features from other modalities. This group of methods may require the training of a 3D encoder and may need many computational resources. For example, the Uni3D [5] leverages a unified vanilla transformer structurally equivalent to a 2D Vision Transformer (ViT) as the backbone to extract 3D features. Downstream tasks can be achieved after feature alignment among different modalities. It is also possible to leverage pre-trained 3D encoders. Point-SAM [6] utilizes the point cloud encoder from the Uni3D to transform the input point cloud into embeddings. It starts by sampling [...]

By |July 14th, 2024|News|Comments Off on MCL Research on 3D Perception with Large Foundational Models|

MCL Research on Prostate MRI Image Segmentation

Magnetic resonance imaging (MRI) is a good way to detect clinically significant prostate cancer and guide biopsies, due to the superior resolution and contrast of imaging, without harming the human body. Based on prostate MRI, prostate segmentation is a process to localize prostate boundaries for radiotherapy and automate the calculation of the prostate volume. Automatic prostate segmentation is an important step in computer-aided diagnosis of prostate cancer and treatment planning [1].

It is very hard to collect and obtain large annotated datasets for AI In Healthcare. We worked with USC Keck Medical School on this project, and they provided us with a large medical dataset, which was very precious and helpful. In addition to this dataset, we also used some public datasets like ISBI-2013 [2] and PROMISE-12[3] to analyze and evaluate our Green U-Shaped Learning (GUSL) methodology.

Our Green U-Shaped Learning (GUSL) framework is a feed-forward encoder-decoder system based on successive subspace learning (SSL), and it consists of two modules: 1) encoder: fine to coarse unsupervised representation learning with cascaded VoxelHop units, and 2) decoder: coarse to fine segmentation prediction with voxel-wise regression and local error correction. Our model is lightweight and totally transparent while keeping comparable performance.

We have done 5 cross-validations for the dataset from USC Keck Medical School. For T2-cube MRIs, the Dice Similarity Coefficient (DSC) of the prostate segmentation was over 93%. The USC Keck Medical School doctors were very satisfied with these impressive results. In the next step, we will apply our GUSL model to some public datasets and then compare and analyze the performance of our method with some state-of-the-art Deep Learning methods. In the future, we aim to develop methods for segmenting other organs like cardiac. I hope our methods [...]

By |July 7th, 2024|News|Comments Off on MCL Research on Prostate MRI Image Segmentation|

Professor Kuo Met MCL Alumni in Thailand

Professor C.-C. Jay Kuo, Director of MCL, visited Bangkok, Thailand, from June 22-26 to reunite with MCL alumni on his Asian trip. There are three MCL alumni in Thailand. They are:• Junavit Chalidabhongse (Lecturer, Faculty of Law, Thammasat University)• Wuttipong Kumwilaisak (Professor, King Mongkut’s University of Technology)• Tanaphol Thaipanich (CEO, Push Media Co., Ltd.)It has been 15 years since Professor Kuo’s last visit to Bangkok, and he received warm hospitality. Professor Kuo was proud of the outstanding performance of MCL alumni in both academia and industry.The ECTI Association in Thailand and the IEEE Thailand Section invited Professor Kuo to deliver a “Recent Developments and Outlook in Green AI/ML” seminar at the Faculty of Engineering, Chulalongkorn University, on June 24. The seminar was well attended. Professor Kuo had several sightseeing tours in Bangkok and enjoyed his spare time in Thailand.

By |June 30th, 2024|News|Comments Off on Professor Kuo Met MCL Alumni in Thailand|

Professor Kuo visited Singapore 

Professor C.-C. Jay Kuo, Director of MCL, visited Singapore from June 17-23 and attended two events in his Asia trip.For the first event, Professor Kuo gave a seminar titled “Demystify Artificial Intelligence and Technology Outlook” at the Nanyang Technological University on June 20 (Wednesday). The abstract of his talk is given below.“The term “Artificial Intelligence (AI)” was coined in 1956. Although the field evolved slowly in the first 55 years, we have witnessed rapid advances in AI in the last decade (e.g., the ChatGPT service). Questions are raised about AI’s role in human society and civilization, e.g., whether AI will replace human intelligence (HI), how HI and AI complement each other, etc. I will shed light on them. Then, I will comment on the limitations of today’s deep-learning-based (DL) AI models. DL-based models are neither interpretable nor sustainable. An alternative methodology is desired. To this end, I have investigated a new statistically based AI/ML framework called “Green Learning” (GL). GL significantly reduces the model size and complexity of DL models while yielding competitive performance and allowing mathematical transparency. GL adopts the feedforward one-pass training pipeline, so all intermediate results are explainable. Finally, I will address the AI role in human society and issues with today’s chatbots, such as bias and fairness.”Professor Kuo’s talk was hosted by Professor Woon Seng Gan at NTU. He also had a reunion with Professor Jiaying Liu of Peking University. Professor Liu was a visiting PhD student at MCL from August 2007 to August 2008.For the 2nd activity, Professor Kuo delivered a keynote speech at the Workshop titled “Emerging Trends and Innovations in Machine Learning and AI” on June 21 (Friday) organized by APSIPA Singapore Chapter and quite a few [...]

By |June 23rd, 2024|News|Comments Off on Professor Kuo visited Singapore |

Professor Kuo met MCL Alumni in Taiwan

Professor C.-C. Jay Kuo, Director of MCL, attended the Picture Coding Symposium (PCS) in Taichung, Taiwan, from June 12-14, 2024. Professor Kuo was invited to be a panelist for a panel program. The discussion topic was “Learned Image and Video Coding: Hype or Hope?” (see https://2024.picturecodingsymposium.org/panel/). All six panelists were optimistic about researching and developing learned image and video coding technologies for various reasons. Professor Kuo emphasized the differences between the classical and the modern learned coding methodologies. The former considers intra-content redundancy removal, while the latter examines inter-content redundancy removal. The former has been researched for four decades and reached its maturity. It isn’t easy to push further. The latter does have more opportunities. On the other hand, Professor Kuo was concerned about the high complexity and black-box nature of neural network codecs. He suggested an alternative non-neural-network-based approach to implement learned image and video codecs.After PCS, Professor Kuo went to Taipei for a reunion luncheon with MCL alums on June 15 (Saturday). Professor Kuo said, “It was a relaxing time during a busy week. Seeing our alums doing well in their careers and families was great.”

By |June 16th, 2024|News|Comments Off on Professor Kuo met MCL Alumni in Taiwan|

MCL Research on Nuclei Segmentation for Histological Images

Nuclei segmentation is a fundamental task required to analyze the underlying nuclei structure of an organ of interest. Cancer starts from the cells, and understanding the nuclei shapes, sizes and distribution can provide cues on whether or not a patient has cancer. Further analysis can also help in cancer grading and prognosis. However, studying whole slide images of biopsied tissues requires a large amount of time and effort. 

Such monotonous and laborious tasks can be simplified by using AI and ML, and can perhaps improve the accuracy of the detections as well. Some challenges in the nuclei segmentation task include inherent staining variations in the WSI, a wide variety of shapes and sizes in nuclei, and irregular boundaries which make it difficult to track the actual contours. Most of the current research in this area involves deep learning based architectures like the U-Net, R-CNN, and even Vision Transformer. These methods require a large number of training samples and high complexity to achieve generalization among the variations inherent in nuclei from different organs.

We propose to use a light-weight, interpretable, and simple Green Learning based approach to perform Nuclei Segmentation. Prior work on highly effective Unsupervised Nuclei Instance Segmentation (HUNIS) [1] forms the first stage of our current approach. To further improve HUNIS results, we now focus on the regions where HUNIS requires the help of labels. We divide our current task into two stages: (i) to identify those areas where we need help and (ii) to correct those areas towards their actual class. With the help of Saab Transform, our main task now is to perform feature engineering to identify the ideal features to implement the above two stages. 

References:

[1] V. Magoulianitis, Y. Yang, and C.-C. J. [...]

By |June 9th, 2024|News|Comments Off on MCL Research on Nuclei Segmentation for Histological Images|