Single image super-resolution (SISR) is an intensively studied topic in image processing. It aims at recovering a high-resolution (HR) image from its low-resolution (LR) counterpart. SISR finds wide real-world applications such as remote sensing, medical imaging, and biometric identification. Besides, it attracts attention due to its connection with other tasks (e.g., image registration, compression, and synthesis). To deal with such ill-posed problem, we recently proposed two methods, LSR[1] and LSR++[2], by providing reasonable performance and effectively reduced complexity.

LSR consists of three cascaded modules:

  1. Unsupervised Representation Learning by creating a pool of rich and diversified representations in the neighborhood of a target pixel.
  2. Supervised Feature Learning by Relative Feature Test (RFT [3]) to select a subset from the representation pool that is most relevant to the underlying super-resolution task automatically, and
  3. Supervised Decision Learning by predicting the residual of the target pixel based on the selected features through regression via classical machine learning, and effectively fusioning the predictions for more stable results.

LSR++ is promoted based on LSR, with emphasis on sample alignment, a more promising sample preparation process which is suitable for all patch-based computer vision problems. As illustrated in Fig 1, based on gradient histograms of patches along the eight reference directions (Fig.1.a), patch alignment utilizes patch rotations and flipping to meet the standard templates of gradient histograms, where D_max is the direction with the largest cumulative gradient magnitude, and D_max_orth_b and D_max_orth_s refer to the orthogonal directions to D_max with big and small cumulative gradient magnitude, respectively. By modifying the set of (D_max, D_max_orth_b, and D_max_orth_s) of a patch, patch alignment can regularize the edge pattern with the patch by directions perpendicular the edge (D_max) and directions along the edge (D_max_orth_b, D_max_orth_s). The process of patch rotations and flippings in patch alignment can be equivalently achieved by modifying the pixel scanning order with one patch, leading to no additional time complexity.

Both LSR and LSR++ show better performance in PSNR/SSIM than the traditional learning-based methods, and comparable performance with the entry-level DL-based method (SRCNN). At the same time, LSR and LSR++ present effective reduction in model complexity (model size and inference speed).

[1] Wei Wang, Xuejing Lei, Yueru Chen, Ming-Siu Lee, C.-C. Jay Kuo. “LSR: Light-weight Perceptual Super Resolution.” accepted by 2023 IEEE International Conference on Image Processing (ICIP 2023).

[2] Wei Wang, Xuejing Lei, Yueru Chen, Ming-Siu Lee, C.-C. Jay Kuo. “LSR++: An Efficient and Tiny Model for Image Super-Resolution.” accepted by 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2023).

[3] Yang, Yijing, Wei Wang, Hongyu Fu, and C-C. Jay Kuo. “On supervised feature selection from high dimensional feature spaces.” APSIPA Transactions on Signal and Information Processing 11, no. 1 (2022).