Volume 16 Issue 5
Sep.  2023
Turn off MathJax
Article Contents
ZHENG Yue-kun, GE Ming-feng, CHANG Zhi-min, DONG Wen-fei. Super-resolution reconstruction for colorectal endoscopic images based on a residual network[J]. Chinese Optics, 2023, 16(5): 1022-1033. doi: 10.37188/CO.2022-0247
Citation: ZHENG Yue-kun, GE Ming-feng, CHANG Zhi-min, DONG Wen-fei. Super-resolution reconstruction for colorectal endoscopic images based on a residual network[J]. Chinese Optics, 2023, 16(5): 1022-1033. doi: 10.37188/CO.2022-0247

Super-resolution reconstruction for colorectal endoscopic images based on a residual network

doi: 10.37188/CO.2022-0247
Funds:  Supported by the National Key R & D Program of China (No. 2021YFB3602200); Suzhou Science and Technology Plan Project (No.SZS201903)
More Information
  • Corresponding author: gemf@sibet.ac.cn
  • Received Date: 29 Nov 2022
  • Rev Recd Date: 23 Dec 2022
  • Accepted Date: 15 Mar 2023
  • Available Online: 04 Apr 2023
  • In this paper, an image super-resolution reconstruction multi-scale algorithm based on a residual attention network (SMRAN) is proposed to solve the problems caused by low resolutions, less texture information and blurred details in colorectal endoscopic images. Images from the colorectal polyp endoscope image dataset PolypsSet are selected as the raw data for these experiments. A convolutional network is built to extract the shallow features of the low-resolution image and a Res-Sobel block is designed to enhance its edge features. A multi-scale feature fusion block MEB is designed by introducing convolution kernels of different sizes to adaptively extract image features of different scales and obtain effective image information. The Res-Sobel block and multi-scale feature fusion module block MEB are connected through the residual attention network. Finally, a high-resolution image is reconstructed at the sub-pixel convolution layer. When the amplification factor is ×4, the performance of the proposed algorithm on the test set are as follows: the peak signal-to-noise ratio (PSNR) is 34.25 dB and the structural similarity (SSIM) is 0.8675. Compared with the traditional bicubic interpolation algorithm and commonly used deep learning algorithms such as SRCNN and RCAN, the proposed SMRAN algorithm shows better super-resolution reconstruction results on colorectal endoscopic images.


  • loading
  • [1]
    刘爽, 田兆星, 李浩然, 等. 一种基于改进YOLOv5s网络的结直肠腺瘤实时检测方法[J]. 河北大学学报(自然科学版),2022,42(3):327-336.

    LIU SH, TIAN ZH X, LI H R, et al. A real-time method for colorectal adenoma detection based on an improved YOLOv5s network[J]. Journal of Hebei University (Natural Science Edition), 2022, 42(3): 327-336. (in Chinese)
    白瑞峰, 江山, 孙海江, 等. 基于编码解码结构的微血管减压图像实时语义分割[J]. 中国光学(中英文),2022,15(5):1055-1065. doi: 10.37188/CO.2022-0120

    BAI R F, JIANG SH, SUN H J, et al. Real-time semantic segmentation of microvascular decompression images based on encoder-decoder structure[J]. Chinese Optics, 2022, 15(5): 1055-1065. (in Chinese) doi: 10.37188/CO.2022-0120
    陈晓冬, 艾大航, 张佳琛, 等. Gabor滤波融合卷积神经网络的路面裂缝检测方法[J]. 中国光学,2020,13(6):1293-1301. doi: 10.37188/CO.2020-0041

    CHEN X D, AI D H, ZHANG J CH, et al. Gabor filter fusion network for pavement crack detection[J]. Chinese Optics, 2020, 13(6): 1293-1301. (in Chinese) doi: 10.37188/CO.2020-0041
    左超, 陈钱. 分辨率、超分辨率与空间带宽积拓展——从计算光学成像角度的一些思考[J]. 中国光学(中英文),2022,15(6):1105-1166. doi: 10.37188/CO.2022-0105

    ZUO CH, CHEN Q. Resolution, super-resolution and spatial bandwidth product expansion——some thoughts from the perspective of computational optical imaging[J]. Chinese Optics, 2022, 15(6): 1105-1166. (in Chinese) doi: 10.37188/CO.2022-0105
    GU X G, ZHOU F X, CHEN R F, et al. . Endoscopic single image super-resolution based on transformer and convolutional neural network[C]. 7th International Conference on Intelligent Life System Modelling, Image Processing and Analysis, Springer, 2021: 24-32.
    YANG X R, CHEN Y, TAO R, et al. . Endoscopic image deblurring and super-resolution reconstruction based on deep learning[C]. 2020 International Conference on Artificial Intelligence and Computer Engineering (ICAICE), IEEE, 2020: 168-172.
    TURAN M. A generative adversarial network based super-resolution approach for capsule endoscopy images[J]. Medicine Science, 2021, 10(3): 1002-1007. doi: 10.5455/medscience.2021.06.218
    TAŞ M, YILMAZ B. Super resolution convolutional neural network based pre-processing for automatic polyp detection in colonoscopy images[J]. Computers &Electrical Engineering, 2021, 90: 106959.
    DONG CH, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. doi: 10.1109/TPAMI.2015.2439281
    雷茂, 郭锋, 秦明伟. 基于改进POCS算法的太赫兹图像超分辨率重建[J]. 传感器与微系统,2022,41(3):122-125. doi: 10.13873/J.1000-9787(2022)03-0122-04

    LEI M, GUO F, QIN M W. Super-resolution reconstruction for terahertz images based on improved POCS algorithm[J]. Transducer and Microsystem Technologies, 2022, 41(3): 122-125. (in Chinese) doi: 10.13873/J.1000-9787(2022)03-0122-04
    孙琰玥, 何小海, 陈为龙. 小波局部适应插值的图像超分辨率重建[J]. 计算机工程,2010,36(13):183-185. doi: 10.3969/j.issn.1000-3428.2010.13.065

    SUN Y Y, HE X H, CHEN W L. Image super-resolution reconstruction of wavelet local-adaptation interpolation[J]. Computer Engineering, 2010, 36(13): 183-185. (in Chinese) doi: 10.3969/j.issn.1000-3428.2010.13.065
    庞博, 张旭东, 徐小红. 自适应图像插值在超分辨率图像重建中的应用[J]. 合肥工业大学学报(自然科学版),2006,29(7):825-829.

    PANG B, ZHANG X D, XU X H. Super-resolution image reconstruction using adaptive interpolation method[J]. Journal of Hefei University of Technology (Natural Science), 2006, 29(7): 825-829. (in Chinese)
    陈楠, 张标. 多尺度半耦合卷积稀疏编码的遥感影像超分辨率重建[J]. 计算机辅助设计与图形学学报,2022,34(3):382-391.

    CHEN N, ZHANG B. Multi-scale semi-coupled convolutional sparse coding for the super-resolution reconstruction of remote sensing image[J]. Journal of Computer-Aided Design &Computer Graphics, 2022, 34(3): 382-391. (in Chinese)
    李方玗, 贾晓芬, 赵佰亭, 等. 高效多注意力特征融合的图像超分辨率重建算法[J/OL]. 小型微型计算机系统: 1-9[2022-11-25]. http://kns.cnki.net/kcms/detail/21.1106.TP.20221111.1852.006.html.

    LI F Y, JIA X F, ZHAO B T, et al. . Efficient multi-attention feature fusion for image super-resolution reconstruction algorithms[J/OL]. Journal of Chinese Computer Systems: 1-9[2022-11-25]. http://kns.cnki.net/kcms/detail/21.1106.TP.20221111.1852.006.html. (in Chinese)
    林敏强, 赵曈, 祝明欣. 基于残差注意力的合成孔径雷达图像超分辨率算法[J]. 电子产品世界,2022,29(11):33-34.

    LIN M Q, ZHAO T, ZHU M X. SAR image super-resolution algorithm based on residual attention[J]. Electronic Engineering &Product World, 2022, 29(11): 33-34. (in Chinese)
    须颖, 刘帅, 邵萌, 等. 一种多尺度GAN的低剂量CT超分辨率重建方法[J]. 西安电子科技大学学报,2022,49(2):228-236. doi: 10.19665/j.issn1001-2400.2022.02.026

    XU Y, LIU SH, SHAO M, et al. Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm[J]. Journal of Xidian University, 2022, 49(2): 228-236. (in Chinese) doi: 10.19665/j.issn1001-2400.2022.02.026
    LIM B, SON S, KIM H, et al. . Enhanced deep residual networks for single image super-resolution[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2017: 1132-1140.
    KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016: 1646-1654.
    ZHANG Y L, LI K P, LI K, et al. . Image super-resolution using very deep residual channel attention networks[C]. Proceedings of the 15th European Conference on Computer Vision, Springer, 2018: 294-310.
    LEDIG C, THEIS L, HUSZÁR F, et al. . Photo-realistic single image super-resolution using a generative adversarial network[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017: 105-114.
    SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. 3rd International Conference on Learning Representations, ICLR, 2015.
    王玉玺, 陈健美. 基于自适应Canny算子和多方向Sobel算子的虹膜边缘检测算法[J]. 计算机与数字工程,2020,48(11):2744-2749. doi: 10.3969/j.issn.1672-9722.2020.11.040

    WANG Y X, CHEN J M. Iris edge detection algorithm based on adaptive canny operator and multi-directional Sobel operator[J]. Computer &Digital Engineering, 2020, 48(11): 2744-2749. (in Chinese) doi: 10.3969/j.issn.1672-9722.2020.11.040
    LIU S R, TANG X Y, WANG D. Facial expression recognition based on Sobel operator and improved CNN-SVM[C]. 2020 IEEE 3rd International Conference on Information Communication and Signal Processing (ICICSP), IEEE, 2020: 236-240.
    HAO F, XU D SH, CHEN D L, et al. Sobel operator enhancement based on eight-directional convolution and entropy[J]. International Journal of Information Technology, 2021, 13(5): 1823-1828. doi: 10.1007/s41870-021-00770-3
    WOO S, PARK J, LEE J Y, et al. . CBAM: convolutional block attention module[C]. Proceedings of the 15th European Conference on Computer Vision (ECCV), Springer, 2018: 3-19.
    SHI W ZH, CABALLERO J, HUSZÁR F, et al. . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2016: 1874-1883.
    LI K D, FATHAN M I, PATEL K, et al. Colonoscopy polyp detection and classification: Dataset creation and comparative evaluations[J]. PLoS One, 2021, 16(8): e0255809. doi: 10.1371/journal.pone.0255809
    JHA D, SMEDSRUD P H, RIEGLER M A, et al. . Kvasir-SEG: a segmented polyp dataset[C]. 26th International Conference on Multimedia Modeling, Springer, 2020: 451-462.
  • 加载中


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(6)

    Article views(411) PDF downloads(198) Cited by()
    Proportional views


    DownLoad:  Full-Size Img  PowerPoint