留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于插值超分辨的双目三维重建方法

刘宇豪 吴福培 吴树壮 王瑞

刘宇豪, 吴福培, 吴树壮, 王瑞. 基于插值超分辨的双目三维重建方法[J]. 中国光学(中英文). doi: 10.37188/CO.2023-0214
引用本文: 刘宇豪, 吴福培, 吴树壮, 王瑞. 基于插值超分辨的双目三维重建方法[J]. 中国光学(中英文). doi: 10.37188/CO.2023-0214
LIU Yuhao, WU Fupei, WU Shuzhuang, WANG Rui. Binocular 3D reconstruction method based on interpolation super-resolution[J]. Chinese Optics. doi: 10.37188/CO.2023-0214
Citation: LIU Yuhao, WU Fupei, WU Shuzhuang, WANG Rui. Binocular 3D reconstruction method based on interpolation super-resolution[J]. Chinese Optics. doi: 10.37188/CO.2023-0214

基于插值超分辨的双目三维重建方法

doi: 10.37188/CO.2023-0214
基金项目: 国家自然科学基金(No. 61573233);广东省自然科学基金(No. 2021A1515010661);广东省普通高校重点领域专项(No. 2020ZDZX2005)
详细信息
    作者简介:

    刘宇豪(1999—),男,重庆人,硕士研究生,主要从事机器视觉和三维测量方面的研究。E-mail:21yhliu@stu.edu.cn

    吴福培(1980—),男,广西玉林人,博士,教授。 2009年毕业于华南理工大学机械工程专业,获博士学位,现就职于汕头大学机械工程系。主要研究方向:自动光学检测和3D测量。E-mail:fpwu@stu.edu.cn

    吴树壮(2000—),男,广东揭阳人,硕士研究生,主要从事机器视觉和三维测量方面的研究。E-mail:23szwu@stu.edu.cn

    王 瑞(2001—),男,河南三门峡人,硕士研究生,主要从事机器视觉和三维测量方面的研究。E-mail:23rwang@stu.edu.cn

  • 中图分类号: TP394.1

Binocular 3D reconstruction method based on interpolation super-resolution

Funds: Supported by National Natural Science Foundation of China (No. 61573233), National Natural Science Foundation of Guangdong, China (No. 2021A1515010661), Special Projects in Key Field of Colleges and Universities in Guangdong Province (No. 2020ZDZX2005)
More Information
  • 摘要:

    基于双目立体匹配重建物体表面三维形貌,其匹配精度往往受限于传感器尺寸、镜头焦距和光源环境等物理条件。针对此问题,论文提出了一种基于插值超分辨的双目表面三维重建方法。首先,在图像预处理阶段,建立基于小波变换与双直方图均衡融合的图像增强方法,克服传统双目视觉受限于复杂环境光干扰等问题;其次,构建基于拉格朗日与三次多项式插值的超分辨算法,提升图像像素密度,为双目匹配代价计算阶段增加图像细节,从而提高匹配精度;最后,基于SLIC算法对目标图像进行分割,并分别针对各分割区域做二次曲面拟合,进而获得与物体实际表面更为贴合的高度曲线,可降低重建误差并可提高重建精度。实验结果表明:5组测量样品的全局高度测量平均相对误差为±2.3%,实验平均测量时长为1.8828s,最大时长1.9362s,较传统方法有明显提升。实验分析结果检验了论文所提方法的有效性。

     

  • 图 1  双目相机深度估计

    Figure 1.  Depth estimation by binocular cameras

    图 2  拉格朗日与三次多项式插值框图

    Figure 2.  Block diagram of Lagrange and cubic interpolations

    图 3  拉格朗日与三次多项式插值组合图。(a)拉格朗日多项式内插;(b)三次多项式内插

    Figure 3.  Combination graph of Lagrange and cubic interpolations. (a) Lagrange polynomial interpolation; (b) Cubic polynomial interpolation

    图 4  基于小波变换与双直方图均衡融合的图像增强算法流程

    Figure 4.  Image enhancement based on wavelet transform and dual histogram equalization

    图 5  图像增强结果。(a)原始图像;(b)直方图均衡;(c)小波变换;(d)论文所提算法

    Figure 5.  The results of image enhancement. (a) Original image; (b) The result of histogram equalization; (c) The result of wavelet transform; (d) The result of the proposed algorithm

    图 6  采集图像增强结果。(a)采集图像;(b)增强图像

    Figure 6.  Enhanced results of acquired images. (a) Acquired image; (b) Enhanced image

    图 7  二维高斯分布。(a)二维高斯分布图;(b)高斯加权卷积块

    Figure 7.  Two-dimensional Gaussian distribution. (a) Two-dimensional Gaussian distribution diagram; (b) The Gaussian weighted convolution block

    图 8  基于插值超分辨的邻域窗口像素值变换结果。(a)左视图邻域窗口;(b)右视图邻域窗口;(c)邻域窗口内像素值变换结果

    Figure 8.  Pixel value transformation results of neighborhood windows based on interpolation super-resolution. (a) Neighborhood windows of left image; (b) Neighborhood windows of right image; (c) Pixel value transformation results in neighborhood window

    图 9  SLIC算法结果。(a)采集图像;(b)超像素分割图;(c)边缘图像

    Figure 9.  Result of SLIC algorithm. (a) acquire image; (b) superpixel segmentation image; (c) edge image

    图 10  算法处理流程图

    Figure 10.  Algorithm processing flowchart

    图 11  (a)采集图像;(b)实验平台

    Figure 11.  (a) Acquiring images; (b) The experimental platform

    表  1  三种图像增强算法对标准图像的指标对比

    Table  1.   Comparison of three image enhancement algorithms with respect to the standard image

    NameMethodDENCCCIIPSNRSSIM
    CameramanHistogram6.77030.98480.803919.22290.6916
    Wavelet4.43010.98880.713117.64520.4708
    Proposed7.11490.99801.013224.52500.8788
    LenaHistogram7.33830.98620.905119.39350.7784
    Wavelet5.09920.98460.734117.05830.5613
    Proposed7.43170.99211.008822.12870.8171
    BarbaraHistogram7.38160.98920.849218.22460.8147
    Wavelet6.17880.98240.700517.93060.5767
    Proposed7.53390.99081.013222.22510.8343
    下载: 导出CSV

    表  3  三维重建实验结果

    Table  3.   Experimental results of three-dimensional reconstruction

    Convex surface Trapezoidal surface Angular surface Semicircular surface Concave surface
    Sample image
    Acquire image
    Reconstruction image
    下载: 导出CSV

    表  2  实验参数

    Table  2.   Experimental parameters

    ParameterValue
    Sample size16 mm×8 mm×6.5 mm
    FOV of CCD camera30 mm×20 mm
    Image resolution640 pixels×480 pixels
    Focal length f6 mm
    Object distance180 mm
    Baseline distance49.38 mm
    下载: 导出CSV

    表  4  三种重建方法的检测结果比较

    Table  4.   Comparison of measurement results based on three reconstruction methods

    Name of Curves Method in paper Method in Ref. [27] Method in Ref. [28]
    Maximum error/mm Average error/mm Detection time/s Maximum error/mm Average error/mm Detection time/s Maximum error/mm Average error/mm Detection time/s
    Convex 0.2047 0.1464 1.8551 0.2756 0.2229 1.3287 0.3856 0.2684 2.5932
    Trapezoidal 0.1468 0.1152 1.9173 0.4078 0.2615 1.3813 0.2378 0.1915 2.7367
    Angular 0.1064 0.0768 1.8436 0.2178 0.1525 1.3102 0.1787 0.1425 2.4353
    Semicircular 0.2538 0.1675 1.9362 0.3612 0.2717 1.3922 0.4101 0.3017 2.8577
    Concave 0.1512 0.1278 1.8619 0.2766 0.2105 1.3247 0.2166 0.1705 2.6019
    Average 0.1726 0.1267 1.8828 0.3078 0.2238 1.3474 0.2858 0.2149 2.6450
    下载: 导出CSV

    表  5  样本高度测量误差

    Table  5.   Sample height measurement error rates

    Name of Curves CD IoU MAE/(mm) RMSE/(mm)
    Convex 0.2634 0.6443 0.1150 0.2742
    Trapezoidal 0.2317 0.6987 0.1050 0.2791
    Angular 0.1448 0.7328 0.0882 0.2488
    Semicircular 0.3163 0.6166 0.1403 0.4348
    Concave 0.2215 0.7065 0.1057 0.2710
    Average 0.2355 0.6798 0.1108 0.3016
    下载: 导出CSV
  • [1] WU F P, ZHU S K, YE W L. A single image 3D reconstruction method based on a novel monocular vision system[J]. Sensors, 2020, 20(24): 7045. doi: 10.3390/s20247045
    [2] NASRULLOH A V, WILLCOCKS C G, JACKSON P T G, et al. Multi-scale segmentation and surface fitting for measuring 3-D macular holes[J]. IEEE Transactions on Medical Imaging, 2018, 37(2): 580-589. doi: 10.1109/TMI.2017.2767908
    [3] WANG Y Y, ZHANG H T, XUE J SH, et al. Measurement of three-dimensional pavement morphology using binocular reconstruction improved by second-order fusion algorithm[J]. Automation in Construction, 2024, 157: 105152. doi: 10.1016/j.autcon.2023.105152
    [4] WANG Y Z, JIA H R, JIA P X, et al. A novel algorithm for three-dimensional shape reconstruction for microscopic objects based on shape from focus[J]. Optics & Laser Technology, 2024, 168: 109931.
    [5] DU F, WANG K, ZHANG G J, et al. Damage characteristics of coal under different loading modes based on CT three-dimensional reconstruction[J]. Fuel, 2022, 310: 122304. doi: 10.1016/j.fuel.2021.122304
    [6] 尹仕斌, 任永杰, 刘涛, 等. 机器视觉技术在现代汽车制造中的应用综述[J]. 光学学报,2018,38(8):11-22.

    YIN SH B, REN Y J, LIU T, et al. Review on application of machine vision in modern automobile manufacturing[J]. Acta Optica Sinica, 2018, 38(8): 11-22. (in Chinese).
    [7] GUO X ZH, SHI ZH Y, YU B, et al. 3D measurement of gears based on a line structured light sensor[J]. Precision Engineering, 2020, 61: 160-169. doi: 10.1016/j.precisioneng.2019.10.013
    [8] SIDKI H M, AMER M. Evaluation of CMM for flatness measurements[J]. Metrology and Measurement Systems, 2008, 15(4): 585-593.
    [9] GUILLORY J, TRUONG D, WALLERAND J P, et al. An SI-traceable multilateration coordinate measurement system with half the uncertainty of a laser tracker[J]. Measurement Science and Technology, 2023, 34(6): 065016. doi: 10.1088/1361-6501/acc26a
    [10] LIN J R, XIN R K, SHI SH D, et al. An accurate 6-DOF dynamic measurement system with laser tracker for large-scale metrology[J]. Measurement, 2022, 204: 112052. doi: 10.1016/j.measurement.2022.112052
    [11] HUANG J Q, SUN M ZH, MA J J, et al. Super-resolution image reconstruction for high-density three-dimensional single-molecule microscopy[J]. IEEE Transactions on Computational Imaging, 2017, 3(4): 763-773. doi: 10.1109/TCI.2017.2699425
    [12] SHI CH Y, LUO X B, GUO J, et al. Three-dimensional intravascular reconstruction techniques based on intravascular ultrasound: a technical review[J]. IEEE Journal of Biomedical and Health Informatics, 2018, 22(3): 806-817. doi: 10.1109/JBHI.2017.2703903
    [13] WANG ZH ZH. Robust three-dimensional face reconstruction by one-shot structured light line pattern[J]. Optics and Lasers in Engineering, 2020, 124: 105798. doi: 10.1016/j.optlaseng.2019.105798
    [14] 张新荣, 王鑫, 王瑶, 等. 基于转动式二维激光扫描仪和多传感器的三维重建方法[J]. 中国光学,2023,16(3):663-672. doi: 10.37188/CO.2022-0159

    ZHANG X R, WANG X, WANG Y, et al. 3D reconstruction method based on a rotating 2D laser scanner and multi-sensor[J]. Chinese Optics, 2023, 16(3): 663-672. (in Chinese). doi: 10.37188/CO.2022-0159
    [15] BEVACQUA M T, SCAPATICCI R. A compressive sensing approach for 3D breast cancer microwave imaging with magnetic nanoparticles as contrast agent[J]. IEEE Transactions on Medical Imaging, 2016, 35(2): 665-673. doi: 10.1109/TMI.2015.2490340
    [16] SUN SH L, WANG X Y, CHEN Y J. Retracted: the application value of CT three-dimensional microscope reconstruction technology in the diagnosis of cervical cancer[J]. Scanning, 2022, 2022: 5648195.
    [17] WANG ZH ZH, ZHOU Q, SHUANG Y C. Three-dimensional reconstruction with single-shot structured light dot pattern and analytic solutions[J]. Measurement, 2020, 151: 107114. doi: 10.1016/j.measurement.2019.107114
    [18] MOONEY S W J, MARLOW P J, ANDERSON B L. The perception and misperception of optical defocus, shading, and shape[J]. eLife, 2019, 8: e48214.
    [19] TIAN X, LIU R, WANG ZH Y, et al. High quality 3D reconstruction based on fusion of polarization imaging and binocular stereo vision[J]. Information Fusion, 2022, 77: 19-28.
    [20] LIU L, ZHU P, GUAN J X, et al. A binocular reconstruction method fused with laplacian image information for pavement texture evaluation[J]. Measurement, 2021, 185: 110039. doi: 10.1016/j.measurement.2021.110039
    [21] LI Y, FU Y J, ZHONG K J, et al. A virtual binocular line-structured light measurement method based on a plane mirror[J]. Optics Communications, 2022, 510: 127974.
    [22] TSENG K K, ZHANG Y Q, ZHU Q L, et al. Semi-supervised image depth prediction with deep learning and binocular algorithms[J]. Applied Soft Computing, 2020, 92: 106272. doi: 10.1016/j.asoc.2020.106272
    [23] BOUKHTACHE S, BLAYSAT B, GRÉDIAC M, et al. Alternatives to bicubic interpolation considering FPGA hardware resource consumption[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2021, 29(2): 247-258. doi: 10.1109/TVLSI.2020.3032888
    [24] 李佳星, 赵勇先, 王京华. 基于深度学习的单幅图像超分辨率重建算法综述[J]. 自动化学报,2021,47(10):2341-2363.

    LI J X, ZHAO Y X, WANG J H. A review of single image super-resolution reconstruction algorithms based on deep learning[J]. Acta Automatica Sinica, 2021, 47(10): 2341-2363. (in Chinese).
    [25] 刘艳, 李庆武, 霍冠英, 等. 结合局部二进制表示和超像素分割求精的立体匹配[J]. 光学学报,2018,38(6):0615003. doi: 10.3788/AOS201838.0615003

    LIU Y, LI Q W, HUO G Y, et al. Local binary description combined with superpixel segmentation refinement for stereo matching[J]. Acta Optica Sinica, 2018, 38(6): 0615003. (in Chinese). doi: 10.3788/AOS201838.0615003
    [26] 冯维, 徐仕楠, 王恒辉, 等. 逐像素调制的高反光表面三维测量方法[J]. 中国光学,2022,15(3):488-497. doi: 10.37188/CO.2021-0220

    FENG W, XU SH N, WANG H H, et al. Three-dimensional measurement method of highly reflective surface based on per-pixel modulation[J]. Chinese Optics, 2022, 15(3): 488-497. (in Chinese). doi: 10.37188/CO.2021-0220
    [27] 吴福培, 余冠霖, 乐聪, 等. 微型相机模组贴装过程画胶表面三维测量方法[J]. 光子学报,2022,51(4):0412006. doi: 10.3788/gzxb20225104.0412006

    WU F P, YU G L, LE C, et al. Three-dimensional measurement method of glue surface during the mounting process of micro-camera module[J]. Acta Photonica Sinica, 2022, 51(4): 0412006. (in Chinese). doi: 10.3788/gzxb20225104.0412006
    [28] 平茜茜, 刘勇, 董欣明, 等. 基于偏振双目视觉的无纹理高反光目标三维重构[J]. 红外与毫米波学报,2017,36(4):432-438.

    PING X X, LIU Y, DONG X M, et al. 3-D reconstruction of textureless and high-reflective target by polarization and binocular stereo vision[J]. Journal of Infrared and Millimeter Waves, 2017, 36(4): 432-438. (in Chinese).
  • 加载中
图(11) / 表(5)
计量
  • 文章访问数:  42
  • HTML全文浏览量:  22
  • PDF下载量:  7
  • 被引次数: 0
出版历程
  • 网络出版日期:  2024-05-20

目录

    /

    返回文章
    返回

    重要通知

    2024年2月16日科睿唯安通过Blog宣布,2024年将要发布的JCR2023中,229个自然科学和社会科学学科将SCI/SSCI和ESCI期刊一起进行排名!《中国光学(中英文)》作为ESCI期刊将与全球SCI期刊共同排名!