Volume 17 Issue 1
Jan.  2024
Turn off MathJax
Article Contents
LIU Ze-long, LI Mao-yue, LU Xin-yuan, ZHANG Ming-lei. On-machine detection technology and application progress of high dynamic range fringe structured light[J]. Chinese Optics, 2024, 17(1): 1-18. doi: 10.37188/CO.2023-0068
Citation: LIU Ze-long, LI Mao-yue, LU Xin-yuan, ZHANG Ming-lei. On-machine detection technology and application progress of high dynamic range fringe structured light[J]. Chinese Optics, 2024, 17(1): 1-18. doi: 10.37188/CO.2023-0068

On-machine detection technology and application progress of high dynamic range fringe structured light

doi: 10.37188/CO.2023-0068
Funds:  Supported by National Natural Science Foundation of China (No. 51975169); Natural Science Foundation of Heilongjiang Province(No. LH2022E085)
More Information
  • Corresponding author: lmy0500@163.com
  • Received Date: 16 Apr 2023
  • Rev Recd Date: 15 May 2023
  • Available Online: 21 Sep 2023
  • Fringe structured light technology is a non-contact measurement method, which has developed rapidly in recent years and provides a new solution for on-machine detection in mechanical processing. However, the accuracy of structured light for on-machine detection is compromised by the convoluted lighting in machining environments and metal parts’ high reflectivity, leading to inaccurate measurements. Applying high dynamic range (HDR) technology to structured light detection can reduce the effect of high reflectivity, achieving the measurement of metal parts in complex scenes. This paper introduces the measurement principle of structured light and summarizes the challenges of on-machine detection for HDR structured light. Subsequently, this paper provides a comprehensive review of HDR structured light technology. In the context of on-machine detection of mechanical processing, the HDR technology based on hardware equipment and the HDR technology based on stripe algorithm are discussed and analyzed, respectively. Following this, different technologies are summarized according to the requirements of on-machine detection. The advantages and disadvantages of various methods are presented, and the applicability of on-machine detection is compared. Finally, the potential applications are analyzed, and the technological prospects will be proposed in combination with the research hotspots of advanced manufacturing technology and precision measurement in recent years.


  • loading
  • [1]
    李文龙, 李中伟, 毛金城. iPoint3D曲面检测软件开发与工程应用综述[J]. 机械工程学报,2020,56(7):127-150. doi: 10.3901/JME.2020.07.127

    LI W L, LI ZH W, MAO J CH. The development and application review of iPoint3D software for surface inspection[J]. Journal of Mechanical Engineering, 2020, 56(7): 127-150. (in Chinese) doi: 10.3901/JME.2020.07.127
    李茂月, 刘泽隆, 赵伟翔, 等. 面结构光在机检测的叶片反光抑制技术[J]. 中国光学,2022,15(3):464-475. doi: 10.37188/CO.2021-0194

    LI M Y, LIU Z L, ZHAO W X, et al. Blade reflection suppression technology based on surface structured light on-machine detection[J]. Chinese Optics, 2022, 15(3): 464-475. (in Chinese) doi: 10.37188/CO.2021-0194
    任明阳, 王立忠, 赵建博, 等. 复杂曲面零件面结构光扫描视点规划[J]. 中国光学(中英文),2023,16(1):113-126. doi: 10.37188/CO.2022-0026

    REN M Y, WANG L ZH, ZHAO J B, et al. Viewpoint planning of surface structured light scanning for complex surface parts[J]. Chinese Optics, 2023, 16(1): 113-126. (in Chinese) doi: 10.37188/CO.2022-0026
    张宗华, 刘巍, 刘国栋, 等. 三维视觉测量技术及应用进展[J]. 中国图象图形学报,2021,26(6):1483-1502.

    ZHANG Z H, LIU W, LIU G D, et al. Overview of the development and application of 3D vision measurement technology[J]. Journal of Image and Graphics, 2021, 26(6): 1483-1502. (in Chinese)
    XU J, ZHANG S. Status, challenges, and future perspectives of fringe projection profilometry[J]. Optics and Lasers in Engineering, 2020, 135: 106193. doi: 10.1016/j.optlaseng.2020.106193
    张宗华, 于瑾, 高楠, 等. 高反光表面三维形貌测量技术[J]. 红外与激光工程,2020,49(3):0303006. doi: 10.3788/IRLA202049.0303006

    ZHANG Z H, YU J, GAO N, et al. Three-dimensional shape measurement techniques of shiny surfaces[J]. Infrared and Laser Engineering, 2020, 49(3): 0303006. (in Chinese) doi: 10.3788/IRLA202049.0303006
    FENG S J, ZHANG L, ZUO CH, et al. High dynamic range 3D measurements with fringe projection profilometry: a review[J]. Measurement Science and Technology, 2018, 29(12): 122001. doi: 10.1088/1361-6501/aae4fb
    ZHANG P, ZHONG K, LI ZH W, et al. Hybrid-quality-guided phase fusion model for high dynamic range 3D surface measurement by structured light technology[J]. Optics Express, 2022, 30(9): 14600-14614. doi: 10.1364/OE.457305
    Zhang S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi: 10.1016/j.optlaseng.2020.106029
    马泽龙, 高慧斌, 余毅, 等. 采用图像直方图特征函数的高速相机自动曝光方法[J]. 光学 精密工程,2017,25(4):1026-1035. doi: 10.3788/OPE.20172504.1026

    MA Z L, GAO H B, YU Y, et al. Auto exposure control for high frame rate camera using image histogram feature function[J]. Optics and Precision Engineering, 2017, 25(4): 1026-1035. (in Chinese) doi: 10.3788/OPE.20172504.1026
    雷经发, 陆宗胜, 李永玲, 等. 基于投影栅相位法和多曝光图像融合技术的强反射表面轮廓检测[J]. 光学 精密工程,2022,30(18):2195-2204. doi: 10.37188/OPE.20223018.2195

    LEI J F, LU Z SH, LI Y L, et al. High reflection surface topography measurement based on fringe projection phase method and multi-exposure image fusion technology[J]. Optics and Precision Engineering, 2022, 30(18): 2195-2204. (in Chinese) doi: 10.37188/OPE.20223018.2195
    ZHANG S, YAU S T. High dynamic range scanning technique[J]. Optical Engineering, 2009, 48(3): 033604. doi: 10.1117/1.3099720
    SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi: 10.1016/j.optlaseng.2017.03.008
    FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi: 10.1016/j.optlaseng.2014.03.003
    CUI H H, LI ZH J, TIAN W, et al. Multiple-exposure adaptive selection algorithm for high dynamic range 3D fringe projection measurement[J]. Proceedings of SPIE, 2019, 11053: 110530M.
    RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
    WU K, TAN J, XIA H L, et al. An exposure fusion-based structured light approach for the 3D measurement of a specular surface[J]. IEEE Sensors Journal, 2021, 21(5): 6314-6324. doi: 10.1109/JSEN.2020.3027317
    SALAHIEH B, CHEN ZH Y, RODRIGUEZ J J, et al. Multi-polarization fringe projection imaging for high dynamic range objects[J]. Optics Express, 2014, 22(8): 10064-10071. doi: 10.1364/OE.22.010064
    平茜茜, 刘勇, 董欣明, 等. 基于偏振双目视觉的无纹理高反光目标三维重构[J]. 红外与毫米波学报,2017,36(4):432-438.

    PING X X, LIU Y, DONG X M, et al. 3-D reconstruction of textureless and high-reflective target by polarization and binocular stereo vision[J]. Journal of Infrared and Millimeter Waves, 2017, 36(4): 432-438. (in Chinese)
    郝婧蕾, 赵永强, 赵海盟, 等. 偏振多光谱机器视觉的高反光无纹理目标三维重构方法[J]. 测绘学报,2018,47(6):816-824.

    HAO J L, ZHAO Y Q, ZHAO H M, et al. 3D Reconstruction of High-reflective and Textureless Targets Based on Multispectral Polarization and Machine Vision[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 816-824. (in Chinese)
    WANG Y H, ZHANG Q, HU Y, et al. Rapid 3D measurement of high dynamic range surface based on multi-polarization fringe projection[J]. Optical Engineering, 2021, 60(8): 084107.
    ZHU ZH M, ZHU W T, ZHOU F Q, et al. Three-dimensional measurement of fringe projection based on the camera response function of the polarization system[J]. Optical Engineering, 2021, 60(5): 055105.
    MAEDA Y, SHIBATA S, HAGEN N, et al. Single shot 3D profilometry by polarization pattern projection[J]. Applied Optics, 2020, 59(6): 1654-1659. doi: 10.1364/AO.382690
    XIANG G, ZHU H J, GUO H W. Spatial phase-shifting profilometry by use of polarization for measuring 3D shapes of metal objects[J]. Optics Express, 2021, 29(13): 20981-20994. doi: 10.1364/OE.427407
    王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学 精密工程,2018,26(5):1014-1027. doi: 10.3788/OPE.20182605.1014

    WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi: 10.3788/OPE.20182605.1014
    ZHUANG Y CH, ZHENG Y M, LIN SH B, et al. Surface shape distortion online measurement method for compact laser cavities based on phase measuring Deflectometry[J]. Photonics, 2022, 9(3): 151. doi: 10.3390/photonics9030151
    GAO F, XU Y J, JIANG X Q. Near optical coaxial phase measuring deflectometry for measuring structured specular surfaces[J]. Optics Express, 2022, 30(10): 17554-17566. doi: 10.1364/OE.457198
    HAN H, WU SH Q, SONG ZH. Curved LCD based deflectometry method for specular surface measurement[J]. Optics and Lasers in Engineering, 2022, 151: 106909. doi: 10.1016/j.optlaseng.2021.106909
    SU P, PARKS R E, WANG L R, et al. Software configurable optical test system: a computerized reverse Hartmann test[J]. Applied Optics, 2010, 49(23): 4404-4412. doi: 10.1364/AO.49.004404
    邵山川, 陶小平, 王孝坤. 基于条纹反射的超精密车削反射镜的在位面形检测[J]. 激光与光电子学进展,2018,55(7):071203.

    SHAO SH CH, TAO X P, WANG X K. On-machine surface shape measurement of reflective mirrors by ultra-precision turning based on fringe reflection[J]. Laser & Optoelectronics Progress, 2018, 55(7): 071203. (in Chinese)
    袁婷. 基于条纹反射法的大口径非球面反射镜面形检测技术研究[D]. 长春: 中国科学院研究生院(长春光学精密机械与物理研究所), 2016.

    YUAN T. Study on fringe-reflection optical surface shape measurement technology for large aspheric mirror[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, CAS, 2016. (in Chinese)
    OH C J, LOWMAN A E, SMITH G A, et al. Fabrication and testing of 4.2m off-axis aspheric primary mirror of Daniel K. Inouye Solar Telescope[J]. Proceedings of SPIE, 2016, 9912: 99120O.
    WOODHAM R J. Photometric method for determining surface orientation from multiple images[J]. Optical Engineering, 1980, 19(1): 191139.
    LU L, QI L, LUO Y S, et al. Three-dimensional reconstruction from single image base on combination of CNN and multi-spectral photometric stereo[J]. Sensors, 2018, 18(3): 764. doi: 10.3390/s18030764
    张颖, 李金龙, 黄趾维, 等. 基于BRDF模型的金属表面反射特性及相变特性研究[J]. 光电技术应用,2017,32(3):32-35. doi: 10.3969/j.issn.1673-1255.2017.03.008

    ZHANG Y, LI J L, HUANG ZH W, et al. Research on reflection and phase shift characters of metal surface based on BRDF model[J]. Electro-Optic Technology Application, 2017, 32(3): 32-35. (in Chinese) doi: 10.3969/j.issn.1673-1255.2017.03.008
    王金海, 李华, 魏力. 基于C-T模型的光学元件加工表面的光学特性研究[J]. 光学技术,2021,47(2):172-177.

    WANG J H, LI H, WEI L. Study on optical properties of machining surface of optical element based on C-T model[J]. Optical Technique, 2021, 47(2): 172-177.
    PEI X H, REN M J, WANG X, et al. Profile measurement of non-Lambertian surfaces by integrating fringe projection profilometry with near-field photometric stereo[J]. Measurement, 2022, 187: 110277. doi: 10.1016/j.measurement.2021.110277
    MENG L F, LU L Y, BEDARD N, et al. Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. IEEE International Conference on Computer Vision, IEEE, 2015: 3433-3441.
    WADDINGTON C, KOFMAN J. Analysis of measurement sensitivity to illuminance and fringe-pattern gray levels for fringe-pattern projection adaptive to ambient lighting[J]. Optics and Lasers in Engineering, 2010, 48(2): 251-256. doi: 10.1016/j.optlaseng.2009.07.001
    LI D, KOFMAN J. Adaptive fringe-pattern projection for image saturation avoidance in 3D surface-shape measurement[J]. Optics Express, 2014, 22(8): 9887-9901. doi: 10.1364/OE.22.009887
    JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
    WANG J H, YANG Y X, ZHOU Y G. 3-D shape reconstruction of non-uniform reflectance surface based on pixel intensity, pixel color and camera exposure time adaptive adjustment[J]. Scientific Reports, 2021, 11(1): 4700. doi: 10.1038/s41598-021-83779-9
    SUN J H, ZHANG Q Y. A 3D shape measurement method for high-reflective surface based on accurate adaptive fringe projection[J]. Optics and Lasers in Engineering, 2022, 153: 106994. doi: 10.1016/j.optlaseng.2022.106994
    BABAIE G, ABOLBASHARI M, FARAHI F. Dynamics range enhancement in digital fringe projection technique[J]. Precision Engineering, 2015, 39: 243-251. doi: 10.1016/j.precisioneng.2014.06.007
    CHEN CH, GAO N, WANG X J, et al. Adaptive pixel-to-pixel projection intensity adjustment for measuring a shiny surface using orthogonal color fringe pattern projection[J]. Measurement Science and Technology, 2018, 29(5): 055203. doi: 10.1088/1361-6501/aab07a
    冯维, 徐仕楠, 王恒辉, 等. 逐像素调制的高反光表面三维测量方法[J]. 中国光学,2022,15(3):488-497. doi: 10.37188/CO.2021-0220

    FENG W, XU SH N, WANG H H, et al. Three-dimensional measurement method of highly reflective surface based on per-pixel modulation[J]. Chinese Optics, 2022, 15(3): 488-497. (in Chinese) doi: 10.37188/CO.2021-0220
    李乾, 薛俊鹏, 张启灿, 等. 利用相机响应曲线实现高反光元件三维面形测量[J]. 光学学报,2022,42(7):0712001. doi: 10.3788/AOS202242.0712001

    LI Q, XUE J P, ZHANG Q C, et al. Three dimensional shape measurement of high reflective elements using camera response curve[J]. Acta Optica Sinica, 2022, 42(7): 0712001. (in Chinese) doi: 10.3788/AOS202242.0712001
    SHAFER S A. Using color to separate reflection components[J]. Color Research & Application, 1985, 10(4): 210-218.
    WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi: 10.1364/AO.57.009172
    CHUA S Y, LIM C C, ENG S K, et al. Improved high dynamic range for 3D shape measurement based on saturation of the coloured fringe[J]. Pertanika Journal of Science & Technology, 2021, 29(2): 759-770.
    YIN Y K, CAI Z W, JIANG H, et al. High dynamic range imaging for fringe projection profilometry with single-shot raw data of the color camera[J]. Optics and Lasers in Engineering, 2017, 89: 138-144. doi: 10.1016/j.optlaseng.2016.08.019
    ZHENG Y, WANG Y J, SURESH V, et al. Real-time high-dynamic-range fringe acquisition for 3D shape measurement with a RGB camera[J]. Measurement Science and Technology, 2019, 30(7): 075202. doi: 10.1088/1361-6501/ab0ced
    LIU Y ZH, FU Y J, ZHUAN Y H, et al. High dynamic range real-time 3D measurement based on Fourier transform profilometry[J]. Optics & Laser Technology, 2021, 138: 106833.
    CHEN Y M, HE Y M, HU E Y. Phase deviation analysis and phase retrieval for partial intensity saturation in phase-shifting projected fringe profilometry[J]. Optics Communications, 2008, 281(11): 3087-3090. doi: 10.1016/j.optcom.2008.01.070
    JIANG C F, BELL T, ZHANG S. High dynamic range real-time 3D shape measurement[J]. Optics Express, 2016, 24(7): 7337-7346. doi: 10.1364/OE.24.007337
    WANG M M, DU G L, ZHOU C L, et al. Enhanced high dynamic range 3D shape measurement based on generalized phase-shifting algorithm[J]. Optics Communications, 2017, 385: 43-53. doi: 10.1016/j.optcom.2016.10.023
    ZUO CH, HUANG L, ZHANG M L, et al. Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review[J]. Optics and Lasers in Engineering, 2016, 85: 84-103. doi: 10.1016/j.optlaseng.2016.04.022
    CHEN B, ZHANG S. High-quality 3D shape measurement using saturated fringe patterns[J]. Optics and Lasers in Engineering, 2016, 87: 83-89. doi: 10.1016/j.optlaseng.2016.04.012
    HE Z X, LI P L, ZHAO X Y, et al. Chessboard-like high-frequency patterns for 3D measurement of reflective surface[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 5009712.
    HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi: 10.1016/j.optlaseng.2019.05.019
    张启灿, 吴周杰. 基于格雷码图案投影的结构光三维成像技术[J]. 红外与激光工程,2020,49(3):0303004. doi: 10.3788/IRLA202049.0303004

    ZHANG Q C, WU ZH J. Three-dimensional imaging technique based on Gray-coded structured illumination[J]. Infrared and Laser Engineering, 2020, 49(3): 0303004. (in Chinese) doi: 10.3788/IRLA202049.0303004
    SONG ZH, CHUNG R, ZHANG X T. An accurate and robust strip-edge-based structured light means for shiny surface micromeasurement in 3-D[J]. IEEE Transactions on Industrial Electronics, 2013, 60(3): 1023-1032. doi: 10.1109/TIE.2012.2188875
    LU L L, WU ZH J, ZHANG Q C, et al. High-efficiency dynamic three-dimensional shape measurement based on misaligned Gray-code light[J]. Optics and Lasers in Engineering, 2022, 150: 106873. doi: 10.1016/j.optlaseng.2021.106873
    ZUO CH, QIAN J M, FENG SH J, et al. Deep learning in optical metrology: a review[J]. Light:Science & Applications, 2022, 11(1): 39.
    REYES-FIGUEROA A, FLORES V H, RIVERA M. Deep neural network for fringe pattern filtering and normalization[J]. Applied Optics, 2021, 60(7): 2022-2036. doi: 10.1364/AO.413404
    JEON W, JEONG W, SON K, et al. Speckle noise reduction for digital holographic images using multi-scale convolutional neural networks[J]. Optics Letters, 2018, 43(17): 4240-4243. doi: 10.1364/OL.43.004240
    ZHANG B W, LIN SH N, LIN J Y, et al. Single-shot high-precision 3D reconstruction with color fringe projection profilometry based BP neural network[J]. Optics Communications, 2022, 517: 128323. doi: 10.1016/j.optcom.2022.128323
    NGUYEN H, WANG ZH Y. Accurate 3D shape reconstruction from single structured-light image via fringe-to-fringe network[J]. Photonics, 2021, 8(11): 459. doi: 10.3390/photonics8110459
    FENG SH J, ZUO CH, YIN W, et al. Micro deep learning profilometry for high-speed 3D surface imaging[J]. Optics and Lasers in Engineering, 2019, 121: 416-427. doi: 10.1016/j.optlaseng.2019.04.020
    LIU Y, BLUNT L, GAO F, et al. High-dynamic-range 3D measurement for E-beam fusion additive manufacturing based on SVM intelligent fringe projection system[J]. Surface Topography:Metrology and Properties, 2021, 9(3): 034002. doi: 10.1088/2051-672X/ac0c62
    彭广泽, 陈文静. 基于卷积神经网络去噪正则化的条纹图修复[J]. 光学学报,2020,40(18):1810002. doi: 10.3788/AOS202040.1810002

    PENG G Z, CHEN W J. Fringe pattern inpainting based on convolutional neural network denoising regularization[J]. Acta Optica Sinica, 2020, 40(18): 1810002. (in Chinese) doi: 10.3788/AOS202040.1810002
    YANG G W, YANG M, ZHOU N, et al. High dynamic range fringe pattern acquisition based on deep neural network[J]. Optics Communications, 2022, 512: 127765. doi: 10.1016/j.optcom.2021.127765
    QIAO G, HUANG Y Y, SONG Y P, et al. A single-shot phase retrieval method for phase measuring deflectometry based on deep learning[J]. Optics Communications, 2020, 476: 126303. doi: 10.1016/j.optcom.2020.126303
    ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi: 10.1016/j.optlaseng.2020.106245
    HU Y, CHEN Q, TAO T Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi: 10.1088/1361-6501/aa5a2d
    陈龙, 王文聪, 张峰峰, 等. 基于双目结构光的术中肝脏表面局部亮度饱和分区投影[J]. 光学 精密工程,2021,29(11):2590-2602. doi: 10.37188/OPE.20212911.2590

    CHEN L, WANG W C, ZHANG F F, et al. Zonal projection based on binocular structured light for localized luminance saturation of intraoperative liver surface[J]. Optics and Precision Engineering, 2021, 29(11): 2590-2602. (in Chinese) doi: 10.37188/OPE.20212911.2590
  • 加载中


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(14)  / Tables(4)

    Article views(512) PDF downloads(289) Cited by()
    Proportional views


    DownLoad:  Full-Size Img  PowerPoint