光栅投影在机三维形貌检测技术研究进展

吕虹毓; 李茂月; 蔡东辰; 赵伟翔

doi:10.37188/CO.2022-0083

Volume 16 Issue 3

May 2023

Turn off MathJax

Article Contents

Chinese Optics > 2023 > 16(3): 500-513.

LV Hong-yu, LI Mao-yue, CAI Dong-chen, ZHAO Wei-xiang. Research progress of grating projection on machine 3D topography inspection technology[J]. Chinese Optics, 2023, 16(3): 500-513. doi: 10.37188/CO.2022-0083

Citation:

LV Hong-yu, LI Mao-yue, CAI Dong-chen, ZHAO Wei-xiang. Research progress of grating projection on machine 3D topography inspection technology[J]. Chinese Optics, 2023, 16(3): 500-513. doi: 10.37188/CO.2022-0083

Citation:

PDF( 6376 KB)

Research progress of grating projection on machine 3D topography inspection technology

doi: 10.37188/CO.2022-0083

Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin 150080, China

Funds: Supported by National Natural Science Foundation of China (No. 51975169) ；the Fundamental Research Fundation for Universities of Heilongjiang Province (No. 2019-KYYWF-0204)

More Information

Corresponding author: lmy0500@163.com
Received Date: 28 Apr 2022
Rev Recd Date: 31 May 2022

Available Online: 28 Sep 2022

Abstract

Abstract

Vision-based measurement has good application prospects and far-reaching development significance for advanced manufacturing fields such as aerospace, the military industry and electronic chips. Among them, on-machine 3D vision detection technology based on structured light is one of the hotspots and challenges in the field of precision machining. Based on the on-machine 3D measurement process of structured light, we discuss and summarize the key technologies, including its technical requirements, methods and principles involved, related research status and existing problems in the measurement calibration, phase optimization solution, on-machine 3D point cloud processing and reconstruction of different feature surfaces. Finally, according to the actual needs of relevant technologies in the future, prospects are made with regard to processing field calibration, dynamic real-time 3D reconstruction, sub-micron and nano measurement, and measurement processing integrated data transmission technology, with the corresponding research ideas put forward.
- grating projection,
- three-dimensional morphology,
- on machine inspection,
- computer vision,
- surface reconstruction

FullText(HTML)

References(62)

References

[1]	DE-DIN. BS EN ISO 25178-6-2010 Geometrical product specifications (GPS) - surface texture: areal - part 6: classification of methods for measuring surface texture[S]. 2010.
[2]	中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 33523.6-2017 产品几何技术规范(GPS) 表面结构区域法第6部分: 表面结构测量方法的分类[S]. 北京: 中国标准出版社, 2017. General Administration of Quality Supervision, Inspection and Quarantine of the People's Republic of China, Standardization Administration. GB/T 33523.6-2017 Geometrical product specifications(GPS)—surface texture: areal—part 6: classification of methods for measuring surface texture[S]. Beijing: Standards Press of China, 2017. (in Chinese)
[3]	WANG Z P, YIN M, OU D Y, et al. Optical measurement method for blade profiles based on blade self-features[J]. IEEE Transactions on Industrial Electronics, 2022, 69(2): 2067-2076. doi: 10.1109/TIE.2021.3062213
[4]	ZHENG ZH, WAN Y, ZHANG Y J, et al. CLNet: cross-layer convolutional neural network for change detection in optical remote sensing imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175: 247-267. doi: 10.1016/j.isprsjprs.2021.03.005
[5]	ZANGL K, DANZL R, HELMLI F, et al. Highly accurate optical µCMM for measurement of micro holes[J]. Procedia CIRP, 2018, 75: 397-402. doi: 10.1016/j.procir.2018.05.098
[6]	KHAN A, MINEO C, DOBIE G, et al. Vision guided robotic inspection for parts in manufacturing and remanufacturing industry[J]. Journal of Remanufacturing, 2021, 11(1): 49-70. doi: 10.1007/s13243-020-00091-x
[7]	LIU J D, SUN W L, HUANG Y. An algorithm for trajectory planning of complex surface parts for laser cladding remanufacturing[J]. Proceedings of the Institution of Mechanical Engineers,Part B:Journal of Engineering Manufacture, 2021, 235(12): 2025-2032. doi: 10.1177/0954405420987712
[8]	陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi: 10.37188/YJYXS20203512.1315 TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi: 10.37188/YJYXS20203512.1315
[9]	REVILLA-LEÓN M, GONZALEZ-MARTÍN Ó, PÉREZ LÓPEZ J, et al. Position accuracy of implant analogs on 3D printed polymer versus conventional dental stone casts measured using a coordinate measuring machine[J]. Journal of Prosthodontics, 2018, 27(6): 560-567. doi: 10.1111/jopr.12708
[10]	朱可, 霍彦文, 武通海, 等. 基于光度立体视觉三维重构算法的微观磨损形貌原位测量原理及方法[J]. 机械工程学报,2021,57(10):1-9. doi: 10.3901/JME.2021.10.001 ZHU K, HUO Y W, WU T H, et al. Principle and method for in-situ measurement of micro-scale worn surface morphology based on 3D reconstruction with photometric stereo vision Algorithm[J]. Journal of Mechanical Engineering, 2021, 57(10): 1-9. (in Chinese) doi: 10.3901/JME.2021.10.001
[11]	李中伟, 张攀, 钟凯, 等. AutoScan系列复杂零件自动化三维测量装备开发与应用[J]. 航空学报,2021,42(10):112-129. LI ZH W, ZHANG P, ZHONG K, et al. Development and application of AutoScan series automated 3D measuring equipment for complex parts[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42(10): 112-129. (in Chinese)
[12]	邱志成, 肖骏. 线结构光视觉系统标定新法及其振动测控应用[J]. 光学精密工程,2019,27(1):230-240. doi: 10.3788/OPE.20192701.0230 QIU ZH CH, XIAO J. New calibration method of line structured light vision system and application for vibration measurement and control[J]. Optics and Precision Engineering, 2019, 27(1): 230-240. (in Chinese) doi: 10.3788/OPE.20192701.0230
[13]	李茂月, 肖桂风, 蔡东辰, 等. 单目结构光测量中参数自适应标定方法[J/OL]. 红外与激光工程 [2022-03-21]. http://kns.cnki.net/kcms/detail/12.1261.tn.20211207.0957.002.html. LI M Y, XIAO G F, CAI D CH, et al. . Parameter adaptive calibration method in monocular structured light measurement[J/OL]. Infrared and Laser Engineering [2022-03-21]. http://kns.cnki.net/kcms/detail/12.1261.tn.20211207.0957.002.html. (in Chinese)
[14]	TAKEDA M, MUTOH K. Fourier transform profilometry for the automatic measurement of 3-D object shapes[J]. Applied Optics, 1983, 22(24): 3977-3982. doi: 10.1364/AO.22.003977
[15]	WU ZH J, GUO W B, LU L L, et al. Generalized phase unwrapping method that avoids jump errors for fringe projection profilometry[J]. Optics Express, 2021, 29(17): 27181-27192. doi: 10.1364/OE.436116
[16]	GUAN F L, XU A L, JIANG G Y, et al. An improved fast camera calibration method for mobile terminals[J]. Journal of Information Processing Systems, 2019, 15(5): 1082-1095.
[17]	YAO Q, KUBOTA A, KAWAKITA K, et al. . Fast camera self-calibration for synthesizing free viewpoint soccer video[C]. Proceedings of 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2017: 1612-1616.
[18]	GAO Z R, GAO Y, SU Y, et al. Stereo camera calibration for large field of view digital image correlation using zoom lens[J]. Measurement, 2021, 185: 109999. doi: 10.1016/j.measurement.2021.109999
[19]	ZHANG J, LUO B, XIANG ZH L, et al. Deep-learning-based adaptive camera calibration for various defocusing degrees[J]. Optics Letters, 2021, 46(22): 5537-5540. doi: 10.1364/OL.443337
[20]	XU X CH, LIU M Y, PENG S, et al. An In-orbit stereo navigation camera self-calibration method for planetary rovers with multiple constraints[J]. Remote Sensing, 2022, 14(2): 402. doi: 10.3390/rs14020402
[21]	FOLTYNSKI P, LADYZYNSKI P. Digital planimetry with a new adaptive calibration procedure results in accurate and precise wound area measurement at curved surfaces[J]. Journal of Diabetes Science and Technology, 2022, 16(1): 128-136. doi: 10.1177/1932296820959346
[22]	MORU D K, BORRO D. Analysis of different parameters of influence in industrial cameras calibration processes[J]. Measurement, 2021, 171: 108750. doi: 10.1016/j.measurement.2020.108750
[23]	管雯璐, 谭逢富, 靖旭, 等. 基于环境温度反馈的卡塞格林望远镜自动调焦[J]. 光学精密工程,2021,29(8):1832-1838. doi: 10.37188/OPE.20212908.1832 GUAN W L, TAN F F, JING X, et al. Automatic focusing of Cassegrain telescope based on environmental temperature feedback[J]. Optics and Precision Engineering, 2021, 29(8): 1832-1838. (in Chinese) doi: 10.37188/OPE.20212908.1832
[24]	潘海鸿, 王耀玮, 徐大斌, 等. 使用非特制靶标实现线结构光平面标定[J]. 光学精密工程, 2021, 29(1): 54-60. PAN H H, WANG Y W, XU D B, et al.. Calibration of line-structure light plane using non-specific target[J] Optics and Precision Engineering, 2021, 29(1): 54-60. (in Chinese)
[25]	CHOI H, RYU J. Design of wide angle and large aperture optical system with inner focus for compact system camera applications[J]. Applied Sciences, 2019, 10(1): 179.
[26]	PAK I J, KIM C S, KANG J C, et al. Verification of phase measurement error sources in phase-shifting interferometry with four step phase-shifting algorithms[J]. Applied Optics, 2021, 60(13): 3856-3864. doi: 10.1364/AO.418495
[27]	LI J, GUAN J T, DU H, et al. Error self-correction method for phase jump in multi-frequency phase-shifting structured light[J]. Applied Optics, 2021, 60(4): 949-958. doi: 10.1364/AO.413506
[28]	LIN H, GAO J, MEI Q, et al. Adaptive digital fringe projection technique for high dynamic range three-dimensional shape measurement[J]. Optics Express, 2016, 24(7): 7703-7718. doi: 10.1364/OE.24.007703
[29]	PINZEK S, GUSTSCHIN A, NEUWIRTH T, et al. Signal retrieval from non-sinusoidal intensity modulations in X-ray and neutron interferometry using piecewise-defined polynomial function[J]. Journal of Imaging, 2021, 7(10): 209. doi: 10.3390/jimaging7100209
[30]	RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
[31]	李茂月, 刘泽隆, 赵伟翔, 等. 面结构光在机检测的叶片反光抑制技术[J]. 中国光学,2022,15(3):464-475. doi: 10.37188/CO.2021-0194 LI M Y, LIU Z L, ZHAO W X, et al. Blade reflection suppression technology based on surface structured light on-machine detection[J]. Chinese Optics, 2022, 15(3): 464-475. (in Chinese) doi: 10.37188/CO.2021-0194
[32]	LIU X J, CHEN W Y, MADHUSUDANAN H, et al. Optical measurement of highly reflective surfaces from a single exposure[J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1882-1891. doi: 10.1109/TII.2020.2991458
[33]	WANG YH, ZHANG Q, HU Y, et al. Rapid 3D measurement of high dynamic range surface based on multi-polarization fringe projection[J]. Optical Engineering, 2021, 60(8): 084107.
[34]	RAMAKRISHNAN V, PETE D J. Savitzky–Golay filtering-based fusion of multiple exposure images for high dynamic range imaging[J]. SN Computer Science, 2021, 2(3): 191. doi: 10.1007/s42979-021-00594-9
[35]	陈龙, 王文聪, 张峰峰, 等. 基于双目结构光的术中肝脏表面局部亮度饱和分区投影[J]. 光学精密工程, 2021, 29(11): 2590-2602. CHEN L, WANG W C, ZHANG F F, et al. . Zonal projection based on binocular structured light for localized luminance saturation of intraoperative liver surface[J] Optics and Precision Engineering, 2021, 29(11): 2590-2602. (in Chinese)
[36]	JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
[37]	LIU G H, LIU X Y, FENG Q Y. 3D shape measurement of objects with high dynamic range of surface reflectivity[J]. Applied Optics, 2011, 50(23): 4557-4565. doi: 10.1364/AO.50.004557
[38]	ZHANG B L, KWOK H S, HUANG H C. Three-dimensional optical modeling and optimizations of color filter liquid-crystal-on-silicon microdisplays[J]. Journal of Applied Physics, 2005, 98(12): 123103. doi: 10.1063/1.2149494
[39]	CHEN M Y, GUO H W, WEI C L. Algorithm immune to tilt phase-shifting error for phase-shifting interferometers[J]. Applied Optics, 2000, 39(22): 3894-3898. doi: 10.1364/AO.39.003894
[40]	FENG SH J, ZUO CH, TAO T Y, et al. Robust dynamic 3-D measurements with motion-compensated phase-shifting profilometry[J]. Optics and Lasers in Engineering, 2018, 103: 127-138. doi: 10.1016/j.optlaseng.2017.12.001
[41]	DENG J, LI J, FENG H, et al. Edge-preserved fringe-order correction strategy for code-based fringe projection profilometry[J]. Signal Processing, 2021, 182: 107959. doi: 10.1016/j.sigpro.2020.107959
[42]	郭逸凡. 运动物体散焦光栅投影三维测量技术研究[D]. 南京: 东南大学, 2019: 30-42. GUO Y F. The research of defocusing fringe pattern projection profilometry on dynamic objects[D]. Nanjing: Southeast University, 2019: 30-42. (in Chinese)
[43]	KIM S S, CHUNG B M. Rapid 3D shape measurement using sine pattern in phase-shifting projection method[J]. International Journal of Precision Engineering and Manufacturing, 2021, 22(8): 1381-1389. doi: 10.1007/s12541-021-00545-7
[44]	WEISE T, LEIBE B, VAN GOOL L. Fast 3D scanning with automatic motion compensation[C]. Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2007: 1-8.
[45]	TAO T Y, CHEN Q, FENG SH J, et al. High-speed real-time 3D shape measurement based on adaptive depth constraint[J]. Optics Express, 2018, 26(17): 22440-22456. doi: 10.1364/OE.26.022440
[46]	GONG M, ZHANG ZH J, ZENG D. A new simplification algorithm for scattered point clouds with feature preservation[J]. Symmetry, 2021, 13(3): 399. doi: 10.3390/sym13030399
[47]	杜钦生, 李丹丹, 陈浩, 等. 结构光3D点云的PIN针针尖提取[J]. 液晶与显示,2021,36(9):1331-1340. doi: 10.37188/CJLCD.2020-0321 DU Q SH, LI D D, CHEN H, et al. PIN tip extraction from 3D point cloud of structured light[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(9): 1331-1340. (in Chinese) doi: 10.37188/CJLCD.2020-0321
[48]	NIE E J, LI J, ZHANG R M, et al. . Three dimensional point cloud hole repairing strategy for binocular stereo reconstruction[C]. Proceedings of 2017 IEEE International Conference on Robotics and Biomimetics, IEEE, 2017: 2456-2461.
[49]	JU T. Robust repair of polygonal models[J]. ACM Transactions on Graphics, 2004, 23(3): 888-895. doi: 10.1145/1015706.1015815
[50]	CHEN H, CUI W. Holes filling of scattered point cloud based on simplification[J]. Multimedia Tools and Applications, 2022, 81(11): 14641-14661. doi: 10.1007/s11042-021-11019-3
[51]	CHU T, YAO W M, LIU J, et al. Hole-filling framework by combining structural and textural information for the 3D Terracotta Warriors[J]. Journal of Applied Remote Sensing, 2021, 15(4): 046503.
[52]	李月雯, 耿国华, 魏潇然. 基于泊松方程的孔洞修补算法[J]. 计算机工程,2017,43(10):209-215,221. doi: 10.3969/j.issn.1000-3428.2017.10.035 LI Y W, GENG G H, WEI X R. Hole-filling algorithm based on Poisson equation[J]. Computer Engineering, 2017, 43(10): 209-215,221. (in Chinese) doi: 10.3969/j.issn.1000-3428.2017.10.035
[53]	刘中玉, 张明锋, 聂雪媛, 等. 一种基于径向基函数的两步法网格变形策略[J]. 力学学报,2015,47(3):534-538. doi: 10.6052/0459-1879-14-280 LIU ZH Y, ZHANG M F, NIE X Y, et al. A two-step mesh deformation strategy based on radial basis function[J]. Chinese Journal of Theoretical and Applied Mechanics, 2015, 47(3): 534-538. (in Chinese) doi: 10.6052/0459-1879-14-280
[54]	LIN H B, WANG W. Feature preserving holes filling of scattered point cloud based on tensor voting[C]. Proceedings of 2016 IEEE International Conference on Signal and Image Processing, IEEE, 2016: 402-406.
[55]	ZHANG H, TIAN ZH G. Failure analysis of corroded high-strength pipeline subject to hydrogen damage based on FEM and GA-BP neural network[J]. International Journal of Hydrogen Energy, 2022, 47(7): 4741-4758. doi: 10.1016/j.ijhydene.2021.11.082
[56]	WANG J SH, GONG Z Y, TAO B, et al. A 3-D reconstruction method for large freeform surfaces based on mobile robotic measurement and global optimization[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 5006809.
[57]	WANG J SH, TAO B, GONG Z Y, et al. A mobile robotic measurement system for large-scale complex components based on optical scanning and visual tracking[J]. Robotics and Computer-Integrated Manufacturing, 2021, 67: 102010. doi: 10.1016/j.rcim.2020.102010
[58]	GUO B Y, WANG J W, JIANG X B, et al. A 3D surface reconstruction method for large-scale point cloud data[J]. Mathematical Problems in Engineering, 2020, 2020: 8670151.
[59]	DE PAOLIS L T, DE LUCA V, GATTO C, et al. . Photogrammetric 3D reconstruction of small objects for a real-time fruition[C]. Proceedings of the 7th International Conference on Augmented Reality, Virtual Reality, and Computer Graphics, Springer, 2020: 375-394.
[60]	张溪溪, 纪小刚, 胡海涛, 等. 基于特征线拟合的微型复杂曲面点云分割方法[J]. 激光与光电子学进展,2020,57(6):061502. ZHANG X X, JI X G, HU H T, et al. Point cloud segmentation method for complex micro-surface based on feature line fitting[J]. Laser &Optoelectronics Progress, 2020, 57(6): 061502. (in Chinese)
[61]	吴恩启, 柯映林, 李江雄. 微细管道内表面光电检测及三维重构系统[J]. 光电工程,2007,34(1):59-64. WU E Q, KE Y L, LI J X. Photoelectric detection and 3D reconstruction system for the inner surface of small-diameter pipes[J]. Opto-Electronic Engineering, 2007, 34(1): 59-64. (in Chinese)
[62]	HARIYAMA T, MARUNO K, WATANABE M, et al. High-accuracy error-reduction method for 3D complex shape measurement with local-maximum-power-based technique of FMCW[J]. Precision Engineering, 2021, 72: 69-82. doi: 10.1016/j.precisioneng.2021.04.004