Volume 14 Issue 3
May  2021
Turn off MathJax
Article Contents
WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry:a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199
Citation: WANG Yong-hong, ZHANG Qian, HU Yin, WANG Huan-qing. 3D small-field surface imaging based on microscopic fringe projection profilometry:a review[J]. Chinese Optics, 2021, 14(3): 447-457. doi: 10.37188/CO.2020-0199

3D small-field surface imaging based on microscopic fringe projection profilometry:a review

doi: 10.37188/CO.2020-0199
Funds:  Supported by National Key Research and Development Program of China (No. 2016YFF0101803); National Natural Science Foundation of China (No. 51805137)
More Information
  • Corresponding author: yhwang@hfut.edu.cn
  • Received Date: 2020-11-10
  • Rev Recd Date: 2021-01-07
  • Available Online: 2021-03-27
  • Publish Date: 2021-05-14
  • Intelligent manufacturing has become more precise, miniaturized and integrated. Representative integrated circuit technology and its derived miniature sensors such as Micro-Electro-Mechanical System (MEMS) have become widely used. Therefore, it is important for intelligent manufacturing development to accurately obtain the surface morphology information of micro-devices and implement rapid detection of device surface defects. Fringe Projection Profilometry (FPP) based on structural light projection has the advantages of being non-contact, highly precise, highly efficient and having full-field measurement, which plays an important role in the field of precision measurement. Microscopic Fringe Projection Profilometry (MFPP) has been developed rapidly during recent decades. In recent years, MFPP has made great progress in many aspects, including its optical system structures, corresponding system calibration methods, phase extraction algorithms, and 3D coordinate reconstruction methods. In this paper, the structure and principle of a three-dimensional measurement system of microscopic fringe projection are reviewed, and the calibration problem of a small field-of-view system that is different from the traditional projection model is analyzed. After that, the development and improvement process of the micro-projection system structure is introduced, and the reflection in the measurment caused by the system structure and metal material is analyzed. On this basis, the prospects of the development of microscopic fringe projection of 3D measurement system are discussed.
  • loading
  • [1]
    WANG J H, YANG Y X, SHAO M W, et al. Three-dimensional measurement for rigid moving objects based on multi-fringe projection[J]. IEEE Photonics Journal, 2020, 12(4): 6802114.
    XIA P, WANG Q H, RI SH E. Random phase-shifting digital holography based on a self-calibrated system[J]. Optics Express, 2020, 28(14): 19988-19996. doi: 10.1364/OE.395819
    屈铭, 郑俊杰, 李敏, 等. 基于扫描白光干涉法的LCOS芯片像素级相位分析[J]. 光子学报,2019,48(9):0911004. doi: 10.3788/gzxb20194809.0911004

    QU M, ZHENG J J, LI M, et al. Pixel-level observation of phase profile in liquid crystal on silicon device by white light scanning interferometry[J]. Acta Photonica Sinica, 2019, 48(9): 0911004. (in Chinese) doi: 10.3788/gzxb20194809.0911004
    MURAKAMI H, KATSUKI A, SAJIMA T, et al. Investigation of factors affecting sensitivity enhancement of an optical fiber probe for microstructure measurement using oblique incident light[J]. Applied Sciences, 2020, 10(9): 3191. doi: 10.3390/app10093191
    李成辉, 田云飞, 闫曙光. 激光扫描共聚焦显微成像技术与应用[J]. 实验科学与技术,2020,18(4):33-38. doi: 10.12179/1672-4550.20190257

    LI CH H, TIAN Y F, YAN SH G. Laser scanning confocal microscopy and its application[J]. Experiment Science and Technology, 2020, 18(4): 33-38. (in Chinese) doi: 10.12179/1672-4550.20190257
    HU Y, CHEN Q, FENG SH J, et al. Microscopic fringe projection profilometry: a review[J]. Optics and Lasers in Engineering, 2020, 135: 106192. doi: 10.1016/j.optlaseng.2020.106192
    LEONHARDT K, DROSTE U, TIZIANI H J. Microshape and rough-surface analysis by fringe projection[J]. Applied Optics, 1994, 33(31): 7477-7488. doi: 10.1364/AO.33.007477
    QUAN C, HE X Y, WANG C F, et al. Shape measurement of small objects using LCD fringe projection with phase shifting[J]. Optics Communications, 2001, 189(1-3): 21-29. doi: 10.1016/S0030-4018(01)01038-0
    PROLL K P, NIVET J M, KÖRNER K, et al. Microscopic three-dimensional topometry with ferroelectric liquid-crystal-on-silicon displays[J]. Applied Optics, 2003, 42(10): 1773-1778. doi: 10.1364/AO.42.001773
    NOTNI G, RIEHEMANN S, KUEHMSTEDT P, et al. OLED microdisplays: a new key element for fringe projection setups[J]. Proceedings of SPIE, 2004, 5532: 170-177. doi: 10.1117/12.560433
    ZHANG SH F, LI B, REN F J, et al. High-precision measurement of binocular telecentric vision system with novel calibration and matching methods[J]. IEEE Access, 2019, 7: 54682-54692. doi: 10.1109/ACCESS.2019.2913181
    ZHANG CH P, HUANG P S, CHIANG F P. Microscopic phase-shifting profilometry based on digital micromirror device technology[J]. Applied Optics, 2002, 41(28): 5896-5904. doi: 10.1364/AO.41.005896
    LIU Y H, ZHANG Q C, ZHANG H H, et al. Improve temporal Fourier transform profilometry for complex dynamic three-dimensional shape measurement[J]. Sensors, 2020, 20(7): 1808. doi: 10.3390/s20071808
    ZHANG H H, ZHANG Q C, LI Y, et al. High speed 3D shape measurement with temporal Fourier transform profilometry[J]. Applied Sciences, 2019, 9(19): 4123. doi: 10.3390/app9194123
    史耀群, 邓林嘉, 王朝旭, 等. 一种基于结构光条纹投影的微小物体测量系统[J]. 应用光学,2019,40(6):1120-1125. doi: 10.5768/JAO201940.0603007

    SHI Y Q, DENG L J, WANG ZH X, et al. Micro-objects measurement system based on structured light fringe projection[J]. Journal of Applied Optics, 2019, 40(6): 1120-1125. (in Chinese) doi: 10.5768/JAO201940.0603007
    ZHONG M, CUI J, HYUN J S, et al. Uniaxial three-dimensional phase-shifting profilometry using a dual-telecentric structured light system in micro-scale devices[J]. Measurement Science and Technology, 2020, 31(8): 085003. doi: 10.1088/1361-6501/ab63b2
    殷永凯, 张宗华, 刘晓利, 等. 条纹投影轮廓术系统模型与标定综述[J]. 红外与激光工程,2020,49(3):0303008. doi: 10.3788/IRLA202049.0303008

    YIN Y K, ZHANG Z H, LIU X L, et al. Review of the system model and calibration for fringe projection profilometry[J]. Infrared and Laser Engineering, 2020, 49(3): 0303008. (in Chinese) doi: 10.3788/IRLA202049.0303008
    ZHANG ZH Y. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334. doi: 10.1109/34.888718
    LI B W, KARPINSKY N, ZHANG S. Novel calibration method for structured-light system with an out-of-focus projector[J]. Applied Optics, 2014, 53(16): 3415-3426. doi: 10.1364/AO.53.003415
    LI B W, ZHANG S. Flexible calibration method for microscopic structured light system using telecentric lens[J]. Optics Express, 2015, 23(20): 25795-25803. doi: 10.1364/OE.23.025795
    LIU H B, LIN H J, YAO L SH. Calibration method for projector-camera-based telecentric fringe projection profilometry system[J]. Optics Express, 2017, 25(25): 31492-31508. doi: 10.1364/OE.25.031492
    安东, 陈李, 丁一飞, 等. 光栅投影相位法系统模型及标定方法[J]. 中国光学,2015,8(2):248-254. doi: 10.3788/co.20150802.0248

    AN D, CHEN L, DING Y F, et al. Optical system model and calibration of grating projection phase method[J]. Chinese Optics, 2015, 8(2): 248-254. (in Chinese) doi: 10.3788/co.20150802.0248
    丁一飞, 王永红, 胡悦, 等. 样本块匹配光栅投影阶梯标定方法[J]. 中国测试,2016,42(8):7-12. doi: 10.11857/j.issn.1674-5124.2016.08.002

    DING Y F, WANG Y H, HU Y, et al. Step calibration method of grating projection based on exemplar matching[J]. China Measurement &Test, 2016, 42(8): 7-12. (in Chinese) doi: 10.11857/j.issn.1674-5124.2016.08.002
    LU P, SUN CH K, LIU B, et al. Accurate and robust calibration method based on pattern geometric constraints for fringe projection profilometry[J]. Applied Optics, 2017, 56(4): 784-794. doi: 10.1364/AO.56.000784
    CHEN Z, LIAO H Y, ZHANG X M. Telecentric stereo micro-vision system: calibration method and experiments[J]. Optics and Lasers in Engineering, 2014, 57: 82-92. doi: 10.1016/j.optlaseng.2014.01.021
    HU Y, CHEN Q, LI H Y, et al. Absolute three-dimensional micro surface profile measurement based on a Greenough-type stereomicroscope[J]. Measurement Science and Technology, 2017, 28(4): 045004. doi: 10.1088/1361-6501/aa5a2d
    Overview: DLP products[EB/OL]. [2020-10-18]. http://www.ti.com/dlp-chip/overview.html.
    肖萍萍. 基于光栅投射的小尺寸物体三维形状测量系统研究[D]. 武汉: 华中科技大学, 2019.

    XIAO P P. Research on 3D shape measurement system of small scale object based on grating projection[D]. Wuhan: Huazhong University of Science and Technology, 2019. (in Chinese).
    VAN DER JEUGHT S, SOONS J A M, DIRCKX J J J. Real-time microscopic phase-shifting profilometry[J]. Applied Optics, 2015, 54(15): 4953-4959.
    CHEN L C, LIAO CH CH, LAI M J. Full-field micro surface profilometry using digital fringe projection with spatial encoding principle[J]. Journal of Physics:Conference Series, 2005, 13: 147-150. doi: 10.1088/1742-6596/13/1/034
    LI A M, PENG X, YIN Y K, et al. Fringe projection based quantitative 3D microscopy[J]. Optik, 2013, 124(21): 5052-5056. doi: 10.1016/j.ijleo.2013.03.070
    LI D, LIU CH Y, TIAN J D. Telecentric 3D profilometry based on phase-shifting fringe projection[J]. Optics Express, 2014, 22(26): 31826-31835. doi: 10.1364/OE.22.031826
    PENG J ZH, WANG M, DENG N N, et al. Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens[J]. Applied Optics, 2015, 54(34): 10055-10062. doi: 10.1364/AO.54.010055
    YIN Y K, WANG M, GAO B Z, et al. Fringe projection 3D microscopy with the general imaging model[J]. Optics Express, 2015, 23(5): 6846-6857. doi: 10.1364/OE.23.006846
    WANG M, YIN Y K, DENG D N, et al. Improved performance of multi-view fringe projection 3D microscopy[J]. Optics Express, 2017, 25(16): 19408-19421. doi: 10.1364/OE.25.019408
    HU Y, CHEN Q, FENG SH J, et al. A new microscopic telecentric stereo vision system-calibration, rectification, and three-dimensional reconstruction[J]. Optics and Lasers in Engineering, 2019, 113: 14-22. doi: 10.1016/j.optlaseng.2018.09.011
    HU Y, LIANG Y CH, TAO T Y, et al. Dynamic 3D measurement of thermal deformation based on geometric-constrained stereo-matching with a stereo microscopic system[J]. Measurement Science and Technology, 2019, 30(12): 125007. doi: 10.1088/1361-6501/ab35a1
    QUAN C, TAY C J, HE X Y, et al. Microscopic surface contouring by fringe projection method[J]. Optics &Laser Technology, 2002, 34(7): 547-552.
    WANG W H, WONG Y S, HONG G S. 3D measurement of crater wear by phase shifting method[J]. Wear, 2006, 261(2): 164-171. doi: 10.1016/j.wear.2005.09.036
    张莲涛, 卢荣胜, 程子怡. 基于相移偏折法的高反射表面面形测量技术[J]. 光子学报,2020,49(1):0112002. doi: 10.3788/gzxb20204901.0112002

    ZHANG L T, LU R SH, CHENG Z Y. Measurement technique of high-reflected surface based on phase measuring deflectometry[J]. Acta Photonica Sinica, 2020, 49(1): 0112002. (in Chinese) doi: 10.3788/gzxb20204901.0112002
    LIU X H, ZHANG Z H, GAO N, et al. 3D shape measurement of diffused/specular surface by combining fringe projection and direct phase measuring deflectometry[J]. Optics Express, 2020, 28(19): 27561-27574. doi: 10.1364/OE.402432
    陶迁, 周志峰, 吴明晖, 等. 基于相位测量偏折术的反射表面缺陷检测[J]. 液晶与显示,2020,35(12):1315-1322. doi: 10.37188/YJYXS20203512.1315

    TAO Q, ZHOU ZH F, WU M H, et al. Detection of reflective surface defects based on phase measuring deflectometry[J]. Chinese Journal of Liquid Crystals and Displays, 2020, 35(12): 1315-1322. (in Chinese) doi: 10.37188/YJYXS20203512.1315
    王月敏, 张宗华, 高楠. 基于全场条纹反射的镜面物体三维面形测量综述[J]. 光学 精密工程,2018,26(5):1014-1027. doi: 10.3788/OPE.20182605.1014

    WANG Y M, ZHANG Z H, GAO N. Review on three-dimensional surface measurements of specular objects based on full-field fringe reflection[J]. Optics and Precision Engineering, 2018, 26(5): 1014-1027. (in Chinese) doi: 10.3788/OPE.20182605.1014
    ZHANG L, CHEN Q, ZUO CH, et al. High-speed high dynamic range 3D shape measurement based on deep learning[J]. Optics and Lasers in Engineering, 2020, 134: 106245. doi: 10.1016/j.optlaseng.2020.106245
    JIANG H ZH, ZHAO H J, LI X D. High dynamic range fringe acquisition: a novel 3-D scanning technique for high-reflective surfaces[J]. Optics and Lasers in Engineering, 2012, 50(10): 1484-1493. doi: 10.1016/j.optlaseng.2011.11.021
    RAO L, DA F P. High dynamic range 3D shape determination based on automatic exposure selection[J]. Journal of Visual Communication and Image Representation, 2018, 50: 217-226. doi: 10.1016/j.jvcir.2017.12.003
    ZHANG S. Rapid and automatic optimal exposure control for digital fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106029. doi: 10.1016/j.optlaseng.2020.106029
    CHEN CH, GAO N, WANG X J, et al. Adaptive projection intensity adjustment for avoiding saturation in three-dimensional shape measurement[J]. Optics Communications, 2018, 410: 694-702. doi: 10.1016/j.optcom.2017.11.009
    WANG J H, YANG Y X. High-speed three-dimensional measurement technique for object surface with a large range of reflectivity variations[J]. Applied Optics, 2018, 57(30): 9172-9182. doi: 10.1364/AO.57.009172
    SONG ZH, JIANG H L, LIN H B, et al. A high dynamic range structured light means for the 3D measurement of specular surface[J]. Optics and Lasers in Engineering, 2017, 95: 8-16. doi: 10.1016/j.optlaseng.2017.03.008
    LIU Y ZH, FU Y J, CAI X Q, et al. A novel high dynamic range 3D measurement method based on adaptive fringe projection technique[J]. Optics and Lasers in Engineering, 2020, 128: 106004. doi: 10.1016/j.optlaseng.2020.106004
    万钇良, 王建立, 张楠. 一种基于相位相关与子图像的偏振图像配准方法[J]. 液晶与显示,2019,34(5):530-536. doi: 10.3788/YJYXS20193405.0530

    WAN Y L, WANG J L, ZHANG N. Polarized image registration method based on phase correlation and sub-graph[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(5): 530-536. (in Chinese) doi: 10.3788/YJYXS20193405.0530
    RIVIERE J, RESHETOUSKI I, FILIPI L, et al. Polarization imaging reflectometry in the wild[J]. ACM Transactions on Graphics, 2017, 36(6): 206.
    FENG SH J, ZHANG Y ZH, CHEN Q, et al. General solution for high dynamic range three-dimensional shape measurement using the fringe projection technique[J]. Optics and Lasers in Engineering, 2014, 59: 56-71. doi: 10.1016/j.optlaseng.2014.03.003
    BENVENISTE R, ÜNSALAN C. Binary and ternary coded structured light 3D scanner for shiny objects[M]//GELENBE E, LENT R, SAKELLARI G, et al. . Computer and Information Sciences. Dordrecht: Springer, 2011: 241-244.
    BENVENISTE R, ÜNSALAN C. A color invariant for line stripe-based range scanners[J]. The Computer Journal, 2011, 54(5): 738-753. doi: 10.1093/comjnl/bxq014
    BENVENISTE R, ÜNSALAN C. Nary coded structured light-based range scanners using color invariants[J]. Journal of Real-Time Image Processing, 2014, 9(2): 359-377. doi: 10.1007/s11554-011-0235-4
    MENG L F, LU L Y, BEDARD N, et al. . Single-shot specular surface reconstruction with gonio-plenoptic imaging[C]. Proceedings of 2015 IEEE International Conference on Computer Vision, IEEE, 2015.
    ZHANG L, CHEN Q, ZUO CH, et al. High dynamic range and real-time 3D measurement based on a multi-view system[J]. Proceedings of SPIE, 2019, 11427: 1142715.
    HU Y, CHEN Q, LIANG Y CH, et al. Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme[J]. Optics and Lasers in Engineering, 2019, 122: 1-7. doi: 10.1016/j.optlaseng.2019.05.019
  • 加载中


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(6)  / Tables(4)

    Article views (747) PDF downloads(195) Cited by()
    Proportional views


    DownLoad:  Full-Size Img  PowerPoint