Attitude compensation and reconstruction methods for single-photon dynamic imaging during UAV flight
-
摘要:
针对单光子深度成像中探测器受散粒噪声和背景噪声的干扰，以及无人机在飞行过程中姿态变化带来的单轴图像偏差问题，旨在提升深度图像在低信号背景噪声比（SBR）或高信号背景噪声比下的重建质量。分析并在经典的SPIRAL-TAP重建框架基础上，提出了一种融合多尺度图像特征与自适应阈值筛选的新型深度图重建方法。该方法首先通过多尺度梯度与局部方差计算生成图像加权矩阵，以刻画图像纹理复杂度；随后结合基于 ROM（Rough Order Map）估计的尺度因子对阈值进行动态调整，以增强噪声鲁棒性；在阈值筛选阶段，提出自适应阈值策略，将尺度平滑与加权矩阵软调融合，限制阈值范围，使筛选更加稳定可靠。实验结果表明，在多种SBR和光子强度条件下，并考虑到无人机单轴姿态偏差影响下，本文方法均优于传统SPIRAL-TAP算法，具有更低的RMSE误差和更好的重建质量。在倾斜角为10°和15°时RMSE分别由0.32降至0.14和从0.43降至0.21。本文方法为无人机载单光子深度图像重建提供了有效的新思路，未来可用在机载高速单光子成像系统中。
Abstract:To mitigate shot noise and background interference in single-photon depth imaging, alongside single-axis image deviation induced by UAV attitude fluctuations during flight, this paper proposes a robust depth reconstruction method. Building upon the SPIRAL-TAP framework, the proposed approach integrates multi-scale image features with an adaptive thresholding strategy. Firstly, an image weighting matrix is constructed via multi-scale gradients and local variance to effectively characterize texture complexity. Subsequently, a dynamic threshold adjustment mechanism, guided by Rough Order Map (ROM) estimation, is implemented to enhance noise robustness. In the screening phase, an adaptive strategy merges scale-space smoothing with weighting matrix soft-tuning to stabilize the filtering process. Experimental results demonstrate that the proposed method significantly outperforms the conventional SPIRAL-TAP algorithm under varying signal-to-background ratios (SBR) and photon intensities. Specifically, at tilt angles of 10° and 15°, the RMSE is reduced from 0.32 to 0.14 and from 0.43 to 0.21, respectively. This method provides an effective solution for UAV-borne single-photon depth reconstruction and exhibits significant potential for high-speed airborne imaging systems.
-
图 1 单光子激光雷达成像系统示意
Figure 1. Schematic of the single-photon counting LiDAR imaging system.
图 2 极少回波光子情形下典型光子到达时间直方图（a） 单个像素的光子到达时间直方图，在低光子计数条件下呈现稀疏分布(b) 所有像素的光子时间分布聚合图，整体仍表现出高稀疏性与背景噪声的均匀性
Figure 2. Typical photon arrival time histograms under photon-starved conditions (a) Photon arrival time histogram of a single pixel, exhibiting sparse distribution under low photon counts (b) Aggregated photon temporal distribution of all pixels, illustrating persistent high sparsity and uniform background noise
图 5 无人机飞行姿态偏差模拟图
Figure 5. Simulation model of UAV flight attitude deviation and its geometric impact
图 6 加权矩阵
$ {\boldsymbol{W}}_{i,j} $ 生成流程图Figure 6. Flowchart of Weighted Matrix
$ {\boldsymbol{W}}_{i,j} $ Generation
图 7 结构中像素相似性及边权分布示意图
Figure 7. Schematic of pixel similarity and edge weight distribution within the graph-based structure
图 8 信噪比为10时，对于算法优化前后SPPP为1、2、5、10的仿真卧室的重建效果
Figure 8. shows the reconstruction results of the simulated bedroom with SPPP of 1, 2, 5, and 10 before and after algorithm optimization when the signal-to-noise ratio is 10.
图 9 信噪比为0.8时，对于算法优化前后SPPP为1、2、5、10的仿真卧室的重建效果
Figure 9. shows the reconstruction results of the simulated bedroom with SPPP of 1, 2, 5, and 10 before and after algorithm optimization when the signal-to-noise ratio is 0.8.
图 10 无人机飞行速度为4m/s时的俯仰角度实时变化
Figure 10. Real-time changes in the pitch angle when the flight speed of the unmanned aerial vehicle is 4 m/s
图 11 无人机飞行速度为8 m/s时的俯仰角度实时变化
Figure 11. Real-time changes in the pitch angle when the flight speed of the unmanned aerial vehicle is 8 m/s
图 12 无人机有无姿态偏差的深度bin值对比
Figure 12. Comparison of depth bin values with and without UAV attitude deviation
表 1 信噪比为10和0.8，对于算法优化前后量化为RMSE前后对比
Table 1. Comparison of RMSE before and after algorithm optimization with SNR of 10 and 0.8
Methods SPPP=1 SPPP=2 SPPP=5 SPPP=10 RMSE/m Shin 0.44 0.37 0.13 0.07 proposed 0.23 0.16 0.07 0.06 Methods SPPP=1 SPPP=2 SPPP=5 SPPP=10 RMSE/m Shin 1.40 1.10 0.43 0.19 proposed 0.76 0.43 0.23 0.09 
下载: 导出CSV
表 2 不同俯仰倾角下的重建误差（RMSE）对比（SBR=10, SPPP=5）
Table 2. Comparison of Reconstruction Error (RMSE) at Different Pitch Angles (SBR = 10, SPPP = 5)
俯仰角度($ \phi $) Shin方法+
(未补偿)Proposed+
（姿态补偿）误差降低
比例RMSE/m 0°（平飞） 0.12 0.12 0% 5° 0.18 0.13 27.8% 10° 0.32 0.14 56.2% 15° 0.43 0.21 51.1% 20° 0.65 0.22 66.1% 25° 0.88 0.23 73.8%
下载: 导出CSV
表 3 算法模块消融实验对比（SBR=10, SPPP=5, ϕ = 15°）
Table 3. Comparison of Ablation Experiments on Algorithm Modules (SBR = 10, SPPP = 5, ϕ = 15°)
实验
方案姿态
补偿多尺度
加权自适应
阈值RMSE 提升贡献 A - - - 0.43 基准 B √ - - 0.31 解决了系统性几何畸变 C √ √ - 0.25 抑制了空间随机噪声 D √ √ √ 0.21 找回了微弱信号特征
下载: 导出CSV
-
[1] HENDERSON R K, JOHNSTON N, ROCCA F M D, et al. A 192×128 time correlated SPAD image sensor in 40-nm CMOS technology[J]. IEEE Journal of Solid-State Circuits, 2019, 54(7): 1907-1916. doi: 10.1109/JSSC.2019.2905163 [2] SHIN D, KIRMANI A, GOYAL V K, et al. Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors[J]. IEEE Transactions on Computational Imaging, 2015, 1(2): 112-125. doi: 10.1109/TCI.2015.2453093 [3] TANNER M G, GALLAND C, ZBINDEN H, et al. Single-photon imaging and its application in biological microscopy[J]. Proceedings of SPIE, 2015, 9504: 95040A. (查阅网上资料, 未找到本条文献信息, 请确认). [4] RECH I, GULINATTI A, GHIONI M. Optical imaging using single-photon detectors: from physics to biomedical applications[J]. Journal of Optics, 2021, 23(10): 103501. (查阅网上资料, 未找到本条文献信息, 请确认). [5] MAIRE M, HENDERSON R K, CHARBON E. SPAD imagers for space applications: trends and perspectives[J]. Sensors, 2020, 20(6): 1451. (查阅网上资料, 未找到本条文献信息, 请确认). [6] HARMANY Z T, MARCIA R F, WILLETT R M. This is SPIRAL-TAP: Sparse Poisson intensity reconstruction ALgorithms—theory and practice[J]. IEEE Transactions on Image Processing, 2012, 21(3): 1084-1096. doi: 10.1109/TIP.2011.2168410 [7] ALTMANN Y, MCCARTHY A, BULLER G S, et al. Bayesian approach to denoising of single-photon binary images[J]. IEEE Transactions on Image Processing, 2016, 25(5): 1935-1946. (查阅网上资料, 未找到本条文献信息, 请确认). [8] CHAN S H, ZICKLER T, LU Y M. Monte Carlo non-local means: Random sampling for large-scale image filtering[J]. IEEE Transactions on Image Processing, 2014, 23(8): 3711-3725. doi: 10.1109/TIP.2014.2327813 [9] KIRMANI A, VENKATRAMAN D, SHIN D, et al. First-photon imaging[J]. Science, 2013, 343(6166): 58-61. doi: 10.1364/cosi.2015.ct3f.1 [10] 朱思宇, 朱磊, 王文武, 等. 基于融合特征分布学习与图像重建的异常检测[J]. 液晶与显示, 2024, 39(8): 1116-1127.ZHU S Y, ZHU L, WANG W W, et al. Abnormity detection based on fusion feature distribution learning and image reconstruction[J]. Chinese Journal of Liquid Crystals and Displays, 2024, 39(8): 1116-1127. [11] RAPP J, GOYAL V K. A few photons among many: Unmixing signal and noise for photon-efficient active imaging[J]. IEEE Transactions on Computational Imaging, 2017, 3(3): 445-459. doi: 10.1109/TCI.2017.2706028 [12] HEIDE F, DIAMOND S, LINDELL D B, et al. Sub-picosecond photon-efficient 3D imaging using single-photon sensors[J]. Scientific Reports, 2018, 8(1): 17726. doi: 10.1038/s41598-018-35212-x [13] REN W, DAI W, LIU Z, et al. Robust image deblurring with structures-preserving via adaptive total variation[J]. Signal Processing, 2016, 122: 63-76. (查阅网上资料, 未找到本条文献信息, 请确认). [14] 张永兴, 连博文, 顾乃庭, 等. 基于多尺度空间注意力互补的红外与可见光图像融合[J]. 光学 精密工程, 2025, 33(7): 1152-1168.ZHANG Y X, LIAN B W, GU N T, et al. Infrared and visible image fusion based on multi-scale spatial attention complementary[J]. Optics and Precision Engineering, 2025, 33(7): 1152-1168. [15] TSAI C, KIRMANI A, SHIN D, et al. Beyond 3D imaging: Reflectance, illumination, and geometry from depth cameras[J]. International Journal of Computer Vision, 2017, 125(1-3): 1-16. (查阅网上资料, 未找到本条文献信息, 请确认). [16] ZHU X, SONG C, DU J, et al. Single-photon 3D imaging using statistical filtering and depth histogram shaping[J]. Optics Express, 2020, 28(6): 8843-8857. (查阅网上资料, 未找到本条文献信息, 请确认). [17] XUE T, XU D, CHEN S, et al. Single-photon imaging via spatiotemporal correlation filtering[J]. Optics Letters, 2021, 46(4): 770-773. (查阅网上资料, 未找到本条文献信息, 请确认). [18] 赵云涛, 邓新辉. 基于多模态数据的注意特征融合姿态估计网络[J]. 液晶与显示, 2025, 40(4): 598-606.ZHAO Y T, DENG X H. Pose estimation network based on attention feature fusion of multimodal data[J]. Chinese Journal of Liquid Crystals and Displays, 2025, 40(4): 598-606. [19] LINDELL D B, WETZSTEIN G. Single-photon 3D imaging with deep sensor fusion[J]. IEEE Transactions on Computational Imaging, 2021, 7: 677-689. (查阅网上资料, 未找到本条文献信息, 请确认). [20] ZHAO Y, LI J, GAO F, et al. Adaptive photon thresholding and fusion for robust single-photon imaging[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-9. (查阅网上资料, 未找到本条文献信息, 请确认). [21] MAIRE M, HENDERSON R K, CHARBON E. SPAD imagers for space applications: Trends and perspectives[J]. Sensors, 2020, 20(6): 1451. (查阅网上资料, 未找到本条文献信息, 请确认)(查阅网上资料, 本条文献与第5条文献重复, 请核对). [22] PAWLIKOWSKA A M, HALIMI A, LAMB R A, et al. Single-photon three-dimensional imaging at up to 10 kilometers range[J]. Optics Express, 2017, 25(10): 11919-11931. doi: 10.1364/OE.25.011919 [23] BULLER G, WALLACE A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2007, 13(4): 1006-1015. doi: 10.1109/JSTQE.2007.902850 [24] BECKER W. Advanced Time-correlated Single Photon Counting Techniques[M]. New York: Springer, 2005. -
下载:
计量
- 文章访问数: 10
- HTML全文浏览量: 8
- PDF下载量: 0
- 被引次数: 0
