Self-supervised learning-based depth completion method using thermal imaging and LiDAR fusion
DOI:
CSTR:
Author:
Affiliation:

Clc Number:

TH811 TP242

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    Depth completion is a technique for generating high-resolution dense depth maps from sparse depth data for environmental perception. Existing methods struggle with accuracy in low-light or dark conditions, performing poorly under extreme lighting. This article proposes a self-supervised method that fuses thermal images and LiDAR data to complete dense depth maps in low-light or no-light scenarios. The network adopts an encoder-decoder structure, using thermal images and sparse LiDAR depth as inputs. Features are fused at multiple scales in the encoder, and the decoder upsamples them to predict dense depth maps. Multi-modal fusion modules based on self-attention and cross-attention are embedded in the encoder to enhance feature fusion with adaptive weighting, improving prediction accuracy. A self-supervised framework is established with temperature reconstruction and sparse depth losses, removing the need for depth ground truth. Experiments on public datasets show that the method generates dense depth maps stably under various lighting conditions. Mean absolute error decreases by 44. 49% on MS2 and 25. 28% on VIVID compared to benchmarks. By leveraging thermal and LiDAR data′s complementary strengths, this method improves depth prediction accuracy and robustness in low-light environments, offering an effective solution for perception in challenging lighting. Keywords:depth completion; multi-sensor data fusion; thermal imag

    Reference
    Related
    Cited by
Get Citation
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:
  • Revised:
  • Adopted:
  • Online: April 08,2025
  • Published:
Article QR Code