LiDAR Resolution Simulation in Synthetic Training Data for 3D Object Detection

Back in 2021, as part of my Bachelor's thesis at Graz University of Technology, I dove into the world of synthetic data for autonomous driving. The project, titled "LiDAR Resolution Simulation in Synthetic Training Data for 3D Object Detection," was all about making it easier and cheaper to train 3D object detectors for self-driving cars. Real-world LiDAR datasets like KITTI are expensive and time-consuming to create — you need fancy sensors, annotated data, and a lot of manual effort. So, I explored using synthetic data from virtual environments to bridge that gap. Let me walk you through what I did, why it mattered then, and how it fits into today's research scene.

The Challenge: Data-Hungry Detectors in a Costly World

The motivation was simple: Traditional 3D object detection models, like those used in self-driving cars, rely on vast amounts of annotated point cloud data from LIDAR sensors. Collecting and labeling this data in the real world is not only expensive and time-intensive but also limited by specific hardware and locations. Synthetic data, generated from game engines or simulators, can be created on demand without real-world hassles. However, there's a "domain gap" — synthetic data often doesn't match real LiDAR scans due to differences in sensor resolution, noise, and environmental factors.

Approach: Simulating Reality in Virtual Worlds

For this thesis, I leveraged the Apollo Synthetic dataset, which provides RGB images, depth maps, and segmentation masks from a virtual driving environment. From these, I generated point clouds by simulating different LiDAR configurations right on the depth images. Instead of ray-casting in a full simulator (which can be computationally heavy), I calculated intersections of virtual LiDAR rays with the image plane to mimic real sensors like the Velodyne HDL-64E.

Here's how it worked:

Key Insights and Results

Experiments revealed that accurate LIDAR simulation significantly boosts detection accuracy. Models trained on mismatched resolutions performed poorly (e.g., 11.85% AP for 32 beams vs. 17.24% for the correct 64 beams on KITTI's car detection). Incorporating noise and drop-out yielded further gains, with one model achieving 23.45% AP on KITTI — demonstrating how these tweaks make synthetic data more robust.

Reflecting on It Now: Where Does This Fit in 2025?

Since 2021, the domain has seen rapid progress in leveraging deep generative models for more realistic synthetic LIDAR point clouds, including diffusion models and advanced GAN variants like L-GAN, which augment imbalanced datasets to improve object recognition for minority classes in driving scenes. [1]

New synthetic datasets have emerged to tackle domain gaps, such as MORDA (2025), which fuses source and target domain characteristics in a simulator to adapt detectors for unseen real-world environments like South Korea while preserving performance on originals like nuScenes. [2]

In terms of environmental factors, we see significant advances with datasets like Argoverse 2 (2023) adding polarization features to LiDAR for material differentiation, reducing false positives by up to 34%. [3]

But it is not only the data sets that have evolved significantly since my bachelor thesis; the architectures are also constantly being improved. Surveys highlight hybrid architectures such as PV-RCNN++ (2022) and transformer-based models (e.g., Voxel Set Transformer, 2022), which deliver ever-increasing performance and efficiency. [3]

As you can see this is a facinating research sector and if you're interested in this stuff I suggest reading one of the cited papers above or start with my bachelor thesis here.

If you want to know more about the implementation, check out the full code and details on GitHub: https://github.com/itsthestranger/synthetic-lidar-sim

Technologies Used

(These relate to the practical implementation available in the GitHub repository)