ReaLiTy: Realistic Environment-Adaptive LiDAR Transformation across Sensors and Weather

The scene-identical pair that no benchmark provides

A LiDAR model trained on one sensor in clear weather degrades when moved to a different sensor, or to rain or snow. Correcting such a shift needs the same scene under two conditions — data that real benchmarks cannot supply, because a clear scan and a snow scan are always of different streets.

ReaLiTy(Realistic LiDAR Transformation) transforms benchmark clear-weather point clouds into physically consistent adverse-weather data, applying geometric distortion and realistic intensity attenuation to produce paired clear-and-adverse point clouds while preserving the original dataset format. It also performs sensor-specific intensity transformation for any dataset, adapting a scan to a different LiDAR. Because the scene is held fixed, a source frame and its transform form the physically consistent pair the field currently lacks.

Two shifts, one physical origin

Starting from the LiDAR range equation for an extended Lambertian target, a measured intensity factorises into a scene-intrinsic term and a domain term. This is what makes “unified” a statement about the problem, not just the code.

Sensor adaptation and weather adaptation are therefore two parameterisations of one operator. ReaLiTy realises it as a composition — project to a range image, apply a physics-based degradation that reduces to the identity in clear weather, translate intensity under physical conditioning, and back-project to 3D:

Conditioning is not optional. Because the sensor response f_s saturates and quantises, unconditional intensity translation is ill-posed — two different scenes can report the same source intensity yet map to different target intensities. Conditioning on the physical modalities (R, cos θ, ρ) restores identifiability, which is the principled reason the model is physics-informed.

A modular, physics-informed pipeline

ReaLiTy orchestrates two physics-informed cycle-consistent GANs — PICGAN for sensor transfer and PICWGAN for weather transfer — within one unified pipeline for LiDAR realism.

Sensor Transfer — PICGAN

Weather Transfer — PICWGAN

The training objective combines three complementary terms, each constraining a degree of freedom the others are blind to: an adversarial term fixes absolute calibration against the target sensor, a cycle-consistency term preserves scene content, and a physics term anchors the output to the radiometric structure implied by the range equation. Removing any one leaves a direction of the solution unconstrained.

The framework enables:

• Realistic LiDAR intensity across different sensors
• Adverse-weather effects on benchmark datasets, creating paired data
• Physics cues: range, incidence angle, and reflectance
• End-to-end preprocessing, training, inference, and evaluation
• Ready-to-use weights for sensor and weather transfer
• Modular, configuration-driven pipelines

What’s in the framework

A single configuration-driven package — dataset adapters, degradation and intensity models, training and inference, and distributional evaluation — with a detailed usage guide in the repository.

# ReaLiTy repository layout
reality/
├─ cli.py                 # entry point for prepare-data / train / generate / evaluate
├─ configs/
│    ├─ sensor/            # sensor-transfer configs (e.g. voxelscape_to_kitti.yaml)
│    └─ weather/           # weather-transfer configs (e.g. kitti_to_cadc.yaml)
├─ core/                  # config loading, pipeline orchestration, determinism, registry
├─ datasets/              # per-dataset adapters (KITTI, nuScenes, CADC, Boreas, VoxelScape, generic)
├─ degradation/           # geometric degradation plugins (physics weather, learned)
├─ preprocessing/         # spherical projection, statistics, disk cache
├─ models/                # intensity models, incl. the vendored PICGAN network and its adapter
├─ postprocessing/        # back-projection from range image to point cloud
├─ inference/             # checkpoint-driven generation and ONNX export
├─ training/              # trainer, dataset wrapper, checkpointing, logging
├─ evaluation/            # distributional metrics against the target sensor/weather
├─ io/                    # output writers (native per-source format)
├─ physics/               # reflectance LUT and ECOSTRESS-derived reflectance lookup
├─ structure/weather_model/ # vendored LISA scattering model (GPL-3.0, see THIRD_PARTY.md)
└─ tests/                 # pytest suite mirroring the package layout

data/          # per-dataset train/test point clouds (see data/README.md)
weights/       # pretrained checkpoints (see weights/README.md)
docs/          # extending the weather model and PICGAN
scripts/       # SLURM training template, run evaluation helper
tools/         # raw-dataset preparation (prepare_data_kitti_cadc.py)

Transformation-ready, paired point clouds

Derived from multiple benchmark datasets using ReaLiTy, LADS provides fully processed, physically consistent sweeps — each in the native format of its source, enabling direct one-to-one comparison with the original scans.

KITTI  Clear → Snow

nuScenes  Clear → Rain

BibTeX

If you find this work useful in your research, please consider citing the relevant papers.

@misc{anand2026reality_lads,
  title         = {ReaLiTy and LADS: A Unified Framework and Dataset Suite for
                   LiDAR Adaptation Across Sensors and Adverse Weather Conditions},
  author        = {Anand, Vivek and others},
  year          = {2026},
  eprint        = {2604.10213},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  note          = {arXiv preprint}
  url           = {https://arxiv.org/abs/2604.10213}
  
}

@article{anand2026sim2real,
  title   = {Toward Closing the Sim-to-Real Gap: A Physics-Guided Learning
             Approach for LiDAR Intensity Simulation},
  author  = {Anand, Vivek and Lohani, Bharat and Kumar, Vaibhav and
             Mishra, Rakesh and Pandey, Gaurav},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year    = {2026}, note = {Early access}, doi = {10.1109/TITS.2026.3681982}
}

@misc{anand2026weather,
  title         = {Simulating Realistic LiDAR Data Under Adverse Weather for
                   Autonomous Vehicles: A Physics-Informed Learning Approach},
  author        = {Anand, Vivek and Lohani, Bharat and Mishra, Rakesh and Pandey, Gaurav},
  year          = {2026}, eprint = {2604.01254},
  archivePrefix = {arXiv}, primaryClass = {cs.RO},
  url           = {https://arxiv.org/abs/2604.01254}
}

@article{anand2025lblis,
  title   = {Advancing LiDAR Intensity Simulation Through Learning With
             Novel Physics-Based Modalities},
  author  = {Anand, Vivek and Lohani, Bharat and Pandey, Gaurav and Mishra, Rakesh},
  journal = {IEEE Transactions on Intelligent Transportation Systems},
  year    = {2025}, volume = {26}, number = {5}, pages = {6493--6502},
  doi     = {10.1109/TITS.2025.3532687}
}

@inproceedings{anand2025snow,
  title     = {Towards Realistic LiDAR Intensity Simulation in Snowy Weather
               Using Physics-Informed Learning},
  author    = {Anand, Vivek and Lohani, Bharat and Mishra, Rakesh and Pandey, Gaurav},
  booktitle = {IEEE Intelligent Vehicles Symposium (IV)},
  year      = {2025}, pages = {2552--2557}, doi = {10.1109/IV64158.2025.11097501}
}