Post by Mahboob Alam

🚗 Built a Real-Time Perception & Sensor Fusion Pipeline for Autonomous Vehicles using the KITTI Dataset 🚀 Recently, I worked on a perception and sensor fusion project where the goal was to recreate a simplified autonomous vehicle perception stack from recorded sensor data. For this project, I used the KITTI Dataset and created a ROS bag pipeline containing synchronized camera images, LiDAR point clouds, IMU, and GPS data to simulate a real vehicle data flow. Pipeline Overview: 🔹 Sensor Synchronization & Fusion Started by synchronizing camera and LiDAR data and aligning both sensors using intrinsic and extrinsic calibration parameters so that the perception system understands the same scene from multiple sensors. 🔹 Object Detection + Depth Understanding Performed object detection on camera images and then estimated object depth using fused LiDAR information to understand the spatial position of objects. 🔹 2D Top View & 3D Bird’s Eye View (BEV) Converted perception outputs into both 2D top-view representation and 3D BEV visualization for better scene understanding. 🔹 Voxel-Based 3D Environment Reconstruction Generated a voxel-style 3D map inspired by occupancy networks: - Free space → remains empty - Occupied space → represented using small voxel cubes This creates a structured representation of the environment that can later be used for planning and navigation. 🔹 Localization Inputs Integrated IMU and GPS data into the pipeline to understand motion and maintain temporal consistency between frames. This project helped me understand how modern autonomous systems combine multiple sensors to move from raw data → perception → environment understanding → decision-ready representation. 🎥 Demo video below ↓ 💻 Source Code & Setup: https://lnkd.in/gpDnKwMr #Robotics #AutonomousVehicles #Perception #SensorFusion #LiDAR #ROS #ComputerVision #DeepLearning #OccupancyNetwork #3DVision #SLAM

Video Content