San Francisco Bay Area
AI research leader (12+ years) driving breakthroughs in 3D/4D generative AI, Gaussian Splatting, world models, and synthetic data. I lead 3D Generative AI research at Meta Reality Labs, shaping multi-year strategy across neural rendering, XR, digital humans, and sim2real. Former Head of ML at AI Reverie, where I built the technical vision that led to the company’s acquisition by Meta. Recognized by Fortune, MIT, Bloomberg, and BusinessWire as a pioneer in simulation-driven AI. Known for seminal work in synthetic data- Domain Randomization, Structured Domain Randomization, and Meta-Sim (2,000+ citations; adopted by OpenAI, NVIDIA, Toyota, Amazon). I build high-performing research teams and deliver 0 → 1 product impact across XR, and large-scale scene understanding.
Leading and supporting the 3D World Models Research team within Reality Labs. In the pas I have also lead the machine learning team within the synthetic data organization at Reality Labs, Meta. My group works on problems at the juncture of machine learning, computer vision and computer graphics. We tackle challenges in domain adaptation, neural rendering and other sim2real problems for mixed reality. My group frequently publishes paper at CVPR, ICCV, ECCV and WACV. Organized second version “XRNeRF- Advances in Radiance Fields for the Metaverse” workshop at CVPR 2024 with Angela Dai (TUM), Fernando Torre (CMU), Daeil Kim (Meta) and Peter Vajda (Meta). Organized “XRNeRF- Advances in NeRF for the Metaverse” workshop at CVPR 2023 with Jon Barron (Google), Angjoo Kanazawa (UC Berkeley), Fernando Torre (CMU), Daeil Kim (Meta) and Peter Vajda (Meta) Gave a talk on synthetic data for mixed reality at CVPR 2022 workshop.
Co-led the acquisition of the company by Meta. Recognized as one of the top AI talent on fortune. I joined AI Reverie as Head of ML, which featured in bloomberg and businesswire Led and supported a team of researchers and engineers on ML based synthetic data generation.
Most of my work is focused on synthetic data and sim2real: learning vision models in simulated environment and applying them in real world and vice versa Details on my webpage https://aayushp.github.io/ Meta-Sim: Learning to Generate Synthetic Datasets (ICCV'19 Oral) https://nv-tlabs.github.io/meta-sim/ My work voted as top 10 coolest paper at CVPR 2018 by towardsdatascience https://towardsdatascience.com/the-10-coolest-papers-from-cvpr-2018-11cb48585a49 Our work Meta-Sim and Structured Domain Randomization featured in OpenAI talk and blog Training Deep Networks with Synthetic Data recognized as breakthrough developments of 2017-2018 by MIT Our work Structured Domain Randomziation featured in TowardsDataScience Previous work: https://www.digitaltrends.com/computing/nvidia-isaac-vr-training-dominoes-siggraph/ https://www.engadget.com/2017/12/05/nintendo-wii-gamecube-nvidia-shield-china/
Leading development effort on implicit Java JIT acceleration using SIMD and GPUs.
Worked on object categorization and shape matching of objects in noisy images. Proposed novel measures of Orientation and Clutter for object categorization.