Varun sharma

Software Engineer at Databricks

Amsterdam, North Holland, Netherlands

About

Software engineer with expertise in distributed systems and real-time stream processing. Experienced in building high-throughput, fault-tolerant pipelines, scalable data platforms, and real-time insights systems. Proven track record of building with an observability-first mindset, ensuring robust monitoring and alerting, and designing highly available, resilient services. Passionate about tackling complex challenges, mentoring teams, and driving continuous learning and growth.

Experience

  • Software Engineer at Databricks
    Apr 2019 - Present · 7 yrs 3 mos

    Billing Platform. Execution Gateways.

  • Software Engineer at Booking.com
    Sep 2016 - Mar 2019 · 2 yrs 7 mos

    Built a scalable, fault-tolerant events and logs pipeline handling 700K+ events/sec and 1M+ logs/sec. * Introduced Apache Kafka as the transport layer, enabling adoption across multiple teams and clusters. * Designed and implemented a highly available, lightweight Go-based proxy/router (40K+ instances) to relay events and logs ensuring reliability and visibility. * Migrated large-scale events pipelines from in-house systems to Kafka, reducing event delivery latency from 13s to 4s. * Enhanced log pipeline stability by replacing Rsyslog relays with Kafka + Logstash. * Defined and implemented SLIs/SLOs, with robust monitoring and alerting. * Contributed to team growth through mentorship, onboarding, interviews, and trainings.

  • Software Engineer at Flipkart
    Aug 2014 - Aug 2016 · 2 yrs 1 mo

    Worked on building the Geo Platform for the Supply Chain arm, enabling real-time, fault-tolerant event processing and insights. * Developed batch and streaming pipelines using Apache Spark, Cassandra, Kafka, and Zookeeper, delivering key insights at scale. * Built application for Hub managers to track and visualise their fleet. * Contributed to address understanding and geocoding algorithms leveraging OpenStreetMaps and the location data ingested. * Built the Insights Platform for real-time events (e.g., shipment state changes, SLA breaches) using Kafka, Spark, Redis, Elasticsearch, and Kibana.

  • Software Engineer at Walmart Labs
    Jul 2013 - Aug 2014 · 1 yr 2 mos

    Developed and enhanced the Customer Service Center Application on the ATG framework with end-to-end ownership of multiple projects. * Led key initiatives including ATG upgrade, REST integrations, unauthorized activity automation. * Collaborated with customer service agents in Cape Town, gathering feedback and integrating enhancements into the application.

  • Software Engineer Intern at Cisco Systems
    Jun 2012 - Jul 2012 · 2 mos

    Contributed to alpha testing of Cisco GSS (Global Site Selector) to improve DNS responsiveness and ensure data center availability, including features like NS/SOA and Keepalives. Automated testing workflows using Expect scripts.