Newark, California, United States
I am currently working as a Software Engineer at Meta Inc in DataInfra org and for the last 7 years, I have worked on multiple distributed compute engines including Hive, Presto and Spark. Currently, I am working on Presto-on-Spark project which is a hybrid offering combining the best of both presto and spark compute engines.
* Developing and maintaining hive and presto inside facebook and striving hard for improving reliability and performance.
• Developing LIVE framework of Unravel to bring profiling data from running pipelines/workflows live over the network. • Identifying probing points and writing low overhead java agents (using JAVA instrumentation technology) for collecting performance metrics. • Understanding Hive and Hadoop code base for understanding probing points and classes that needs to be profiled. • Developing a Recommendation engine to recognize patterns from various data sources and give insights to improve latency and throughput of running workloads/queries/jobs. • Analyzing Hadoop counters/logs to determine inefficiencies (configurations) at application/cluster level and writing algorithms to point them out with proper suggestions to mitigate them. • Understanding HIVE and underlying data storage format in order to analyse user queries and give recommendations based on data layout and patterns. • Responsible for designing, coding, testing the new algorithms and tuning old ones so as to make them better and smart.
Querying XML data using XPath and XQuery. Querying mySQL database for mining relavant data. Develop reports, metrics, dashboards, adhoc pivot tables, analyze data, and present executive summaries
• Full Stack Developer in product development maintenance team for 2 Years and 2 months. • Part of Product development team for Corporate Banking product. Worked as a full-stack developer with Java at the back end and JSP’s with struts framework at the front end and DB2 as the database. • Responsible for feature enhancements, performance improvement, debugging and defect fixing. • Worked on Batch and real-time frameworks. Also, worked on Performance issues faced by the application. • Learnt FreeMarker, JavaScript and HTML Tags while working on front end of the application.
Worked as Project Intern for 7 months. Worked on the Project named "QuicX: A Light weight protocol for effective communication in Linux Clusters". The goal was to design an efficient transfer protocol in a closed intra-cluster network. It consisted of two main modules. 1st module eliminated the extra buffer copies while the 2nd module bypasses TCP/IP layer processing using RAW sockets The project was an effort to show the overhead the TCP/IP protocol stack poses in a closed cluster.