Tracy, California, United States
Software Developer with a passion for solving and debugging complex problems. Hands on experience working with big data technologies, including building an ingestion streaming pipeline from scratch.
Helped built many of the charging components from ground up and deployed to production along with interacting/integration with third party services. All our tech stack was on AWS • Implemented serverless AWS lambda with CDC on a DynamoDB table that would capture charger data and send the OCPI compliant data in real time to external service. • Introduced Elastic Search cluster into the charging ecosystem for faster retrieval of data rather than relying on DynamoDB. Wrote an extensible Java Library that was easily integratabtle into Java micro services for retrieval of data from Elastic Search. • Implemented a configurable AWS lambda that would capture data from DynamoDB (via CDC) and index it into Elastic Search. • Implemented a Java Microservice that would send the OCPI compliant Rivian charger data to NREL on a daily basis. • Introduced API Gateway and lambdas that integrated with Chargepoint so Rivian customers could link their account with Chargepoint via the Rivian app, enabling them to access the Chargepoint EV chargers. • Developed Java Microservice that would consume 6-7GB worth of data per day from various Kafka topics and index it into Elastic Search. Later the service was also enhanced to persist the data into S3 for analytics team.
• Helped design and develop CCM service that was used to fetch list of funding instruments that can be leveraged to charge a seller based on rule policies with which the CCM service would communicate with. Also introduced monitoring graphs in Kibana to capture some key metrics. • Implemented historical variables that leveraged Spark-SQL to query the DWs and join the necessary tables to fetch the relevant info to build the historical variables. The Spark Job was run on a daily basis • Designed and implemented a near real time consumer that would consume the event’s payload, massage the data by calling external REST service and send it to another platform for data storage. This was the real time processing component of Lambda Architecture, there was a spark application for processing historic data as well. • Designed a java web service that returned a JSON strategy regarding money movement steps needed to be taken after a transactions had taken place. This service had to be written in a very extensible manner so made heavy use of OOP concepts. • Built a UI dashboard for monitoring that displayed data fetched from oracle. The data was fetched using Java RESTful apis. Front end was written in angular. • Contributed to implementing multiple endpoints that were responsible for fetching data from MongoDB, manipulating the data and also persisting the data in another collection in MongoDB. Made use of Google Guice for Dependency Injection and Spring Framework.
•Implemented the ingestion pipeline that transported data from MES (edge) to GE’s Predix cloud for storage and analytics. •Captured the data via HVR (change data capture tool) from SQL which was then streamed into Cassandra and S3 using Kafka and RESTful APIs. •Implemented Spark Batch jobs that were responsible for loading data from S3 and inserting the correct state of the data into Cassandra •Introduced Spark Streaming for a more real time processing layer which inserted the correct state of the data into Cassandra, moving away from the batch processing. This lead to a much faster of data transportation from edge to source, meeting the customer’s SLA (reduced from 2 hours to about 5 minutes). •Revamped the Cassandra data model for storing our denormalized data which resulted in significant improvement when fetching the data via the REST API (reduced from 13 seconds to 3 seconds) •Created a transformation layer that loaded data from Cassandra to project multiple views such as denormalized view and a fact-dimensional model view.