New York, New York, United States
I'm an engineer at MongoDB on the Distributed Systems - Replication team. My team originally managed MongoDB's C++ implementation of the Raft Consensus Algorithm, but today, my team is involved in several projects beyond the consensus algorithm. I enjoy studying and working on things like compilers, operating systems, and distributed systems and I'm fortunate enough to be doing what I enjoy. I'm still learning The Art of Computer Programming (TAOCP)! Also, will Chris Paul ever win a ring? You can view some of my projects here: https://github.com/kauboy26
I'm on the Distributed Systems - Replication Team, which manages the C++ implementation of the Raft Consensus Algorithm. Today, the team is involved in several projects beyond the core consensus algorithm. My contributions are visible here: https://github.com/mongodb/mongo/commits?author=kauboy26
I'm a Teaching Assistant for CS 2110, a class that introduces students to logic circuits, assembly language and finally C. In this class, I help students understand the structure and operation of a computer from the most fundamental building blocks such as logic circuits to more advanced computer concepts such as the run-time stack.
I was on the Distributed Systems (Replication) Team, where I worked on the Initial Sync project. Initial Sync is the process of cloning data from an existing node to a new, blank node. A replica set is a family of nodes where each contains the same data; thus, when a new node is added to the replica set, it needs to populate itself with the same data the other nodes in the set contain. • Sped up sync process between two nodes by 15% on workloads with complex, nested documents by avoiding deserialization on sync source (optimization deployed to production). The sync source sends documents over to the destination (the new, blank node). A bottleneck on the sync source that involved the sync source unnecessarily deserializing data was removed, leading to a 15% drop in total sync time. • Experimented with a cross-layer optimization on the sync destination utilizing the WiredTiger storage engine’s bulk mode, and saw a performance gain of 5%. I utilized an alternate "bulk" document insertion mode on the sync destination. Bulk mode is a property of the storage engine, one of the lowest layers of the stack. Therefore, implementing this optimization was difficult because it involved meddling with code starting from the transport layer (the layer that receives data from the sync source) to the storage layer. This mode also has certain disadvantages (there is a strict set of rules dictating when the bulk mode can be used) that had to be worked around. • Identified further bottlenecks using dtrace and flame-graphs for performance gains of up to 5%. We examined the code and broke the initial sync process down, and found opportunities for improvement. • Contributed to replication (Raft consensus algorithm based) codebase. Initial sync falls under the replication team. • Learned about formal verification and played with TLA+, a formal verification tool.
I had a great experience at Ultimate Software and learned a lot: • Developed RESTful event sourced, antifragile microservice endpoints using Springboot, Axon and RabbitMQ: Event sourcing maintains a list of events that contain state changes. As a result, the state of our data at any point in time can be rebuilt by replaying events until that point. An antifragile system is one that will respond positively to failure (better than robust and resilient). • Wrote contract and integration tests to continuously deploy microservices to the cloud: In the continuous deployment strategy, changes are pushed directly to production provided they pass a series of tests. Therefore, these tests are critical and ensure that a microservice behaves well alone and with its peers. • Built a Mule API prototype to transform and move data between two unrelated data systems: Mule is an Extract-Transform-Load (ETL) tool to convert data from one format to another. We used Mule to build a prototype REST endpoint that when triggered, fetches data from one point, transforms that data, and then deposits the data at another point. • Designed a queuing system to solve concurrency requirements and reduce idle time. • Reviewed code and worked with business analysts to improve user stories.
At Skylark Drones, I developed several tools that are now in use internally by the company. This development was done primarily in Python 3. • Saved roughly 2-3 hours of manual planning per flight set, by creating a tool to generate a flight path over a given corridor. This tool takes a KML file that contains a corridor such as a planned highway as its input (typically provided by the client), parses that file and splits it into several segments. The purpose of this segmentation is to comply with the Indian government's law regarding maximum flight path length of a drone. Each segment is also "padded" by extrapolation. Finally, in order to maintain a fixed altitude over the terrain, I programmed the tool to fetch Shuttle Radar Topography Mission (SRTM) data and inserted a new calculated altitude at each waypoint that the drone will fly through. I also created a front-end GUI for this project using PyQt, a Python wrapper library for the Qt framework. • Developed a library that verifies the presence of ground control points within images by calculating the image’s footprint on the ground. The library reads EXIF data from images using ExifTool, from which data such as the pitch, yaw, roll, altitude, focus length, latitude, longitude etc are extracted. Using this data, I calculated the trapezoidal footprint the image would have on the ground, and ascertained whether a ground control point (ground reference point, GCP) existed in the image. A list of GCPs is then outputted. • Isolated the libraries used by the code and their dependencies by setting up a virtual environment. • Packed Python code into a .exe, and then cleaned it up through a Batch script. The tool we were using to pack the Python code into an executable by default added every library installed on the computer. To get around this, I set up a virtual environment and later created a Batch script that deletes unwanted libraries.