San Francisco, California, United States
Built out creator sourcing systems for 50+ customers Introduced agentic frameworks, replacing simple structured LLM calls Rewrote vetting platform, reducing costs by 5x and improved reliability from 80% to 99.5%.
Part of YC W23 batch. SOTA stateful stream processing engine: https://github.com/ArroyoSystems/arroyo Acquired by Cloudflare to power Cloudflare Pipelines.
Continuing to lead development on our proprietary dataset. Lead overhaul of data processing rearchitected across the offline and real-time modeling stacks.
Tech lead and primary developer on an in-house analytics engine responsible for delivering realtime audience insights to customers about their web properties and advertising campaigns. Contributions include * Several compounding 5-10x performance improvements. * Greatly increased the size and variety of datasets, now including over a trillion facts. * Replaced over two dozen batch pipelines in favor of interactive computation. * introduced time-series capabilities.
Oversaw a zero-downtime migration of 500+ map-reduce pipelines from a physical datacenter into AWS. Leveraged algorithmic abilities and distributed systems expertise to release a variety of optimizations and improvements to batch computing pipelines and the real-time bidding stack saving hundreds of thousands of dollars annually.