United States
Experienced Data Engineer with a demonstrated history of working in the tech industry (healthcare, social media, marketing, finance, oil & gas), predominantly for small companies and startups
• Evaluation, planning, and execution of migrating extract and load portion of data pipeline from homegrown scripts to open source system (Airbyte) - ~95% reduction in failure and error alerts to date for migrated clients - Managed another data engineer throughout the project • Development (Python) of multiple Airbyte connectors (both API and ODBC) • Handling of orchestration layer (Airflow), connecting to transform layer (DBT) • Development of new config and orchestration layer (Django and Celery)
• Creation of an ELT pipeline to load any data source from S3 (AWS) to Snowflake • Frontend app development for data ingestion (React & Redux) • Project lead on integration with client API endpoints for low-latency requests • Ad-hoc pipeline development for internal BI/data science team
• Productionalized a standardized data pipeline (Python/Pandas) to validate, clean, transform, and load all client data into Postgres database (hosted on AWS) • Automation of numerous tasks required within data operations • Various sprint based projects within the data science team • Direct interaction with both clients and end-users to better understand data analysis/processing needs
• Creation of data pipeline (Python) to ingest customer build data, clean, run error logic engine, store needed data (MongoDB), and return build issues via web app (Flask)