Australia
A professional and passionate Cloud data solutions Architect who has more than 16 years of experience in several IT operational roles, leading end to end data automation and modern data solution across a wide range of industries including Consulting, Cloud Computing, Public Sector, Wagering and Online Fashion Retailer, over 15+ years in Data Architecture Database Management, Data Engineering and Consulting experience. Specialties: Data Platform Solutions, Data Platform Delivery, Database Administration, Data Warehouse, Data Lake, Data Modeling, Architecture, Installations, Migrations, Upgrades, High Availability, Performance Tuning, Automation and Securities. NoSQL: Cassandra, HBase, DynamoDB, MongoDB, ElasticSearch. RDBMS: SQL Server, PostgreSQL, MySQL, Oracle. Data Analytics: BigQuery, Presto, Redshift, Athena, Power BI, QuickSight, HIVE, Spark and SSAS. Data Platforms: Hadoop, Spark, Elk, Databricks, Azure HDInsight, Amazon EMR, AWS Glue and AWS Lake Formation. DevOps: CI/CD, Docker, Containerization, Azure Kubernetes Service, EKS, Jenkins, Slack, Jira, Confluence. Orchestration: Apache Airflow, Azure Data Factory and SSIS. Cloud Storage and Data Lake: ADLS Gen1&2, Azure Storage Account, Google Cloud Storage, Amazon S3, HDFS, DBFS, etc. Cloud platforms: Azure, AWS, GCP. Languages: SQL, Scala, Python, Java, VB.NET, etc.
• Manage Distributed Data Stores including Apache Cassandra, SQL Server, Azure RDS PostgreSQL, MySQL and MongoDB and support 80+ of Geo-distributed clusters including SQL Server Always-on, Apache Cassandra, MongoDB and define Replication Strategies across Multiple Azure regions. • Manage Big Data Platforms and Distributed Query Engines including BigQuery, Presto, SparkSQL, Hive, Azure HDInsight(Hadoop, Spark and Hive Clusters), Azure Data Factory, Azure Data Lakes. • Set up the Metabase + Prestosql on HDI Foundation for Interactive Query Engine for China Teams to gain data insights. • Implement the BigQuery data ETL pipelines to China Azure Data Lake at PB scale to support multiple projects like Annual Billing, Customer DNA(CDP), DMP and Recommendations, which are critical for China's Marketing Growth. • Enable historic data archiving and OLAP analytics for Cassandra, ElasticSearch in Spark. • Logical and physical data modeling, database security and information security policies. • Assist China backend engineering team to plan new APIs and support DB clusters that underpin iOSApps/wechat mini program microservices. • Ensure that data is clearly defined, secure and remains consistent across the databases. • Work closely with Backend engineers to define service blueprints for all database dependencies and support CI/CD Pipelines and automate database provisioning processes using tech stacks including GitLab, Docker Tooling, Saltstacks, Terraform, Jenkins, Ansible, etc. • Support China Dev/QA environments running on Docker containers. • Responsible for planning, development and troubleshooting all database related issues. • Working closely with engineers to set up Database Monitoring&logging stacks including Prometheus, Grafana, NewRelic, HA Proxy, Elk, Kafka and Nagios XI. • Familar with Database Internal mechanisms and protocols such as consistent hashing, WAL, B-Tree, B+Tree, LSM tree, Merkle Tree, 2PC, Paxos and Gossip.
• Provide Databases and Big data consultancy for the could provisioning. • Linux Administration(CentOS Server), Windows Server administration. • User requirement gathering and analysis, DB conceptual, logical and physical modeling. • Build Apache Hadoop, HBase and ZooKeeper Cluster for a social network web traffic analysis. • MapReduce Development in Java. • Azkaban/Flume/Sqoop/MySQL/Hive/Elk for Data Analysis.