Greater Toronto Area, Canada
Capable of processing large sets of structured,unstructured & semi-structured data. Expertise on Data Ingestion. Experience with Hadoop Eco System. Experience on Hive,Pig,Sqoop,Kafka,Zookeeper,Oozie,Scala,RDD,Dataframe,Dataset. Experience on Ambari and Hue. Handling data export import tools. Experience on Spark in memory computation. Experience on Sql,NoSql databases. Strong competency in HIVE Schema design, Partitions, Bucketing and high-speed query engine Impala, Data imports and Analysis. Proficiency in understanding of cloud computing virtualization technologies, storage architecture & AWS technologies: EC2, ECS, S3, SNS, ELB, EBS, EMR, EFS, AMIs, Route53 (DNS), Redshift,IAM. Experience on working different domains like Banking,Retails,Insurance,Logistics.
Overall 8+ years of experience in design and deployment of Data Management and Data Warehousing Projects in various roles as a Data Modeler and Data Analyst on Big data technologies. Possesses 2+ years of rich Hadoop experience in design and development of Big Data applications, which involves Apache Hadoop Map/Reduce, HDFS, Hive, HBase, Pig, Oozie, Sqoop, Flume and Spark. Expertise in developing solutions around NOSQL databases like MongoDB and Cassandra. Experience with all flavor of Hadoop distributions, including Cloudera, Horton works. Excellent understanding of Hadoop architecture Map Reduce MRv1 and Map Reduce MRv2 (YARN). Developed multiple Map Reduce programs to process large volumes of semi/unstructured data files using different Map Reduce design patterns. Worked extensively over semi-structured data (fixed length & delimited files), for data sanitation, report generation and standardization. Strong Knowledge of Hadoop and Hive and Hive's analytical functions. Good knowledge on data analysis with SAS. Good knowledge on executing Spark SQL queries against data in Hive. Hands-on experience with AWS (Amazon Web Services), using Elastic MapReduce (EMR), creating and storing data in S3 buckets and creating Elastic Load Balancers(ELB) for Hadoop front end Web UI’s. Extensive knowledge on creating Hadoop cluster on multiple EC2 instances in AWS and configuring them through ambari and using IAM (Identity and Access Management) for creating groups, users. Experience working on Version control tools like SVN and Git revision control systems such as GitHub and JIRA to track issues and crucible for code reviews. Strong Database background with Oracle, PL/SQL, Stored Procedures, trigger, SQL Server, MySQL. Experienced in monitoring Hadoop cluster using Cloudera Manager and Web UI.