Responsibilities:
Design and Development
o Work with Cloud Architect to identify data components and process flows
o Design and Develop data ingestion processes into Hadoop/AWS Platform
o Collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy
o Identify, analyze, and interpret trends or patterns in complex data sets
o Innovate new ways of managing, transforming and validating data
o Establish and enforce guidelines to ensure consistency, quality and completeness of data assets
o Apply quality assurance best practices to all work products
Qualifications:
o BS/BA degree or equivalent experience
o 5+ Experience in a Big Data technologies (Spark, Impala, Hive, Redshift, Kafka, etc.)
o 3+ years of Experience with Python is preferred
o Experience performing data analysis (NOT DATA SCIENCE) on AWS platforms is preferred
o Experience in implementing complex ETL transformations on big data platform like NoSQL databases (Mongo, DynamoDB, Cassandra)
o Familiarity with relational database environment (Oracle, SQL Server, etc.) leveraging databases, tables/views, stored procedures, agent jobs, etc.
o Strong development discipline and adherence to best practices and standards
o Demonstrated independent problem solving skills and ability to develop solutions to complex analytical/data-driven problems
o Experience with data management process on AWS is a huge Plus
o Experience of working in a development teams using agile techniques