Senior Cloud Data Engineer
Our client is building a “born in the cloud” data platform to enable sales, finance, master data management, and analytics enterprise applications. We are seeking an experienced Data Engineer to solve our most difficult challenges as we expand our platform capabilities.
The Cloud Platforms Senior Data Engineer is responsible for processing structured and unstructured data, validating data quality, and developing and supporting data products. She/he also plays a leadership role in extending the platform capabilities, Agile planning, and providing mentorship and guidance to counterpart engineers.
Tech Stack: AWS Event Driven Serverless Architecture (Lambda, Glue, Glue Crawler, Athena, Step Functions), DMS, Redshift, RDS (Aurora, Oracle, MySQL, Postgres), Splunk
Complexity: Performs an extensive range and variety of complex technical and/or professional work activities. Undertakes work which requires the application of fundamental principles in a wide and often unpredictable range of contexts.
Expertise: Analyses, designs, plans, executes and evaluates work to time, cost and quality targets. Assesses and evaluates risk. Demonstrates creativity, innovation and ethical thinking in applying solutions for the benefit of the customer/stakeholder.
- Provide hands-on technical leadership in data engineering design and implementations including the scaled application of data ingestion, data models, data structures, data storage, high-throughput data processing, data pipelines, machine learning, and platform monitoring.
- Deliver work products built within and supported by microservice, API, event driven and serverless architectures, and built within enterprise security and enterprise networking’s policies.
- Create and maintain Continuous Integration / Continuous Deployment (CI/CD) processes to deploy cloud-based analytics applications
- Develop and follow data engineering best practices with considerations for high data availability, computational efficiency, data latency, data security, cost, and quality.
- Build and maintain environments (in cloud platforms), processes, functionalities, and tools to improve all stages of analytics solution development, e.g., proof of concepts and production.
- Develop frameworks (in cloud platforms) which optimize the scalability, supportability, and reliability of future Data Engineering solutions.
- 10+ years of experience in data modeling, data warehousing, and big data architectures
- 7+ years of experience in a data engineering role
- Experience in the following AWS Services: Data Migration Service (DMS), Glue, Elastic Map Reduce (EMR), Lambda, S3, Simple Notification Service (SNS), Step Functions, Redshift, Oracle RDS, Postgres SQL language, Glue Crawler, Athena, EC2
We'd love to see:
- Strong experience with writing complex programs, implementing architectures, and enabling automation in these environments
- Strong programming skills and ability to utilize a variety of software/languages/tools; e.g., Informatica, Python, Hive, Analytical Function in SQL, SAS, Tableau, PySpark, PL/SQL, Vertica, OBIEE
- Experience in the following AWS Services: DynamoDB, Secrets Manager, Key Management Service, CloudTrail, CloudWatch, Simple Queue Service (SQS)
- Experience working with various data source like Oracle, MySQL, ServiceNow, SFDC, EBS
- Understanding of Amazon network and security essentials such as VPC, IAM, Subnets, Route 53, Policies
- Understanding of Version Control and Deployment tools like GIT, Code Commit, Code Deploy, Code Pipeline
- Experience working with leading cloud vendors such as Amazon
- Strong analytical and problem-solving skills
- Strong communication skills
A human approach to staffing.
Our Company is committed to the principles of equal employment. We are committed to complying with all federal, state, and local laws providing equal employment opportunities, and all other employment laws and regulations. It is our intent to maintain a work environment which is free of harassment, discrimination, or retaliation because of sex, gender, race, religion, color, national origin, physical or mental disability, genetic information, marital status, age, sexual orientation, gender identity, military service, veteran status, or any other status protected by federal, state, or local laws. The Company is dedicated to the fulfillment of this policy in regard to all aspects of employment, including but not limited to recruiting, hiring, placement, transfer, training, promotion, rates of pay, and other compensation, termination, and all other terms, conditions, and privileges of employment.