We are looking for a Data Engineer to join our team and help us drive innovation across our business. In this role you will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.
The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support our software developers, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self- directed and comfortable supporting the data needs of multiple teams, systems and products.
The right candidate will be excited by the prospect of optimizing or even redesigning our company’s data architecture to support our next generation of products and data initiatives.
- Assembling large, complex sets of data that meet non-functional and functional business requirements.
- Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
- Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using big data, cloud and SQL technologies for real time and offline applications.
- Building to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition.
- Build monitoring capabilities and analytical tools to give insight into data pipeline performance metrics using it to ensure optimal performance of our data pipelines.
- Working with stakeholders including data, software development and leadership teams and assisting them with data-related technical issues.
- Bachelor’s degree in Computer Science, Information Systems, Engineering or equivalent degrees or equivalent in relevant work experience.
- 1-2 years data engineering experience
- Strong experience in coding languages like Python, Scala & Java
- Knowledge of data pipelines, data stores, kafka or similar streaming infrastructure
- Experience within Linux/Unix based operating systems
- Advanced degree in Computer Science, Information Systems, Engineering or equivalent degrees
- 3+ years data engineering experience
- Deep understanding of advanced SQL techniques in Hive, Spark, Redshift and other traditional database applications.
- Experience with big data tools such as Hadoop, Hive, Spark, Scala, PySpark, EMR, GCP.
- Experience with the ELK stack (Elasticsearch, Logstash, Kibana)
- Experience with in-memory data technologies such as Apache Ignite, Redis etc.
- Experience in Workflow orchestration tools such as Airflow etc.
- Experience leveraging monitoring tools such as Grafana, CloudWatch, Datadog etc.
- Experience with container management frameworks such as Docker, Kubernetes, EKS, GKE etc.
- Experience working on CI/CD processes such as Jenkins, Octopus etc. and source control tools such as GitHub, etc.
- Strong understanding of database technologies and management systems
A Human Approach to Staffing
Our Company is committed to the principles of equal employment. We are committed to complying with all federal, state, and local laws providing equal employment opportunities, and all other employment laws and regulations. It is our intent to maintain a work environment which is free of harassment, discrimination, or retaliation because of sex, gender, race, religion, color, national origin, physical or mental disability, genetic information, marital status, age, sexual orientation, gender identity, military service, veteran status, or any other status protected by federal, state, or local laws. The Company is dedicated to the fulfillment of this policy in regard to all aspects of employment, including but not limited to recruiting, hiring, placement, transfer, training, promotion, rates of pay, and other compensation, termination, and all other terms, conditions, and privileges of employment.