Job Description
This job appears on /jobs/1671-Data-Engineer- Minimum 5+ years’ data engineering experience working with designing data models in both column store and relational databases, incorporating disparate data to solve complex business needs.
- Experience building and optimizing data pipelines, architectures, database modeling and data sets that meet business requirements.
- Experience in building the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources in ‘big data’ technologies or similar. Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Experience in building processes supporting data transformation, analytics tools, data structures, metadata, dependency, and workload management. Assist Data Science team that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and optimizing our product into an innovative industry leader.
- Experience in using Azure DevOps, or Jira project management/requirements management.
- Experience with big data tools: Hadoop, Spark, Kafka, or a related technology.
- Proficient in programming languages such as SQL, Python, R, Shell Scripting
- Coordinating and collaborating with cross-functional teams, stakeholders, and vendors for the smooth functioning of the enterprise data system.
- Managing end-to-end data architecture, from selecting the platform, designing the technical architecture, and developing the application to finally testing and implementing the proposed solution.
- Identifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
- Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition.