Azure Databricks Data Engineering Lead

About this role

Job Description

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.

• Lead large-scale, complex, cross-functional projects to build the technical roadmap for the WFM Data Services platform.

• Lead and review design artifacts.

• Build and own the automation and monitoring frameworks that showcase reliable, accurate, easy-to-understand metrics and operational KPIs to stakeholders for data pipeline quality.

• Execute proof of concept on new technology and tools to pick the best tools and solutions.

• Support business objectives by collaborating with business partners to identify opportunities and drive resolution.

• Communicate status and issues to Senior Starbucks leadership and stakeholders.

• Direct project team and cross-functional teams on all technical aspects of the projects.

• Lead with engineering team to build and support real-time, highly available data, data pipeline and technology capabilities.

• Translate strategic requirements into business requirements to ensure solutions meet business needs.

• Define and implement data retention policies and procedures.

• Define and implement data governance policies and procedures.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.

• Enable team to pursue insights and applied breakthroughs, while also driving the solutions to Starbucks scale.

• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of structured and unstructured data sources using big data technologies.

• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.

• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

• Perform root cause analysis to identify permanent resolutions to software or business process issues.

MUST HAVE:

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.
Originally posted on Himalayas