Data Engineer - Azure Databricks & Application Development

Over deze functie

**Data Engineer - Azure Databricks & Application Development**

**Position Overview**

DigiTribe is looking for a **Data Engineer** with deep expertise in Azure and Databricks to design, develop, and maintain scalable data engineering solutions. You will build production-grade ETL/ELT pipelines, develop data processing applications, and contribute to a modern lakehouse architecture. Working cross-functionally with Architecture, Data, AI, Cloud, and CI/CD teams, you will translate business and analytical requirements into robust, reusable, and cost-efficient data solutions.

**Key Responsibilities**

* Design, develop, test, and maintain scalable data engineering solutions on Microsoft Azure and Databricks, using Python, PySpark, Scala, and SQL. * Implement reliable batch and streaming data workflows using Azure Databricks, Delta Lake, and modern data lakehouse patterns. * Develop data processing applications for ingestion, transformation, enrichment, validation, and serving of analytical datasets. * Build CI/CD pipelines for data applications using Azure DevOps YAML pipelines and apply software engineering best practices: modular design, clean code, version control, code reviews, and documentation. * Develop reusable libraries, utilities, and frameworks to accelerate data product delivery and support less experienced engineers through coaching and knowledge sharing. * Collaborate with Cloud, DevOps, and Platform teams on deployment, environment configuration, and operational readiness.

**Skills & Qualifications**

Technical Skills

* 5+ years of hands-on experience in data engineering, application development, or cloud-based data platform development. * Strong hands-on development experience with Python, PySpark, Scala, SQL, and ETL/ELT development. * Solid experience with the Azure Databricks ecosystem: notebooks, jobs, Spark clusters, Delta Lake, lakehouse architecture, and performance tuning. * Good knowledge of Azure data services: Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure monitoring/logging capabilities. * Experience with CI/CD pipelines (Azure DevOps YAML), version control, automated testing, modular development, and code reviews. * Understanding of cloud security, access management, observability, and cost-aware development practices. * Working knowledge of Terraform or infrastructure-as-code concepts is a plus.

Preferred Qualifications

* Experience with large-scale distributed data processing using Spark and streaming/real-time ingestion patterns. * Experience building reusable data engineering frameworks, libraries, or shared components. * Knowledge of data quality frameworks, metadata management, lineage, and governance practices. * Experience integrating data pipelines with AI, machine learning, or analytics use cases; familiarity with MLOps is an advantage. * Preferred certifications: Microsoft Azure Data Engineer Associate, Azure Developer Associate, Azure Solutions Architect Expert, Databricks Certified Data Engineer Associate/Professional, or Databricks Certified Developer for Apache Spark.

Soft Skills

* Strong communication and documentation skills, with the ability to explain technical data engineering concepts to diverse audiences. * Strong problem-solving mindset with a focus on automation, reliability, and maintainability. * Team-oriented, proactive, and comfortable working in a cross-functional environment. * Continuous improvement mindset and willingness...

...

\-