Senior Data Engineer

Sobre esta vaga

<p><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Job Title: </strong>Senior Data Engineer</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Key Skills:</strong> DBT, Python, SQL, Snowflake, Databricks</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Location:</strong> Brazil</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;"><strong>Mode: </strong>Remote</span><br><br><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">We at Coforge are hiring <strong>Senior Data Engineer</strong> with the following skill set.</span><br><br><strong><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Key Responsibilities</span></strong></p> <ul> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Delivery & Ownership</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Own the end-to-end delivery of data pipelines and data products, from design through deployment and production support</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Develop, maintain, and optimise scalable data pipelines using Azure Databricks, DBT, and Python</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Ensure solutions are production-ready, reliable, and meet defined SLAs</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Migration & Platform Engineering</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Execute and support migration of data from Snowflake (AWS) to Azure using Databricks, DBT, and Lakebridge</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Build and manage infrastructure-as-code using Terraform to enable scalable and repeatable deployments</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Ensure seamless integration between legacy systems and the Azure data platform</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Data Quality, Observability & Governance</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Implement data quality frameworks (e.g., DBT testing, validation rules) to ensure trusted data pipelines</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Enable observability through logging, monitoring, and alerting of pipelines and data workflows</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Support data governance standards including data lineage, consistency, and compliance requirements</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Architecture & Engineering Excellence</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Contribute to the design and architecture of scalable data solutions, aligned with lakehouse principles</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Apply best practices in data modelling, performance optimisation, and cost efficiency</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Follow and promote engineering standards, including modular design and reusable components</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Production Support & Reliability</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Monitor and maintain data pipelines in production, ensuring high availability and performance</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Troubleshoot issues and perform root cause analysis (RCA) for incidents</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Continuously improve system stability, resilience, and operational processes</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Collaboration & Technical Leadership</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Work closely with cross-functional teams (data, product, architecture, business) to translate requirements into scalable technical solutions</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Provide guidance and mentorship to junior engineers through code reviews and best practices</li> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Contribute to documentation of architecture, pipelines, and operational processes</li> </ul> <p> </p> <p><strong><span style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Required Skills & Qualifications</span></strong></p> <ul> <li style="font-family: arial, helvetica, sans-serif; font-size: 12pt;">Strong hands-on expertise with: Azure Databricks, DBT, Snowflake, Python and SQL</li> <li…