X - Data Architect - Government of Alberta Pipeline

About this role

Sr. Data Architect - GOAPRDJP00000616
Eleventh Floor
9942 - 108 Street
Edmonton
Primarily work remotely but must be available for on-site meetings as required (Meetings may occur up to 3-4 times per fiscal month, but the actual frequency will depend on the specific initiative and will be determined on an on-demand basis.)
Duties:

• Design and implement scalable, secure, and high-performance data architecture on Microsoft Azure, supporting both cloud-native and hybrid environments.

• Lead the development of data ingestion, transformation, and integration pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

• Architect and manage data lakes and structured storage solutions using Azure Data Lake Storage Gen2, ensuring efficient access and governance.

• Integrate data from diverse source systems including ServiceNow, and geospatial systems, using APIs, connectors, and custom scripts.

• Develop and maintain robust data models and semantic layers to support operational reporting, analytics, and machine learning use cases.

• Build and optimize data workflows using Python and SQL for data cleansing, enrichment, and advanced analytics within Azure Databricks.

• Design and expose secure data services and APIs using Azure API Management for downstream systems.

• Implement data governance practices, including metadata management, data classification, and lineage tracking.

• Ensure compliance with privacy and regulatory standards (e.g., FOIP, GDPR) through role-based access controls, encryption, and data masking.

• Collaborate with cross-functional teams to align data architecture with business requirements, program timelines, and modernization goals.

• Monitor and troubleshoot data pipelines and integrations, ensuring reliability, scalability, and performance across the platform.

• Provide technical leadership and mentorship to data engineers and analysts, promoting best practices in cloud data architecture and development.

• Other duties as needed

Must-Have's:

• A college or Bachelor's degree in Computer Science or a related field of study.

• Hands-on experience managing Databricks workspaces, including cluster configuration, user roles, permissions, cluster policies, and applying monitoring and cost optimization for efficient, governed Spark workloads - 3 years

• Experience as a Data Architect in a large enterprise, designing and implementing data architecture strategies and models that align data, technology, and business goals with strategic objectives - 8 years

• Experience designing data solutions for analytics-ready, trusted datasets using tools like Power BI and Synapse, including semantic layers, data marts, and data products for self-service, data science, and reporting - 4 years

• Experience in Github/Git for version control, collaborative development, code management, and integration with data engineering workflows - 4 years

• Experience with Azure services (Storage, SQL, Synapse, networking) for scalable, secure solutions, and with authentication (Service Principals, Managed Identities) for secure access in pipelines and integrations - 5 years

• Experience in Python (including PySpark) and SQL, applied to developing, orchestrating, and optimizing enterprise-grade ETL/ELT workflows in a large-scale cloud environment - 6 years

• Experience building scalable data pipelines with Azure Databricks, Delta Lake, Workflows, Jobs, and Notebooks, plus cluster management. Extending solutions to Synapse Analytics and Microsoft Fabric is a plus - 3 years

Nice-to-Have's:

• Certification in The Open Group Architecture Framework (TOGAF).

• Use of AI-Experienced in using AI for code generation, data analysis, automation, and enhancing productivity in data engineering workflows - 1 year

• Direct, hands-on experience performing business requirement analysis related to data manipulation/transformation, cleansing and wrangling - 8 years

• Experience and strong technical knowledge of Microsoft SQL Server, including database design, optimization, and administration in enterprise environments - 8 years

• Experience building scalable ETL pipelines, data quality enforcement, and cloud integration using TALEND technologies - 2 years

• Experience in data governance, security, and metadata management within a Databricks-based platform - 2 years

• Skilled in building secure, scalable RESTful APIs for data exchange, with robust auth, error handling, and support for real-time automation - 3 years

• Experience in Message Queueing Technologies, implementing message queuing using tools like ActiveMQ and Service Bus for scalable, asynchronous communication across distributed systems - 3 years

• Experience working with cross-functional teams to create software applications and data products - 5 years

• Experience working with ServiceNow- Azure-based Data Management Platform Integrations - 1 year

Originally posted on Himalayas