Data Engineer

About this role

Key Responsibilities

Data Engineering & Pipelines

• Build and enhance data pipelines and automation workflows on Azure for large-scale data ingestion and processing

• Maintain and optimise existing toolkits, databases, dashboards, and API-based solutions

• Develop end-to-end automated data solutions across commodities (Gas/LNG, Power, Weather)

• Design and implement web scraping pipelines (e.g. US LNG data)

Analytics & Real-Time Systems

• Support real-time and batch analytics use cases in a dynamic trading environment

• Enable and support analytics dashboards and GUI tools with reliable backend infrastructure

• Develop real-time flow tracking solutions at critical market points

Data Modelling & Architecture

• Design and manage Snowflake data models, ensuring performance and scalability

• Integrate external data sources including ENTSOE, EPEX, KPLER, MetDesk, WoodMac, and others

• Collaborate with analytics teams to structure data for models and advanced analytics use cases

• Build and maintain data platforms supporting ML models such as demand forecasting

Key Deliverables

• LNG Sendout Optimisation Model

• Ship Tracking & Flow Monitoring Tools

• Prompt Price & Forward Curve Bootstrapping

• Web scraping pipelines for US LNG and other market data sources

• Real-time flow tracking infrastructure at critical market points

• Data platforms supporting ML and demand forecasting models

Required Skills & Experience

Technical

• Strong proficiency in Python for data engineering and pipeline development

• Hands-on experience with Azure data services (Data Factory, Databricks, Blob Storage, etc.)

• Experience designing and optimising Snowflake data models

• Proven track record building and maintaining data pipelines at scale

• Experience integrating third-party market data APIs and external data feeds

• Familiarity with SQL and data warehousing concepts

Domain

• Experience or strong interest in energy, commodities, or financial markets data

• Understanding of real-time and batch data processing patterns in a trading context

• Familiarity with market data providers such as ENTSOE, EPEX, KPLER, or similar

Nice to Have

• Experience with web scraping frameworks (e.g. Scrapy, BeautifulSoup, Playwright)

• Knowledge of LNG, Gas, or Power market fundamentals

• Experience supporting ML model pipelines or demand forecasting workflows

• Exposure to GUI/dashboard tools such as Grafana, Power BI, or Streamlit

• Knowledge of ship tracking data or AIS feeds