Platform Staff Engineer - UK

About this role

Your Impact

The Staff Software Engineer (Platform) leads large-scale projects and is a key contributor to their group's success. They are involved in long-term strategic planning, mentor senior engineers, and are influential in technical decision-making across multiple teams or a department. They focus on implementation and system design, as well as maintaining and evolving DISCO’s distributed data platform.

What You'll Do

• Technical Leadership and Mentorship: Provides technical leadership in project teams, guiding engineers, and contributing to technical decision-making. Sets technical direction for large projects or teams, driving innovation, and solving organization-wide problems.

• Performance Optimization: Focuses on designing systems for high performance, scalability, and reliability.

• Ownership: Proactively identifies areas of products and systems that currently have problems or will develop problems as well as makes plans to address those problems.

• Stakeholder Engagement: Works with stakeholders, including product managers, designers, developers, and executives, to align architectural designs with business needs and customer expectations.

• Software Development: Designs, develops, and maintains sophisticated, scalable, high-performance data software solutions, ensuring they are efficient, scalable, and meet business
equirements with architectural excellence for our big data platform at petabyte scale. Responsible for the boundary identification, system design and grooming for one or more sub-systems.

• Code Quality and Optimization: Upholds and enhances code quality through rigorous review processes, optimization techniques, and adherence to best practices.

• Technical Problem Solving: Addresses intricate technical issues through innovative problem-solving strategies.

• Collaboration and Communication: Works effectively within the engineering team and cross-functional teams, collaborating with other engineers, product managers, and stakeholders, contributing to project planning and execution.

• Continuous Learning and Adaptation: Stays updated with the latest technology trends and advancements, continually enhancing skills and knowledge, bringing innovative ideas to the team.

• Technical Documentation: Creates and maintains comprehensive technical documentation, ensuring clarity and accessibility of information.

• Data Platform Development: Builds and maintains scalable and robust data platforms to support analytics, machine learning, and business intelligence.

• Data Integration and ETL: Develops and optimizes ETL (Extract, Transform, Load) processes and data integration pipelines. Ensures efficient data flow across various sources and systems.

• Advanced File Processing: Develops extensible file processing pipelines that offer flexible, self-service artifact generation. Researches and evaluates file processing software based on fidelity, reliability, and performance criteria. Models file processing outcomes within an observability framework to provide insights for engineering teams and business leaders. Develops engineering systems that facilitate rapid and automatic evaluation of file processors.

• Data Management: Crafts and orchestrates scaleout data pipelines that optimize serverless and containerized compute to balance cost, latency, and duration for big-data solutions. Contributes to algorithmically intensive data engines operating on streaming, large, or multi-tenant datasets. Troubleshoots systems, identify root causes, and rectify bugs.

Who You Are

• 10+ years of relevant experience in backend engineering, with a substantial focus on big data platforms.

• 7+ years of professional experience with each of the following: Service-Oriented Architecture, Docker-based services, microservices, API, and Data Model Design or Implementation.

• Experience with Large-Scale Systems: Involvement in developing, monitoring, and maintaining large-scale, high-availability software systems.

• Experience designing and developing from a Security Perspective.

• Demonstrated experience in leading or playing key roles in significant software projects, showing an ability to work effectively both independently and as part of a team.

• Demonstrated expertise in designing, implementing, and maintaining (through operational observability) high-availability, high-performance, distributed data processing systems.

• Experience with gRPC and Protocol Buffers for efficient, language-agnostic service-to-service communication.

• Proven ability to deliver well-crafted, tested, and maintainable code solutions to complex technical challenges.

• Experience with a variety of software stacks including, but not limited to, JVM-focused tech stacks, particularly server-side Kotlin and gRPC.

• Deep expertise with both relational and non-relational data stores: Elasticsearch, Cassandra, MySQL, and PostgreSQL.

• Experience delivering data platform software solutions (including designing, implementing, and operating) in the cloud.

• Experience selecting and extending 3rd party components (commercial or open source) that provide operational leverage.

• Familiarity with Agile development methodologies and practices.

• Experience using feature or release toggles as a code branching strategy.

• Experience leveraging observability systems (logging, metrics, traces) to monitor and debug production subsystems and generate business-level metrics.

• Experience with algorithmically intensive data engines operating on streaming, large, or multi-tenant datasets.

• Experience with designing, modifying, and operating multi-tenant systems.

• Experience with tools such as Envoy or AWS App Mesh.

• Experience with CI/CD pipelines and tools (like Jenkins, GitLab CI, or CircleCI) for automated testing and deployment of microservices.

Even Better If You Have...

• Knowledge of microservices architecture principles, including decentralized data management, service discovery, and bounded contexts.…