Sr. Staff Observability Engineer (GPU Cloud & Telemetry Platform)

About this role

<p><strong>About Coupang</strong></p> <p>We exist to wow our customers. We know we’re doing the right thing when we hear our customers say, “How did we ever live without Coupang?” Born out of an obsession to make shopping, eating, and living easier than ever, we’re collectively disrupting the multi-billion-dollar e-commerce industry from the ground up. We are one of the fastest-growing e-commerce companies that established an unparalleled reputation for being a dominant and reliable force in South Korean commerce.</p> <p>We are proud to have the best of both worlds — a startup culture with the resources of a large global public company. This fuels us to continue our growth and launch new services at the speed we have been since our inception. We are all entrepreneurial surrounded by opportunities to drive new initiatives and innovations. At our core, we are bold and ambitious people that like to get our hands dirty and make a hands-on impact. At Coupang, you will see yourself, your colleagues, your team, and the company grow every day.</p> <p>Our mission to build the future of commerce is real. We push the boundaries of what’s possible to solve problems and break traditional tradeoffs. Join Coupang now to create an epic experience in this always-on, high-tech, and hyper-connected world.</p> <p> </p> <p><strong><span data-ccp-props="{"134233279":true,"134245417":false,"335559738":240,"335559739":240}">Role Overview</span></strong></p> <div data-olk-copy-source="MessageBody">We are seeking a <strong>Sr. Staff Observability Engineer</strong> to lead the design and evolution of our observability platform for a <strong>GPU-as-a-Service (GPUaaS)</strong> infrastructure. This role will own the end-to-end telemetry strategy—from high-throughput metric ingestion to log pipelines and real-time visualization—powering deep insights into GPU clusters, datacenter systems, and distributed workloads.</div> <div>You will architect and operate <strong>planet-scale telemetry pipelines leveraging Grafana Alloy, Mimir, Loki, and Vector</strong>, ensuring high-fidelity observability across GPU workloads, Kubernetes clusters, and datacenter infrastructure.</div> <div> </div> <div><hr></div> <div><strong data-olk-copy-source="MessageBody">Key Responsibilities</strong></div> <div> </div> <div><strong><Architectural Leadership & Strategy></strong></div> <div> <ul> <li><strong>End-to-End Observability Platform Ownership</strong>: Design and scale telemetry pipelines using: <ul> <li><strong>Grafana Alloy</strong> for metrics collection (Prometheus-compatible pipelines)</li> <li><strong>Datadog Vector</strong> for high-throughput log ingestion and transformation</li> <li><strong>Grafana Mimir</strong> for scalable time-series storage</li> <li><strong>Grafana Loki</strong> for log aggregation and querying</li> </ul> </li> <li><strong>Strategic Roadmap</strong>: Define the multi-year vision for GPU infrastructure observability, transitioning from reactive monitoring to <strong>SLO-driven, predictive, and automated observability</strong>.</li> <li><strong>High-Cardinality Telemetry Design</strong>: Optimize pipelines for GPU workloads characterized by: <ul> <li>High-cardinality labels (GPU IDs, tenants, workloads)</li> <li>Burst-heavy workloads (ML training, inference spikes)</li> <li>Multi-tenant isolation requirements</li> </ul> </li> </ul> </div> <div><strong><Telemetry Pipeline Engineering></strong></div> <ul> <li>Architect <strong>low-latency, high-throughput pipelines</strong> capable of ingesting:</li> <ul> <li>GPU metrics (utilization, memory, thermals, MIG partitions)</li> <li>Kubernetes and container telemetry</li> <li>Distributed system logs and traces</li> </ul> <li>Build and optimize <strong>metric pipelines (Alloy → Mimir)</strong> ensuring:</li> <ul> <li>Efficient remote_write tuning</li> <li>Cost-effective retention strategies</li> <li>Horizontal scalability and compaction tuning</li> </ul> <li>Design <strong>log pipelines (Vector → Loki)</strong> with:</li> <ul> <li>Structured logging and enrichment</li> <li>Intelligent filtering/sampling</li> <li>Stream partitioning for high-ingest environments</li> </ul> </ul> <div><strong><GPU & Infrastructure Observability></strong></div> <ul> <li>Establish deep observability into:</li> <ul> <li>GPU hardware (NVIDIA DCGM, MIG, NVLink, PCIe)</li> <li>Kubernetes GPU operators and scheduling behavior</li> <li>Network fabric (RDMA, InfiniBand, TCP performance)</li> </ul> <li>Define <strong>GPU-specific SLIs/SLOs</strong> such as:</li> <ul> <li>GPU utilization efficiency</li> <li>Job scheduling latency</li> <li>Cluster fragmentation</li> <li>Thermal and power…