Staff Software Engineer, GPU Inference

About this role

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud inference services.

This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.

Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership https://openai.com/index/cerebras-partnership/ with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.

ABOUT THE ROLE

Cerebras is building a new generation of disaggregated AI inference systems https://www.cerebras.ai/press-release/amd-and-cerebras-announce-industry-leading-ultra-low-latency-and-high-throughput-ai-inference that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

We are hiring a Software Engineer to productionize and optimize our GPU serving stack, working across our custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable, numerically correct, observable, and exceptionally performant.

You will write production code, establish operational practices for a new accelerator fleet, and drive improvements in time to first token, throughput, tail latency, and capacity efficiency. This is a hands-on role requiring deep debugging and optimization across application, runtime, distributed systems, and hardware layers.

RESPONSIBILITIES

- Productionize the GPU inference stack. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.

- Own GPU operational readiness. Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet. Build automation that makes driver, firmware, runtime, model, and container compatibility explicit and reproducible.

- Drive reliability in production. Define service-level indicators and objectives for GPU-backed inference. Improve fault isolation, graceful degradation, automated recovery, incident response, and post-incident remediation across the serving stack.

- Improve inference performance. Profile and optimize time to first token, request throughput, tokens per second per GPU, tail latency, GPU utilization, memory efficiency, and rack-level capacity under representative production workloads.

- Optimize model-serving behavior. Tune and improve scheduling, continuous batching, prefix caching, KV-cache management, tensor and expert parallelism, request admission, quantization, graph execution, and distributed communication.

- Debug across system layers. Diagnose complex failures and performance regressions across application code, vLLM, PyTorch, ROCm/HIP, collective communication libraries, kernels, drivers, firmware, networking, and hardware.

- Ensure numerical correctness. Build validation and regression infrastructure for model quality, numerical accuracy, precision changes, quantization, determinism, and compatibility across software and hardware releases.

- Build performance and correctness infrastructure. Develop representative benchmarks, workload replay tools, profiling automation, release qualification, dashboards, and regression gates. Turn one-off investigations into repeatable engineering systems.

MINIMUM QUALIFICATIONS

- 8+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.

- Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.

- Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.

- Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.

- Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling methodology.

- Experience debugging distributed systems across multiple layers rather than treating the serving framework or accelerator runtime as a black box.

- Experience with Linux, containers, Kubernetes or comparable orchestration systems, observability, CI/CD, and operating latency-sensitive services in production.

- Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.

- Strong communication and technical leadership skills, with a demonstrated ability to drive ambiguous cross-functional projects to completion.

- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.

PREFERRED QUALIFICATIONS

- Experience with AMD Instinct accelerators and the ROCm ecosystem, including HIP, RCCL, rocprofiler, AMD SMI, AITER, hipBLASLt, Composable Kernel, or related libraries and tools.

- Deep CUDA experience that demonstrates an ability to transfer GPU systems knowledge across accelerator platforms.

- Experience modifying or contributing to vLLM, SGLang, PyTorch, Triton, TensorRT-LLM, or another open-source ML systems project.

- Experience optimizing prefill-heavy or disaggregated prefill/decode inference architectures.

- Understanding of KV-cache transfer, prefix caching, continuous batching, chunked prefill, request scheduling, and…