Senior DevOps Engineer

Role, briefly
Our data plane moves hundreds of billions of rows a day across hundreds of thousands of Kafka topics, with Spark, Flink, Ray, and Redis running on EKS and GKE. You would own its reliability as an engineering problem rather than an operations one: the SLOs that define what working means, the instrumentation that proves it, and the software you write to remove the manual work in between. This is not a cloud-provisioning or CI/CD role. It is Senior level, so your influence is expected to reach past the systems you personally touch.

What you’ll own

• What reliable means, in numbers. Availability, freshness, and latency SLOs that platform and runtime teams actually agree to, error budgets that govern release pace, and the data-aware instrumentation behind them: consumer lag, throughput, partition skew, job latency, error rates. Not host and CPU metrics.
• The stateful data plane itself. Kafka via Strimzi at hundreds of thousands of topics, Spark, Flink, and Ray workloads, Redis clusters, and the EKS and GKE clusters underneath. Broker health, checkpointing, backpressure, failover, upgrades, capacity.
• Toil, treated as an engineering problem. Measure the manual work the team does to keep things running, then write software to remove it. Terraform-provisioned infrastructure, runbooks, and automation, including for air-gapped and private enterprise installs.
• Incidents and what comes after. Lead root-cause analysis on distributed-systems failures, run blameless postmortems, and drive the durable fix. Mitigate fast, then design out the recurrence.

Senior scope: you set the reliability bar other engineers design against. Changing how teams outside SRE build is part of the job, not a side effect of it.

The hard problem
The failures we hate are the silent ones: a sink that stopped writing and told nobody, a consumer group quietly drifting behind, a checkpoint that stalled. At hundreds of thousands of topics, host and CPU metrics tell you nothing about whether data is actually moving. Making the data plane’s behaviour legible before it breaks is the standing problem on this team.

Must-haves

• 8+ years in SRE or infrastructure engineering, most of it operating production distributed data systems rather than application or cloud infrastructure.
• Deep hands-on Kafka at scale in production, ideally on Kubernetes via Strimzi: upgrades, topic and partition management, performance tuning, and TLS and secret rotation.
• Redis and stateful workloads on Kubernetes in production: StatefulSets, PVCs, probes, operators. Plus strong hands-on EKS, including cluster lifecycle and version upgrades.
• You write software, not just scripts. Python or similar, advanced Terraform, and enough range to build tooling other engineers depend on.
• You have owned SLOs, error budgets, or an equivalent reliability contract, and you can describe a time one of them changed what a team shipped.
• You use AI tools in your actual work and can say specifically where they help and where they mislead you.