Root cause analysis guides
Root cause analysis guides for the failures that page on-call engineers. Each covers one failure mode end to end: what the error means, the commands that confirm it, and the configuration change that stops it recurring.
sre@prod-bastion ~ $ column -t ~/oncall/last-four.txtSTACK SYMPTOM ROOT CAUSEkubernetes pod Evicted node ephemeral-storageaws EBS latency 4x gp3 throughput ceilingdatabases pool exhausted plan regression, migrationlinux-vm / at 99% logrotate rule dropped sre@prod-bastion ~ $ cat ~/oncall/kubernetes-eviction.md
Start with your stack
(04 Stacks, 05 Guides Published)- 05 guides
Kubernetes
Root cause analysis guides for Kubernetes production failures. Each covers one failure mode: what the pod state means, the commands that confirm it, and the configuration change that stops it recurring.
Read guides - Coming soon
AWS
Managed AWS services enforce limits per dimension, so a resource can saturate on one while showing headroom on every other. A gp3 volume at 21% of provisioned IOPS can still be throttled at its throughput ceiling.
- Coming soon
Databases
Database incidents are usually reported by the application rather than the database: a request timeout, an exhausted connection pool, a worker killed for memory. The pool is the symptom; a plan regression is the more common cause.
- Coming soon
Linux VMs
On long-lived virtual machines, failures accumulate rather than recur, and the change responsible often predates the alert by weeks. That defeats the standard approach of correlating against the most recent deploy.
Looking for worked incidents instead?
A guide explains a class of failure. The examples section is the other half: real production incidents investigated end to end, including the hypotheses that were ruled out along the way.
See Sherlocks AI in action
Watch an AI SRE work a real incident from alert to root cause, on your stack, in 30 minutes.