AI SRE for Redis and ElastiCache
Turn Redis alerts into a draft root cause analysis in minutes. Watson reads your Redis instances and ElastiCache clusters read-only, correlates connection and memory pressure with recent deploys and scaling events, and posts the root cause in Slack with the evidence attached.
sre@prod-bastion ~ $ sherlocks investigate --source redisALERT SIGNAL CORRELATEDcurrconnections 280 -> 1,788 HPA pod churncurrconnections 370 -> 1,988 WS reject loopcelery memory 91% of limit broker healthy sre@prod-bastion ~ $ sherlocks explain elasticache-websocket-churn
Why Redis alerts are hard to explain
A Redis alarm rarely says what caused it. The hard part is not the fix, it is finding which of hundreds of possible causes is the real one, and it usually sits in one of three places.
Reconnect loops, autoscaling that opens a new pool per pod, a client rejected and retrying. Connections climb while Redis itself sits idle.
Data growth, fragmentation, client output buffers or one hot key. The same used_memory number can mean any of them.
Redis runs commands on one thread, so a single O(N) command or a fork for persistence stalls every other client.
So the on-call engineer runs INFO, opens the ElastiCache metrics, checks the deploy history and the autoscaler, and rules out guesses one at a time. Sherlocks AI takes over that checking: 16+ agents test hypotheses against the data and rank them, so the engineer verifies the reasoning instead of redoing it.
Redis incidents, worked end to end
Real published investigations, grouped by failure class, with the RCA guide and upstream documentation for each.
Connection pressure
- Reconnect loop after a deploy
- Connection fan-out from autoscaling
- WebSocket reject and reconnect storms
- Connection pool that never releases
- maxclients exhaustion
Read NewConnections next to CurrConnections. A leak grows CurrConnections slowly at a low NewConnections rate; both spiking in the same interval is a churn loop. Two ElastiCache clusters here crossed their alarms that way, one from HPA pod churn and one from a WebSocket reject loop, while the cache itself sat near idle.
Redis as a queue broker
- Task backlog growth
- Workers holding tasks
- Broker slow to dispatch
A growing queue does not mean the broker is slow. A Celery worker reached 91% of its memory limit as its backlog grew, the Redis broker's CPU, memory and evictions were healthy, and the backlog traced to an Aurora load surge that slowed task completion.
Other Redis failure modes
What to look for in each, from the Redis documentation.
Memory pressure
- maxmemory reached with eviction
- Memory fragmentation
- One hot key holding a large share of memory
- Pub/Sub buffer growth from slow subscribers
- Replica output buffer growth
Once maxmemory is reached, the eviction policy decides what happens: an allkeys or volatile policy evicts keys, noeviction rejects writes with an error. If used_memory_rss sits well above used_memory, the gap is fragmentation rather than data, and INFO memory reports it as mem_fragmentation_ratio.
Slow commands and latency
- O(N) command blocking the event loop
- Latency spikes during BGSAVE or AOF rewrite
- CPU saturation from a hot command
- Cross-slot errors in cluster mode
- Oversized pipelines
Redis executes commands on a single thread, so one KEYS or HGETALL on a large hash stalls every other client for as long as it runs. SLOWLOG measures execution time only, not network or queueing time, and the fork behind BGSAVE or an AOF rewrite can add latency on large datasets.
Replication and failover
- Replica lag after a write burst
- Full resync loading the primary
- Failover that did not complete
- Replica diverged from primary
Lag is the gap between master_repl_offset and each replica's offset in INFO replication. A replica that falls further behind than the replication backlog can cover needs a full resync, and the primary forks to produce an RDB snapshot for it, which costs CPU and memory at the worst moment.
Caching patterns
- Cache stampede after a mass invalidation
- Thundering herd on a cold cache
- Hot key contention
- TTL change after a deploy
When many keys expire or are invalidated at once, every miss falls through to the database behind the cache, so the first alert often names the database rather than Redis. The hit ratio from keyspace_hits and keyspace_misses in INFO stats is the quickest way to see it.
A real investigation, step by step
A published case from the examples section, not a mock-up. CurrConnections on an ElastiCache cluster rose more than fourfold while the cache sat near idle. Here is how the investigation got from the alarm to a WebSocket reject loop.
- 1The alertWarningCloudWatch2026-04-06 07:17:33 UTC
CurrConnections on eaze-websocket-001 exceeded threshold
1,988 maxCurrConnections6,377NewConnections6.6%Redis CPU - 25 hypotheses ruled out
Redis-side resource exhaustion (CPU, memory, or maxclients)
CPU 6.57% (max 9.26%), EngineCPU 5.03%, DB memory 2.34% (max 2.44%). All well within safe limits the whole window. Redis is a passive victim, not the cause.
Slow accumulation of idle / leaked connections
NewConnections surged to 6,377 in the same 5-minute window as the CurrConnections jump. A leak shows gradual CurrConnections growth with a low NewConnections rate; the simultaneous spike in both is a churn loop, not idle accumulation.
Recent deploy to eaze-channels introduced the regression
Rollout history could not be confirmed from available data; no image change or deploy correlation could be proven in the window. Cannot be used as the cause. Ruled out here; tracked as an open caveat.
Client-side bug forces reconnects regardless of route status
Clients behave correctly by retrying; the server-side WSREJECT is the root issue driving the loop.
Load balancer health checks are creating transient connections
Health checks run at a steady low frequency and do not account for the surge.
- 3Root cause
WSREJECT on /ws/expert_availability triggered a reconnect loop
Repeated WSREJECT on /ws/expert_availability interleaves with the connect/disconnect churn. Each reject makes the client re-attempt with no backoff, opening a fresh Redis connection per attempt: ~21 reconnects/sec drove CurrConnections to 1,988 and NewConnections to 6,377 while Redis stayed healthy.
- 4The fix
- Investigate the root cause of WSREJECT on /ws/expert_availability: check recent auth, CORS, or allowed-host changes.
- Add connection rate limiting or backoff logic to websocket clients to prevent reconnect storms from cascading to Redis.
For the general method, read how to debug ElastiCache connection spikes, or browse every AWS investigation.
How it works
(04 Steps)- 01Connect Redis, read-only
Access is the INFO command and read-only key metadata. ElastiCache metrics come through the AWS integration.
- 02Install the Slack app
The Redis alarms your team already sends to Slack are where an investigation starts.
- 03An alert fires
Watson reads Redis, CloudWatch, deploys and scaling events. Sherlocks AI tests hypotheses against the data and ranks the causes.
- 04The RCA lands in Slack
Root cause with a confidence level, timeline, remediation and links to the evidence. Ask follow-ups in the thread.
What Watson reads
Supported today: Redis, including Amazon ElastiCache for Redis. Running Redis next to PostgreSQL, MySQL or MongoDB? The databases page covers the rest of the data layer.
From Redis. INFO. Plus read-only key metadata: memory usage, key counts, replication status and evictions.
From CloudWatch, for ElastiCache. CurrConnections, NewConnections, CPUUtilization, EngineCPUUtilization and DatabaseMemoryUsagePercentage. Read through the AWS integration, and each appears as evidence in a published investigation.
Around Redis. kubectl rollout history, HPA events and application logs. The clients are usually where a Redis alert starts, so the deploys, scaling events and client logs are read too.
Security and data handling
Read-only. For Redis that is the INFO command and read-only key metadata. Sherlocks AI cannot write or delete keys, change configuration or disconnect clients.
Stays in your environment. Watson runs inside your network. Only aggregated metrics and metadata reach the Sherlocks AI platform, or nothing at all with the self-hosted deployment.
SOC 2 Type 2. The Sherlocks AI cloud environment is audited every year against the SOC 2 Type 2 framework.
Why Sherlocks AI for Redis incidents
In both published ElastiCache cases the cache stayed near idle while connections climbed. Sherlocks AI reads the client side, the deploys and the scaling events alongside Redis, so the alert is traced to whatever is actually opening the connections.
Redis's own stats and the ElastiCache metrics in CloudWatch are read together, with the Kubernetes and application signals around them.
The INFO command and key metadata, nothing that writes, deletes or reconfigures.
Every root cause comes with a confidence level and the evidence behind it, including the hypotheses that were ruled out.
Frequently asked questions
(07 Questions)Can Sherlocks AI investigate Redis and ElastiCache incidents?
Yes. The published investigations on this site include an ElastiCache CurrConnections breach caused by HPA pod churn, a CurrConnections spike from a WebSocket reconnect and reject loop, and a Celery memory alert where the Redis broker was checked and ruled out. Each is worked from the alert to a root cause, with the hypotheses that were ruled out.
Which Redis deployments does Sherlocks AI work with?
Redis, including Amazon ElastiCache for Redis, where Sherlocks AI also reads the cluster's CloudWatch metrics through the AWS integration.
Does Sherlocks AI need write access to Redis?
No. All permissions are read-only. For Redis that is the INFO command and read-only access to key metadata.
Why does CurrConnections spike when traffic has not changed?
Usually because clients are reconnecting, not because there are more of them. Compare NewConnections with CurrConnections: a leak grows CurrConnections slowly at a low NewConnections rate, while both jumping in the same interval is a churn loop, often from a deploy, an autoscaling event or a client that is rejected and retries.
Does a connection spike mean Redis hit maxclients?
Not necessarily. If maxclients were the limit, CurrConnections would plateau at it and new clients would be refused. A spike that peaks and falls back, with low CPU and memory, points at the clients rather than at Redis.
How does a Redis alert become an investigation?
Sherlocks AI monitors your Slack workspace for incident conversations and alerts. An investigation can be triggered manually, by an alert, or proactively. Watson reads the Redis, CloudWatch and client-side signals, and Sherlocks AI generates hypotheses, tests them against the data, and ranks them by likelihood and impact.
How do I know whether to trust a root cause?
Every root cause comes with a confidence level, and every claim cites the logs, metrics or commits it rests on. The published investigations also show the hypotheses that were checked and ruled out.
Get started
Start free with 30 investigations per month, or book a 30-minute demo and we will walk you through a live incident replay on your own Redis setup.
Questions? Get in touch.
Sherlocks AI is used by Fynd, Topmate, Wafeq, TradeIndia, and Lokal to investigate production incidents. SOC 2 Type 2 certified. Watson runs read-only.