The checks your best engineer always runs, answered before the page is opened

Write a check once: what to check, why it matters, and how to run it. Sherlocks AI binds it to the alert type and attaches the answers to every future investigation.

Finding

Spike or plateau over three hours?

Plateau. Flat at 4.1% since 14:10, not a step change.

A check is three things

Write it once. Sherlocks AI binds it to the alert type and attaches the answers to every future investigation.

  1. What to check

    The question in plain English, scoped to one signal.

    Compare the 5xx rate over the last three hours.

  2. Why it matters

    The reasoning your best engineer would apply, written down once.

    A plateau points at a dependency. A spike points at a change.

  3. How to run it

    The query, the dashboard, or the command that answers it.

    sum(rate(http_requests_total{status=~"5.."}[5m]))

Example: High 5xx rate

Bound to the alert type, so the answers are already attached when the page is opened.

Alert typeHighHttp5xxRate3 of 3 answered
  • Spike or plateau over three hours?

    Finding

    Plateau. Flat at 4.1% since 14:10, not a step change.

  • Did P50 latency move downstream?

    Normal

    payments-db and redis P50 unchanged across the window.

  • What shipped in the previous 30 minutes?

    Inconclusive

    No deploys. Two config changes with no diff captured.

Every check returns one of three
  • NormalChecked, and there is nothing here.
  • InconclusiveChecked, and the data cannot answer it.
  • FindingChecked, and this is worth your attention.

Because a trustworthy agent must be able to say I do not know.

Answers above are illustrative. Yours come from the check as you wrote it, run against your own signals.

The problem

The best triage routine on your team is not written down

Every team has one engineer whose first three questions are always the right ones. They see a 5xx alert and they already know whether to start at the deploy log or the dependency graph, because they have seen this shape of failure enough times to have compressed the experience into a routine they run without consciously thinking about it.

That routine is one of the most valuable things your on-call rotation owns, and it exists in exactly one place. When that person is asleep, on holiday, or in a meeting they cannot leave, whoever is holding the pager starts from zero. Not because they are less capable, but because the routine was never written anywhere they could find it at three in the morning.

The usual answer is a runbook, and runbooks fail for a specific and predictable reason. A runbook is a document, and a document has to be remembered, found, opened, and followed by someone who is already under time pressure and already three tabs deep. The checks inside it are usually correct. The moment they are needed is the moment nobody has the spare attention to go looking for them.

Why binding matters

The answer should be waiting, not merely available

Writing the check down is only half of it. The other half is that its answer has to arrive before the person does, and that is the half most tooling skips.

Consider what actually happens between an alert firing and an engineer forming their first hypothesis. The page arrives. They find a laptop. They authenticate to something. They open the right dashboard, set the right time range, and work out which of the four similarly named services is the one that alerted. Only then do they start thinking. The queries themselves are seconds of work wrapped in several minutes of setup, and that setup is paid again in full on every single occurrence, for questions whose answers were entirely predictable from the alert type alone.

Binding a check to an alert type deletes that setup. When the alert fires, Sherlocks AI already knows which questions matter for this class of failure, runs them, and attaches the answers to the investigation before anyone opens it. The engineer arrives at interpretation, which is the part of triage that genuinely needs a human, rather than at data collection, which is not.

What this buys you

The first ten minutes of an incident stop being about assembling context and start being about deciding what to do with it.

Calibration

Inconclusive is the verdict that makes the other two trustworthy

An agent that always produces a confident answer is not confident, it is uncalibrated. Given a question its data cannot settle, it will produce something anyway, and because that something reads exactly as fluently as a real finding, you have no way to tell the two apart from the outside. One wrong answer delivered with certainty costs more than ten honest refusals, because it sends a tired engineer down a path with their guard lowered.

So every check returns exactly one of three things. Normal means the check ran and there is nothing here. A finding means the check ran and this is worth your attention. Inconclusive means the check ran and the data cannot answer it, which is not a failure but a real and useful result: it tells you the gap is in your instrumentation rather than in your understanding.

That third verdict is also a routing decision. Normal closes a branch of the investigation and stops the agent spending effort there. A finding opens one and pulls more evidence toward it. Inconclusive says a human has to look, and says precisely which question they are looking at, which is a far better handoff than a summary that quietly omitted the thing it could not determine.

Getting started

What is actually worth writing a check for

Start with the alert that most often turns into a real investigation. Not the noisiest one, which usually needs tuning rather than checks, but the one where somebody ends up genuinely digging, because that is where the same three questions are being re-asked from scratch every time.

Good checks are specific, bounded, and answerable from data you already collect. Is the service healthy is not a check, because there is no single query behind it and no answer that changes what anyone does next. Did the 5xx rate step or ramp over the last three hours is a check, because one query answers it and the two possible shapes point in different directions: a step implicates a change, a ramp implicates saturation.

You do not need many. Three or four checks against your top two alert types will cover most of what a competent first responder does in the opening ten minutes, and each one you write is answered automatically on every future occurrence for as long as that alert exists. The work is front-loaded once and amortised across every incident after it.

Frequently Asked Questions

What engineering teams ask before they write their first check.

A runbook waits to be opened. A check runs on its own and delivers its answer into the investigation before anyone reads anything. The content can be identical, the difference is who does the work and when. A runbook says here is what you should go and look at, which is a task added to somebody's queue at the worst possible moment. A bound check says here is what we already looked at and here is what we found, which is context, not homework. Teams often write their first checks by reading their existing runbooks and turning each numbered step into one.

You write what to check and why it matters in plain English. How to run it is where you point at the actual source, which can be a PromQL query, a log filter, a dashboard, or a command, depending on where the answer lives. The plain-English half is not decoration: it is what lets the agent decide whether a returned number is Normal or a finding, and it is what a human reads when they want to know why this question was worth asking in the first place.

It returns Inconclusive rather than guessing, and it says why. A metric that has been renamed, a dashboard that was deleted, or a time range with no samples in it all produce the same honest outcome: the check ran, and the data could not answer it. That is deliberately visible rather than silent, because a checklist that quietly degrades into nothing is worse than no checklist at all. Repeated Inconclusive results on the same check are a signal that the check, or the instrumentation behind it, needs attention.

Yes, and the good ones usually are. What shipped in the previous 30 minutes is worth asking on almost any alert, because a recent change is a plausible cause of almost any failure. Narrow checks stay bound to the alert class they were written for. Binding is a mapping rather than a hierarchy, so a given alert type ends up with the union of every check bound to it, which is generally what you want.

No. Your checks are additional, not a replacement for the investigation. Sherlocks AI still forms and tests its own hypotheses against the signals it is connected to. What your checks change is the starting point: they encode the questions your team has already learned are worth asking about your specific system, which is knowledge no general agent can derive from telemetry alone. The two compose, and the checks tend to be what makes the first pass feel like it was done by somebody who knows your stack.

One is enough to feel the difference on the alert it is bound to, and most teams see the value on the first incident where the answer was already attached. There is no threshold to cross before it starts working, because each check is independently useful the moment it exists. The realistic ceiling is lower than people expect too: past roughly a dozen well-chosen checks, most teams find the marginal one is answering a question that rarely changes anyone's next move.

Start your free month

Cancel anytime, fully refundable.

A note from the founder

You'll talk to me, not a sales rep. Bring your worst incident from last quarter.
Gaurav ToshniwalGaurav ToshniwalFounder and CEO, Sherlocks AI

Tell Me About Your Worst Incident

We use cookies

Only the strictly necessary ones are set when you arrive. Analytics and marketing cookies are set solely if you allow them, and you can change your mind at any time. See our Privacy Policy.