NewReplay any alert rule over thirty days

All of production, one query away.

Sonde keeps every event your services emit for thirty days, unsampled, and answers a query over all of it in about a second.

Start freeRead the docs

brew install sonde

The console below runs on a fixture dataset. / edits the query, Enter runs it.

prod-events
Fixture

99,499 of 47,384,918 events · 0.24 s · 6 groups

Matching events per minute

max 3K/min

Count by route

Showing 5 of 6 groups

Last hourvs first 30 min

Matched events

99,499↑ +147%

Share of traffic

0.21%↑ +0.17 pt

p95 latency

1.17 s↑ +195%

Chosen route

/checkout
Events
46,943
Share
47.2%
p95
2.00 s

By status

  • 50232.4K
  • 50412K
  • 5002.6K
$ sonde tail 'status >= 500 and route = "/checkout"'

sonde 0.18.2 · last run 14:33 · 0.24 s

/ EditEnter Run↑↓ Choose

Kept unsampled by on-call teams at

Fictional companies running Sonde

ferncastleMOSSBANKferrowPlinthMARROWBYcobblefordbrightwirebrindlewickParcelhop

01sonde tail · trace · compare

From the count to the cause.

We keep your filter and let you change the verb. Tail the matching events as they land, open one as a trace, or set this hour against the last deploy.

$ sonde tail 'status >= 500 and route = "/checkout"'
followingFixture
502 8504 3500 127 matching events a second, all regions
TimeStatusRegionEventTook
14:32:07.118502us-eastPOST /checkout upstream=payments req=7f3ac21e1.92 s
14:32:07.402502eu-westPOST /checkout upstream=payments req=a01c9e442.04 s
14:32:08.260504us-eastPOST /checkout upstream=payments req=3be80f193.00 s
14:32:09.733502ap-southPOST /checkout upstream=payments req=c7d2a5b01.88 s
14:32:10.412500us-eastPOST /checkout order rollback failed req=91e0d3aa1.71 s
14:32:11.305502eu-westPOST /checkout upstream=payments req=5f16be021.97 s
14:32:12.846504us-eastPOST /checkout upstream=payments req=e4a7730c3.00 s
14:32:13.520502us-eastPOST /checkout upstream=payments req=0b9f61d71.90 s
14:32:14.077502eu-westPOST /checkout upstream=payments req=d38c02e52.11 s

02sonde query --days 30

Thirty days. Every event.

We sample nothing and roll nothing up. Pick any hour from the last month and the raw events are still there, with every field you sent.

Peak 5xx share by hour

route = "/checkout"Aug 26 to Sep 24, 720 hours, none sampled

<0.1%1.6%+
Fixture

Thu, Sep 24

Partial day, 15 of 24 hours

Events
45.9M
Peak 5xx share
2.31%
Worst hour
14:00
p95 latency
248 ms

Incident at 14:33, ongoing

Worst event, 14:00 hour

time      2026-09-24T14:33:22.112Zservice   checkout-apiroute     /checkoutstatus    502region    ap-southupstream  paymentstook_ms   1548trace     3e31fe2fdeploy    v2.42.0sampled   false
$ sonde query --day 2026-09-24 --route /checkout 'status >= 500'

03sonde alert test

An alert is a saved query.

Rules live as plain files in your repository and get reviewed like any other change. Before one goes live, we replay it over the last thirty days and show you every time it would have paged.

  • Reviewed with the code

    The rule ships in the pull request it watches.

  • Replayed before it merges

    Thirty days of traffic show how often it fires.

  • Paged by ownership

    Pages go to the owner the rule file names.

alerts/checkout-5xx.sonde
main · 2 reviewsFixture
# Upstream timeouts show up as# 502 and 504 on /checkout first.alert "checkout 5xx" {  over    route = "/checkout"  match   status >= 500  when    share > 1% for 5m  every   1m  notify  "#payments-oncall"  owner   "team-payments"}
Try a threshold
$ sonde alert test --over 30d
Fixture

4 pages in 30 days at share > 1%

StartedLastedPeak share
2026-09-24 14:33ongoing2.31%
2026-09-19 18:0622 min4.12%
2026-09-12 03:416 min1.34%
2026-09-02 14:1311 min2.40%

replayed 43,200 minutes in 1.9 s

It holds when production doesn’t.

04sonde bench

We publish our benchmark suite. Every figure below comes with the command that reruns it on your own hardware.

Count over 30 days
0.84s
sonde bench count --over 30d
Group by route, 30 days
1.31s
sonde bench group --by route
Trace lookup by ID
38ms
sonde bench trace --by id
Sustained ingest, one node
410k/s
sonde bench ingest

Sample results from a synthetic dataset of 1.2 billion events on one 16-core node, median of 20 runs.

05sonde notebook

On call, the query stays open.

A notebook keeps every query from an incident in order, with its answer beside it, so whoever takes the next page starts where you stopped.

Dana Okafor, a fictional engineer, in a charcoal knit shirt, looking off to the side.

At two in the morning I don’t want a dashboard. I want to ask every event one question and have the answer before the next page.

Dana OkaforStaff SRE, Ferncastle

People, quotes and companies are fictional.

Checkout on-call, Ferncastle
DOFixture
Pages this week
2
↓ 3 vs last week
Time to cause
25 min
↓ 41% vs median
Queries run
6
↓ 11 vs last time
Events searched
61.2M
→ none sampled
Recent incidents on this rotation
SeverityIncidentStartedLasted
SEV1Checkout 5xx, payments timing outSep 24 14:33ongoing
SEV2eu-west checkout 503sNotebook belowSep 17 02:0628 min
SEV3Slow search in ap-southSep 14 09:1212 min
eu-west checkout errors2026-09-17 02:06
› status >= 500 and route = "/checkout" | count by region02:06 · 14,153 of 61.2M events · 0.31 s
  • eu-west13,842
  • us-east214
  • ap-south97
5xx per minute, eu-west
› region = eu-west and status = 503 | count by upstream02:09 · 13,790 events · 0.22 s
  • resolver-eu13,738
  • payments52

Every 503 went through the eu-west resolver. Rolled its config back at 02:31, and errors were at baseline by 02:34.

note · 02:34

sonde init

Start with one service.

Point Sonde at one service and its events start arriving within a minute, with all thirty days kept from the first one.

  • Free up to 50 GB a month
  • Events stay in the region you choose
  • Export any day as Parquet

Then, in your terminal

sonde init --service checkout-api
sonde 0.18.2 · linking checkout-api to prod-events✓ wrote sonde.toml✓ found 3 emitters in checkout-api✓ retention set to 30 d, sampling off→ first events arriving in prod-events14:33:02POST /checkout200212 ms14:33:02GET /cart/items20048 ms14:33:03POST /checkout5021.92 snext sonde tail 'service = "checkout-api"'

sonde.tomlWritten
[service]name      = "checkout-api"dataset   = "prod-events"[retention]days      = 30sampling  = "off"[emitters]http      = "src/server.ts"worker    = "src/jobs/charge.ts"queue     = "src/queue/orders.ts"