A runbook for the dashboard that is wrong at 6 a.m.

It is 06:10 and a message arrives: “the revenue number on the dashboard looks low”. You have no alert, and you have a meeting at 07:00 that uses the number.

This runbook is the routine we teach customers for that moment. It assumes nothing about your tools beyond a warehouse and a way to run a query. Each step either rules out a cause or points to the next step, and most incidents end by step 4.

Before you start

Write down three things so you do not lose them later: the name of the dashboard, the number that looks wrong and the number you expected. Everything else follows from those.

Step 1: Is the data old?

Most wrong numbers are old numbers. Find the table behind the chart and check its newest row.

select max(updated_at) as newest, current_timestamp as now
from analytics.revenue_daily;

If the newest row is older than your normal arrival time, the table is late and you can skip to step 3. If it is current, move on.

Step 2: Is the data incomplete?

A table can be current and still be missing rows, for example when one region’s feed has not arrived. Compare today’s row count by source with the same weekday last week.

select
  region,
  count(*) filter (where order_date = current_date) as today,
  count(*) filter (where order_date = current_date - 7) as last_week
from analytics.orders
group by region
order by region;

A region with a count near zero is the likely cause. A region at half of last week’s count might be a partial load.

Step 3: What feeds this table?

If the table is late or incomplete, find what is upstream. In Tidewater, open the table and select Lineage. You can also ask the API:

curl -s "https://api.tidewater.example/v1/lineage/analytics.revenue_daily?direction=upstream&depth=2" \
  -H "Authorization: Bearer $TIDEWATER_TOKEN" | jq '.nodes[] | {name, state, owner}'

Look for the first node in the chain whose state is late or failed. That is the root cause. Everything after it is a symptom.

Step 4: Who owns it and what do they know?

The same response includes an owner for each node. Message the owner with the facts you have: the table, the expected time, the actual time and what is affected. A message that includes those four things is answered much faster than “is something wrong”.

Step 5: Tell the people who rely on the number

You do not need a fix to communicate. In the channel where people ask, post:

• what is wrong, in plain words (“revenue for Europe is missing since 05:00”)
• what it affects (“the executive dashboard and the daily email”)
• what you know and do not know yet
• when you will post again

Posting every 30 minutes, even to say there is no change, stops the same question being asked ten times.

Step 6: Fix, then verify

After the owner fixes the feed or re-runs the job, check the same query from step 1 and compare the dashboard number with your expected value. Do not close the incident until both match.

Step 7: Make it less likely next time

Spend ten minutes while it is fresh:

1. Add or tighten the freshness expectation on the table that was late.
2. Make sure the source table has an owner and an alert route.
3. Write the cause in one sentence in the incident note.

A late table that nobody was told about is a monitoring gap. Closing the gap is the real fix.

A short version to pin

1. Is the newest row old?
2. Are any sources missing?
3. What is upstream, and what is the first late node?
4. Who owns it? Send the four facts.
5. Tell the readers, then update on a schedule.
6. Verify, then close.
7. Close the gap.