A compact decision dashboard

Assemble a one-screen view carrying its measures, its coverage, its as-of date and its assumptions together, write a warning that appears only when its condition holds, and choose a headline the data supports.

A dashboard is where a number stops being a calculation and becomes a decision. Somebody reads it for thirty seconds and acts.

This lesson's dashboard reports that mean attendance per session rose 41.5% between October and November. Every measure behind that is computed correctly. The comparison mixes changing coverage with changing attendance: November describes only Cedar, while October describes all three sites. Cedar's own mean also rose, from 23.0 to 25.0. The reported 41.5% cannot establish a programme-wide increase.

Nothing on a normal dashboard would show that. The measures are right, the chart is right, and the conclusion a reader reaches is wrong. What is missing is not accuracy; it is the two lines saying how much of the programme the figures cover, and a warning that fires when that changes.

The habit this lesson is for: a screen that says what it knows and beside it, in the same size type, what it does not.

A dashboard worth trusting carries four things, and most carry only the first.

Measures. Four to six of them. Enough to answer the decision the dashboard exists for, and no more — every additional number spends attention that the important ones need. If a measure would not change anybody's action, it is decoration.

Coverage. How much of the thing is in these figures? For a programme across three sites, that is how many sites are represented in each period being compared. This is the block that would have caught the 41.5%, and it is two cells.

Coverage is not the same as sample size and it is not the same as a total. It is the answer to "what proportion of what I am claiming to describe is actually in here", and it is almost always available cheaply from data you already have.

Assumptions. Any figure on the screen that was not measured. "Places per session: 30, taken from the room booking" is an assumption, and putting it on the dashboard rather than in a footnote is what stops a reader treating it as an observation.

Freshness. An as-of date. A dashboard without one cannot be compared with a printout of itself from last month, and somebody will try.

Then the warnings, which are the part everybody skips.

A warning is a cell that is empty most of the time and says something specific when a condition holds:

=IF(B14<B13, "Coverage fell: " & (B13-B14) & " sites stopped reporting", "")

Three properties make it useful.

It is conditional. A permanent line of text that says "note: coverage may vary" is a disclaimer, not a warning. Readers learn to skip it within two viewings.

It says how much. "Coverage fell" is a nudge; "two sites stopped reporting" is something to act on.

It can clear. This is the one to insist on, and the only way to know it is true is to make it clear at least once. Change the data so the condition stops holding and watch the cell empty. A warning that has never been observed to clear is indistinguishable from a warning that cannot.

Counting how many warnings are showing gives the screen a single number that can be scanned without reading, and it is a number a person can be asked about in a meeting.

Finally, the headline. The dashboard supports one sentence, and the difference between the sentence the measures suggest and the sentence the data supports is the whole of this module:

• "Attendance rose across the programme." Not supported. The programme is three sites and November covers one.
• "Mean attendance per session at the sites that reported rose from 17.7 to 25.0." Supported, longer, and less exciting.

Choosing the second is not pessimism, and it is not a failure to find a result. It is the only one of the two that will still be true when somebody checks.

Nine sessions were reported across two months:

October    Cedar 24, Cedar 22, Ridge 14, Ridge 12, Hill 18, Hill 16
November   Cedar 26, Cedar 25, Cedar 24

The measures:

Sessions recorded            9
Total attendance           181
Mean per session, October   17.67
Mean per session, November  25.00
Change                     +41.5%

Every one is correct. The coverage block is where the story is:

              October  November   Reported in both
Cedar               2         3   yes
Ridge               2         0   no
Hill                2         0   no

Sites reporting in October     3
Sites reporting in November    1

November covers only Cedar, the site with the highest October mean. The 41.5% mixes a change in the represented sites with Cedar's own rise from 23.0 to 25.0. Comparing Cedar with Cedar gives about 8.7%, which still describes only its reported sessions.

The sessions at Ridge and Hill ran. The dashboard's note column says so. The registers are missing. The sheet identifies which sites did not report, but does not establish their November attendance or how many session registers are missing. Missing reports are not zero attendance.

The two warning cells are written as:

B21  =IF(B14<B13,"Coverage fell: "&(B13-B14)&" sites stopped reporting","")
B22  =IF(B14<B13,"The change in mean per session is not comparable across the two months","")
B23  =COUNTIF(B21:B22,"<>")

Predict what each shows now, and then what each shows if Ridge and Hill both submit one November register.

B21   Coverage fell: 2 sites stopped reporting
B22   The change in mean per session is not comparable across the two months
B23   2

B21   (empty)
B22   (empty)
B23   0

A dashboard as most people first build it:

Sessions recorded            9
Total attendance           181
Mean per session, October   17.67
Mean per session, November  25.00
Change                     +41.5%

Five correct measures and one false impression. Three additions fix it, and none of them is a measure.

The coverage block. Sessions per site per month, and a count of how many sites appear in each. Two formulas copied over a small block, plus two counts.

B16   =COUNTIFS(Data!$B$2:$B$10,$A16,Data!$C$2:$C$10,B$15)
B13   =COUNTIF(B16:B18,">0")

The warnings, conditional on that coverage.

The as-of date and the assumption, written on the screen rather than known by whoever built it.

Then one deletion. The change figure — the 41.5% — stays, because removing it does not make it untrue and somebody will compute it anyway. What changes is that it now sits above two lines explaining why it is not comparable, and beside a headline cell that says what may be published.

That is the difference between a dashboard that hides a problem and one that reports it: not fewer numbers, but the context that makes the numbers readable.

A dashboard has shown "all clear" every week for eight months. Somebody asks whether it works.

That is the right question and it has an answer.

1. Read each warning's formula, not its cell. A cell showing nothing is either a warning that is not firing or a warning that cannot fire, and they look identical.
2. Find the condition and make it true. On a copy, change whatever the warning is about until it should fire. If it does not, the warning is broken and has been for eight months.
3. Check the ranges. A warning over rows 2 to 200 on a table that now reaches row 240 stops noticing anything happening in the last forty rows, silently.
4. Check for a warning that cannot be false. The mirror image: a cell that always shows text, which readers have learned to ignore. Both failures produce a dashboard nobody looks at.
5. Check the coverage block against reality once. Not the formulas — the world. Are there really three sites? Did a fourth open? A coverage block listing three sites cannot notice a fourth, exactly as the summary grid in the last lesson could not notice a new category.
6. Write down the date of the test. "Warnings verified 5 December, coverage list checked against the site register" is a line that belongs on the dashboard, and it is the only evidence that any of this was ever done.

The general shape: an indicator that has never been observed to change state carries no information. Making it change is a five-minute job that can be done on a copy, and it is the only way to distinguish a working dashboard from a decorative one.

Build the measures, then build the coverage block and the warnings, then choose what should be published.

The last check restores the missing registers and requires both warnings to clear. That check is the point of the exercise.

Take a dashboard or a regular report you receive — at work, from a service you use, from a device — and audit it in four questions.

1. Coverage. What proportion of the thing it claims to describe is actually in it? If it does not say, estimate it, and note that the estimate was yours.
2. Freshness. What is the as-of date? Is it on the screen, or did you have to ask?
3. Assumptions. Which numbers on it were measured and which were assumed? Anything derived from a rate, a capacity, a standard cost or a conversion factor is assumed.
4. Warnings. Is there anything on it that would look different if something were wrong? If yes, work out what would have to happen for it to change, and whether that has ever happened.

Question 4 usually has the answer "no". A report with no state that can change is a picture of numbers, and pictures do not tell you when to act.

Sign in to track your progress on this exercise.

This module is finished when you can do all of these.

• Name the four things a trustworthy dashboard carries, and say which one is most often missing.
• Explain what coverage means and why it is not the same as a total or a sample size.
• Write a warning cell that appears only when its condition holds, and say why it returns an empty string rather than a word.
• Say how to establish that a warning works, and why "it has never fired" is not evidence either way.
• Given a measure that rose 41.5% while coverage fell from three sites to one, write the headline the data supports.
• Say what an indicator that has never been observed to change state tells you.

Sign in to track your progress on this exercise.

A decision screen is finished when its measures answer the decision, its coverage says how much of the thing they describe, its assumptions and its as-of date are on the screen rather than in somebody's head, its warnings have been watched to clear at least once, and its headline is the sentence the data supports rather than the one the measures suggest.

That completes the fifth module, and with it everything the capstone needs.

The capstone is a real decision made badly by default: three suppliers, an order history nobody has cleaned, a rate table that disagrees with itself, and a budget that does not obviously cover any of the options. Three lessons: assemble the workbook, compare the alternatives, and deliver a recommendation somebody else can check.