Brier Score: Check Sales Probabilities Against Outcomes

The binary Brier score measures how far a set of probability forecasts was from what happened. For each forecast, subtract the outcome—1 if the event happened, 0 if it did not—square the difference and average the results. Lower is better on the same set of events.

For a sales team, the question might be: “When we recorded a 70% chance of winning by month-end, how well did that probability hold up?” The score lets you evaluate the probabilities you actually recorded. It doesn't tell you why a deal changed or how much revenue you missed.

This walkthrough uses six fictional opportunities and a fixed deadline. It is a spreadsheet exercise, not a Trellus report or a benchmark for sales teams.

Define the event before opening the spreadsheet

“Will this deal close?” is incomplete. A deal that closes in October can count as a success for an eventual-win forecast and a failure for a September-win forecast. Both records could be correct because they answer different questions.

In this example, a fictional team freezes its probabilities on September 1, 2026. Every row answers the same question: Will this opportunity be recorded as won by September 30, 2026, at 23:59:59 UTC? The team later checks the dated records after that deadline.

The outcome is 1 when that event occurred and 0 when it did not. An opportunity can still be open after September 30 and have outcome 0 for this particular question, provided the records establish that it was not won by the cutoff. A later win doesn't change the September result.

A missing or contradictory record is different. Keep an unresolved outcome separate until it can be established; don't turn missing information into 0. Record how many rows were excluded and why. A score on the cases you could resolve may not represent the ones you couldn't.

Keep the forecast as it stood on September 1. Replacing an original 70% with the final CRM probability would evaluate information that arrived later. If you want to score later snapshots too, label their dates and horizons separately.

Calculate the score, one row at a time

Here, p is the frozen probability between 0 and 1, y is the binary outcome and N is the number of eligible forecasts:

Brier score = sum of (p − y)² / N

This is the binary convention with a range from 0 to 1. A score of 0 means every forecast matched its outcome exactly. A score of 1 means every prediction was certain and wrong. The Japan Meteorological Agency's verification appendix gives this definition in section A.3.1.

Take a forecast of 0.8. If the event occurs, its squared error is (0.8 − 1)² = 0.04. If it doesn't, the squared error is (0.8 − 0)² = 0.64. A confident miss contributes more error than a hesitant miss.

Enter 80% as 0.8, not 80. Keep the positive event consistent: every probability below refers to winning by the deadline, never sometimes winning and sometimes losing.

Download the six-row example CSV to reproduce the calculation in a spreadsheet.

The six errors total 1.38. Divide by six to get 0.23. That is a mean squared probability error; it isn't a 23% loss rate, a 77% accuracy rate or an average error of 23 percentage points.

For a spreadsheet with probability in column D and outcome in E, put =(D2-E2)^2 in F2, fill it through F7 and use =AVERAGE(F2:F7). First check that all six outcomes are present and binary. A blank cell silently treated as zero can change the result.

Compare two forecasts on the same rows

For an arithmetic reference, suppose a second fictional method assigned 0.5 to each of these same six opportunities at the same snapshot. Every squared error would be 0.25, so its Brier score would be 0.25.

The recorded probabilities score 0.23 against 0.25 for that reference: a difference of 0.02 on these rows. This demonstrates the comparison. Six invented cases do not establish that a real forecasting process is better.

A useful operational comparison might be a rep's frozen estimate versus a stage-based estimate saved at the same time. Both methods need the same opportunity set, event definition and cutoff. Don't compare one method's easiest opportunities with another method's whole pipeline.

Choose a reference before seeing the outcomes. If you use a historical win frequency, establish it from an appropriate earlier period and keep that source period visible. Fitting a reference to the very outcomes being scored answers a different question from evaluating a forecast made in advance.

Different periods can have different event frequencies and difficulty. The JMA appendix also explains why the event frequency affects the score. A lower score in a new month isn't, by itself, proof that the forecasting method improved.

A lower score doesn't prove better calibration

Calibration asks whether events occur at the rates the probabilities suggest. Across many comparable 70% forecasts, you would look for roughly 70% observed wins. Discrimination asks whether the forecasts distinguish opportunities that experience the event from those that don't.

The scikit-learn probability-calibration guide explains that Brier loss reflects calibration, discrimination and uncertainty together. A lower total can come from better separation even when calibration is worse. You need more than the single score to diagnose the change.

Look at groups of similar probabilities alongside their row counts and observed event rates. If a group contains two opportunities, one outcome changes its observed rate by 50 percentage points. A dot on a calibration chart can look precise while resting on very little information.

Our six-row example is too small to establish reliable calibration. It is useful for checking the calculation and finding rows worth discussing. A02 contributes 0.49 because it had a 0.7 forecast and no win by the deadline. That doesn't establish dishonesty or a bad decision: a 70% event can fail to occur.

Probability calibration is also different from a call-review calibration session, where reviewers align how they apply a scoring rubric. Agreement between managers' call ratings doesn't validate opportunity win probabilities.

Keep the score attached to its question

Save the snapshot date, cutoff and timezone with the score, plus the event definition, eligible-row count, exclusions, probability source and comparison method. Those details let someone reproduce the result rather than argue about two numbers that describe different samples.

Give each opportunity equal weight for this example. Weighting by deal value would change which mistakes matter most and produce a different summary. Even an unweighted Brier score says nothing directly about the size of won or missed revenue. Keep revenue forecast error as a separate measure.

If you're choosing an overall method rather than evaluating saved probabilities, start with the broader sales forecasting techniques guide. If the problem is missing outcome history, fix the record first; an extra decimal place won't repair it.

For a calling workflow inside a CRM or sales engagement platform, apply the same distinction to recorded outcomes: a conversation, an agreed meeting and a won opportunity are different events. Pick the event you can actually establish, preserve what was known at the forecast date and calculate the score for that question.