Ecological Fallacy: What Team Totals Leave Unknown
Two tables can fit the same team totals and give different answers about individual reps. Check the missing connection before writing the conclusion.
On this page
On this page
- Name the record behind the number
- A fictional summary with one missing connection
- The same totals fit two different pairings
- State what the totals do constrain
- Return to linked records when the question needs them
- A linked result still needs a careful interpretation
- Use this check before writing about a rep
By Trellus Team ·
Suppose a team report says six reps recorded a follow-up plan and four met an agreed deadline. It doesn't say whether the four were among the six. Before describing what the plan writers did, check whether those facts are actually linked to the same people.
This is a useful place to recognize the ecological fallacy: carrying a conclusion from group-level information into a claim about individuals without enough support. A correct team summary can leave an individual question unanswered.
You can keep the summary. Just make the next decision at the level the available records can support.
Name the record behind the number
Start by asking what one row represents. It might be a rep, a call attempt, a conversation, an account or a whole team. Those are different units, even when the report uses the same label, such as “follow-up rate.”
Suppose a team has a high share of completed follow-ups. That describes the included work under the report's definition. It doesn't establish that a particular rep completed theirs. Nor does a comparison between team averages, by itself, establish the relationship between two behaviors among individual reps.
The EU's data visualisation guide describes the mistake of making an individual claim from group averages. David Freedman's Berkeley report on ecological inference addresses the related assumption that relationships observed for groups necessarily hold for individuals. Neither source provides a finding about sales teams.
The practical question is whether your records contain the connection your sentence assumes. “The team recorded plans and completed follow-ups” is different from “the reps who recorded plans completed their follow-ups.” The second sentence needs to connect those facts for the same reps.
A fictional summary with one missing connection
Imagine ten reps in one reporting week. Each has one preselected, agreed follow-up due during that week. For every rep, the team knows whether a written follow-up plan was recorded before the week began and whether that particular follow-up was completed by its agreed due time.
These are invented records for a calculation. They aren't customer data, a recommended planning routine or evidence that writing a plan changes a result. “Plan recorded” also doesn't establish that the rep used the plan.
The report presents only these separate totals:
| Recorded fact | Reps | Share of the ten reps |
|---|---|---|
| A plan was recorded before the week | 6 | 60% |
| The selected follow-up was completed on time | 4 | 40% |
Both totals are accurate, and both concern the same people. Yet they don't reveal how many plan writers met the deadline. The original records contain that connection; this summary has left it out.
The same totals fit two different pairings
In one possible arrangement, all four on-time follow-ups belong to reps who recorded a plan:
| Rep classification | On time | Not on time | Total |
|---|---|---|---|
| Plan recorded | 4 | 2 | 6 |
| No plan recorded | 0 | 4 | 4 |
| Total | 4 | 6 | 10 |
In another possible arrangement, all four belong to reps who didn't record a plan:
| Rep classification | On time | Not on time | Total |
|---|---|---|---|
| Plan recorded | 0 | 6 | 6 |
| No plan recorded | 4 | 0 | 4 |
| Total | 4 | 6 | 10 |
Check the edges of both tables. Six reps recorded a plan, four didn't, four completed the selected follow-up on time and six didn't. Each arrangement produces the original 60% and 40% summary.
The tables are alternative possibilities, not two reporting periods or an experiment. We haven't shown that either arrangement actually occurred. They demonstrate the information missing from the separate totals.
State what the totals do constrain
The overlap could be anywhere from zero to four reps. Four is the largest possible overlap because only four follow-ups were on time. Zero is possible because all four on-time results could belong to the four reps without a recorded plan.
That range describes possible counts consistent with this summary. It isn't a confidence interval, an estimate of the most likely count or a probability that planning helps. Don't choose the midpoint and report that two plan writers met the deadline.
The same restraint applies to names. Membership in the team doesn't tell you which result belongs to a particular rep. You need the relevant person's record before attributing an outcome to them.
Here, the totals rule out an overlap of five or more. Other totals may narrow the possibilities further. State those limits without claiming the missing connection.
Return to linked records when the question needs them
If the question is “How many plan writers completed their selected follow-up on time?”, return to the permitted records for these same ten reps. For each rep, check whether a plan was recorded beforehand and whether the selected follow-up was on time. Keep the reporting week and inclusion rule unchanged.
A manual review might need only a rep reference, the two classifications and the source of each. It doesn't require copying prospect details, full call notes or unrelated personnel information into another document. Use the access and sharing arrangements your team already has.
Check that you're matching the right work. A plan recorded this week shouldn't be paired with a follow-up from another period merely because the rep's name matches. Two people with the same display name also aren't necessarily the same person.
Keep “not known” separate from “no.” If a plan record is unavailable, you haven't established that no plan was recorded. Likewise, a missing completion timestamp doesn't automatically establish that the follow-up was late. The fictional example assumes complete classifications; a real review may need to stop and resolve a gap.
Once the linked counts are available, label the population carefully. A share of reps is not a share of all their calls. If you need to combine call outcomes with unequal amounts of work, use the guide to pooling call counts for that different question.
A linked result still needs a careful interpretation
Suppose the permitted records eventually show the first arrangement. You can report that four of the six plan writers completed the selected follow-up on time in that week. The linkage has answered the descriptive question.
It hasn't established why those outcomes occurred. The follow-ups may have differed, the reps may have had different support, and recording a plan wasn't assigned as an experiment. Don't turn a completed table into a claim that the plan caused better performance.
Nor does the table tell you whether someone demonstrated a particular coached skill. To review an observed behavior, define when it was relevant and what would count as use. The coaching adoption guide covers that separate task.
If you want to generalize beyond the observed work, there is another question about how much the available observations support. The small-sample uncertainty guide explains a model-based calculation for call outcomes. More precise estimates alone can't recover a connection omitted from the data supplied to them.
Use this check before writing about a rep
Read the proposed conclusion beside these questions:
- What does one source row represent, and what does the conclusion describe?
- Are the facts connected for the same people or work, or do we have separate group totals?
- Which individual conclusion remains possible but unconfirmed?
- What is the smallest permitted record check that would answer the actual question?
For the fictional summary, a useful reporting note could read:
Ten reps each had one selected follow-up due in the reporting week. Six recorded a plan beforehand; four completed the selected follow-up on time. The summary doesn't connect the two classifications. Between zero and four reps could appear in both totals. We haven't established which plan writers met the deadline.
If an authorized reviewer needs that connection, assign the specific record check. If the report only needs the team's separate totals, keep those totals and remove the individual claim. There is no reason to gather more personal information merely to decorate a summary.
A team total can be a useful description of the team. Let the individual record answer the question about the individual.