Precision and Recall: Read a Call-Intent Review Table
Use a fictional four-cell table to calculate precision and recall, compare review workloads and keep unresolved reference labels outside the result. Includes a reusable manual worksheet.
On this page
By Trellus Team ·
Precision starts with the calls that received a positive flag. Recall starts with the calls your reference review marked positive. A report can show 60% precision and 80% recall because the two calculations use different denominators.
Work through 100 fictional calls with the label “explicitly requests a callback.” The counts and dialogue below are invented. Use the worksheet to record your own review rule, included calls and original counts before interpreting a percentage.
Define the reference label before counting
For this exercise, one call is one item. A reference reviewer marks it positive only when the available conversation contains a clear request from the other person for a return phone call. A complete, understandable conversation without that request is reference-negative. Missing context or unclear audio makes the reference unresolved.
That rule is deliberately narrow. A request for a brochure is not a callback request. A positive tone is not one either. The label describes the reviewed conversation under this rule; it does not establish buying intent, qualification, permission for a future action or a completed booking.
Keep the rule fixed while comparing configurations. Agree on how reviewers will resolve disagreements, and record what supports each label. Don’t change the rule after seeing which configuration it favors. A reference label is a review judgment. It can be mistaken, so document how it was assigned.
If you need the broader background, intent detection explains the category, while call classification covers organizing call labels and outcomes. Here the task is narrower: read one binary review table correctly.
Put the 100 resolved calls into four cells
Imagine that 100 calls have both a resolved reference label and a yes-or-no flag from configuration A. The reference review identifies 15 callback requests and 85 calls without an explicit request. Configuration A flags 20 calls.
| Configuration A | Reference: callback requested | Reference: no explicit callback request | Row total |
|---|---|---|---|
| Flagged positive | 12 true positives (TP) | 8 false positives (FP) | 20 |
| Not flagged | 3 false negatives (FN) | 77 true negatives (TN) | 80 |
| Column total | 15 | 85 | 100 |
“True” and “false” here mean agreement or disagreement with the defined reference review. They are not judgments about the person or the value of the call. A false positive is a flag the reference does not support. A false negative is a reference-positive call the configuration did not flag.
Check the totals before doing the division. Every resolved call belongs in exactly one cell. The four cells must sum to 100, the flagged row to 20, and the reference-positive column to 15. Repeated rows or calls left out of the count will change the answer.
Precision starts with flags; recall starts with reference positives
Precision = TP / (TP + FP). It asks what share of positive flags agrees with the reference. Recall = TP / (TP + FN). It asks what share of reference positives received a positive flag. These definitions and the treatment of zero denominators are explained in Google’s classification metrics guide.
For configuration A, precision is 12 / (12 + 8) = 12 / 20 = 60%. Start with the 20 flagged calls: the reference supports 12 of them and does not support eight.
Recall is 12 / (12 + 3) = 12 / 15 = 80%. Start with the 15 reference-positive calls: 12 were flagged and three were missed. The numerator is the same 12; the denominator changes because the question changes.
A useful report sentence is: “In these 100 resolved calls, A flagged 20; 12 agreed with the callback-request reference, and A captured 12 of the 15 reference-positive calls.” That sentence keeps the counts visible. “A found 80% of interested buyers” changes the label and goes beyond the exercise.
A high accuracy figure can hide missed requests
Accuracy counts agreement across both positive and negative labels: (TP + TN) / total. A’s figure is (12 + 77) / 100 = 89%.
Now imagine a different configuration that never flags any of the same 100 calls. It agrees with all 85 reference negatives, giving 85% accuracy. It captures none of the 15 reference positives, giving 0 / 15 = 0% recall. Its precision has a zero denominator: 0 / 0 is undefined, not a demonstrated perfect score.
Show “not calculable: no positive flags” beside that precision result. Likewise, recall is not calculable when there are no reference positives. Keep the counts beside the explanation so the missing percentage has a clear meaning.
Compare the second configuration’s review workload
Keep the same 100 fictional calls and the same reference labels. A second illustrative configuration, B, has TP 14, FP 26, FN 1 and TN 59. The reference still has 15 positives and 85 negatives.
| Measure | A | B |
|---|---|---|
| Positive flags | 20 | 40 |
| Reference-supported flags | 12 | 14 |
| Unsupported flags | 8 | 26 |
| Missed reference positives | 3 | 1 |
| Precision | 12 / 20 = 60% | 14 / 40 = 35% |
| Recall | 12 / 15 = 80% | 14 / 15 = 93.33% |
B puts 40 flags in a potential review queue rather than 20. Its table contains more supported flags, more unsupported flags and fewer missed reference positives. Those are different consequences to discuss with the person responsible for the review process. Flag counts describe queue size here, not measured handling time or business value.
The totals don’t identify which individual calls changed labels, or whether B was made by lowering a threshold in A. The two configurations are invented for this calculation. To choose between real configurations, inspect the actual records, the work created by their flags and the consequences of each kind of disagreement. These counts establish neither a product’s controls nor a causal improvement.
Keep 12 unresolved calls outside the matrix
Suppose the eligible group also contains 12 calls whose reference labels could not be resolved. There are now 112 eligible calls, of which 100 have resolved references. Reference-review coverage is 100 / 112 = 89.29%.
The matrix still describes the 100 resolved calls. Do not put the other 12 in the negative column, assume they resemble the resolved calls or report the displayed precision and recall as results for all 112. Record why they remain unresolved and whether each had a flag, if that is known. Missing reference information limits what the table can establish.
Inspecting only flagged calls creates another gap. You can describe precision among the inspected flags, subject to how those flags were selected. You cannot find missed reference-positive calls without examining unflagged calls too. If a report contains no review of unflagged calls, ask where its false-negative count came from before accepting a recall figure.
For a real sampled review, keep the selection method and sample size with the results. A review of difficult calls selected for troubleshooting will answer a different question from a sample drawn across the calling program. The small call sample guide explains why a recorded share and an estimate for a wider group are different.
Apply the same rule to ordinary language
- “Please call me back Tuesday.” In a clear, complete fictional conversation, this meets the reference-positive rule. A positive flag is a TP; no positive flag is an FN.
- “Send the brochure by email.” With no callback request elsewhere in that complete fictional conversation, this is reference-negative. A positive callback flag is an FP; no positive flag is a TN.
- “Maybe after…” followed by missing audio. The available conversation does not resolve the reference. Record it as unresolved, outside the four cells, rather than guessing a positive or negative label.
Keep this manual review sheet beside the percentages
Copy the fields below into your report notes. Complete the definition and selection fields before interpreting the metric fields.
| Record | Fill in |
|---|---|
| Unit and positive label | One call or another stated unit; exact positive-label rule |
| Reference review | Reviewer method, evidence available, disagreement handling and unresolved rule |
| Group and selection | Eligible group, inclusion/exclusion rules and how calls were selected for review |
| Configuration | Exact fixed configuration represented; keep comparisons on the same reference group |
| Coverage | Eligible count; resolved count; unresolved count and reasons; resolved / eligible |
| Four cells | TP ___; FP ___; FN ___; TN ___; sum ___ |
| Precision | TP ___ / positive flags (TP + FP) ___ = ___; note a zero denominator |
| Recall | TP ___ / reference positives (TP + FN) ___ = ___; note a zero denominator |
| Review consequences | Flag count, unsupported flags, missed reference positives and unanswered questions |
Finish with a sentence naming the reviewed group and both denominators: “In these 100 resolved calls, 12 of 20 flags were supported, and 12 of 15 reference-positive cases were flagged.” If a needed count is missing, leave the metric unavailable and assign someone to resolve that gap. Another reader should be able to reproduce your percentage from the saved table.