Nonresponse Bias: What Missing Survey Answers Can Change

By Trellus Team · November 1, 2026

Eight of ten survey respondents say they want a callback worksheet. You invited twenty people. You can report that 80% of respondents said yes. You can't yet report that 80% of the invited group wants it.

The missing ten answers could change that conclusion. Nonresponse bias is the problem that arises when the people who answer differ from those who don't in ways that distort the result you're trying to estimate. Counting the replies tells you how much is missing; it doesn't tell you what the missing people would have said.

Start with the group your conclusion describes

A survey summary needs a target group. Are you describing everyone invited, everyone on the team, or only the people who answered? Those groups can have different opinions even when the calculation is correct.

If the question is about the whole team, first check who received an invitation. Leaving a regional team off the list is a coverage problem. Inviting them and receiving no answers from them is a nonresponse problem. A confusing question can create a measurement problem even among people who reply. Each calls for a different investigation.

AAPOR's survey definitions distinguish these sources of error. Its explanation also makes a useful point for ordinary workplace reports: the response rate alone can't establish whether nonresponse error exists or how large it is.

Work through a twenty-person survey

Here's a fictional example. An enablement manager has a list of twenty eligible reps and invites each person once. Everyone receives the same question: “Would you like a callback worksheet for your next call block?” The choices are yes and no. The survey has a stated closing time.

By that time, ten distinct people have answered. All ten answered this question: eight said yes and two said no. The other ten people provided no survey answers. Nobody's eligibility is uncertain, and there are no duplicate or partial replies in this example.

Two calculations are available. The response fraction is 10 ÷ 20 = 50%. The yes share among respondents is 8 ÷ 10 = 80%. They answer different questions, so keep both denominators in the report.

To calculate the yes share across all twenty invitees, you would need the other ten opinions. The following possibilities all fit the replies already received:

The smallest possible total is eight yes answers: the eight already known, with none among the missing people. The largest is eighteen: the eight known plus all ten missing people. Under this deliberately simple setup, the full-list share could therefore be anywhere from 40% to 90%.

That range lists what the missing answers could permit. It isn't a confidence interval or a prediction. It doesn't assign probabilities to the possibilities, and its midpoint isn't an estimate you can quietly put into a slide.

A low response fraction doesn't settle the direction

The middle row matters. If eight of the missing ten would also say yes, the full-list share would be 80%, exactly the respondent share. A 50% response fraction can coexist with no difference in this particular percentage.

In the first row, using the respondent share for everyone would overstate interest. In the last row, it would understate interest. The ten observed answers don't tell you which situation you're in.

Don't fill that gap with a story about people's motives. Silence doesn't establish dissatisfaction, indifference or agreement. Someone might not have seen the invitation; someone else might have chosen not to answer. The dataset hasn't supplied an explanation for any particular person.

These arithmetic limits concern this fixed invitation list only. If the manager also wants to describe future hires or other teams, that's a further question. Even complete answers from these twenty people wouldn't automatically describe a different population.

Keep skipped questions separate from missing people

A person can return a survey and leave one question blank. That's item nonresponse. When an eligible invited person provides no usable survey response, you're dealing with unit nonresponse. The Census Bureau's response-rate definitions explain unit nonresponse in the context of its American Community Survey; that survey has its own eligibility and interview rules.

For a workplace report, retain the number of usable answers to the particular item you're discussing. A count of returned forms may be larger than that item's denominator. Explain exclusions and partial answers rather than treating every returned form as an answer to every question.

The example above avoids those complications. Its 50% is a simple response fraction for twenty known eligible invitees, not a formula for a survey with unknown eligibility, partial interviews or mixed outcomes. Once those cases enter the data, define them before calculating a rate.

Look for a useful comparison before adding an explanation

Start with information the team already has permission to use. If you already have permitted aggregate counts for both the invitation list and the respondents, compare their region or role mix. A group that was invited but supplied few answers deserves attention in the interpretation.

That comparison can reveal an imbalance. It can't prove that respondents and nonrespondents with the same role have the same opinion. Matching a few visible characteristics doesn't make the missing answers known.

Choose the next step around the decision. If you need to learn where callback preparation breaks down, voluntary comments or an appropriately selected review of existing work may help identify concrete problems. The guide to choosing calls for coaching explains how to select useful examples. Those examples would inform a different question; they wouldn't turn the survey percentage into a whole-team estimate.

If an authorized survey owner decides that a reminder or another collection method is appropriate, keep the question, closing time and resulting counts clear. Respect a person's choice not to participate. An anonymous survey also needs an approach that preserves its anonymity; don't try to identify respondents from small group totals.

Use the answers you have for a proportionate decision

The fictional manager has learned something useful: eight respondents asked for a worksheet. Offering an optional worksheet can be a reasonable response to that expressed interest. A team-wide claim about demand requires more than those eight requests.

The survey also hasn't tested whether the worksheet improves follow-ups. A preference question supplies opinions about an offer. If the manager later evaluates the worksheet's use or results, that needs a separately defined observation and comparison.

Keep statistical precision separate too. A narrow interval calculated from a large set of respondents can't, by itself, establish that missing people are comparable. The small-sample uncertainty guide covers a model-based calculation for call outcomes; its precision calculation doesn't resolve missing survey opinions.

Copy a reporting note with the denominators intact

For the fictional survey, a useful note could read:

We invited twenty eligible reps once and closed the survey at the stated deadline. Ten people answered the callback-worksheet question: eight yes and two no. The response fraction was 50%; 80% of respondents said yes. Ten invitees' answers are unknown. The yes share across the full invitation list could be 40%–90% under this binary example. We haven't estimated where it lies in that range or established the direction of nonresponse bias.

For your own report, replace the example with five concrete details:

1. The target group, invitation list and collection deadline.
2. The number of eligible people invited and how duplicates or uncertain eligibility were handled.
3. The usable answer count for the specific question, plus the number of skipped or unusable answers.
4. The recorded answers and the denominator beside every percentage.
5. The conclusion still unsupported, and the next useful check if the decision needs one.

Don't replace missing answers with no. Don't assign the respondent average to them merely to complete the table. Weighting or imputation needs a defensible method and assumptions; an automatic fill won't supply the missing opinions.

A clear report can preserve an encouraging response without enlarging its audience. Eight people said yes. Start there, and keep the unanswered question about everyone else visible.