When Two Customer References Give Different Answers

Suppose one customer reference says the software made their weekly review easier. Another says they still needed a spreadsheet. Before deciding which answer to trust, find out what each person needed the review to do.

Different experiences can both be useful. Ask how each team used the software, then compare that work with your own, including the tasks a cheerful recommendation might leave out. Keep the accounts separate, ask about the part that matters, and give the seller a specific question to answer.

You don't need to make the references agree. You need to understand whether the difficulty they describe would matter in your setup.

Compare the work behind the recommendation

“Would you recommend it?” can open a conversation, but the answer needs context. A manager may be pleased with a product that still requires substantial work from an administrator. Someone who maintains a complicated setup may describe a problem that your team would never encounter.

Ask what the person actually does with the software and which part they know firsthand. Were they involved in choosing it, getting it running, using it each day or maintaining it afterward? A job title won't answer that for you.

Then narrow the praise or criticism. If someone says reporting is difficult, ask which report they needed and what made it hard. If another person calls reporting excellent, ask what decision it helps them make. Those answers are easier to compare than two ratings.

Dates matter too. An account of a rollout several years ago may raise a useful question without describing the current product. Ask when the experience occurred; let the seller establish whether the relevant behavior has changed.

Two answers about the same weekly report

Consider a fictional buyer evaluating software for managing inbound sales enquiries. The sales lead wants a weekly view of requests that still need a response, without maintaining a separate list.

The first reference is a sales manager. Asked about reporting, they say, “We stopped keeping the side spreadsheet. The weekly review became much simpler.” Their team handles requests from one website form, and the manager checks whether each new request received a first response.

The second reference is an operations colleague at another company. They say, “We still maintained a spreadsheet because the report didn't give us the view we needed.” Their review includes website requests, partner introductions entered by the team and further enquiries from existing accounts. They need to distinguish a new request from another message about an older one.

The buyer initially wrote “reporting easy” beside the first conversation and “reporting unreliable” beside the second. Neither note is precise enough to guide the evaluation. The references were describing different work, and the second person hasn't necessarily said the underlying records were inaccurate.

The useful question becomes: would the proposed setup include all the request types this buyer needs to review, with a clear explanation of what remains outside it? That question belongs in a product check. Neither reference has answered it for the buyer.

The second account may be especially relevant if the buyer also receives partner introductions. The first still helps explain a simpler use case. There is no need to discard either experience or invent an explanation that makes the product look consistent.

Keep a comparison note that preserves the differences

Write the note around the requirement. Avoid a score that reduces each person to “positive” or “negative.” You may want to return to a detail after another part of the evaluation changes.

Decision we need to make

What part of our work would this experience help us assess?

What the reference described

Record the relevant observation in their terms. Mark a paraphrase as a paraphrase; don't turn remembered wording into a quotation.

Conditions that mattered

Which request types, responsibilities, period or setup shaped that experience? Leave unconfirmed details open.

Why it matters here

Where does our work resemble that situation, and where does it differ?

Remaining question

What would we still need to learn before relying on the recommendation?

Next check

Who has agreed to answer it, and what would a useful answer show?

In the fictional example, the note could say: “One reference removed a spreadsheet used for first-response reviews of website enquiries. Another retained a spreadsheet for a review spanning several request types. Our requirement includes partner introductions. Need the seller to show how those requests would appear in the intended weekly view, including anything requiring separate work.”

Keep the original conversation notes in the place and within the sharing arrangements your team agreed. The comparison should help colleagues make sense of the evidence, rather than become a new collection of private customer material.

Ask about the incident that would change your decision

A follow-up is useful when it can fill an important gap. Ask permission through the agreed introduction process and keep the request small. The reference has already given you time; they don't owe you a second evaluation meeting.

For the fictional operations colleague, you might ask: “Could you describe one request that needed the separate spreadsheet? What did you need to know that the report didn't show?” That leaves room for a missing input, a different reporting question, a workaround or a limitation.

Listen for the work around the problem. Who noticed it? Who corrected the record? Did the difficulty recur, or was it part of getting started? You are trying to understand what happened, not collect a dramatic quotation for your internal presentation.

Don't ask a participant to defend their experience against another customer's account. You can ask about your own requirement without naming the other person or forwarding their notes. If a document or screenshot would help, let the reference decide what they can appropriately share; a description may be enough.

When no follow-up is available, preserve the unanswered question. If you need another introduction, the guide to arranging a relevant customer reference covers matching the experience and obtaining agreement. Another enthusiastic customer is useful only if they can address something you still need to learn.

Give the seller a question they can verify

The seller should help explain the product, not decide which customer you are allowed to believe. Ask them to separate a confirmed difference in configuration or product version from an explanation they haven't checked.

“That customer set it up incorrectly” is not a complete answer. What was different, how has that been established, and what would the buyer have to do to avoid the same difficulty? A workaround may be reasonable, but someone still needs to own it.

For the fictional reporting requirement, propose harmless example requests: a new website enquiry, a partner introduction and another enquiry from an existing account. Ask what the intended review would include and how someone would notice an omission. Use those examples to make the requirement concrete. The seller still needs to show what the current product can do.

If the seller doesn't know, use a specific product-question handoff. Keep the required result intact while someone checks it. A promise to investigate is useful progress, but it doesn't establish that the report will meet the requirement.

Decide what the feedback changes

Sometimes the difference is harmless. You needed a straightforward first-response review, and the harder case concerns a process you don't use. Record that reason for giving the difficult experience less weight. “Our situation is different” should identify the difference, rather than dismiss an inconvenient answer.

Sometimes it exposes work you had overlooked. If a colleague would have to reconcile requests every week, ask whether they can assess that task. Give them the requirement and the relevant context, following the new-attendee evaluation guide. Their practical objection may matter more than another general recommendation.

A genuine limitation may also be a reason to stop. You don't have to keep arranging references until the balance sounds positive. Equally, unresolved feedback is not proof that the product is unsuitable. Decide whether there is a feasible check that could settle an important requirement, and who would do it.

If the vendor selected the references, don't treat the conversations as a survey of all its users. Counting two recommendations against one concern doesn't tell you how frequently the concern occurs. It can still reveal a difficulty important enough to investigate.

Write the conclusion you can use

For the fictional buyer, a useful evaluation update might read: “The references described different reporting needs. The simpler case supports the usefulness of a first-response review. The broader case raises an unresolved question about partner introductions, which we also need to include. Our operations colleague will review the seller's example before we decide whether the extra work is acceptable.”

That update gives the next discussion a purpose. It doesn't say the software has passed, that the critical reference was wrong or that the buyer has agreed to proceed.

After the check, add what was actually established. You might accept a known extra step, reject an unsuitable workflow or leave the requirement unresolved. The references have done something valuable in each case: they have helped you see the work behind the recommendation.