Social desirability bias: why your best customer feedback may be your least honest

Part 1 of Trust Your Own Data, a four-part series on whether you can believe the numbers your business collects about itself.

Social desirability bias is the habit people have of answering a question the way they think will please or protect them, rather than the way they actually feel. It is quiet, it is close to universal, and it is probably sitting in your best survey results right now.

Here is where I ran into my own version of it.

For years I have built training and then stood in the room while people take it. After each session I survey the group, and the answers come back warm. Strong delivery. Useful. Would recommend. For a long time I filed that as evidence the training had worked. Then I would remember the newsrooms I started in, where an editor leaned over every green reporter and repeated the line I still carry: people lie, usually to be kind.

Look at the moment I collect those answers in. By the time the survey goes around, I have bought a good dinner, handed out the shirts and the certificates, and made a fuss over people who do not often get one. Some of them had never had a steak dinner, let alone one in Niagara Falls, until it went on my card. Then, a day or two later, I ask them what they thought, and I write the warm answers down as findings. How much of that is the training, and how much is the steak?

The instinct that produces a kind answer

None of those people are lying to me. They are giving the answer that fits the moment, which is a different thing. Researchers who study survey behaviour have a name for it. In a well-known 1985 review, the psychologist Anton Nederhof mapped how social desirability bias creeps into answers, and the methods researchers use to catch it and correct for it. People lean toward the flattering answer. They lean hardest when the person asking has just been generous to them, or holds some power over them.

That second half is the part I almost missed myself: who is holding the clipboard changes the answer. A researcher’s own position shapes what people are willing to tell them, which is why careful qualitative researchers are now expected to name their position out loud before they gather a word, a practice the field calls positionality. An employee rating their manager’s workshop is not in a neutral seat. A customer answering a survey stapled to a discount is not either. Neither am I, standing at the front of a room I just fed.

Where your business collects praise it manufactured

Once you see the pattern, you see it everywhere in the ordinary machinery of business feedback. The satisfaction survey goes out while the box is still new and the buyer still feels good about the decision. You measure your Net Promoter Score® right after a win, or right after you have waived a fee. You run the engagement survey, and the invitation comes from the boss whose team is being asked to grade him. You ask your happiest customer, the one who just thanked you, to leave the review.

Every one of those is real data about something. It is just not data about what you think. It is a faithful record of how people behave when they want to be kind or want to be safe, collected in the exact conditions most likely to produce a kind and safe answer. You did not set out to rig it. The instinct did that for you.

The part that unsettles me is that none of it feels like a mistake while it is happening. It arrives in a spreadsheet, it carries a number, and it flatters you, so you treat it as evidence and you act on it. That is how a company ends up genuinely confident in a strategy that rests on politeness.

When to distrust a warm answer

Before you act on a piece of feedback, look at the moment it was collected in. A few signs the answer may be kind rather than true:

• It arrived right after you gave something: a discount, a gift, a good dinner, a favour.
• The person asking holds some power over the person answering, like a manager, a landlord, or an instructor.
• The response was not truly anonymous, or the person answering could not be sure it was.
• It came from a single touchpoint at a single moment, with no way to check it against how people actually behaved later.

None of these makes the feedback worthless. Each one means the number is telling you about goodwill or safety, not necessarily about the thing you meant to measure.

What a number is actually worth

Here is the idea the rest of this series is built on. A piece of feedback is only as trustworthy as the moment it was collected in. Good evidence cannot be separated from the conditions that produced it, and so much business feedback is produced in moments that were built, without anyone intending it, to return a nice answer instead of a true one.

That does not mean your data is worthless. It means you have to know which kind you are holding. The score that comes back after the steak dinner tells you something honest about goodwill and something close to nothing about whether the work changed anyone’s behaviour. Those are different facts, and trouble starts the moment you mistake the first one for the second.

So before you trust any piece of feedback enough to build on it, run one question over it: was this collected in a moment designed to get a real answer, or a kind one? If you cannot say, you do not yet have evidence. You have a warm impression wearing a number’s clothes.

Seeing the problem is where it starts. In the rest of this series I get to the harder and more useful part: why consent is the safeguard most feedback quietly skips, the common ways businesses fool themselves with their own research, and the practical checks that make your numbers worth trusting. If you only read one, read the last.

Trust Your Own Data is a four-part series: (1) the steak-dinner problem, (2) consent you can trust, (3) how businesses fool themselves, (4) data worth trusting. It runs alongside my weekly newsletter, artificial intelligence (AI) Learning, Keeping It Human, where I take the same sober eye to the week’s AI and learning news. If you want to be the person in the building who can tell a real number from a flattering one, that is the whole project.

Christopher Ross builds training and advises businesses on measurement they can trust. If this landed, the two ways teaching with AI goes wrong comes at the same problem from the learning side. Or see how I can help.