When researchers surveyed 1,690 male US adolescents about drug use and sexual behaviour in 1998, the survey mode decided the result: when the questions came from a computer through headphones rather than from a paper form, reporting rates for the most sensitive behaviours rose by a factor of 3 or more. The study appeared in Science, a quarter of a century before AI interviews existed. The mechanism behind it shapes the quality of qualitative data more than any choice of model.
I have known this moment for over 20 years: the interview is over, the recorder is off, one hand is already on the door handle, and the participant says: "One more thing..." What follows is often more honest than the 45 minutes before it.
The reason sits deeper than the sincerity of the person. As soon as a human is listening, we answer two questions at once: the one that was asked and the unasked one, namely what the person opposite is thinking about us right now. The second wins more often than we researchers would like.
Why are people more honest with an AI? Because the social judgement of the counterpart falls away. As soon as a human is listening, people edit their answers towards what is socially desirable; Tourangeau and Yan (2007) describe this polishing of one's own answers (social desirability bias) as a process that takes hold above all in the presence of an interviewer. Once the perceived evaluator is gone, the answer moves closer to the truth - documented meta-analytically since the 1990s and the central quality advantage of AI-led interviews.
The interviewer is part of the measurement
Survey research has known for over half a century that who asks changes the answer. Schuman and Converse showed in 1971 that Black respondents in Detroit answered differently depending on whether a Black or a white interviewer sat across from them - most clearly on the most politically charged questions, hardly at all on pure questions of fact.
Distortions like these are a property of the conversation itself, not a craft error. Age, gender, demeanour, the company logo on the badge: everything about the person opposite signals which answer is likely to land well here; 50 years of survey research never managed to train that away completely.
I have seen this from the inside often enough. When the product owner sits in on a usability test, the closing verdict comes out reliably friendlier than the minutes before, in which the same person failed at the product.
The line of research begins in 1969
Whether people confide more in a machine than in a human is a question research examined long before ChatGPT. A first meta-analysis summarised in 1996 what 39 studies had measured since 1969: where a computer asked the questions, people disclosed more about themselves, most clearly in comparison with the face-to-face interview.
The hardest single piece of evidence came from the study in Science mentioned at the start. Its 1,690 adolescents were randomly assigned to survey modes and received identical questions; for injecting drug use and other highly sensitive behaviours, the computer group's reporting rates exceeded the paper group's by a factor of 3 or more.
Where exactly the gain comes from is what Richman, Kiesler and colleagues measured in 1999 across 61 studies: between computer and paper questionnaire there was on average almost nothing (effect size 0.05), while the drop in socially desirable answering appeared where the computer replaced the face-to-face interview (-0.19). The computer wins against the observing human, not against the paper.
That the Science study still found a factor of 3 against paper resolves itself in the details: its adolescents answered the most sensitive questions of all, through headphones instead of on a form that parents or an interviewer in the room could see; Richman's paper comparisons mostly measured harmless personality scales. The more sensitive the question and the more present a potential onlooker, the wider the gap.
A 2015 meta-analysis with data from 125,672 respondents found exactly this layering again: the largest effects on the most sensitive topics and where people were surveyed one at a time. Ask about someone's favourite feature and the survey medium barely matters; ask about money or failure and it decides the result.
The observer in your head makes the difference
How the mechanism works in detail is something an experiment at the University of Southern California isolated. There, 239 adults talked to the virtual interviewer Ellie about difficult subjects such as mental health; the technology was identical for everyone. The only thing that varied was what participants believed: one half took Ellie for a fully automated computer, the other assumed a human was observing and steering the figure.
Those who believed they were alone with a machine showed less fear of being judged, engaged in less impression management and allowed more sadness; independent observers rated these conversations as more open. Same software, same questions, a different audience in the head.
For me this is the core finding of the entire line of research: the distortion arises from perceived human judgement, and the technology behind it is interchangeable. That is why every model upgrade attacks the wrong spot as long as the setting places an evaluator in the room with the respondent.
Where the advantage tips
The social reflex never disappears entirely, even in front of screens. Nass and Moon documented in 2000 that people mindlessly apply social rules to computers: they are polite to machines, apply stereotypes, return favours. The bias gets smaller, not zero.
And it returns as soon as the machine gets a face. In a laboratory experiment by Lind, Schober and colleagues (2013), 235 participants disclosed most under a plain audio interface, more even than towards two animated virtual interviewers. The mere hint of a watching face was enough to push openness down.
Then there is what human conversations are simply better at. At a music festival, researchers had 286 visitors talk about intimate topics with either a chatbot or a human: fear of being judged was lower with the bot, but visitors trusted the human more - and in the coded analysis they even told the human the more intimate things. Rapport, genuine interest, the feel for the moment when silence draws out more than a follow-up question: that remains human terrain.
We all know this debate, usually under the heading "AI interviews are soulless". For topics that need a relationship, say grief counselling or executive coaching, I would still send a human into the conversation; for topics that touch on shame, the evidence says otherwise - and the biggest knowledge gaps in product and CX work sit exactly there: money, health, failure, frustration with your own employer.
What this means for the design of AI interviews
Concrete concept decisions follow from this research, and several of them went directly into QUALLEE.
Our interviews are deliberately a conversation in text form, with no animated face and no humanlike avatar. The Lind study is the reason.
Participants always know that an AI is asking; since 2 August 2026 the EU AI Act requires this anyway, and the Ellie experiment suggests that precisely this clarity is what creates the advantage in the first place. Our analysis works on patterns across all conversations rather than named individual profiles - because anyone who has to fear that their statement will land on their manager's desk with their name attached starts editing again.
The follow-up questions, in turn, dig into the substance: adaptive, specific, and persistent until the example is on the table. Why a conversation reaches places a form field never will is something I have described in detail here.
That leaves the question of whether participants want the format at all. In a working paper by Chopra and Haaland that analysed 381 AI-led text interviews on why people do not invest in shares, 71 percent of respondents preferred the AI interviewer to a human one after their own conversation; over 96 percent would take part in such an interview again.
The industry currently spends a lot of energy debating models: which LLM asks better follow-up questions, understands more deeply, summarises more cleanly. The older, bigger lever sits in the setting itself. No model upgrade brings back a truth that politeness removed from the conversation.
The uncomfortable consequence lands on us researchers in the end. Once the polite version falls away, we get answers that hurt: about products, processes, leadership. The sentence that used to come out only at the door handle now sits in the report - and the real question is whether your organisation is ready to read it.
Frequently asked questions about honesty in AI interviews
Are people really more honest with an AI than with a human?
On sensitive topics, yes: the meta-analyses of the 1990s already found the clearest effects where a computer replaced the face-to-face interview - people disclose more about themselves as soon as no human is asking. On everyday, harmless topics the difference is small to unmeasurable.
Why does social judgement fall away with an AI?
What distorts answers most is the fear of being evaluated, and that fear is tied to a perceived human observer. In the USC experiment, the mere belief that a human was watching was enough to increase impression management and reduce openness - with identical technology.
Does social desirability bias disappear completely with an AI?
No. People carry social reflexes such as politeness over to machines; the effect shrinks but does not vanish. It rises again as soon as the interface gets a face or leans heavily on human likeness.
Where do human-led interviews remain superior?
Wherever relationship and rapport form the core of the method: contextual observation, co-creation, grief counselling, coaching. A human counterpart can captivate, comfort and stay silent at the right moment; an AI interview scores where precisely this social presence pushes honesty down.
How does QUALLEE put these findings into practice?
We run text-based interviews without a face avatar, label the AI transparently, analyse patterns instead of individual profiles and let the follow-up questions dig into the substance. With us, the honesty of the answers is a design decision.
Sources
- Weisband, S. & Kiesler, S. (1996): Self Disclosure on Computer Forms: Meta-Analysis and Implications. CHI '96.
- Turner, C. F. et al. (1998): Adolescent Sexual Behavior, Drug Use, and Violence: Increased Reporting with Computer Survey Technology. Science, 280(5365).
- Richman, W. L., Kiesler, S., Weisband, S. & Drasgow, F. (1999): A Meta-Analytic Study of Social Desirability Distortion. Journal of Applied Psychology, 84(5).
- Gnambs, T. & Kaspar, K. (2015): Disclosure of Sensitive Behaviors Across Self-Administered Survey Modes: A Meta-Analysis. Behavior Research Methods, 47(4).
- Tourangeau, R. & Yan, T. (2007): Sensitive Questions in Surveys. Psychological Bulletin, 133(5).
- Schuman, H. & Converse, J. M. (1971): The Effects of Black and White Interviewers on Black Responses in 1968. Public Opinion Quarterly, 35(1).
- Lucas, G. M., Gratch, J., King, A. & Morency, L.-P. (2014): It's Only a Computer: Virtual Humans Increase Willingness to Disclose. Computers in Human Behavior, 37.
- Lind, L. H., Schober, M. F., Conrad, F. G. & Reichert, H. (2013): Why Do Survey Respondents Disclose More When Computers Ask the Questions?. Public Opinion Quarterly, 77(4).
- Nass, C. & Moon, Y. (2000): Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues, 56(1).
- Croes, E. A. J., Antheunis, M. L., van der Lee, C. & de Wit, J. (2024): Digital Confessions: The Willingness to Disclose Intimate Information to a Chatbot. Interacting with Computers, 36(5).
- Chopra, F. & Haaland, I. (2023): Conducting Qualitative Interviews with AI. CESifo Working Paper.
Try it yourself
The quickest way to find out how a conversation without social judgement feels is to have one yourself: 20 minutes, no login, right in the browser. And if you want to run your own interviews afterwards: the first three are free.
