The headline is the thesis statement. Do not require answers to survey questions. That is, never make any answer to any question required for completing a survey.
Never.
Don’t do it. Here’s why.
Humans should not be unwitting subjects in scientific experiments. Yet, our history is cluttered with examples of this. The Tuskeegee Experiment. The Nazis. I’m not going to recount those atrocities here. If you fancy yourself a responsible researcher, though, you should dig in to understand the deep meaning that led to a thorough research reckoning in the post-WWII era.
A group of researchers and policy leaders convened to discuss the ethics of research and practice. The resulting document, The Belmont Report, is the culmination of their efforts.
The Belmont Report specified three principles to guide research. First, beneficence is the tenet of doing no harm. But not just that; research should maximize benefits. Second, research needs to be just. Justice involves ensuring processes are fair and equal. It’s the third principle that’s especially relevant to survey questions here.
Respect for persons: People’s ability to choose needs to be prioritized. This is where the concept of informed consent comes in. But informed consent is more than saying “yes” at the beginning of a study. Informed consent can be revoked at any time for any reason.
Another implication is that researchers must be upfront and truthful with participants. People should know the reason they are participating and giving up their information. This is where informed consent comes in. This is also where those long, long terms of service agreements also feature.
But what about deception?
Many of the most famous studies in psychology used deception to mask what they were studying. The threshold for deception, though, needs to be justified by minimizing the risk to the participants. Things like the famous Milgram experiment would not fly at all today. Less dramatic but equally impactful studies on implicit bias in racial attitudes rely on masking the true hypothesis.
What can I do about my survey?
Look, we all like data. Good ideas all over the place could benefit from greater exposure. We like to collect data. There’s a survey at the end of this post! But, it is necessary to allow people the opportunity to opt out of any and all parts of any inquiry effort. That shows respect for persons.
Many of our friends are researchers and community-based activists who rely on collecting survey data to help learn, evaluate, and improve. And sometimes, to meet the requirements of the project sponsor or funder. But, those requirements do not revoke the need to show respect for persons.
What if I need the answer to a question? Get informed consent. And provide sufficient incentive for the information. My colleague Bob Atkins constantly emphasizes the importance of reimbursing people for their time. If we value the information we are getting, we should be willing to compensate justly for it.
Another digression. Sometimes, missing data tells you a lot of interesting things. If answers to a certain question are routinely missing, the patterns of that missingness may be related to some other unmeasured variable. But that’s a whole other post.
Okay, what if a funder requires SOME answer to a question?
My colleagues and I just ran into this problem. Answers to questions about gender, sexuality, and other potentially identifying information (especially in small communities) were required. We were able to push back and get I prefer not to answer as a potential option for questions. It’s not perfect, but it was a good middle-ground solution.
Look, the easy way is not always the right way. And frankly, requiring answers may not always yield the most accurate information.