Double-barreled questions are one of the most common but easily avoidable mistakes we see at PickFu. In this article, you’ll learn to recognize and avoid double-barreled questions.
What are double-barreled questions?
A double-barreled question, also known as a compound question or a double-direct question, is a question that touches upon two different issues, but only allows for a single answer. Essentially, a double-barreled question makes the mistake of combining what should be two questions into one.
Double-barreled question examples in surveys
The following are examples of double-barreled questions:
How satisfied are you with your work environment and compensation?
In this example, a respondent could potentially be satisfied with her work environment but feel she is underpaid. Or, she may be satisfied with her paycheck, but hate her workplace. So how should she answer this question? And how should a pollster interpret her answer? What did she truly intend to say?
Is this video clear and interesting?
In this example, clear and interesting are both positive attributes. However, the two attributes are not interchangeable. The video might be clear but dull at the same time. Or it might be interesting to watch but not make sense.
What’s the problem with double-barreled questions?
The problem with double-barreled questions is that they lead to confusion. Respondents are confused because they’ve been asked two questions, but aren’t sure which one to answer. Then, when it comes time to analyze the responses, survey creators are confused by the results. Because you can’t be sure which question the respondents answered, you also can’t be sure exactly what is being measured.
Sometimes survey creators ask a double-barreled question in order to make their surveys shorter. While this is well-intentioned, asking double-barreled questions returns inaccurate or unreliable results. This, in turn, makes the survey a waste of time and money. Worse, if the pollster doesn’t realize the results are skewed, the survey could lead them to make unwise business decisions, such as changing something that didn’t need to be changed or focusing on an area that isn’t the most pressing problem.
How to avoid double-barreled questions
Double-barreled questions are one of the most common surveying mistakes, but also probably the easiest to fix. You correct a double-barreled question by separating the combined question into two distinct questions.
How satisfied are you with your work environment and compensation?
How satisfied are you with your work environment?
How satisfied are you with your compensation?
Is the video clear and interesting?
Is the video clear?
Is the video interesting?
A shorthand way to identify double-barreled question is carefully examining the use of the word and. Though this is not a fool-proof test (the use of and does not always indicate a double-barreled question), it is a good watchword to double-check yourself.
Additionally, another way to correct a double-barreled question is to ask follow-up questions. For example, if you ask, “Is the video clear?” and the respondent answers affirmatively, you could then follow up by asking whether it was also interesting.
Double-barreled questions on PickFu
Sometimes, in the interest of maximizing a poll’s value, a surveyor will inadvertently ask a double-barreled question. Usually, the person will try to test a title and a subtitle in a single poll, or a logo and a tagline in a single poll.
But because a title is distinct from a subtitle, it’s judged by different criteria. Each has different considerations. Same goes for a logo and tagline. Or, in the case below, a logo and business name:
When a double-barreled question asks two separate things, respondents usually only respond to the one that means the most to them.
In the poll’s results, you can see the divide. Some respondents commented on both the name and the logo. Some respondents commented only on the name. Some only on the logo treatment. But even in cases where the respondent only reacted to one element, the reaction was likely subconsciously influenced by the other: Those who commented on the name were influenced by their feelings about the logo, and those who commented on the logo were influenced by their feelings about the name. And in the end, how did everyone determine which to vote for — by which name was better, or by which logo was better?
You only want to test one thing at a time. So if you’re testing your business name, test your business name only. Leave the logo design, tagline, and any other considerations for later tests.
Double-barreled questions with authors
Authors often make the mistake of asking double-barreled questions by testing their titles and subtitles together.
What’s the problem? Some people might like 101 Real Estate Tips You Wish You Knew (the title in Option A), but prefer “The Ultimate Collection of Tips, Tactics, and More for Getting a Great Deal” (the subtitle in Option B). So how should the reader judge the appeal?
To fix this double-barreled question, the author should test the two titles against each other, without a subtitle, i.e.:
Which book title is more appealing?
Option A: 101 Real Estate Tips You Wish You Knew
Option B: 101 Real Estate Tips for Buying a House
Then, once a title had been decided upon, the author could run a second test with the subtitle options paired with the winning title.
Let’s suppose that Option A won the first test. The second test would look like this:
Which subtitle is more appealing?
Option A: 101 Real Estate Tips You Wish You Knew: The Ultimate First-Time Homebuyer’s Collections if Tips, Tactics, and More to Get the Best Deal Possible
Option B: 101 Real Estate Tips You Wish You Knew: The Ultimate Collection of Tips, Tactics, and More for Getting a
A final example for authors
A second example shows an author who tried to test a subtitle and a book cover design at the same time.
Even though the design difference is subtle, some respondents reacted to the arrow design, while others responded to the subtitle text. And once again, those who only wrote responses to one element were likely still influenced by their feelings about the other. Every respondent used different criteria to make a choice, meaning that the winning vote tally is unreliable.
Here’s a better example of a book cover design poll because the design is the only variable. The title, subtitle, and author attribution are the same, so respondents are only reacting to the visual elements.
Have you made the mistake of asking a double-barreled question? If so, don’t feel bad — it’s a common error. But now that you know what a double-barreled question is, how to recognize one, and how to fix it, you can poll with confidence.
Want more tips? Read How to Write an Unbiased Poll Question now!
Also published on Medium.
Get 50 instant poll responses for $50.
Start uncovering actionable, audience driven data today.