This guide will help you recognize whether biases may be influencing your results. Avoid these common mistakes and poll respondents will answer more openly and honestly.<\/p>\n\n\n\n\n\n\n
Contents<\/h2>\n\n\n- Contents<\/a><\/li>
- Polling Question Mistake #1: Leading words<\/a><\/li>
- Polling Question Mistake #2: Making assumptions<\/a><\/li>
- Polling Question Mistake #3: The double-barreled question<\/a><\/li>
- How to create good unbiased survey questions with PickFu<\/a><\/li>
- Summary: Unbiased questions<\/a><\/li><\/ul>\n\n\n
Polling Question Mistake #1: Leading words<\/h2>\n\n\n
Your question may include a positive or negative bias \u2014 words that consciously or unconsciously lead the respondents toward a certain kind of answer.<\/p>\n\n\n\n
Examples of biased survey questions: leading words<\/strong><\/p>\n\n\n\n\u2022 How much did you enjoy this YouTube video?<\/em> (positive bias – implies that the viewer enjoyed it, and leads respondents to answer more favorably)
\u2022 Should responsible parents vaccinate their children?<\/em> (puts respondents on the defensive by insinuating that parents who do not vaccinate are irresponsible)
\u2022 A recent poll found 80% of Americans disagreed with this government policy. How unhappy are you about this policy?<\/em> (negative bias – not only is the question phrased negatively, but it also includes a statistic that shows many unfavorable views, leading the respondent to feel as though he or she should feel that way, too.)
How to fix leading words<\/strong><\/p>\n\n\n\nRemove leading words and phrases and structure the questions as objectively as you can:<\/p>\n\n\n\n
\u2022 On a scale of 1-5, with 1 being the worst and 5 being the best, please rate this YouTube video.<\/em>
\u2022 Do you think children’s vaccinations should be required?<\/em>
\u2022 Please rate your level of agreement with this government policy. <\/em><\/p>\n\n\n\nRemember, biased questions lead to biased results<\/strong>. And if you’re using polling to make important business decisions, you want your feedback to be as objective as it can be.<\/p>\n\n\nPolling Question Mistake #2: Making assumptions<\/h2>\n\n\n
Sometimes questions include an opinion. For example, “Do you agree that dog owners should be able to walk their dogs off-leash in the park?” This assumes that dog owners want<\/em> to walk their dogs without leashes, which may or may not be true. But by including this assumption, you are leading more people to respond in agreement. <\/p>\n\n\n\nExamples of biased survey questions:<\/strong> Making Assumptions<\/strong><\/p>\n\n\n\n\u2022 Is your favorite color blue?<\/em> (assumes the respondent’s favorite color is blue)
\u2022 As long as nobody minds, is it okay to smoke indoors?<\/em> (assumes that nobody minds)
\u2022 Where do you like to party?<\/em> (assumes respondents like to party)<\/p>\n\n\n\nHow to fix assumptions in poll questions<\/strong><\/p>\n\n\n\nStructure your questions around facts, not opinions. Make sure you’re not painting respondents into a corner so that they can only answer one way. In the first example, it would be better to ask, “Do you think dog owners should be allowed to walk their dogs off-leash in the park?” Similarly, these questions eliminate assumptions:<\/p>\n\n\n\n
\u2022 What color do you like the best?<\/em>
\u2022 How do you feel about smoking indoors?<\/em>
\u2022 What do you like to do in the evenings?<\/em><\/p>\n\n\nPolling Question Mistake #3: The double-barreled question<\/a><\/h2>\n\n\nSometimes questions ask about two disparate things but only allow for a single answer. These double-barreled questions do not make good survey questions because respondents will often only concentrate on the one topic that means the most to them.<\/p>\n\n\n\n
Examples of biased survey questions: double-barreled questions<\/strong><\/p>\n\n\n\n\u2022 How do you feel about our two new flavors, French Vanilla and Hazelnut Cream?<\/em>
\u2022 How have teachers and students responded to the new dress code?<\/em>
\u2022 Which title and subtitle do you prefer?<\/em>
How to fix double-barreled questions<\/a> in polls<\/strong><\/p>\n\n\n\nDouble-barreled questions should be separated into two distinct questions and polled separately from one another. When questions are worded similarly in a multi-question poll, it’s also good to distinguish them with underlined words or italics, as below:<\/p>\n\n\n\n
\u2022 How do you feel about our new flavor French Vanilla<\/em>?
\u2022 How do you feel about our new flavor Hazelnut Cream<\/em>?
\u2022 How have teachers<\/em> responded to the new dress code?
\u2022 How have students<\/em> responded to the new dress code?
\u2022 Which title<\/em> do you prefer?
\u2022 Which subtitle<\/em> do you prefer?<\/p>\n\n\nHow to create good unbiased survey <\/a>questions with PickFu<\/h2>\n\n\nThe same rules apply to preference testing on PickFu as in general surveying: avoid leading words, don’t make assumptions, and only ask about one thing at a time.<\/strong><\/p>\n\n\n\n![\"\"](\"https:\/\/www.pickfu.com\/blog\/wp-content\/uploads\/2017\/03\/app-icons.png\")
<\/a><\/figure><\/div>\n\n\nFor example, if you’re testing mobile app icons, the simplest unbiased question would be, “Which app icon do you prefer?” To embellish the question with anything else may introduce bias. Sometimes pollsters want to know whether their icons read a certain way, so they’ll ask something like, “Which icon seems more kid-friendly?” While not biased, per se, this question asks respondents to judge your icons differently than they might otherwise.<\/p>\n\n\n\n
Because PickFu respondents include comments, one approach to avoid any hint of bias is to ask the simple question, “Which app icon do you prefer?” and then read the comments to see if kid-friendliness is something the respondents took away by themselves, rather than being led to see kid-friendliness or not.<\/p>\n\n\n\n
Another approach is to include factual statements (unbiased, of course!) to explain what you’re testing, and then ask the respondent’s preference. For example, “Our app is a game aimed at children under 10. Which icon do you prefer?” <\/p>\n\n\n\n
Polling Question Mistake #1: Leading words<\/h2>\n\n\n
Your question may include a positive or negative bias \u2014 words that consciously or unconsciously lead the respondents toward a certain kind of answer.<\/p>\n\n\n\n
Examples of biased survey questions: leading words<\/strong><\/p>\n\n\n\n \u2022 How much did you enjoy this YouTube video?<\/em> (positive bias – implies that the viewer enjoyed it, and leads respondents to answer more favorably) Remove leading words and phrases and structure the questions as objectively as you can:<\/p>\n\n\n\n \u2022 On a scale of 1-5, with 1 being the worst and 5 being the best, please rate this YouTube video.<\/em> Remember, biased questions lead to biased results<\/strong>. And if you’re using polling to make important business decisions, you want your feedback to be as objective as it can be.<\/p>\n\n\n Sometimes questions include an opinion. For example, “Do you agree that dog owners should be able to walk their dogs off-leash in the park?” This assumes that dog owners want<\/em> to walk their dogs without leashes, which may or may not be true. But by including this assumption, you are leading more people to respond in agreement. <\/p>\n\n\n\n Examples of biased survey questions:<\/strong> Making Assumptions<\/strong><\/p>\n\n\n\n \u2022 Is your favorite color blue?<\/em> (assumes the respondent’s favorite color is blue) How to fix assumptions in poll questions<\/strong><\/p>\n\n\n\n Structure your questions around facts, not opinions. Make sure you’re not painting respondents into a corner so that they can only answer one way. In the first example, it would be better to ask, “Do you think dog owners should be allowed to walk their dogs off-leash in the park?” Similarly, these questions eliminate assumptions:<\/p>\n\n\n\n \u2022 What color do you like the best?<\/em> Sometimes questions ask about two disparate things but only allow for a single answer. These double-barreled questions do not make good survey questions because respondents will often only concentrate on the one topic that means the most to them.<\/p>\n\n\n\n Examples of biased survey questions: double-barreled questions<\/strong><\/p>\n\n\n\n \u2022 How do you feel about our two new flavors, French Vanilla and Hazelnut Cream?<\/em> Double-barreled questions should be separated into two distinct questions and polled separately from one another. When questions are worded similarly in a multi-question poll, it’s also good to distinguish them with underlined words or italics, as below:<\/p>\n\n\n\n \u2022 How do you feel about our new flavor French Vanilla<\/em>? The same rules apply to preference testing on PickFu as in general surveying: avoid leading words, don’t make assumptions, and only ask about one thing at a time.<\/strong><\/p>\n\n\n For example, if you’re testing mobile app icons, the simplest unbiased question would be, “Which app icon do you prefer?” To embellish the question with anything else may introduce bias. Sometimes pollsters want to know whether their icons read a certain way, so they’ll ask something like, “Which icon seems more kid-friendly?” While not biased, per se, this question asks respondents to judge your icons differently than they might otherwise.<\/p>\n\n\n\n Because PickFu respondents include comments, one approach to avoid any hint of bias is to ask the simple question, “Which app icon do you prefer?” and then read the comments to see if kid-friendliness is something the respondents took away by themselves, rather than being led to see kid-friendliness or not.<\/p>\n\n\n\n Another approach is to include factual statements (unbiased, of course!) to explain what you’re testing, and then ask the respondent’s preference. For example, “Our app is a game aimed at children under 10. Which icon do you prefer?” <\/p>\n\n\n\n
\u2022 Should responsible parents vaccinate their children?<\/em> (puts respondents on the defensive by insinuating that parents who do not vaccinate are irresponsible)
\u2022 A recent poll found 80% of Americans disagreed with this government policy. How unhappy are you about this policy?<\/em> (negative bias – not only is the question phrased negatively, but it also includes a statistic that shows many unfavorable views, leading the respondent to feel as though he or she should feel that way, too.)
How to fix leading words<\/strong><\/p>\n\n\n\n
\u2022 Do you think children’s vaccinations should be required?<\/em>
\u2022 Please rate your level of agreement with this government policy. <\/em><\/p>\n\n\n\nPolling Question Mistake #2: Making assumptions<\/h2>\n\n\n
\u2022 As long as nobody minds, is it okay to smoke indoors?<\/em> (assumes that nobody minds)
\u2022 Where do you like to party?<\/em> (assumes respondents like to party)<\/p>\n\n\n\n
\u2022 How do you feel about smoking indoors?<\/em>
\u2022 What do you like to do in the evenings?<\/em><\/p>\n\n\nPolling Question Mistake #3: The double-barreled question<\/a><\/h2>\n\n\n
\u2022 How have teachers and students responded to the new dress code?<\/em>
\u2022 Which title and subtitle do you prefer?<\/em>
How to fix double-barreled questions<\/a> in polls<\/strong><\/p>\n\n\n\n
\u2022 How do you feel about our new flavor Hazelnut Cream<\/em>?
\u2022 How have teachers<\/em> responded to the new dress code?
\u2022 How have students<\/em> responded to the new dress code?
\u2022 Which title<\/em> do you prefer?
\u2022 Which subtitle<\/em> do you prefer?<\/p>\n\n\nHow to create good unbiased survey <\/a>questions with PickFu<\/h2>\n\n\n
<\/a><\/figure><\/div>\n\n\n
![\"\"](\"https:\/\/www.pickfu.com\/blog\/wp-content\/uploads\/2017\/03\/biased-questions.png\")
<\/a><\/figure><\/div>\n\n\n![\"example](\"https:\/\/www.pickfu.com\/blog\/wp-content\/uploads\/2017\/03\/unbiased-question.png\")
<\/a><\/figure><\/div>\n\n