Designing Questions
As mentioned previously, speakers will need to decide early on if they need open-ended questions for individual feedback or closed-ended questions to gather more generalized information about the audience. Closed-ended question types include either-or, multiple choice, and scaled questions, but each type carries with it a different set of considerations.
Either-or questions include true/false, yes/no, or any other pair of polar opposite responses. Use these types of questions only if two available options exist from which to choose. Consider the following poor examples of either-or questions:
- What political affiliation are you: Democrat / Republican
- People are more likely to identify as having tendencies toward both parties or neither. A multiple-choice question that gave the audience more options would work better here.
- What religion are you: Christian / Other
- This question implies that any religion besides Christianity is not worthy of naming. Lumping all other religions into a general category could unintentionally insult some members of the audience, particularly the ones who practice anything besides the aforementioned religion.
- Would you consider yourself: Healthy / Unhealthy
- Consider the social desirability bias here—who would rate themselves as unhealthy?
- How often do you exercise: All the time / Never
- The either-or dichotomy presented here does not represent the full range of possibilities.
By altering these questions into better uses of the either-or question type, they may look like the following:
- Do you generally describe yourself politically as: Left-leaning / Right-leaning
- Do you consider yourself strongly religious: Yes / No
- I can recite the Bill of Rights: Yes / No
Notice how the generalized nature of the questions lead the respondents to choose between two equally attractive options?
Multiple-choice questions can provide helpful information because they offer a wider range of answer choices, however, a poorly designed question might unfairly lead them to answer in a way they normally would not. Avoid unintentionally inserting bias into the lead-in question or statement, and also ensure that each response has an equal a chance of being selected as the others. Avoid inserting one improbable response for the sake of humor, because then people may choose that response in an attempt to play along with the joke, at which point the data collected has become compromised.
Note to Self
Scaled questions provide the best possible insight into how an audience feels about the subject of a question. Ask the audience to place their knowledge or interest level on a scale from 1 to 10, and once completed, analyze the results to capture an overall picture of that audience. For example, if, on a knowledge-level question like “how comfortable would you feel building a computer,” the audience surveys resulted in an average response of 7.2, this means they possess reasonable knowledge already, requiring the speaker to prepare a more advanced presentation since the audience already has demonstrated they have more than a basic understanding of the subject. Other uses of scaled questions include Likert-scale questions, which feature responses on a qualitative scale of agreement, such as those having “Strongly Disagree” on one end of the scale, “Strongly Agree” on the other end, and “Neutral/No Opinion” in the middle of the scale. These types of questions prove particularly helpful for determining audience attitudes, beliefs, and opinions.
However, as with any other type of survey question, make sure to keep any potential bias out of the question prompt, as well as the scale itself. For example:
Poorly worded scaled question prompt: “The president’s speech last night was full of lies and deceit.”
Re-worded scaled question prompt: “The president delivered an effective and informative speech last night.”
In the first example, the survey taker knows immediately that the person who wrote the survey does not hold the president in high regard. For a fellow critic of the president, that could lead to an answer of stronger agreement than normal, and for a fan of the president, that could lead to greater disagreement than normal. By rephrasing the question neutrally as shown, the initial statement sets up a survey respondent with the best possible opportunity to agree or disagree, thereby capturing the most honest and accurate responses possible.
