Let’s say you want to know something from your prospective juror: “Do you tend to think that corporations are basically dishonest, or not?” You get a chance to talk to them in court during oral voir dire, or even better, you get to give them a questionnaire that they’ll fill out in advance. Their response will be the basis for that consequential choice: Do I strike or do I keep? Now, if you have a social scientist near you, they might whisper in your ear that it matters how you ask, and it matters a great deal. And one of the first things they would say is, “Don’t make it a yes/no. Instead, give them options.”

On written questionnaires at least, the gold standard since 1932 has been the Likert scale, with response options ranging from “Strongly Agree” to “Strongly Disagree.” However, the format and number of options tends to vary quite a bit, and there’s been little research on how many options are best. It matters to the precision of your response: Include too few options and you’re leaving real differences unexplored, but include too many options and you are increasing confusion and random noise in the response. So what’s the sweet spot? Well, that “Six” at the top of the post isn’t there by accident. Based on recent research (Simms et al., 2019), “Six appears to be the magic number.” After testing scales ranging from two options (yes/no) to eleven options, a team of researchers from the University of Buffalo found that we lose valuable information when we have fewer than six options, but gain no greater precision with more than six options. In this post, I will take a look at what that means when designing surveys, and even when developing questions for oral voir dire.

Have Options, But Not Too Many

When judges and, to some extent, attorneys think about jury selection, they have what I call a “binary bias.” There are two options. A potential juror is either biased or they aren’t. A potential juror is either a strike candidate or they aren’t. At the end point, the choices certainly are binary: the juror is either kept or removed. But on the road to that decision, the assessments need to be more nuanced so as to create, at any given moment, a running sense of who is worse for your client and your side of the case. That’s what helps you prioritize your strikes and cause challenges. For that process, you’ll add more precision to the task if you or the questionnaire you’re using, pose questions that provide multiple options for response. “Yes or no” gives you greater control when you’re taking a deposition, but when you’re questioning a potential jury, you’re reducing your measurement precision if you confine yourself to those options.

At the same time, there’s a limit to the number of options. As the study authors write, “humans’ ability to make fine-grained distinctions about relatively fuzzy and complex psychological constructs is not without limits.” So, there’s no reason to have more than six.

Even in Oral Voir Dire

While six options is ‘psychometrically ideal,’ it is probably unrealistic in the setting of oral voir dire, where your pool is unlikely to remember a large number of options. The good news is that with four options you are capturing the vast majority of power that comes from multiple options. In other words, based on the research, having four options is a lot better than having two, and only a little worse than having six.

It is still a little daunting to list off the options to the jury. They likely can follow you if you’re asking, for example, if they are “very trusting,” “somewhat trusting,” “somewhat distrusting” or “very distrusting” toward insurance companies. Alternately, you can ask in two steps: “Raise your hand if you tend to distrust insurance companies” as a first step, and then, “For those of you who raised your hand, who would say you’re ‘very distrusting?’ And who would say ‘somewhat?'” In that way, you end up getting a response on a four-point scale. You obviously won’t be able to do that on every question (not without trying the venire’s patience), but for the truly important attitudes, it is worth thinking about how you can ask with greater precision so as to single out the most extreme, which is after all, the point of voir dire.

Eliminate the Middle Option

Seems natural and fair to have a middle option, and that is why many response sets are in an odd number with a neutral choice in the middle. From a measurement standpoint, however, there are two problems with this. One, the researchers showed that there is no advantage in measurement precision in adding that odd number (5 versus 4, or 7 versus 6). But the bigger problem is that, from a practical standpoint, people have different reasons for selecting that middle option. Someone who circles the option in the center on a survey could mean any of the following:

• • • They are truly neutral
    • They have difficulty deciding
    • They are confused about the question
    • Their response is a conditional “it depends”
    • They’re signaling neutrality because it is “socially desirable”

The problem with the easy middle option is that, without additional follow-up, you do not know which of the five messages they are actually sending. The last possibility is especially likely in a setting, like voir dire, where people are motivated to avoid the appearance of bias. So by eliminating that option, you may end up with a response that is more honest. And even if they are truly neutral, you still learn something about them in seeing which way they lean when the question forces them to lean.

So the bottom line is this: For the most precise attitudinal responses, provide a number of options and eliminate the middle if you can. How sure are we that this is better? Well, on a six-point scale ranging from “No Idea” to “Very Sure,” I’d pick option six: Very Sure.

____________________

Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological assessment, 31(4), 557.

Image credit: 123rf.com, used under license