Zeev's Turnips Talk Politics (ZTTP)

[arizona1]

How to Poll on the Public Option
by Nate Silver @ 12:40 PM

**UPDATED [1:56 PM] with additional polls.

It shouldn't be that hard to conduct a decent poll on the public option, but so many pollsters seem to be getting it wrong. Let's go through what I'd consider to be five essential ingredients in conducting a good poll on the public option:

1. Make clear that the 'public option' refers unambiguously to a type of health insurance, and not the actual provision of health care services by the government.

In general, this is a concept that a lot of people seem to be unnecessarily confused by (although I suspect that a lot of the "confusion" is deliberate). The recent NBC/WSJ poll gets this wrong, referring to a "public health care plan" rather than a "public health insurance plan". So does this Rasmussen poll, which refers to a "public health insurance company".

2. Make clear that by "public", you mean "government".

The phrase "public" is somewhat vague and could conceivably be confused for something like a non-profit cooperative, which is a far different approach to tackling the health care problem. Including the phrase "public" in the question wording is probably fine, but it should be supplemented by the term "government". The Kaiser Family Foundation Tracking Poll, for instance, adds some ambiguity to what is otherwise a very well-worded question by describing "[A] public health insurance option to compete with private health insurance plans". I don't think this provides quite enough information to the respondent.

3. Avoid using the term 'Medicare' when referring to the public option.

On the other hand, it's quite possible to provide too much information to the respondent. Medicare is a popular program, and I'd consider using it as a reference frame when referring to the public option to be somewhat leading. This is particularly so given that the public option proposals actually being considered by the Congress would have some important distinctions from Medicare, such as probably having their premiums benchmarked to those charged by private providers (if the proposal were literally to make anyone eligible for Medicare, I might feel differently). Kaiser has found that support for the public option is a few points higher when it is referred to as a Medicare-like program. The CBS/NYT poll, among others, violates this premise.

4. Make clear that the public option is, in fact, an option, and that private insurance is also an option.

Here's another issue that a lot of pollsters are having trouble with. The Rasmussen poll blurs the distinction most blatantly, by referring to "[A] government health insurance company to compete with private health insurance companies". It is not at all clear from this wording that eligible consumers would have a choice between government-provided and privately-provided insurance policies. Likewise, the ABC/Washington Post poll ("Would you support or oppose having the government create a new health insurance plan to compete with private health insurance plans?"), would be better if it made clearer to the respondent that the public/government option provides consumers with a choice of health insurance alternatives. The FOX News poll suffers from a similar flaw.

5. Ask in clear and unambiguous terms whether the respondent supports the public option -- not how important they think it is.

As I wrote before:

"Importance" is a notoriously vague concept in public opinion polling and may be separate and distinct from asking someone whether or not they support a particular policy. How might someone respond to this question, for instance, if they had particularly strong feelings against a public option? Would they say that it was "not at all important", or would they say that it was "extremely important"? Conversely, how would someone respond if they had a weak preference for a public option, but didn't consider it an especially important component of health care reform?

NBC/WSJ had previously used this phraseology, which I criticized at the time. Unfortunately, it was resurrected by MoveOn.org in a poll conducted through SurveyUSA. "Importance" and "support" are not by any means synonymous concepts.

So, who gets it right?

Regrettably, almost all of the polls on the public option succumb to one or more of these sins. However, there are two exceptions. One is the Quinnipiac poll, which asks:

Do you support or oppose giving people the option of being covered by a government health insurance plan that would compete with private plans?

This is a perfect question. It makes clear that the public option is an insurance program, rather than a program to provide health care services. It uses the less ambiguous phrase "government" rather than the more ambiguous phrase "public". It makes clear that the public option is a choice. It avoids leading the respondent by comparing the public option to Medicare. And it asks in unambiguous terms whether the respondent supports or opposes the proposal.

62 percent of people support the public option in Quinnipiac's August 5th poll, versus 32 percent opposed.

Another is the Time/SRBI poll, which asks:

Would you favor or oppose a healthcare bill that creates a government sponsored public health insurance option to compete with private health insurance plans?

This poll has 56 percent in favor of a public option, and 36 percent opposed.

The Economist/YouGov poll is in something of a first-runner-up category, asking:

Do you favor or oppose having a "public option" which would allow individuals to purchase health insurance coverage from the government?

In the Economist/YouGov poll, 41 percent favor the public option and 33 percent are opposed.

This is not a bad question, but it arguably violates criterion #4, as it leaves it somewhat ambiguous as to whether individuals would continue to have a choice of private insurance in addition to the "public option", a fact perhaps reflected in the unusually high number of undecideds (27 percent) in the poll. In addition, this is an Internet-based poll, and at the present time I do not consider Internet-based polls to be especially trustworthy.

I would strongly urge people to cite the Quinnipiac or Time/SRBI polls when making reference to public sentiment on the public option. Not coincidentally, these come up with broadly similar results, showing between 56 and 62 percent in support of the public option, and 32 to 36 percent opposed. The other polls run the gamut from slightly to deeply flawed.

Errata: I had originally also described the ABC/WaPo poll as passing all of our tests, but failed to recognize that it in fact violated criterion #4. My apologies for the confusion.

There's More...
http://www.fivethirtyeight.com/2009/08/how-to-poll-on-public-option.html