The Perils of Polling
A majority of the people of this state-no, make that a huge if not an immense, majority-favor raising taxes on either tobacco, the very wealthy, or both, “in order to keep Catamount Health, Dr. Dynasaur and other state health care programs affordable for low income Vermonters.”
It’s in a poll. The poll was taken by Macro International of Burlington, a respected firm whose surveys have been used by businesses and advocacy groups in Vermont for years.
Here are the results: Seventy-seven percent of those surveyed would support a temporary state income tax surcharge on those earning more than $500,000 a year. Eighty-two percent would support raising the cigarette tax by a dollar to subsidize the health care programs.
That sounds impressive. Actually, it sounds unbelievable. It’s hard to get a 77 percent majority-much less 82 percent-for almost anything. Asking a random sample of people whether they approve of motherhood and apple pie would probably get more negative responses than these two questions did.
The questions on the income tax surcharge the cigarette tax were inserted into a broader survey that Macro takes four times a year on behalf of various clients, according to Stephanie Ezzo, the company’s assistant research manager.
“I bought these two questions,” said Peter Sterling, the Executive Director
Vermont Campaign for Health Care Security.
Sterling bought them and wrote them. He is an advocate, not a pollster.
“It is not a neutral question,” Sterling acknowledged. “The question is worded in a way that elicits a greater understanding of the issue. I honestly say that because I don’t believe people think about their taxes going to specific programs. They think their taxes are going to some guy sitting behind a desk.”
The wording of the questions breaks one basic rule of polling-asking respondents only if they would support the higher taxes. A polling question should ask whether the respondents support (or favor) or oppose. Presenting only the favorable option “leads the respondent by suggesting the position … of an authority with which it might be difficult for the respondent to disagree.”
That’s the fancy language the American Association for Public Opinion Research (AAPOR) uses to explain that some respondents, when hearing only one option, tend to assume that it is the “correct’ or expected choice.
In addition, the questions linked the proposed tax hikes to Catamount Health and Dr. Dynasaur, two of the most popular programs in the state. Dr. Dyanasaur, which has provide health coverage for low-income children for more than a decade, has acquired a reputation in the state close to that of…well, motherhood and apple pie. Catamount Health is much newer. But according to a poll taken for the state by Lake Research Associates a year ago, it is overwhelmingly popular.
Had the questions just asked about “health programs for low-income people” without mentioning the popular Dr. Dyanasaur and Catamount Health “brands,” the results might have been different.
Furthermore, respondents can be influenced by the questions that came earlier in the survey. These are not being released. Stephanie Ezzo said she could not divulge the other questions in the survey, taken for other clients, mostly businesses. She would not even say whether any of the earlier questions had dealt with health care, poverty, or tobacco, subjects that could have altered the outlook of some respondents.
“Other clients can participate so (the survey) oftentimes jumps from subject matter to subject matter,” she said.
According to the AAPOR, earlier questions can set up a “context effect.” For example, according to its web site, “if you ask questions about a specific issue like the economy before asking what the most important problem is facing the nation, respondents will be more likely to name the economy in that subsequent question then they would have been without having that context set up for them.
In that statement, the AAPOR was talking about deliberate distortion. That is not the case here. Neither Sterling, who said his organization paid $1,000 to get the two questions in the poll, nor Macro International is guilty of unethical conduct. Sterling does not seem to have been trying to pull a fast one. He apparently did not know the basic rule about asking “support or oppose.”
Nor does there seem to be any reason to doubt Ezzo’s assertion that “everything we do is methodologically sound.” Macro International is a reputable company, and “piggy-backing” questions into a larger poll seems to be standard practice in Vermont.
It’s just that the results aren’t really credible thanks to the flawed wording and the mystery about what questions may have preceded the two about tax hikes and health care.
From various polls it’s reasonable to assume that a majority of Vermonters-but not three quarters or 80 percent majorities — would in fact favor both those tax increases to keep the health care programs affordable for poor and low-income people. In neither case would the tax hikes violate the fabled wisdom of the late Sen. Russell Long of Louisiana, for years the chairman of the Senate Finance Committee. The typical American’s tax preference, Long said, was “Don’t tax you. Don’t tax me. Tax that fellow behind the tree.”
Most people in the state do not smoke, and only a tiny percentage earn close to $500,000 a year. Few, then, are that fellow behind the tree.
And clearly most Vermonters are pro-health care. That Lake Research Partners study found that a large majority agreed that “the state should help people get affordable health coverage if they cannot afford health coverage on their own or get it through a job.”
Still, Vermonters, like other Americans, retain a visceral distaste for higher taxes, even if they are not the ones being taxed. So the huge pro-tax margins in this poll seem…well, too huge.
But Sterling did at least try to find out whether voters would consider some selective tax increases to finance social programs. On the other side of the debate. Gov. Jim Douglas and his aides simply keep asserting that Vermonters are opposed to any and all tax hikes, making no attempt whatever at providing anything resembling evidence.
Tags: health Care, Peter Sterling, polling
January 27th, 2009 at 7:30 am
“Had the questions just asked about ‘health programs for low-income people’ without mentioning the popular Dr. Dyanasaur and Catamount Health ‘brands,’ the results might have been different.”
True. But the question as phrased has its relevance and integrity. 1) Legislators aren’t faced with cutting back “health programs for low-income people” as some sort of abstract policy decision, they are specifically faced with cutting funding for these two popular brands. 2) I suspect that it is easier to deceive a poll respondent (intentionally or unintentionally) with an abstraction than with a concrete example. The ability for a respondent to recognize the personal and political significance of the question is likely to be enhanced by mentioning the recognizable brands.