This Is Not a Poll
Wednesday, January 28th, 2009Did you know that Vermonters are less satisfied with their jobs than workers in any other state and are the most likely to be planning to move away?
It’s right there in the encyclopedia. It’s in Wikipedia, in the article about Vermont, where it says, “A 2007 survey claimed that Vermonters were the least satisfied with their job in the nation and were the most likely to be making plans to leave.”
Nor is this a mere assertion. Not on your tintype. This sentence ends with a footnote. Footnote number 70, just in case you’re interested, which provides the source of the information.
The source of the information is something called Salary.com . Salary.com, a company in the Boston area, “builds on-demand software around a deep domain knowledge in the area of compensation to help customers win the war for talent by simplifying the connections between people, pay and performance,” whatever that means.
The company’s press contact is a friendly fellow named Rob Halpin, of a Boston firm called Version 2.0 Communications, who readily acknowledged something about the Salary.com poll.
It isn’t a poll.
OK, Halpin didn’t say that in so many words. What he did say is that it was not a random sample survey. It was one of those on-line displays that asked a question to be answered by any computer-user who stumbled onto it. It had, Halpin said, “no methodology” to speak of, and was not “terribly scientific.”
It isn’t a poll.
A poll-if it’s a real poll-must be a survey of a “random sample,” defined as a “set of items that have been drawn from a population in such a way that each time an item was selected, every item in the population had an equal opportunity to appear in the sample.” (from the Internet Glossary of Statistical terms).
If respondents choose to be respondents, then every item (in this case, employed Vermonter) in the population (in this case, all employed Vermonters) does not have an equal opportunity to appear in the sample.
Not to mention that there are employed Vermonters who don’t use computers to begin with, meaning they would have no opportunity at all to appear in the sample.
It isn’t a poll.
What is it then? It is nothing. Nada. Rien de tout. Bupkiss. Gornish. A nullity.
At least as far as its conclusion meaning anything. It might serve as a promotional device for the company. It might be fun for the folks who click their answers to the questions.
It isn’t a poll.
There are two problems illustrated here, one cosmic and one local. The cosmic relates to but transcends Wikipedia, which, to its credit, has been trying to police the (mis-)information its readers/contributors often insert. Just the other day one of them inserted an obituary of Sen. Edward M. Kennedy, saying that he “died shortly after” his Inauguration Day collapse.
Kennedy is, of course, alive, and Wikipedia is re-examining its free-wheeling ways as “the free encyclopedia that anyone can edit.” The whole “anyone can edit” outlook is essential to the on-line-world philosophy which disdains “gate-keepers” and other elites as relics of another day.
Perhaps. But sometimes a “gatekeeper” is someone who knows what he is talking about. Or who has standards of intellectual honesty which will prevent her from approving a sentence saying Vermonters are more dissatisfied with their jobs than workers elsewhere, an assertion for which there is not a scintilla of evidence.
The local problem is that several Vermont news organizations, including the three major newspapers-Burlington Free Press, Rutland Herald, Barre-Montpelier Times-Argus—put these pseudo-polls into their on-line versions. Currently, the Herald and the T/A are asking whether students should be allowed to carry backpacks or bags during the school day. The Free Press question asks whether the state should cut spending more, raise taxes, or do some of both.
At about 8PM last night, “cut more” had a big lead, 64 percent, to 28 percent for some of both and eight percent for raise taxes.
Sounds like a blow-out, but only 272 votes had been cast, not enough of a sample had it been a random sample, which it was not.
It’s not a poll.
Which did not stop the editorial writer at St. Johnsbury’s Caledonian Record from proclaiming the other day that most Vermonters must agree with Gov. Jim Douglas about cutting school spending, because such was the result of what the writer called a “poll” in the on-line version of the Herald and Times-Argus.
It was not a poll, which might have embarrassed the editorial writer, who, however, seems unembarassable.
Just because these on-line devices are not polls, does not mean they are nothing. They are something. From the perspective of the bean-counters at the newspapers, they are no doubt part of their marketing and promotion campaigns. And at least somebody at the newspaper understands that they are not real polls. “This is not intended to be a scientific sample of local and national opinion,” the Free Press acknowledges under its “poll question.” The other papers use similar disclaimers.
In short, the newspapers are admitting that these are not polls. They are lies. Really, news organizations shouldn’t do that.




