Saturday, January 19, 2008
Lies, Damn Lies, and Statistics
Unlike many of my fellow bloggers, I find it difficult to comment on events immediately. For one thing, doing so smacks of knee-jerking, so I'm rather loathe to jump in too quickly. As I age, I also find it necessary to stew longer in order to form a cogent response. So although news of the recent polling debacle in New Hampshire is shortly to be overshadowed by whatever comes out of Nevada and South Carolina, I'm still miffed about the roll of polls in the so-called "democratic process," and about the way statistics are currently being used and interpreted, and I'm just now getting around to posting about it.
Benjamin Disraeli, who provided the title for the post (through Mark Twain for many of us), probably had little experience with polls, since Victorian England was short on instantaneous media outlets (although there was that enviable twice-a-day mail service and lots of messengers running around). But I am becoming increasingly angry at the intrusiveness of proliferating numbers-gatherers, whether for political or commercial purposes. I remember being called by a Gallup rep once when I lived in Chicago and possessed more patience than I do now. I can't remember the initial question, but it offered three possible answers, none of which even came close to describing my opinion. When I complained about this to the pollster, he proceeded with some gibberish about how the questions were designed to take that into account and that whatever came closest would provide a statistically valid sample within an acceptable margin of error. That's when I hung up. Now I always hang up, after loosing a few choice words about what I think of polls and the people who take them.
Perhaps this is why I have developed such a suspicious nature and have lost so much of my faith in human intelligence. I don't care how many people you survey; unless you get every one of us, your "margin of error" is not going to reflect what's actually going on in peoples' minds. It might give you a range of possibilities, but if the choices are very limited and specific, errors will creep in, and the fewer people asked these questions, the greater the possibility the poll will contain significant errors. I have no statistical basis for saying this. I only have experience and common sense on my side--having lived in several locations on this planet, among a large number of divergent cultural influences, for a long time. Democracy is not about numbers; it's about people. Focus groups pretend to be about people, because they're composed of small, face-to-face "samples" of "target audiences" in specific "demographics" (since when did "demographic" become a noun?)--but they're even worse, precisely because their numbers are so tiny. They do not really represent me; but then, being older than the preferred 18-34 age-group, I apparently don't deserve representation.
The trouble is, thanks to indoctrination by Madison Avenue through every medium that provides us with news, "we the people" have become extremely malleable and stupifyingly gullible. Folks who can be persuaded that they need to buy dish-washing liquid with air freshener pellets in the bottom of the bottle (probably the topic of a later blog) can be persuaded of any uncritical claptrap that shows up on the telly--from the presence of UFOs in Stephenville, Texas, to unsuspecting twins' marrying one another in England. And I'm certainly not convinced that exit polls conducted during primaries or general elections and reported on while the elections are still in progress don't affect what later voters do.
What I truly do not understand is why we just can't simply sit back, have a beer, and wait for the bloody results! It is in no way important to the outcome of any election for voters to know who won until after the polls have closed and the votes have been tallied. Most ballots are now cast electronically (thanks to previous disasters), and can be counted almost instantaneously. But our addiction to "breaking news" and instant gratification--or at least the perception by news media that we just can't wait--has led us to the point where people are accosted right and left, fresh out of the voting booth, and asked to answer intrusive questions which will be gathered into a spurious database, the numbers crunched, and then reported by the polling agency to the newspapers and television outlets as the Word of God.
It's not that I think numbers themselves are useless or all statistics baseless. In fact, I've only recently discovered some very interesting websites and material of a statistical nature that seem to be responsibly and helpfully constructed. I started looking into the topic when my son (who was in town this week doing a statistical audit of Star Wars Pocket Model packets, which are manufactured locally) suggested that I locate some of Edward Tufte's books on design. When I did so, I was intrigued by the fact that Tufte writes not only on the visual delivery of information in general, but on effective (and, it seems, ethical) delivery of quantitative information as well.
As it turns out, I was already familiar with Tufte, because he authored one of my most oft-quoted truisms: "Power Corrupts; PowerPoint Corrupts Absolutely"--the title of an article he wrote for Wired magazine (September 2003). Since I'm known for my bipolar relationship with the technologies I use (at once loving them and hating them), I frequently seek out critical assessments so that I'm aware of both the benefits and problems associated with tools like the internet and varieties of software. In a later article, "PowerPoint Does Rocket Science: Assessing the Quality and Credibility of Technical Reports," he provides an in-depth analysis of how this particular software is constructed and used--often badly, and on occasion disastrously--and how its use can be improved. At the end of the article he lists some examples of well-designed presentations, which led me to Gapminder.
According to its information page, "Gapminder is a non-profit venture promoting sustainable global development and achievement of the United Nations Millennium Development Goals by increased use and understanding of statistics and other information about social, economic and environmental development at local, national and global levels." So here we have (finally), an organization dedicated to gathering statistical information, but which (using clear, engaging presentations) actually helps the general public understand what all the numbers mean. It even features a PowerPoint presentation--one that doesn't participate in the software's implicit hierarchy--to help viewers understand why interpreting statistics is important in the first place. And this brings me to something of an epiphany: numbers can be illustrative; they actually can tell us something about what's going on in the world. But they have to be sensibly presented and thoughtfully interpreted in order to be meaningful and/or useful.
In fairness to political pollsters, I did run across an interesting blog, Prof. Charles Franklin's Political Arithmetik, which keeps track of election poll results, presidential approval rates, etc. I'm not convinced that this stuff really means anything, but if you want to know what's going on, he's got links and charts galore. In my utopia, however, there are no polls. That's mostly because in my utopia, we'd all know what had to be done, we'd work through consensus, and we wouldn't have to vote. But since we live here, on our imperfect planet earth, what we can do when confronted with pollsters is what the anti-drug campaigners are always urging: just say "no."