The Damnedest Lies

Feature / by Joe Kloc /

The success of is a credit not only to statistical prowess but also to keen intuition about social habits.

Electoral predictions have traditionally been the territory of pundits and politicians, but as polling results become more widely accessible on the web, a new group of analysts have emerged. These statisticians gather polling data from across the country and run it through mathematical models that predict the victor on November 4. And while the accuracy of these models is still debatable, one analyst seems to be doing something right: Nate Silver of constructed an election model that out-predicted the pros in the democratic primary last May, and now he’s looking to repeat this feat on Tuesday.

The success of Silver’s model is due in part to his ability to use the information collected from national polls and heavily polled states in order to inform and strengthen the data on less frequently polled states. He also accounts for regional and national trends that individual polls might overlook.

Silver writes of one particular example, “The late February SurveyUSA polls had Barack Obama four points ahead of John McCain in North Dakota, but behind by four points in South Dakota. Since North Dakota and South Dakota are very similar, it is unlikely that there is a true eight-point differential in the polling in these states. The regression estimate is able to sniff out such discrepancies.”

Illustration by Joe Kloc.

As Silver points out, “Polls are an imperfect measure of voter sentiment.” Polling organizations employ different techniques to try to minimize this imperfection, and some do it better than others. He looks at the historical performance of each poll in order to determine how reliable its polling methodology is. Then he weights each poll accordingly. His weights also factor in the sample size and age of each poll.

Of course, there is no foolproof mathematical method for determining how these weights should be assigned. And doing so successfully is as much of an art as it is a science. Modeling real-world systems, particularly those influenced by human behavior, also presents statisticians with another problem. There are an overwhelming number of factors that influence the behavior of the system. And attempting to keep track of and account for all these influencing factors would be something akin to trying to herd a large group of cats across Manhattan at rush hour.

So inevitably statisticians must determine which factors to consider and which to disregard. This ultimately amounts to making guesses. And while these guesses are educated, they are still essentially guesses.

Once these initial assumptions are made, we can trust the theory to provide us with an accurate model of the system based on the assumptions, but this is only an approximation of the actual system as many factors — sometimes quite influential — are not considered. The objectivity of statistics comes after the assumptions are made. Thus Silver’s success is a credit not only to his statistical prowess but also to his keen intuition about social habits and relationships of the population he is analyzing.

We tend to overlook this more creative element involved with statistical modeling perhaps because we associate with math a particular degree of truth. And usually this is a sound approach, because by the time an idea wiggles free from the mathematical community and permeates the public sphere, we can be relatively certain that concept has been reinforced by a substantial amount of rigorous proof.

But in this way statistical models can be deceptive. We often conflate the objectivity of statistical theory with the uncertainty of statistical results. The British politician Benjamin Disraeli once commented that as a consequence of this confusion, “there are three kinds of lies: lies, damned lies, and statistics.”

So in less than a week voters will head to the polls, testing Silver’s assumptions and determining if the predictions of his election model are valid — or just more of those damnedest lies.

Originally published October 30, 2008

Tags data decision making democracy information math theory

Share this Stumbleupon Reddit Email + More


  • Ideas

    I Tried Almost Everything Else

    John Rinn, snowboarder, skateboarder, and “genomic origamist,” on why we should dumpster-dive in our genomes and the inspiration of a middle-distance runner.

  • Ideas

    Going, Going, Gone

    The second most common element in the universe is increasingly rare on Earth—except, for now, in America.

  • Ideas

    Earth-like Planets Aren’t Rare

    Renowned planetary scientist James Kasting on the odds of finding another Earth-like planet and the power of science fiction.

The Seed Salon

Video: conversations with leading scientists and thinkers on fundamental issues and ideas at the edge of science and culture.

Are We Beyond the Two Cultures?

Video: Seed revisits the questions C.P. Snow raised about science and the humanities 50 years by asking six great thinkers, Where are we now?

Saved by Science

Audio slideshow: Justine Cooper's large-format photographs of the collections behind the walls of the American Museum of Natural History.

The Universe in 2009

In 2009, we are celebrating curiosity and creativity with a dynamic look at the very best ideas that give us reason for optimism.

Revolutionary Minds
The Interpreters

In this installment of Revolutionary Minds, five people who use the new tools of science to educate, illuminate, and engage.

The Seed Design Series

Leading scientists, designers, and architects on ideas like the personal genome, brain visualization, generative architecture, and collective design.

The Seed State of Science

Seed examines the radical changes within science itself by assessing the evolving role of scientists and the shifting dimensions of scientific practice.

A Place for Science

On the trail of the haunts, homes, and posts of knowledge, from the laboratory to the field.


Witness the science. Stunning photographic portfolios from the pages of Seed magazine.

Sites by Seed Media Group: Seed Media Group | ScienceBlogs | Research Blogging | SEEDMAGAZINE.COM