Credit: Elaine Lanmon
Michelle Kwan's much-lamented exit from Torino may not be the only tragedy that befalls this year's Olympic figure skating competitions. Jay Emerson, an assistant professor of statistics at Yale University believes the new rules for scoring could create results that may be unfair and potentially contestable. Emerson published his critique of the new system—inaugurated at the 2004 World Championships— on his website earlier this week.
In past Olympics, scores were more subjective and prone to bias, often based on the order when a skater skates or a judge's idea of what medal a particular skater deserves rather than what points they scored for certain elements in their program.
Advertisement
The reformed system, introduced in response to a controversy at the 2002 Salt Lake City Olympics that resulted in the awarding of two sets of gold medals during the ice skating pairs competition, is harder to manipulate. It awards points for technical elements, based on a prior analysis of each element's level of difficulty— a triple axel gets more base points than a double axel—as well as five other criteria: skating skills, transition or linking footwork, performance/execution, choreography/composition and interpretation.
In the new system, a computer algorithm randomly disregards three of the 12 judges' scores. Then, from the nine-score subset, the highest and lowest scores are dropped, a relic from the old system, which was installed to correct for favoritism.
Emerson claims that this judging process could lead to undeserved rewards or punishment if, for example, the computer happened to pick the nine judges most in favor of one skater, or most opposed to another. He also pointed out that he is not the first one to find fault with the changes to the scoring system. Katherine Godfrey, a professional statistician with a PhD from Harvard, lodged a complaint in 2003 about an interim system that fell on deaf ears.
"Different combinations of nine judges would yield slightly different scores," Emerson said via e-mail. "If a competition is close, the medal standings could change. So far, I've shown this has been a possibility in past competitions. Fortunately, the actual medal results for competitions I've studied have successfully reflected the consensus of the full panel of twelve judges. But this is, to some extent, no more than good luck."
To illustrate his objection, Emerson examined the scores for the Ladies' 2006 European Figure Skating Championships. Russia's Irina Slutskaya walked away with first place, but the scores for the next four competitors—Yelena Sokolova, Sarah Meier, Elena Gedevanishvili and Carolina Kostner— were very close to one another. The scores were calculated according to the new judging rules, and out of the 220 possible groups of nine judges, Emerson says, only 50 would have awarded the medals in the same order as the actual group did.
"Only Slutskaya's standing was secure," Emerson writes in his critique. "Each of the other skaters could have been placed as high as 2nd or as low as 5th in the short program."
As it turned out, the group of nine judges picked the same order that the entire group of 12 would have picked, lending some credence to the official results. Still, as Emerson notes, such a close race could potentially have painful results at the Olympics this year.
"I think that's something people would find pretty dissatisfying," said Eric Zitzewitz, assistant professor of economics at Stanford and author of an earlier paper on nationalist bias in figure skating scoring.
Zitzewitz agrees with Emerson that the new rules could result in skewed standings.
"Basically what you're doing when you take 12 and throw out three is you're taking the average of the 12 and adding noise to it," he said. "That doesn't seem to make sense."