When Precision Is Only 92.11567% Accurate

The Numbers Behind Cash-for-Clunkers Imply Exactitude, but a Look Under the Hood Reveals Guesswork and Assumptions
By CHARLES FORELLE
Pity the owner of a 1987 Plymouth Sundance. Maybe in its youth the Sundance's pinched nose exuded sportiness and verve, maybe its vinyl-everywhere interior felt shiny and new. But nearly a quarter-century on, the thing is a clunker.
Still, someone hoping to dump an '87 Sundance in the cash-for-clunkers program was shocked recently when the Environmental Protection Agency re-checked fuel economy figures. In the new math, some Sundances got 19 miles per gallon, just ahead of the clunker-cutoff of 18. It and 77 other cars were bumped from the bad-enough-for-cash list.
In a statement, the EPA said "more precise" data calculated "to four decimal places" caused the revisions.
Just how precise can a fuel-economy test be? Not that precise. After all, 0.0001 miles is about six inches, and, if you could count it, a car getting around 18 miles to the gallon would consume about half a drop of fuel in that distance.
Such precision is futile when dealing with essentially unknowable quantities derived from rough real-world experiments and seasoned with debatable assumptions.
A Clunker or Not?
Some models were disqualified as others became eligible for the cash-for-clunkers program after fuel-efficiency ratings were recalculated.
"It is ludicrous to suggest that you can get fuel-consumption accuracy anywhere past the first decimal place, let alone the second," says Peter de Nayer, an independent U.K. auto tester.
Still, decimal places lend the aura of authority and the veneer of verisimilitude. So the modern world is awash in squishy numbers wearing the many-figured garb of faux precision. There are 307,085,556... no, wait 307,085,557 people in the U.S., according to the Census Bureau's ticking population clock. The Energy Information Administration says the vast continent of Africa has 117.064 billion barrels of proven oil reserves. The firm eMarketer "estimates that in 2009 96.6 million US Internet users will read a blog at least once per month."
The EIA says it gets its figures from a trade journal. An eMarketer analyst says the 96.6 million blog readers is accurate to "within 5% to 10%."
The state of Montana reported three weeks ago that its unemployment rate for June "increased 0.1% to 6.4%."
Did it? Maybe. The unemployment data come from the federal Current Population Survey, which interviews tens of thousands of people across the nation but only 1,200 in Montana -- too few for a precise figure. The rate is derived from a statistical model. The state says that the margin of error is plus or minus 0.8 percentage points at a 90% level of confidence. (It's actually plus or minus 0.9 percentage points.)
Barbara Wagner, a state economist, admits the figure isn't precise. But, she says, "it is still our best guess at the exact rate."
County numbers are even squishier. Nobody provides margins of error. What to make of the estimate that Petroleum County (labor force: 229) had an unemployment rate of 3.9% in June? Ms. Wagner says the county rates largely come from divvying up the state's overall rate, adjusting for things like unemployment claims. For small places, "there's no guarantee that there is someone in that county getting interviewed."
Faux precision has a number of causes. Wide error margins can render multi-digit precision meaningless. Other times, number crunchers run afoul of the rule of significant figures.
The principle is simple: When combining measured numbers, the final answer is only as precise the least-precise piece of data that went into it; you can't just add a tail of decimal places, even if they show up on the calculator. So a room that's 2.5 meters (two significant digits) by 3.87 meters (three) has an area of 9.7 square meters, though the two numbers multiply to 9.675.
But that is often messed up. In March, the New Scientist magazine published a short item questioning calculations of salt content in snack foods. One variety of cheese puffs "contains 0.4 grams of salt per pack, which the panel says amounts to 9% of a child's [recommended daily allowance]. This would mean that the [allowance] is 4.444 grams a day," the magazine wrote. A sharp-eyed reader wrote a scolding letter: Given those figures, the answer can have only one significant digit: four grams. The decimal places are unwarranted. The magazine ate humble pie in an editor's note.
The rule has even ended up in court. In 1991, a would-be lawyer, Frank Bettine, failed the Alaska bar exam, missing by 0.5 point the threshold needed for a re-evaluation of his test. Perhaps jump-starting his career, Mr. Bettine sued.
An engineer by training, he fought the math: Graders scored essay questions only with integers -- 1, 2, 3, and so on. But the score is an average of two graders' marks -- 1.5, for instance, if the graders gave 1 and 2. That's too many digits. Mr. Bettine argued that the essays should have either been graded with two significant figures in the first place, or rounded in the end to one -- pushing 1.5 up to 2.
Mr. Bettine lost at the Alaska Supreme Court, but the justices found his critique "convincing from a purely mathematical standpoint." Mr. Bettine passed the bar, practiced law for 15 years and then went back to electrical engineering.
Fuel mileage doesn't come from pumping in a gallon of fuel and seeing how many miles a car will travel. Instead, car makers record tailpipe emissions and, knowing how much carbon dioxide to expect after burning fuel, compute how much gas was consumed. Experts say that's more accurate than directly measuring the volume of fuel, which varies with temperature.
For decades, the EPA prescribed two fuel-economy tests performed on a dynamometer -- a pair of rollers on which the wheels spin without the vehicle going anywhere -- in a lab. A driver accelerates and stops 23 times for the city test and cruises at an average 48.3 miles per hour for the highway. The two results are recorded to four decimal places -- the law demands it, the EPA says.
Recently, the EPA added tests meant to mimic things people actually do with cars, such as turn on the air conditioning or drive them in cold weather. But these weren't done for older cars, so the EPA created a formula that estimated from the old data what would happen had the new tests been run. Karl Simon, an EPA scientist, says the results are "effectively equivalent."
When preparing its public database for the clunkers program, the EPA realized some figures going into these formulas came from rounded numbers, and others were at four-decimal-place precision. The agency updated the database so all cars were scored from the more precise numbers.
"Repeatability and accuracy is something we spend a lot of time on," says Mr. Simon.
Mr. de Nayer says the pursuit of precision is misplaced. "There is no such thing as a single figure you can attach to a car," he says. How and where you drive, on what kinds of roads, in what kinds of traffic -- all of it makes a difference far greater than variances in a lab test.
"When we are talking about first decimal places, we are rather straining at gnats and swallowing camels," Mr. de Nayer says. "In the real world, nothing, but nothing, is spot on."
Write to Charles Forelle at charles.forelle@wsj.com