In Infallibility of Nate Silver

He's just like Paul Krugman, but with politics

How poor have the election forecasters been this year? It is a topic many are discussing given the large number of upsets we've had during the Primaries. For example, statistician Nate Silver (who started the campaign season proclaiming Trump had Silver then predicted on March 8 with >99% probability that Clinton would win Michigan (she lost). Silver again predicted on May 3 with 90% probability that Clinton would win Indiana (she lost). But there is another issue besides being wrong, which is how much model flip-flopping is occurring just up to these elections. The most proximate example is Silver stating this past Sunday that Cruz had a 65% chance to win Indiana; the next day (Monday, the eve of the election) and with little new data, he "adjusts" that to Trump having a 69% chance to win! That's horrible!

When at first you don't succeed, flip-flop.

Now let's look at the 63 states where Silver has provided a time series of his forecasted winner. We documented each of these states in the map below, separating out those which were/are Democratic races from those that are Republican races. We show the Republican states, in green on the right map below. For 7 of 32 states, Silver completely flipped his opinion on which Republican would win and who would lose that state's primary. We instead color those red. Only in Missouri did we color the state yellow, for a partial flip-flop. Since the initial probability Silver gave to his predicted winner was beyond cut in half (from 14% advantage, to 6%). For Republicans, Silver flip-flops in nearly 1 of 4 states! Any general election poll without flip-flopping could have similar odds for the weaker of the two candidates, and now we cut this forecaster's reliability in half by flip-flopping.

FLIP.jpg

For Democrats, things are about the same too, in the 31 states so far, suggesting that this year's difficulties are not a Trump-only phenomenon. In 4 states (red) we see that Silver completely flip-flopped, and in 3 states (yellow) he at least partially flip-flopped. For Democrats, Silver flip-flopped in nearly 1 of 5 states. Different states too versus those on the Republican party, further suggesting there are serious problems at the core of how Silver and other forecasters are modeling and misleading the public as to their accuracy.

I've got nothing against Nate Silver. It's the worship of him at TOP that bugs the sh*t out of me.

Tags: 
Share
up
0 users have voted.

Comments

...is that Silver is letting his personal agenda color his results. That's got to be a cardinal mistake of pollsters.

up
0 users have voted.
Pricknick's picture

has changed from political science to who will pad his bank account the most.
Create a story line and when it doesn't fit, change it when nobody's watching. It's a paid gig.

up
0 users have voted.

Regardless of the path in life I chose, I realize it's always forward, never straight.

lunachickie's picture

When I think of Nate Silver now, I think of Chuck Todd in the same thought. Both of them started as numbers geeks with polling/forecasting. Both had a banner year in the 2007/2008 cycle, as I recall--Silver got a noticeable bump in exposure largely because of TOP--he was good at it then. We all know how Todd fared, of course, ending up at MSNBC, and going on to host Meet The Press. He can make the same claim, he was good at crunching numbers. Once upon a time, that is.

And that's precisely why "it's a paid gig" now, because they both have some fatcat benefactors. And now they're mouthing the words of those benefactors instead of what the damn numbers actually tell them....

up
0 users have voted.
riverlover's picture

Baseball for some of us who are not interested has become odds arcana. Ho hum.

up
0 users have voted.

Hey! my dear friends or soon-to-be's, JtC could use the donations to keep this site functioning for those of us who can still see the life preserver or flotsam in the water.

tapu dali's picture

He "aggregates" polls (much like RCP), but then "adjusts" them (which he calls "polls plus") to take into account various subjective factors, like superdelegate preferences, early voting tendencies, age-skewing (he polls well with the young, but they won't vote), weather forecasts, the phase of the moon, etc. to come up with his own version of "unskewing".

It's all hocus-pocus, AFAIAC.

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

... "according to historical trends".

So, just like predicting the volatility of mortgage house defaults, it works as long as nothing happens that is outside of historical experience.

Except, we have not experienced this kind of partisan gamesmanship in the post-WWII era, and so the careful cultivation of the ignorance of the Republican base electorate runs into their personal experience and for far more than those inside the beltway can understand, Trump makes more sense than all the Jeb Bush's and Marco Rubio's that the GOP could throw into the race.

It is certainly understandable based on the experience of the majority of people when the top 0.1% garners most national income gains of the past two decades, but that reality doesn't fit with the personal experience of those in the next 9.9% who have been getting the balance of the gains are part of the oil that greases the wheels of our corrupt society.

up
0 users have voted.

-- Virtually, etc. B)

cardboardurinal's picture

who confirms the world view of the TOP, will be viewed irreverently. Another example is Charles Blow.

up
0 users have voted.

Michigan weatherman sticking his hand out of the window. They have no clue of what the weather will be until it is.

up
0 users have voted.

"Religion is what keeps the poor from murdering the rich."--Napoleon

tapu dali's picture

we did use the raised wet finger technique ...

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

PastorAgnostic's picture

And pissing out the window.

Then again, how drunk are skunks to begin with?

up
0 users have voted.

at 2am.

Then again, how drunk are skunks to begin with?

up
0 users have voted.

hitting the bottle, I've also merger seen Superman and a skunk in the same place at the same time so what does that'd tell you?

up
0 users have voted.

Arrr...the laws of science be a harsh mistress -Bender B. Rodriguez

How did I lost that 3 times?!

up
0 users have voted.

Arrr...the laws of science be a harsh mistress -Bender B. Rodriguez

I must be skunk as a drunk!

up
0 users have voted.

Arrr...the laws of science be a harsh mistress -Bender B. Rodriguez

progdog's picture

We're working hard to improve the site and resolve the issues with repeated posts. In the meantime, I tidied up the extra comments for the sake of readability.

(didn't want you thinking you'd hallucinated comments in your impaired state :P)

up
0 users have voted.

prog - weirdo | dog - woof

featheredsprite's picture

Smile

up
0 users have voted.

Life is strong. I'm weak, but Life is strong.

lotlizard's picture

Does the Pope s*t in the woods?

up
0 users have voted.
PastorAgnostic's picture

up
0 users have voted.
Bollox Ref's picture

and then there's Fools Ag.

People will support anything they want to believe.

up
0 users have voted.

Gëzuar!!
from a reasonably stable genius.

tapu dali's picture

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

lotlizard's picture

up
0 users have voted.
PastorAgnostic's picture

Or . . . Deplane?

up
0 users have voted.
featheredsprite's picture

continue to lean heavily on "likely voters." By definition, the newly registered voters aren't included in that group.

And what's with this "polls plus" nonsense? Is it statistical talk for "I fudged the numbers"?

up
0 users have voted.

Life is strong. I'm weak, but Life is strong.

tapu dali's picture

wherein I opine that he

"adjusts" them (which he calls "polls plus") to take into account various subjective factors, like superdelegate preferences, early voting tendencies, age-skewing (he polls well with the young, but they won't vote), weather forecasts, the phase of the moon, etc. to come up with his own version of "unskewing". - See more at: http://caucus99percent.com/comment/78626#comment-78626

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

karl pearson's picture

On some blogs, Silver's method is called "his secret sauce." So maybe he cooks the numbers? LOL Seriously, I don't care for the Silver adoration crowd and he should not be making predictions unless the polls are accurate in the first place. Silver's Indiana predictions were really awful. And why did Silver predict HRC with such a lead and then suddenly flip-flop on his Live Blog that evening? Silver said that for Indiana he should have used the model that correctly predicted the Northeast primaries. So why didn't he use that model then? IMHO, Silver achieved the height of his career in the 2008 election. Does anyone know why he the NYT parted company? Just curious.

up
0 users have voted.

go back to the days when poblano was just a sabermetrician who thought he had an edge on traditional pollsters because they were wrongly focused.

Well now he has a biz and is playing with the big boyz. Congratulations and good luck with that.

up
0 users have voted.

a couple new prognosticators have arisen.

Benchmark Politics is one:

http://www.benchmarkpolitics.com

I think that these are Bernie enthusiasts, however, they have base their model on demographic analysis.

Tyler Pedigo is another:

https://tylerpedigo.com

His formula involves analyzing Google Trends (search words) and Facebook likes at the state level. He's had some good success with the approach.

up
0 users have voted.
ngant17's picture

see link:

https://richardcharnin.wordpress.com/tag/nate-silver/

Nate wants us to ignore exit polling. And assume there is no such thing as election fraud.

up
0 users have voted.
tapu dali's picture

necromancy, astrology, and alchemy to "unskew" polls.

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

...reading his own crap.

He almost has seizures when he reads it a day after gobbling coleslaw.

up
0 users have voted.

******************************

Muerte al fascismo. Muerte a la tiranía. colapso total de los que promueven tampoco. A la pared con el unico porciento%

lotlizard's picture

Notice the absence of the standard disclaimer that “no animals were harmed” during the production of these poll results.

https://en.wikipedia.org/wiki/Haruspex

(edit: this was supposed to be a reply to tapu dali’s remark above about Nate using necromancy and astrology)

up
0 users have voted.
Alison Wunderland's picture

up
0 users have voted.
Sandino's picture

to deny it afterwards, even more so.

up
0 users have voted.
tapu dali's picture

the pencil, you're overdoing your humanity.

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

Dragonkat's picture

"There are three kinds of lies: lies, damned lies and statistics."

up
0 users have voted.

See, their morals, their code... it's a bad joke. Dropped at the first sign of trouble. They're only as good as the world allows them to be.

-The Joker-

I quit paying attention to him before he incorporated himself. I knew he'd become a legend in his own goddamned mind.

up
0 users have voted.

******************************

Muerte al fascismo. Muerte a la tiranía. colapso total de los que promueven tampoco. A la pared con el unico porciento%

tapu dali's picture

to his own satisfaction.

up
0 users have voted.

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – the ones we don't know we don't know.

Or do we have to call over the turd monitor first?

up
0 users have voted.

******************************

Muerte al fascismo. Muerte a la tiranía. colapso total de los que promueven tampoco. A la pared con el unico porciento%