Archive for the 'Elections' Category

Quick GE2017 Lessons

June 16, 2017

Some thoughts on larger lessons to be drawn from the surprise result of the UK 2017 general election.

Brexit

Brexit is, thankfully, at risk. Hard Brexit is probably off the table and the difference between most soft Brexit’s and staying in is that we’re worse off. A Norway style deal means most of the obligations (including budget contributions) but no say.

But hard Brexit is worse than all these. The best deal for the UK (indeed, a wildly unfair deal) would be to stay on the old terms. This is unlikely.

Was it a Brexit election? Did the Remainers fail to get their revenge? The LibDems campaigned on a second referendum and lost! vote share (while gaining a few seats). Does this mean Remain is dead?

We can’t quite use the LidDem failure to say Remain is dead. I’m a hard core Remainer and I voted Labour. I may still vote against Corbyn in a leadership election on Remain grounds. I’d guess that most Remainers are not single issue and a lot of Remainers are anti-Conservative/hope to avoid hard Brexit types. Clegg probably lost in spite of his excellent Remoaning, not because of it.

Furthermore, LibDems are a third party. In a two party consolidation election they are likely to do poorly. And this is roughly what we see from the Ashcroft post election survey. Consider the party shift graph:

q2-party-by-2015-vote

What’s striking is that both Labour and the Conservatives held onto around 80% of their 2015 voters. There wasn’t a lot of coming home of 2015-Conservative-voting 2010-Labour voters. UKIP collapsed mostly into the Conservatives (but they feel more like a generally spent party). LibDems held on to only 50% with a contingent hitting Labour. SNP had main party level retention, but few gains.

Compare with the party split of Referendum voters:

q2-vote-by-eu-result

Labour is the party of Remain while the Conservatives are the party of Leave. But in neither case was it dominant. It seems that Labour is underperforming (compared to the Conservatives) for Remain voters who seem to defect to the LibDems (seems!).

The depressing chart is the “feeling toward Brexit” (enthused, accepting, resistent):q6-brexit-enthusiast-1024x923

The “all” line is the problem…only 28% are resistant. The Conservatives are definitely the party of Brexit, but Labour has a mixed bag.

Events may change things, but non-Brexit is a long shot given these results.

Bernie woulda won

I don’t go so far as Scott in saying that all “x woulda won arguments are useless”. Counterfactual reasoning is tough and the more divergent the counterfactual from the real would the weaker the possible evidentiary constraints. Analogical evidence from across different elections and election systems is also weak. Combine them and you are in speculation land.

Many of these arguments (pro and con) that I’ve seen are facile: “Corbyn did better than expected so Bernie would have done better than expected and won”. “Corbyn lost so Bernie woulda lost”. None of these are great. Trump was a much better campaginer than May. Republicans were structurally favored (a bit) whereas I think the Conservatives were structurally disadvantaged.

The interesting analogical argument (and I don’t have a source ready to hand, so perhaps I’m making it up) would be of the following form:

  1. Corbyn started from a ≈20 point hole.
  2. Corbyn ran a Bernie style campaign with a Bernie style manifesto and youth enthusiasm.
  3. Corbyn made up a big chunk of that 20 point hole.
  4. So Bernie would have seen a similar gain and won a landslide.

Of course, the problem, in general, is that in polarised two party systems, each marginal gain gets more difficult and acceleratingly so as you get to the rough party parity mark (50% in the US, 40-45% in the UK). Roughly, making up large amounts of ground from a low mark can be much easier than gaining a lead from near parity.

People looking at Bernie’s approval numbers (compared to, say, Clinton’s) might take as a cautionary note how May’s absurdly (we now know) approval numbers didn’t save her. One lesson is that approval (or disapproval) may be more tricky and less sticky than we thought.

My modest conclusion remains: Being Bernie/Corbyn left on policy, rhetoric, and perception just isn’t electoral poison in either the US or the UK and we should stop thinking that it is. Trump made lefty (if lying) noises! It doesn’t mean that left policy alone means victory…as we have direct evidence thereof. It just means that arguments about policy shouldn’t be dominated by a “if left, then unelectable”.

Interpreting the 2017 UK General Election

June 15, 2017

We had yet another election that was yet again rather a surprise, though in a good way this time. The Conservative government intended and were expected to extend their majority in Parliament, perhaps by quite a bit (400 seats seemed possible). They would have 5 years with a commanding majority to execute Brexit and ride out some of the aftermath. The humiliating defeat of a Corbyn-led Labour party would reify his unelectability and might lead to his replacement with someone more electable…probably from a somewhat more center part of the party. This would, of course, depend on Corbyn stepping down and the Parliamentary Labour Party (PLP) not having enough Corbynites to renominate them.

Well THAT didn’t happen. Instead, the Conservatives lost their majority. Labour is back up to around 2010 levels in number of seats. Third parties either lost (SNP) or had only minor gains (Liberal Democrats…big relative, small compared to their 2010 levels). May will try to form a government, but it’ll be tricky. The Conservatives are damaged. Mayesqe Brexit is damaged. (Yay to both.) We have a second own goal/absurd folly by a Conservative PM in as many years.

Excitement! In 2015, the pollsters said, “hung parliament” and we got a Conservative majority. In 2017, the pollsters (mostly) said “Conservative majority” and we got a “hung parliament”. Whatever else, this shows the dangers of mixing polling with time travel.

Corbynites claim vindication! Some Sanders supporters do too! Grouchy people say, “Well, Labour didn’t win!” The anti-Corbynites and Corbyn skeptics in the PLP are making friendly noises! Lefty folks are talking about how the party is Blairite no more!! YOUNG PEOPLE!!!

Here’s my 1.5th take. My 0.5-0.7th takes can be found on various LGM threads.

Caveats

We don’t yet know what happened. The data hasn’t been synthesized or in some cases gathered. So there’s a lot of speculation. This post is mostly speculative or “frameworky”, that is, I’ll try to articulate ways of thinking about the election rather than make specific claims about what happened.

Analyses are emerging and quickly. I started to try to sort through them but they are coming a bit too quickly and voluminously.

I’m pretty left tempered with serious commitments to harm mitigation or reduction. Thus, I am generally against heighten the contradictions strategies or attempting for (unlikely) maximal results. I therefore tend to be a two partier side taker (Labour in the UK and Democratic in the US). In the pre Bush area I experimented with third party politics but that’s largely done (esp. in the US).

I have technocratic inclinations…I care a ton about policy. I have a strong antipathy to Jeremy Corbyn because of his pro-Brexitness and esp his horrible campaigning unto sabotage for Remain. (That is likely the most consequential act he’ll ever do and it’s for a lot of harm.) That being said, the Conservatives are so much worse on everything, I will reliably support (and vote for) Labour. I formally joined the Labour Party a couple of days before the election (when I thought that the  best likely result was “only” a 30-70 seat gain for May).

I don’t have a big hate for the “Blairite” wing of the party, to the degree there is one, even though I disagree with so much of what Blair did (even things that weren’t as harmful as supporting Bush in Gulf War II). I think Brown did some great things including keeping us out of the Euro (I’m super pro-EU, but the Euro has severe problems; cf Krugman) but also a more stimulus based response to the Great Recession (both of these things are what kept Cameron’s contractionary policy from screwing us).

Final caveat: that’s a lot of caveats!

What Happened?

Short answer:

  • Labour didn’t win, but it wasn’t destroyed. The pre election bet was Labour would be hurt through destroyed. We exceeded expectations by a lot.
  • The Conservatives lost their majority but remained the largest party by seat (by a lot) and vote share (by a little). They fell short of expectations by a lot.
  • May is seriously damaged, having lost a majority in a snap election she called.
  • Third parties are on the decline even though the LibDems made some gains. Scottish Tories did very well.
  • We are not in a stable situation. A Conservative minority government with DUP support is unlikely to last 5 years.

Even with my loathing of Brexit loving Corbyn and McDonnell, I’ll take this result over a Labour crash that dislodged them (and definitely over a Labour crash that did).

What Grounded Expectations?

There are two points-in-time to consider:

  1. Pre-calling the election (i.e., before April 2017).
  2. During the election esp. close to the date.

1 is relevant because it grounded May’s decision to call the election. There were two key poll numbers that made calling an election with an expectation of strengthening her majority reasonable:

  1. Overall opinion polls which showed the conservative with around a 20 point lead and gaining.

    UK opinion polling leading up to the 2017 election. The top blue line is the Conservatives and Labour is the red line. Note the vertical line is the moment the election was called. (From Wikipedia.)

  2. Corbyn, specifically, had poor polls esp as a possible Prime Minister, wheres May had really good polls on those.

Traditionally, the opposition party has high polls between elections which narrows when the election is called. Approval of a party leader as possible Prime Minister generally is seen to have a substantial effect (or at least be predictive of) final seat total.

(Remember, the Prime Minister is not directly elected. The only way to vote against someone as PM is to vote against your local MP.)

In 2015, Corbyn started out with the worst initial approval rating of an opposition leader since such polling began (after WWII). He was at -8 net approval. Contrariwise, Ed Miliband started out in 2010 at +26. (Check out this opposition polling at midterms (whole article).)

Note, I’m not offering any explanation of why. Unfair press. PLP backstabbing (though that wouldn’t explain his initial approval per se). It wasn’t clear that it could be overcome. For example, the press wasn’t going to change absent an exogenous event.

Add in his lackluster performance in the referendum, and it wasn’t clear that Corbyn wasn’t electoral poison.

2017 showed that he wasn’t. But it was a surprise.

Why Did Expectations Get Overturned?

In US presidential elections, we have evidence that campaigns and candidates have, at most, marginal effects. Structural factors, esp. given strong polarization and party identity voting pattern, tend to dominate. I’m not sure that this is as true in the UK, but I think we can say that the UK is seeing some hardening of party idea (in the respective bases), but there’s also a lot of party migration/leader sensitivity going on (cf “traditional Labour voters” who voted Leave and speak heavily against Corbyn).

So, I don’t know, but I think looking at structural factors is a bad idea. Brexit is a big deal. The economy and recovery is a big deal. May was never elected as Prime Minister and the winning leader, Cameron, resigned after a wacky defeat. We’re into a second, eventful term.

Let’s look at the seat/vote spread from the last three elections (with some structural points):

Election Lab. Vote Lab. Seats Con. Vote Con. Seats Structural Factors
2010 29.0% 258 36.1% 306 Labour was 3 term govt and were in charge during the Great Recession
2015 30.4% 232 36.9% 330 Re-election campaign of a 1 term govt with a popular leader; Labour collapses in Scotland to the (very left) SNP because Scottish nationalism
2017 40.0% 262 42.4% 317 Snap election (3rd in 2 years) for replacement PM after popular PM suffered a epochal defeat; worst/most botched Conservative campaign in quite some time; UKIP collapses post-Brexit (and Scottish Conservatives do very well on anti-Scottish Nationalism)

If we look at the structural factors, the election looks less exciting. Indeed, if we had a generic Labour leader (with normal opposition Leader numbers in April) this could be seen as an expected or even weak result. But with a generic Labour leader with normal numbers, May would never have called this election. Some combination of polls and perhaps personal underestimation of Corbyn prompted May’s folly.

So Corbyn is Super Awesome, Right?

It’s still inconclusive.

What this election proved is that he’s not electoral poison. Given the right circumstances, he can perform at least as good as a generic Labour leader with normal numbers. Unlike lots of prior performances, almost everything after the first week or two of the campaign was very strong, from the manifesto to his personal campaigning. May refused to debate him, which may have been to his advantage (his PMQ sessions aren’t so wonderful). Hi campaign was strong while May’s was really bad.

There’s some thought that Corbyn mobilised the youth vote to turn out. If so, he didn’t do so on top of bringing back traditional Labour voters, as far as I could tell. That would have been victory, I’d guess. I’m waiting for numbers that show a substitutional effect…making up in increased youth turnout what we lose in traditional Labour voters.

Also, many MP candidates ran away from Corbyn. Or at least kept quiet.

So, it’s not conclusive. But we do know that 1) Corbyn wasn’t poison (that’s good), 2) there seems to be a chance for a hard left to centre intra-party unity, and everyone’s making the right moves (also good), and 3) Corbyn is a different figure than before the election. Success tends to breed success.

That being said, Corbyn still needs to show that he can be a functional parliamentary leader. The great PLP rebellion and leadership challenge was as much about his shambolic management and leadership as ideology and worries about electability. Some moves look good and maybe he’ll do better now. But we need to see.

So UK polls are worthless, then?

UK polls are definitely less accurate overall. Though YouGov was predicting a hung parliament. I suspect Corbyn will turn out to be an outlier.

Approval is odd. In a polarized setting it doesn’t necessarily tell us too much about voter intention. So who knows?

Conclusion

There’s lots more to say even before we get some more data. The short answer remains: Good news for Labour; very good news for Corbyn. May and the Conservative are seriously damaged. They situation is more dynamic than before the election. May screwed the pooch, however rational it may have seemed at the time. It shouldn’t have seemed a no brainer esp given that she had promise no election until the term was up. Breaking that promise “just because” adds an insane amount of uncertainty. The fact that it was structurally a bit unfavourable meant she was relying on Corbyn Numbers and Personal Contempt for Corbyn and didn’t bother to assemble a proper campaign. She knew it was coming but didn’t put one together!

May and Cameron…two of the worst Prime Ministers…ever.

The Comey II Effect

January 16, 2017

Background

US presidential elections are extremely complex events. There are a lot of a lot of moving parts from candidate selection to the Electoral College. The occur only every four years and there’s been under 60 in total, with maybe a third in a reasonably modern era (e.g., with mass communication). Furthermore, the US, in general and in the political scene, exhibit complex dynamics. It’s changing all the time! This makes them hard to study. This XCKD cartoon is a good reminder to be epistemicly humble.

However, this doesn’t mean we should just shrug our shoulders. Predicting winners is going to be hard we predict because increasing polarization, plus some structural features, means elections will be close. When elections are close, they are hard to call (at least individually). That doesn’t mean we aren’t right to predict that they will generally be close and that gaps between the popular vote and the electoral vote will increase and tend to benefit Republicans. These seem quite true.

However, autopsy is different that predicting.  When we look backwards, we are dealing with one set of events, not many possible ones. Now, of course, we can’t go back in time and rerun stuff. Plus evidence degrades quickly with time. But we are looking at something static. And by careful study we can learn surprising things! For example, the consensus is that Ross Perot did not spoil the 1992 election for George Bush. See this excellent blog post by Samuel T. Coop that has good links to the literature as well as a nice discussion both of the direct and possible indirect effects of Perot on the 1992 and 1996 elections. Note that the kind of study one does (and confidence one has) depends critically on how one frames the question. “Did Perot spoil the election for Bush?” has one kind of answer if you operationalise it as “Were Perot voters such that they would have voted for Bush if Perot had been removed from the ballot the week before the election in numbers and patterns sufficient to change the election outcome?” A much harder question to answer is, “If Perot never entered the race, would Clinton still have one?”

The former question might nevertheless be unanswerable or difficult to answer if we don’t have good data. The latter question might well be simply unanswerable absent singularity level simulations.

In general, effect questions about events close to an election are more tractable than ones about events further away. This shouldn’t be surprising.

Comey had two big interventions in the election: The first was in July 2016 when he cleared Clinton of wrong doing, closed the election, and (wrongly and inappropriately) bad mouthed her. The second was when he released a letter a week before the election saying, “Hey! We might have something to look at!” (then another letter just before the election saying, “Oops, nothing here”). There is no disputing the wrongness and inappropriateness of the second intervention. The “best” interpretation is that he was trying to get ahead of rogue agents in New York. Of course, this is several levels of failure including that rogue agents shouldn’t be rogue and the way he did things was strongly biased against (a totally innocent!) Clinton. In any case, it was against policy and precedent, and it was definitely biased toward Trump. We can see this in his current refusal to discuss the FBI’s investigation of the Trump-Russia issue with congress even in private. This is the Bush v. Gore “We only intend this discussion/principle to hold for this one case where it steals the election” bit all over again.

One thing that is conclusively established is that Comey did some very wrong things and that Comey’s FBI is in the running for one of the worst FBIs ever (which is saying something).

But not all wrong actions have bad consequences. (Luck can intervene.) A critical question, thus, is whether Comey threw the election to Trump. Scott Lemieux thinks that the answer is yes, partly based on a recent Vox article by McElwee,  McDermott, and  Jordan which looks at 4 pieces of evidence that “the Comey effect was real, it was big, and it probably cost Clinton the election”.

Some Qualifications

Both Scott and the Vox folks try to disarm one of the standard counter arguments, to wit, that even if Comey II had an effect, it was dwarfed/only made possible by the badness of Clinton and/or her campaign. Their attempt is roughly, “Yes, the campaign made mistakes, but all campaigns do.” Meh. I want to see more affirmative evidence that the Clinton campaign was bad other than “it shouldn’t have been close enough for Comey to affect” or even “She lost to Trump.” The overall evidence is that Clinton is a pretty good candidate (nearly won against Obama and won the popular vote in a year when the fundamentals, the press, and the FBI and Russia/wikileaks, etc were against her). In 2008, we had very specific evidence that campaign competency was a problem (critically, they didn’t pay enough attention to caucuses and delegate math). There’s little such evidence this year. So, pfft. She ran a pretty good campaign in adverse circumstances. Qua candidate, she has vulnerabilities, but clearly also strengths: It seems that it took fundamentals, plus the press, plus Russia, plus Wikileaks, plus the FBI to beat her narrowly.

So, meh, to that.

We’re looking for a change in voter behavior from what it would have otherwise been without Comey II but with no new events. Thus, if some hackers held back some Comey II equivalent info because they thought Comey II was sufficient, then the counterfactual “But for Comey II, Clinton wins” is false. But this isn’t the right standard since we are trying to determine the effect of Comey II on the actual election. This is an autopsy. If someone gets hit in the brain by three bullets spaced three seconds apart, each of which were sufficient to kill them, we don’t say that the first bullet wasn’t the cause of death just because if it hadn’t been fired the person would still be dead.

Finally, even if Comey II had no effect, it would still be unjustified and a serious failing on Comey’s part. If it had some effect that doesn’t seem quite enough to throw the election, it’s still very bad. However, obviously, throwing the election aligns intent and horrific effect.

The Vox Case

The conclusion is:

the Comey effect was real, it was big, and it probably cost Clinton the election

The evidence is in four “exhibits” (and I grabbed their headings to make this list):

  1. “Exhibit 1: the state polls.”
    This is a weird one because it should say something like, “Looking at the state polls, we see a decisive, unusually large shift toward Trump in key states. Indeed, the average is from Clinton +3 on the 28th (Comey II day) to Trump +1.2…4 points! in a week!” However, they are focused on the “surprise” aspect of the election and how some polls just before the election undershots Trump’s actual win. I think this is important (see below), but it’s not part of the first order case for the Comey Victory. They are convincing that some states where the polling didn’t show the trend were underpolled. But this is sorta beside the point. The idea is that there was a swing in votes that was captured by a swing in the polls (and between the polls and the actual outcome).
  2. “Exhibit 2: the national polls”
    Basically, every account of the national polls showed a big hit (2-3 points) against Clinton in reaction to Comey II. Comey I also produced a direct swing in the polls against her. So Comey announcement affecting the polls seems reasonable. Note that this does not yet generate a Trump win…national polls still had Clinton up. But, she did win the popular vote!
  3. “Exhibit 3: The early voting numbers compared with the late deciders”
    Clinton led in a lot of the early voting. That could be a biased sample, but  Clinton had a huge drop between early and election day voting was in blue states like RI. Obama saw gains in such circumstances.
  4. “Exhibit 4: media coverage of email, email, and more email”
    EMAILZ!!! dominated the news coverage and, correlatively, voter perceptions of Clinton, “While 79 percent of registered voters had heard “a lot” about Clinton’s emails, only 23 percent heard “a lot” about Trump’s housing discrimination, 27 percent heard “a lot” about the Donald J. Trump Foundation’s illegal political contribution to the Florida attorney general, and, surprisingly, only 59 percent had heard a “a lot” about the Hollywood Access tape.”

Only 59% of voters heard “a lot” about the Trump tape?! Whoa.

We see some undeniable Comey II effects. 4 (media coverage) is just plainly evident. (Note: Obviously the media were complicit. They could have treated Comey II correctly and didn’t. How this absolves Comey is a mystery to me. He knew or should have known what would happen.) 1 & 2 seem probable both based on the timing and on past effects. 3 needs a bit of work to directly establish the relationship, but as a supporting consideration is quite alright. That all these things march together strengthens the story. As the Vox piece puts it:

Instead, the evidence is clear, and consistent, regarding the Comey effect. The timing of the shift both at the state and national levels lines up very neatly with the publication of the letter, as does the predominance of the story in the media coverage from the final week of the campaign. With an unusually large number of undecided voters late in the campaign, the letter hugely increased the salience of what was the defining critique of Clinton during the campaign at its most critical moment.

Challenges

Let’s recall what has to have happened for a Comey Victory:

  1. Before Oct 28th, Clinton had to have been really ahead. That is, enough people in the right places would have voted for her or, at least, not voted for Trump (e.g., by splitting their ticket or staying home).
  2. On election day, we have the result we have.
  3. People changed their voting behavior (e.g., stayed home, changed their vote, or came out for Trump).

And, of course, this change had to be caused by Comey II. But I think if we can establish 1-3 then we’ve done rather well. Direct polling on this (i.e., “Did Comey II change your voting behavior”) would be welcome but will get less reliable the further out we go (due to recall bias).

So, we have a shift in the polls with the right timing plus a mechanism (coverage). Isn’t this enough?

There are some possible alternatives.

  1. The shift was due to some other factor like awesome Trump ground game. (Unlikely.)
  2. Before or around Oct 28th, Trump was ahead or close to being so and the Clinton lead was a polling illusion.
  3. Trump’s mostly was ahead but there was a strong, consistent polling illusion.

1 suggests a different dynamic. 2 and 3 suggest that the race was more static than a Comey Victory requires.

Could the race have been more favourable to Trump than the polls suggested? Event driven swings in polling, esp. large ones in early polls, have been viewed with skepticism for quite some time. Convention bounces, for example, tend to be bounces: They boost the candidates polls for a short while then fade. The idea that voting intention would be so fickle seems improbable, esp. in this day and age. Recall the myth of the independent voter, that is, in spite of increasing numbers of Americans identifying themselves as independent, only a small fraction are “true” independents, that is, exhibit voting behavior markedly different than some partisan. (And of course, “independent” doesn’t mean “indecisive”.) One rising explanation of such poll volitility without real change is “differential response rates“.

Roughly speaking, given the low response rates typical of modern polling, shifts in polling results can come from a systematic change in who is likely to respond. Note that this is different than standard sampling issues and isn’t addressed by, for example, larger sample sized. It’s also different from problems in your likely voter models. (Assuming your likely voter model is stable, it could give a consistent error but is unlikely to yield big swings.)

Exhibit 3 suggests that the change was real: We have an unusual difference in actual voting behavior from early voting to election day. We have a large number of “undecided” who broke strongly for Trump. So we have a clear causal story.

One challenge with that story is whether the shift from undecided to Trump would have happened anyway. This is similar to the “shy Trump” effect sometimes posited. But basically, people tend to vote identity. Identity doesn’t tend to shift over a campaign, so who people will end up voting for is pretty predictable. But who people think or say they are going to vote for is more flexible. And this isn’t because they are shy (shy people lie) but because they are genuinely conflicted.

Imagine a Bernie supporting, Green curious Democrat. They might well flirt with the idea of voting Green the whole election season but when it comes down to the crunch they vote Clinton because, well, Trump! (I was that sort of Nader supporter in 2000. Even though I though Gore would win and had bought some of the “not a dimes worth of difference” line, I couldn’t risk being part of a Bush win.)

So it is possible that Republicans who were “concerned” by Trump always were going to vote for him. We saw this in a lot of “Never Trumper” elites who declined to support him until they did.

Worse, experientially, many of these folks will take Comey II as the causal reason for their vote. But, the argument goes, that’s a cognitive illusion. In reality, they would have always found a way to vote for Trump. This distinction will be hard to sort out, if even possible.

What Should We Believe About Comey II?

Scott advances a strong (but slippery) version of the Vox conclusion:

As I’ve said before, at this point to deny the effects of Comey’s interventions is essentially trooferism. There is no serious alternative explanation that can account for the data. The “durrrr, correlation is not causation, durrr” argument loses any plausibility when you consider that every Comey intervention caused a wave of negative media coverage about Clinton and was followed by a significant decline in national polls numbers. The “polls can’t account for Trump being a celebrity” response fails to explain why Election Day voters were more affected by Trump’s celebrity status than early voters although he didn’t become more famous in the interim (but people were treated to an obsessive wave of negative coverage about Clinton.) Even if Comey had not sent the letter on October 28, we can be as confident that Clinton would have won as we could ever be confident in such a counterfactual.

The strongest line against it is that voting intentions are fairly stable and we have some mechanisms to explain that polling produced shifts are illusory.  But, let’s not, that there is no direct evidence for this in the current situation. Yes, differential response, shy Trump, and dithery Trump could explain everything, but even for early event responses evidence is thin on the ground. It’s mostly the general thesis that voting intention is stable. And clearly, there’s some truth to that! I was never going to vote for the Republican candidate, ever, in 2016. Ever. There are lots of similar sorts. The general closeness of elections is suggestive as is the research on nominally independent voters.

One interpretation of the election is that campaign quality doesn’t matter at all, given Trumps weak traditional campaign. Specifically, campaign quality doesn’t have a causal effect (as opposed to a balance of causes). However, given Russian hacking, Wikileaks dribbling, Comey, and a compliant press, a rival interpretation is that narrow Republican campaign quality matters less because the Republican campaign includes parts of the US government as well as state and non-state actors as well as the press.

The very closeness of the results in key states is also suggestive. Comey doesn’t have to swing or consolidate a lot of votes in the Clinton firewall to break it. Unlike conventions which are 1) standard, expected events and 2) way early in the campaign before voter intentions have solidified, Comey II was imminent to the election. So it’s more reasonable to suspect real changes rather than differential response.

So, I lean toward the Comey II effect was real and likely made the difference.

Now, there are some definite bias risks here. Confirmation bias (partly due to anchoring) is strong. I’ve been a proponent of a version of the stability hypothesis (though, usually of the “polls far out aren’t super reliable; polls in Nov tend to be” sort). I think differential response rates are fascinating and provide an elegant explanation of convention bounces. (I think all polls should publish their response rates!) I feel Trump is illegitimate on many fronts. Comey’s actions (and the press reaction) are clearly indefensible. So there’s a lot of room for motivated reasoning here.

That being said, it’s clearly possible to swing too far the other way. The evidence for a Comey Steal are more direct and multifaceted. (Early to late voting behavior isn’t subject to polling illusion!) The idea that that coverage had no effect seems pretty bonkers. You’d want some very strong evidence for that.

So, trooferism? Maybe? The Vox picture (however awkwardly put) is pretty compelling. We’ll see how the evidence evolves. It’s still the case that saying, “Pretty compelling, but we still need some details to know how big the Comey effect was” is reasonable, but perhaps on the edge of reason. The “you can’t know!!!!!” folks are clearly way out of line.

Starting to Sort Out the Election

November 29, 2016

There’s lots to sort out. Obviously, that it was a disaster is not something we need to sort out: We know that it was a disaster. The precise contours of the disaster will only become clear over time, but it will be pretty damn bad.

We do need to come to an understanding of what happened, including what we can or cannot know about it. Joe from lowell suggests that the election was analogous to the Iraq war from a punditry (and maybe political science?) perspective: That is, people got it very wrong and those people should have some humility and be treated as at least somewhat unreliable.

I said stuff before the election about the election outcome, so I need to figure out if I’m unreliable (or to what degree I’m unreliable). Note that this isn’t a flagellation exercise, or at least, not intended to be primarily one. If I’m relying on flawed models I should update my models! This election does seem to potentially provide some interesting new data, in any case.

Before the election:

And I really don’t think Karen24’s Llama Drama is because she’s a Rat Rogerer. I think she’s a Nervous Nellie who keeps panicking in the comments here at least partly because she can be so authoritatively reassured by other commenters. For those of us who are freaked out by the reminders that it’s not an automatic total blowout even with Donald Trump on the ballot, because (1) the MSM won’t do its fucking job and (2) so many American voters are still such horrid reactionary dumbshits, I think you underestimate how soothing it is for Bijan Parsia et al. to set us straight.

Here’s an example in that thread of me “setting them straight” with “authoritativeness”:

Look at the pattern though. “Trend” doesn’t mean “current slope”.

And look at several:

http://polltracker.talkingpointsmemo.com/contests/us-president-2016

http://elections.huffingtonpost.com/pollster/2016-general-election-trump-vs-clinton

It’s annoying, but there’s no evidence that he’s winning or that his line will cross hers. If it does, that will be a surprise.

I think that what I said is consistent with what happened, so I guess I would say it again in relevantly similar situations. Should the people who trusted me before trust me in the future? I…guess? I’m not going to overpredict worst case scenarios, but I don’t think I was overconfident.

One thing that did come out of this election was the idea that many aspects of campaigns matter even less than we thought. The classic view about “fundamentals” based models of elections is that modern campaigns are fairly evenly balanced so tend to cancel each other out. So all that’s left are things like the state of the economy.

One thing that was clear in this election was that the campaigns were very lop sided on some key features like get out the vote operations. This suggested that it would be a powerful natural experiment!

I would guess this range underestimates Clinton’s chances, because the models can’t account for Trump’s unusually unprofessional campaign

I’ve felt this a few times, but then I start to worry that perhaps an unusual (or unusually unprofessional) campaign might not be the sort of drag we’ve thought it would be.

Just consider two factors:

1) Advertising, esp. television
2) GOTV, esp. day of ground game

Up until now, HRC has had the airwaves to herself. It’s not clear that it’s done any good. Or, I’d like to know how it’s done good. Maybe it’s too early. Maybe it just hasn’t shown up in polling. Who knows. In the primaries, the rest of the candidates held off and Trump never collapsed. So we don’t really know what advertising will do to this heavily exposed, extremely strange candidate.

What is the effect of GOTV efforts? I believe that it will have an effect, but it might be pretty small compared to random turn out effects. It’s good bit of body english, but it doesn’t seem to be the makings of a blow out. (I’d love pointers to literature on this.)

Maybe campaigns *really* don’t matter?

wtjs formulated an interpretation of possible outcomes:

wjtssays:

But what would show us that the inept vs. ept campaign didn’t matter?

Certainly a Trump win (absent a major exogenous shock like 9/11) would. A win for Clinton along the lines of Obama’s reelection might. A convincing win for Clinton in the popular vote plus a sweep or near-sweep of the toss-up states and a win in one or two “lean R” states in the Electoral College would be pretty good evidence that campaigns do matter.

So, there we go?

One potential confounding factor is “the media” and, frankly, the FBI. One alternative story is that the last minute Comney letters were fairly strong events (given all the media priming on EMAILZ!!!) that worked to the Trump campaign’s advantage.

In general, Democrats seem to have a disadvantage in turn out (as we see dramatically in midterm elections). There are almost certainly loads of factors, but that might start the scales strongly against them. This would explain that the “vote against Trump” effect wasn’t so very strong.

Of course, the simplest explanation is just that fundamentals set the stage and a certain amount of randomness completes the job. If you make it to a major party nomination, you can win. Trump was slightly favored by the fundamentals, strongly hampered by his campaigned, but had a bit of luck.

This isn’t very satisfying! It’s particularly unhelpful for any future planning. (Lots of post mortem analysis is actually quite worthless for future planning. “Nominate a better candidate” doesn’t really help as no one is really in control of the nominations! This conclusion suggests that Bernie would not have done better. If campaign effects are small they are small in both directions. My guess is that he would have been “as likely” to lose…that is, not very likely, but it would be possible.)

Turn out is obviously critical, but our understanding of how to goose turnout is really poor. Voter suppression efforts are pure evil, but it’s unclear that they are turning elections (yet). I can’t find the reference, but apparently the Clinton campaign contacted twice the voters that the Trump campaign did. Now, some are suggesting that they ended up turning out Trump voters, but I don’t think we know the actual effects yet. If we go back to Romney-Obama, we see a large gap in effort but not a dramatic advantage:

We estimate that the presidential campaigns increased turnout by more than 10 percentage points among targeted subgroups, indicating that modern campaigns can significantly alter the size and composition of the voting population.

In this paper, we exploit the 2012 presidential campaign to assess the aggregate effects of a large-scale campaign on the size and composition of the voting population. We take advantage of variation in ground campaigning across state boundaries, extensive information on Romney and Obama campaign tactics, and detailed information on every voter in the United States to estimate the effects of the entire campaign. Our results suggest that the aggregate mobilizing effect of a presidential campaign is quite large. We estimate that the 2012 campaign increased aggregate turnout by approximately 7 percentage points

This analysis also allows us to compare the relative effectiveness of the Obama and Romney campaigns. The Obama campaign of 2012 has been championed as the most technologically-sophisticated, evidence-based campaign in history while the Romney campaign was more traditional (e.g., Issenberg 2013). When we began this project, we surveyed 46 academics, and they predicted that Obama’s campaign was almost 3 times as effective as Romney’s in mobilizing supporters.17 Do these perceptions manifest themselves in the data?

As discussed above, this analysis allows us to roughly compare the effectiveness of the Obama and Romney campaigns in mobilizing their respective supporters. Despite the purported technological sophistication of the Obama campaign and its devotion to a data-driven, evidence- based campaign, we see similar mobilization effects on both sides of Figure 3. The two campaigns were roughly comparable in their ability to turn out supporters.

One interpretation of the Romney campaign’s slight advantage with their own partisans is that Democrats are simply harder to mobilize than Republicans. Indeed, previous research suggests that, on average, GOTV interventions are more effective for conservative and high-socioeconomic- status citizens (Enos, Fowler, Vavreck 2014). The amount of effort and resources needed to mobilize Democratic supporters may be greater than that needed to mobilize Republican supporters. With this in mind, even if the Obama campaign was more advanced than the Romney campaign, this difference was not great enough to overcome this structural disadvantage

If this is confirmed in the 2016 data, it points a direction, at least, for the Democratic party: It needs needs needs to solve the GOTV problem. This would solve all sorts of problems including midterm elections, statehouse controls, etc. etc.

The problem is that we really don’t know how.


On a related note, Nate Silver is sorta claiming that his prediction was more accurate because it gave more weight to Trump’s chances of winning (plus some details of why his model gave such weight…roughly, the scenario that played out was one his model could and did contemplate). I thought that 538’s coverage and the Silver models were poorer than alternatives, esp. Sam Wang‘s. Now , Wang’s prediction was much more heavily weighted toward Clinton (i.e., 99% chance vs. ≈70% in Silver’s).

I need to think more about this.

Epistemic humility and existential dread

November 25, 2016

Health

I’ve been under the weather for months. Fatigue, light headedness, limb tingling, coughing, phlegm…all in different mixes. I date it to February which makes it about 10 months.

Some of this is “standard” for me. I’m immuno-strange both due to my arthritis and my arthritis meds (or so we believe). But this period has been noticeably worse. The past few weeks have been particularly annoying as the “cough/flu” symptoms have been particularly continuous (i.e., runny nose and productive cough and just so tired).

Back in February, we thought it was just some bacterial infections. My sweetie had a respiratory one (confirmed by culture) and we both had some antibiotics (two rounds for me!). There seemed close correlations between the medication and (some) improvement of the symptoms, which is some sort of indicator.

But then I didn’t get any better. And I didn’t get better. And still no. And so on.

We went through a battery of tests. We ruled out diabetes (which I was scared of having for a few weeks while waiting for the results). We also ruled out thyroid problems and B vitamin deficiency. We did a lung xray (it was clear). We did a testicle ultrasound (it was clear).

After a battery of tests we concluded that I was perfectly healthy except for all the crap I was experiencing. These could be partly stress related so I’m on a beta blocker. That’s helpful, but the symptoms have persisted through two months of beta blockers. One might have thought that the regulation of stress the beta blockers bring would have mitigated the symptoms to some degree, but no. The past few weeks testify otherwise.

At the end of all this I’m left in the following state: I don’t know what’s going on; I feel like crap; but I’m probably not going to die or have a severe hospitalisation or anything like that. I do have to figure out how to function, but it’s not the first time I’ve had some long term, debilitating, mysterious condition I had to cope with.

Epistemically, I’m at a loss. No one has any ideas or interest in pursuing further investigations (including me, at this point). Existentially, I’m pretty calm. I don’t feel any acute dread, but my ennui levels are high.

Politics

Trump won the presidential election. Unlike Brexit, I’m hard pressed to think that there will be any way out of it. The Brexit vote was  disaster, but it wasn’t the event itself. The referendum is advisory and didn’t present a specific deal. We have some further legal groundwork which supports the need for more consideration. There are people who are determined to make it happen come hell or high water and the loser opposition continues to be loser (the Labour leadership should be ashamed of itself). But it’s not wholly unreasonable to hope for the longshot.

The Trump adminstration and, thus, unified Republican government is coming. There’s no reasonable hope here. The electoral college is not going to install Clinton. There will not be an overturning recount. There’s not a shadow of a chance. Trump is the next president and the next Congress is Republican and the vacant Supreme Court seat will be filled by them (shameful though that may be procedurally).

I don’t know the precise mix of bad things coming, just that there are lots of bad things coming. Unlike in George W. Bush’s first term, there isn’t a big foreign policy idee fixe a la Iraq. But we have Ryan and the Assault on Medicare (as well as the ACA) to look forward to. Governance (i.e., the basic execution of governmental functions) is likely to be very bad, i.e., Michael Brown FEMA bad is going to look pretty good. We have a dangerous tendency toward personal profit and favoritism that goes well beyond mere incompetence.

Scary stuff.

We learned some stuff in this election, perhaps. Campaigns don’t seem to matter very much. Fundamentals based models did ok (but that’s sort of boring; the Republican won and most predicted a Republican win). Disentangling the causal picture is going to be tricky and take a while (to the degree that we can do so at all), but I think it’s fair to say that, absent some internal campaign data that dramatically shows otherwise, advertising and ground game are not big movers. Clinton’s campaign was far ahead on both of these and it didn’t do the job.

The result wasn’t “impossible” given the polling…just unlikely. The fact that Clinton has a huge popular vote win is rather surprising (FiveThirtyEight consistently gave very low probabilities (≈1%-5%) to this outcome).

It goes back to what I was saying all along: If you are the nominee of one of the two major parties, you can win. Polarization  and voting identity makes this fairly inevitable. The big issue is whether the elite interventions (esp. by the FBI and media follow up on email non-scandals) against Clinton were decisive or just noise.

With both the Brexit vote and the US election, there was reason to believe that the better outcome was coming. I’m trying not to feel too broken about that.

The outcomes are bad enough, but the knowledge failures (even if not strictly failures, due to hedging) are adding a level of additional pain. I strive to be epistemically humble. I’m well aware of the vast panoply of cognitive biases. Furthermore, I’m acutely aware of my own (specific) limitations.

And yet.

Core to my identity is being a knower, however humble, fragmentary, and fallible. “Know” has many shades of meaning and many of them are under assault in the world we are making (even if against our will). I don’t want to know the bad stuff if only because my individual capacity to resist is so limited. People with with positions and platforms a thousand times larger than mind have nearly zero ability to even mitigate these bad things.

Being a witness to disaster is terrible. Not as terrible as being the target of that disaster, but terrible enough.

Connecting

In both cases, we have complex phenomena that resist successful manipulation. There’s a sense that if we just knew the right levers to pull, we could fix things. For health issues, that almost surely is the case. With enough knowledge about the whole of my body, we should be able to alter how it works to fit a desired profile. That level of knowledge and control is far beyond us now, but it seems a bit more possible.

With the political world, it’s less obvious to me. Perhaps we’ll have a wave election in 2018 that sweeps in a Democratic congress, but this is very hard to ensure…maybe impossible. It might still happen, but it’s hard to see what we can do to further it. (Well, other than the bad things happening…but even bad things don’t seem to motivate people in the right way, i.e., against the real causes.)

It is bewildering.

Spinning post-election market reaction

November 8, 2006

Dan Drezner makes this judicious claim:

It would seem that the markets ain’t thrilled with the midterm elections

(The post has a not-so-judicious title.)

Amusingly, I noticed these two headlines on (different pages at) Yahoo!

Yahoo! listing of a “markets are down due to dems” article (from the BBC):

And:

Yahoo! listing of a “Dow is up due to dems” article (AP article)

Fun!

Sober Joy

November 8, 2006

My joy wasn’t all that sober when I saw Santorum and Weldon lose, and lose hard. There may have been delighted yelping plus a little dance in public.

Weldon was one of the first pols I worked against back as a high school student. I remember the bitter taste of his taking office and holding it and holding it and holding it… The victory is very sweet. I want to give it up for my old family, neighbors, and friends who finally kicked him out.

Good job.