At Deck, we build predictions using an approach we call “contextual inference.” Most predictive models in politics are trained on survey responses. Responses are then generalized to a broader audience based on the demographic and socioeconomic traits of the respondents.

Our approach instead captures real data on individuals’ past behaviors and the context around those decisions to anticipate what people in new contexts might do in the future. Rather than primarily relying on survey data, we anchor our models on evidence of past behaviors and robust data on the context around those behaviors.

In this paper, we’ll discuss how we use that approach to determine which people are most likely to shift away from the political party they previously supported for a specific office. These scores are then used in the Deck web app to help users identify persuasion audiences.

In our model, partisan elasticity is the degree to which a voter is willing to shift away from a political party she previously voted for in favor of another. A voter who cast a ballot for Mitt Romney for President in 2012 but voted for Hillary Clinton in 2016 would have demonstrated partisan elasticity. A voter who supported the Democrat both times would not have.

The available data for this exists at the precinct level. If 36% of voters in Precinct A voted for a Democratic state senate candidate in 2014, and 47% of the voters in that precinct supported a Democrat for state senate the next time the office was up in 2018, that precinct would have a partisan elasticity score of 0.11 for 2018.

Note: the voting populations of individual precincts are not stable across time. To address this, we weight our training data by the degree to which pairs of precincts across time are similar.

This model uses traits of voters, traits of candidates, election-related media coverage, campaign finance reports, and election results.

The first step in preparing this model is to assemble its training data. Since elasticity is describing a change that has taken place across elections, we represent information from both the preceding contest for a given office and the current election in our training samples.

We further link the traits of the campaigns within a given contest so that our models can understand the differences between the choices previously and currently in front of voters.

Next, we use our training data to identify the features most likely to have high predictive power -- either alone or in combination with others -- and those most likely to confuse a model into overfitting or diminish the impact of other features. At this stage, we prune highly correlated features and features without meaningful variation, use a technique called VSURF (variable selection using random forest) to better interpret how features will interact with each other, and impute missing data.

Finally, we iteratively design a deep learning architecture to predict our outcome. In this case, we’ve built a six-layer neural network. The model uses the Adam optimization algorithm, optimizing for a low mean squared error.

To validate this model, we trained a version of it with no knowledge of data from 2018. The model was instead only trained on data from 2010 through 2016 -- containing thousands of unique contest pairs and millions of precinct-level result pairs.

The result was a model with significant predictive power. When validated on 10,000 testing samples from 2018, we found that the model’s area under the ROC curve was 0.75, indicating that the model’s ranking of elasticity aligned with the actual ranking of elasticity in our testing samples 75% of the time. The mean absolute error of our testing predictions was 1.5%. And in a lift chart organized by decile, the top decile had a lift of 335 while the bottom had a lift of 31. This means the people with the top decile of scores were 3.4 times more likely than a random person to have elastic partisanship. Those in the bottom decile were less than a third as likely to have elastic partisanship.

Digging into specific contests, we found interesting examples of both good and bad predictions. On the good side (which, consistent with our evaluation metrics, is the large majority of cases),

the swingiest district in our testing data was Kentucky’s HD 87. The average predicted elasticity in this district was 23%. The actual swing was 26%. The least swingy district in our testing data was California’s 15th Congressional District. Our average predicted elasticity in this district was 2%. The results actually shifted by 1%.

On the flip side, our biggest errors mostly occurred in districts that went one or more cycles without a challenge (so the baseline for measuring elasticity was further out). For example, in New Mexico’s HD-68, the average elasticity score was 6%, while the seat actually saw a 13% swing from 2014 to 2018. This is an issue we’ll work to address in a future version of the model.

At the office level, we found that the highest elasticity scores were associated with state legislative races (mean: 8.4%) and state executive races (mean: 8.6%). The lowest elasticity scores were associated with congressional races (mean: 7.1%).

While it’s difficult to measure variable importance in deep learning models, we can use the output of our VSURF runs to estimate which variables have the most predictive power. The most significant variables are described below, grouped by category.

Voter traits

Candidate traits

The goal of these predictions is to improve persuasion targeting. We believe that our context- driven approach allows us to give campaigns the most dynamic look available at which voters may be open to a persuasion message or at risk of switching from supporting a Democrat to supporting a Republican in a given contest. Our reliance on historic data rather than survey data also means that these predictions can be generated, and updated, at a relatively low price point.

However, for well-resourced campaigns, we believe an experiment-informed program would offer a better approach -- running surveys alongside advertisements and voter contact efforts to understand in real time how your specific messages, in your specific context, are landing with specific audiences. This approach is not for most campaigns, since it is expensive and requires significant time and expertise.