At Deck, we build predictions using an approach we call “contextual inference.” This approach uses hard data on individuals’ past behaviors, and the context around those behaviors, to predict what new people in new contexts might do. In this documentation, we’ll discuss how we are using that approach to determine which registered voters are most likely to cast a ballot in a given election.

WHAT MAKES OUR APPROACH UNIQUE

Most turnout predictions in Democratic politics use a “flash forward” approach. With this method, modelers identify past elections that might be similar to an upcoming election, then train a model to predict which traits predicted a voters’ decision to cast a ballot in the past. That model is then used to predict individual-level voter turnout in an upcoming election.

However, this approach doesn’t always work well. In 2016, 2018, and 2020, the size and composition of the electorate broke from historic precedent in significant ways. And in less high-profile elections, we’ve seen enormous variability. This can even be true when two elections are on the same day in the same region for the same office.

On June 20, 2017, special elections for US House seats were held in Georgia’s 6th district and South Carolina’s 5th district. The race in GA-6 was expected to be very close and it became a national flashpoint for assessing where the country’s politics were headed just months after Donald Trump was inaugurated. Campaigns and outside groups spent over $55 million and 260k voters cast a ballot — 24% more than in the 2014 midterm election. In the end, Karen Handel beat Jon Ossoff by 3.6pp.

On the same day, 250 miles to the east, the special election in SC-5 attracted 96% less media attention. Campaigns and outside groups just over $2 million. In the end, 88k ballots were cast and Ralph Norman, the Republican, beat Archie Parnell, the Democrat, by just 3.1pp — a smaller margin than the race in GA-6! In SC-5, the electorate included only the most engaged voters, who at that time were Democrats. In GA-6, the high level of spending and media coverage had engaged a much larger electorate.

To us, this illustrates that each person’s likelihood of voting hangs on a number of contextual factors related to each specific election and the campaigns they’ll find on the ballot. We’ve built an approach that recognizes this — incorporating all of the contextual factors related to each election that may have influenced a registered voter’s decision to cast a ballot.

THE DATA WE USE

This model uses traits of voters, traits of candidates, election-related media coverage, campaign finance reports, and election results.

  • Voter traits — We use historic snapshots of each state’s voter file from TargetSmart (for soft side customers) and the DNC (for hard side customers) to determine a set of demographic and socioeconomic traits for American adults registered to vote at the time of various past elections.

  • Candidate and election traits — We rely on Ballotpedia, VoteSmart, Open States, Reflective Democracy, and state election agencies to collect information on the candidates who ran (or are running) for office — including their incumbency status, endorsements from issue advocacy organizations, demographics, history of holding elected office, and more. We also use these sources to identify when elections take place and what rules govern voting eligibility and access.

  • Media — We license historic and current online, print, and TV news content from Critical Mention, Aylien, and the Google Programmable Search API. We then identify articles and clips related to specific candidates and elections, and match them to the appropriate campaigns. We also use natural language processing tools (both licensed and built by our team) to parse the topics discussed in coverage and the sentiment of candidate mentions.

  • Finance — We gather itemized and summary campaign finance data from state campaign finance portals and the National Institute on Money in Politics. We then match contribution records to individual registered voters, allowing us to track how the traits of a campaigns’ contributors are changing over time.

  • Results — Finally, we gather election results at the district, precinct, and Census block level from Voting and Election Science Team, the MIT Election Lab, MGGG, Open Elections, Statewide Database, and a number of state and county public election result portals to validate the vote totals found in our historic voter file snapshots.

HOW THE MODELS ARE BUILT

The first step in preparing this model is to assemble its training data. To do this, we combine a registered voter’s traits (including their demographics and previous turnout record) with the details of each election (including whether it’s a primary or special election, the ease of casting a ballot, and any eligibility constraints), the traits of the candidates on the ballot (e.g., their incumbency status, demographics, and ideologies), and contest-level traits for all of the races on the ballot (e.g., which demographic groups are making contributions to campaigns, how much coverage of each race is in local media, and how much is being spent) to take a holistic view of a potential voter and the factors that might have influenced their decision to cast a ballot.

Next, we use our training data to identify the features most likely to have high predictive power -- either alone or in combination with others -- and those most likely to confuse a model into overfitting or diminish the impact of other features. At this stage, we (1) prune highly correlated features and features without meaningful variation, (2) use a technique called VSURF (variable selection using random forest) to better interpret how features will interact with each other, and (3) impute missing data.

Finally, we design a deep learning architecture to predict our outcome. In this case, we’ve built a six-layer neural network. The model uses a sigmoid activation function and a stochastic gradient descent optimization algorithm, optimizing for a low Kullback–Leibler divergence.

EVALUATING ACCURACY

To validate this model, we suppressed a randomly selected batch of elections from our training data. We then used the data associated with those elections to test the model’s accuracy.

Last updated Nov 4 at 11pm ET

When validated on over 35,000 testing samples, we found that our turnout model’s area under the ROC curve was 0.91, indicating that the model’s ranking of ballot return likelihood aligned with actual ballot returns in our testing samples 91% of the time. The overall binary accuracy of our testing predictions was 82%. And in a lift chart organized by decile, the top decile had a lift of 224 while the bottom had a lift of 2. This means that voters with the top decile of scores were 2.2 times more likely than a random voter to return their ballot. Those in the bottom decile were about 2% as likely to return their ballot.

Observations

Actual mean

turnout

Predicted mean

turnout

Lift

3,500

0.97

0.95

224

3,500

0.90

0.85

207

3,500

0.78

0.71

179

3,500

0.64

0.57

147

3,500

0.46

0.41

106

3,500

0.30

0.28

69

3,500

0.18

0.17

41

3,500

0.09

0.10

20

3,500

0.03

0.05

6

3,500

0.01

0.01

2

MOST VALUABLE PREDICTORS

While it’s difficult to measure variable importance in deep learning models, we can use the output of our VSURF runs to estimate which variables have the most predictive power. The most significant variables are described below, grouped by category.

Voter traits

  • Turnout in past elections

  • Contact info stability

  • Marital status

  • Party affiliation

  • Age

  • Presence of children

  • Religion

  • Race and ethnicity

  • Education attainment

  • Urbanicity

Election traits

  • Volume of media coverage

  • Election stage

  • Offices on the ballot

  • Ease of voting

  • Number of open seats on the ballot

  • Number of campaign contributions

  • Demographics of contributors

  • Demographics of candidates

  • Generic congressional ballot polling

  • Election timing

Did this answer your question?