The latest Florida Atlantic poll is so bad, we’ve revived the saltshaker test for it. The whole shaker.

Oh, Canada!

If you are reading the latest poll from FAU/Mainstreet Research, STOP!

It seems compelling, with apparent insights into the presidential race, including how former President Donald Trump and current President Joe Biden would perform in a head-to-head match-up and in a race with independent and third-party candidates. In a good poll, that would give readers some indication of which candidate the spoilers are spoiling. 

The poll also surveys respondents on U.S. Sen. Rick Scott’s re-election prospects, and the Democratic Primary to challenge him (though it’s Alan Grayson in the poll and, as we now know, he’s not running in that race anymore.) It also looks forward at possible gubernatorial candidates to replace Gov. Ron Desantis, including his wife, Casey DeSantis.

So, yes, it’s a tempting read. But, again, STOP!

Several years ago, we stopped writing our ‘Saltshaker’ analysis of polls for various reasons, but the latest poll from FAU screams for correction. So, let’s break out the 2024 version of the Saltshaker test!

This poll is so poorly conceived and constructed that frankly it embarrasses — or at least it should embarrass — an otherwise fine Florida-based institution of higher learning.  

How bad is this poll?  Let us count just some of the ways…

It surveys “adults” 

Any election poll that begins with “among a sample of X adults” should be immediately discarded, for several reasons. First, in Florida only registered voters can vote. Seems obvious, doesn’t it? (Maybe not so much to a Canadian-based polling firm.) When you consider that only about 80% (or so) of adults are registered to vote and, at most, about 80% of them will vote in November, back-of-the-napkin math suggests that about 36% of adults will not vote in November.  So more than a third of the sample could essentially be non-voters. Why bother talking to those who can’t or won’t vote?

Second, if you don’t use the voter file to identify and find actual active voters (which they apparently did not), then you must rely on self-description of how individuals are registered. For a variety of reasons (people forget, people don’t want to disclose, people wish they were a different party, etc.), this also leads to skewed results.  

Party balance

Ugh! The survey had more Democrats than Republicans! All Florida pollsters agree that given current voter registration trends, coupled with past turnout trends, we should see what we call a +7 or +8 GOP model. What that means is that on Election Day the final tallies should end up with about seven or eight percentage points more Republicans than Democrats. Democratic optimists might be shooting for a +5 or +6, but no serious operative thinks there will be more Democrats than Republicans voting.  

Party balance, Part Deux

Also, ugh! The overall balance of the poll is ridiculously off, with more Democrats than Republicans polled. So, the pollster re-weighted the sample in a way that is “intended to represent the voting population in Florida.” That could be kind of OK if done correctly, except it was not done correctly. Instead, the pollsters re-weighted the sample (post-survey re-weighting has problems in and of itself) but they didn’t make it look like the likely voting electorate at all. They re-weighted it to show a +2 GOP model. As noted above, not even optimistic Democrats think that will happen because there isn’t a shred of data that suggests it will. To further make the point, their newly weighted sample has only 19% “independent” voters when, in reality, it should be around 26%-28% “NPA/Minor Party” voters.  

The bottom line

We presume Florida Politics readers are not the typical audience and are more politically active and engaged. With that, we would like to offer a word of caution when reading polls released by either a media outlet or a university. Most of these polls are designed to generate clicks and, therefore, are generally done on the cheap, meaning they simply must cut corners.  

If you are hiring McLaughlin, Tyson, Eldon or some of the other well-established pollsters (like our own Clearview Research) who know and understand Florida, we can’t afford to cut those corners because we wouldn’t last in this business. A good poll costs money and a good poll requires (more than ever) a strict adherence to proven methodologies. In the current circumstance, this simply is not a good poll.

Sorry FAU and Mainstreet, you get the whole shaker of salt.