Opinion / Politics

Hard Look at Political Polls – What They Miss

pollsBrace yourself for wave upon wave of … political polls.  The latest show President Trump lagging presumptive Democratic presidential candidate Joe Biden, as usual.  But stop. Let us do what no one does – look BEHIND these polls, understand the process and bias in polling.

Take two recent polls, by NPR/PBS/Marist (NPR) and ABC/Washington Post (ABC).  Both report new anti-Trump findings.  Take a moment and unpack them.

MSN.com – like may non-technical, left-leaning sites – summarizes both.  A stinging line punctuates a leading paragraph.  Says MSN, NPR’s poll “has Biden beating Trump 60 percent to 35 percent among suburban voters.”  ABC’s poll “is the latest to show former Vice President Joe Biden on a roll … up by a 52 percent to 43 percent margin among suburban voters …”

Okay, so narrative is set – “Biden beating Trump,” “Biden on a roll,” stark numbers.  Most readers go no further, but MSN inserts links to suggest you can – so let us do it.  See:  https://www.msn.com/en-us/news/politics/new-polls-show-joe-biden-is-winning-suburbanites-by-a-historic-margin/ar-BB16Vu0y.

Go to links.  What you see is “methodology,” on NPR’s poll some 61 categories defining 1,640 interviewees.  Who is going to study 61 categories?  Not many – but we are.  Here is what you learn.

On process, the survey was done in English and Spanish (no word how many of each), demographic data lags to 2017, interviews were not all “registered voters,” sample size of small groups were unreported, cell phones for business were not accepted, and no response rate is listed.  That is, no report of how many calls resulted in no contact, or “no thank you, not interested in NPR’s survey.”

Any bias here?  What if the survey was just English – our official language? How were nationalized Americans distinguished from illegals? What if demographics were current?  Doesn’t old data change findings?  Which subgroups were “too small to report?”  What if they tipped for Trump?

What effect did omitting “business cell phones” have on findings?  What if owners of the nation’s 30.2 million small businesses have one cell for both business and personal use?  After all, 50 percent of small businesses operate from “home.”  What if these small businesses are pro-Trump entrepreneurs?

Surprise: They are.  Another survey reports “President Trump’s approval rating among small business owners hit an all-time high of 64 percent” in 2020.  If cells used by business owners are omitted, does that affect results? See:  https://www.cnbc.com/2020/02/20/trumps-approval-rating-among-small-biz-owners-hits-64percent-survey-shows.html#:~:text=Small%20business%20owners%20are%20a%20consistent%20source%20of,%E2%80%9CPart%20of%20this%20is%20just%20partisanship%2C%E2%80%9D%20Wronski%20said.

Dig deeper.  In the NPR sample, 38 percent are Democrats, only 31 percent Republicans, with 29 percent Independents.  Does that overrepresent Democrats v. Republicans, and Independents v. both?  What effect does that have on accuracy?

Look closer.  Among Democrats in NPR’s survey, one third more are women than men.  Do Democrat men favor Trump more than Democrat women?  Maybe.  Slice the pie differently.  In NPR’s survey, 52 percent are women, 48 percent men.  Does that affect findings?

How about the fact that older voters – who tend to be more conservative – vote more often?  How does NPR account for that?  Pew Research reported in 2020, while “older voters accounted for 43 percent of eligible voters, they cast 49 percent of ballots.” See: https://www.pewsocialtrends.org/essay/an-early-look-at-the-2020-electorate/.  Moreover, 54 percent of older Americans approve of Trump, compared to 30 percent of youngest voters.  See: https://www.pewsocialtrends.org/2019/01/17/generation-z-looks-a-lot-like-millennials-on-key-social-and-political-issues/.

With respect to age, only 26 percent of NPR’s respondents were aged 45 to 59.  This may be a major flaw, as 35 percent of voters in 2016 were over 50, and NPR’s survey appears to wholly omit voters over 59.  This is particularly meaningful, as 25 percent of all voters in 2016 were over 65.

Another unreported fact should trigger critical thinking – especially on this NPR survey.  The June 26th NPR survey says Trump’s approval fell from 43 percent in March 2020 to 40 percent in June 2020. But Trump’s overall approval rating was 37 percent in March 2017 – five months after winning election.

Pivot to ABC’s latest survey, more bias.  The media does not expect you to ask, but let us ask. ABC’s headline – to which MSN links: “Pandemic Surge Damages Trump, Boosting Biden’s White House Bid.”  But is that what ABC’s data really says?

To get methodology and qualifiers, you must search.  Stressing credibility, they note: “We do accept some probability-based surveys that do not meet our own methodological standards … recommend cautious use of such data.”  Then on shorter survey methodology, the link just pops an error message.

We learn 65 percent of interviews were cell phones, 35 percent landlines.  Taking that as gospel, demographic data show landlines are held by older Americans, cells held by younger Americans.  Does a 2:1 bias for cells tip results?  Maybe.  See, e.g., https://nypost.com/2017/05/04/why-nearly-46-percent-of-household-still-have-landlines/.

How about this:  ABC uses a “cleaning process” that “excludes respondents who have home-based business-listed phones.” Recall more than 50 percent of small businesses operate from “home,” and 64 percent favored Trump.  Does omitting such lines affect survey results? Maybe.

Other biases are unaccounted – or just undercounted.  The ABC poll omits phones in “institutional” settings, but what of senior living communities? Most are conservative – and residents vote.   Numbers are not small.  In fact, the nation has thousands of apparently undercounted senior living communities. See, e.g. https://www.statista.com/statistics/895322/senior-housing-communities-usa-by-region/.

Last, like other surveys, this one is ambiguous on “response rate” – that is, number of failed contacts, and calls ending in “no, thank you, click.”  Despite contrary arguments, response rates matter.  For several years, telephone survey response rates have been falling.  They now stand at a shocking – six percent.  See: https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/.

What does that mean?  Most people just do not answer.  If they answer, they hang up. Those who talk – when told NPR or ABC is calling – might have a motive.  So, net-net do not put great faith in polls.

Even when luminaries at NPR, ABC, and elsewhere say they have 2020 figured out – they probably do not.  They have results they want, based on questions, demographics, and methods supporting the narrative – and a nice, juicy headline.

So, brace yourself for wave on wave … of polls.  But do not put great stock in them.  The reservations often swallow their findings.  Only one poll really counts.  You know it – November 3, 2020.

If You Enjoy Articles Like This - Subscribe to the AMAC Daily Newsletter
and Download the AMAC App

Sign Up Today Download

If You Enjoy Articles Like This - Subscribe to the AMAC Daily Newsletter!

Notify of
Oldest Most Voted
Inline Feedbacks
View all comments
2 years ago

Just caught some of the eulogy for John Lewis on the tube today. President Clinton, Bush, and Obama. Honest Praise for a great man who fought against the injustice faced by blacks. Knowing that he was dying wrote an essay. A line from his essay, “Let them say that it was your generation who laid down the heavy burdens of hate”. A added quote, Together we can redeem the soul of our nation. We as a nation are ready to do that. There are some small segments of the population that may never be ready for that, too small a number to stop us. Listen carefully to Obama’s comments from his eulogy. Are these comments to bring us together or keep the dying embers of division alive.

2 years ago

If the polls mentioned in the article showed Trump slipping against Biden because of the corona virus response, I wonder if the poll asked if respondents knew FIVE vaccines were in development? Likewise, if respondents knew about the development of treatments for those with COVID-19 and paid for by the federal government. The Fox poll from two weeks ago didn’t ask if respondents knew about vaccines being developed, even though Trumps poor polling against Biden was corona virus related. The talking-heads on Fox’s Special Report didn’t mention vaccine development when discussing poll results.

Pat R
2 years ago

They obviously didn’t learn anything from 2016 when their polls showed Hillary winning by a wide margin. Duh….

Lee McQuillen
2 years ago

Polls mean nothing. I don’t answer them. I don’t answer my phone if I don’t recognize the number. I know a great many people who feel the same way. And, fear may lead to speaking out for Biden but voting for Trump. I live where it’s safe to support Trump and most do but I’m afraid when I get out of this area to show any type of support for Trump – fear for my own safety and that of any property I may own. Polls mean nothing!

Bill Brown
2 years ago

Become a Counter-Revolutionary!  Demand Authority ENFORCE OUR LAWS!!  Stand-up for America!!  Remove Every Democrat!!!!!

Bill Brown
2 years ago

Democrats were the Slave Owners and ran the Slave Trade.  They are trying to tear-down their own monuments to cover-up their SINS!!!!!

Bill Brown
2 years ago

The Democrats are Communists!!  The RINO’s are also Communists!!  The Republican Party as a hole is useless!!  We need a new Political Party – The Freedom Party founded from FreedomWorks.org!!  Stand-up for America!!  Make the TRUMP Movement grow – Freedom NOW!!!!!

Nero Wolfe
2 years ago

These so called “polls”that show Joe Biden leading must have been conducted in mental institutions.

Jeannie Huppert
2 years ago

Christians voting plus God = overwhelming majority!

Paul Kline
2 years ago

Your email subject line asks what is my confidence level in domestic travel. I said I would not hesitate to travel by air or land. However, any travel would need to be something I must do. I would not just jump on a plane and travel for pleasure.
Nor would I travel by car unless it was a necessity.

I know there are bad people in our country and we keep hearing about this political football, Covid-19. But I have confidence we will get past all this nonsense and get our lives back again.

2 years ago

In another article, nevertrumper Senator Mitt Romney candidly observed that youngsters, who are more leftist and tech savvy to respond to online polls and ready to interact on cellphones with pollsters, than conservative seniors,are less likely to actually vote. And in 2016 the editor of the New York times stated that it is so essential to defeat, then remove Trump from office after his victory, that journalistic standards of impartiality and truthful fact finding should be discarded to achieve that end of influencing rather than informing the readers. Polling organizations are part of the establishment legacy mainstream media, therefore are on that same mission. About 85% are registered Democrats. ‘Down with you Deplorables’, “Shut up Silent Majority”. We’ll see about that on November 3rd.

Gloria P. Sterling
2 years ago

I have never had any confidence in polls. I vote the way my conscience tells me, not what someone else tries to get me to do. I still remember how Truman was way behind in those “so-called” polls and won the presidency. (I didn’t vote for him, but greatly admired his honesty and taking responsibility for all that he did, instead of blaming someone else).

2 years ago

Polls do not and will never vote!
They were so wrong in 2016 and in the UK they were WAY wrong with Brexit and the last election, Johnson won in a landslide. Largest margin since Thatcher won. The sampling is usually 1,000 or less. Seriously?
Polls can be manipulated and they are continually.
Just saying.

2 years ago

A little cheating here….A little cheating there! These polls are taken by media with an agenda – a leftist agenda! I don’t trust them! Get out and vote!

Tommy Molnar
2 years ago

I think there a lot of ‘old time’ Dems will vote for President Trump but will never admit it to their left wing socialist Marxist friends. The secret voters?

Wendy Smith
2 years ago

I believe most people getting calls from pollsters don’t answer. Also the way questions are posed leave no room for clarification or explanation. In other words question format can be biased from the get-go. I don’t believe the polls, I don’t participate in the polls, but one thing for sure – I vote.

2 years ago

The silent majority will show up in force for OUR PRESIDENT and for OUR COUNTRY

Les Jones
2 years ago

I’m always amused when a) the sample is under 1000 people and b) the headline movement is a 2% shift and the accuracy is
+/- 5%! How is it possible to reach any conclusion from such faulty data?

Bic is my pen name
2 years ago

I would love to see the polls show that Biden is going to win by a landslide. So the Democrats do not even need to go and vote. Just stay at home not need to worry. they have won already. Good job NPR, CNN, MSNBC; the are really doing another Hillary has won repeat.

Jane S
2 years ago

The liberal media must think we’re all imbeciles and that we have a very short memory. They were positive Trump would lose in 2016.

Would love your thoughts, please comment.x