Online opt-in polls recruit people through advertising, email lists, or self-enrollment rather than random sampling. That makes them vulnerable to bogus respondents: people who answer carelessly or dishonestly to finish quickly and collect a reward. Pew Research Center tested whether three screening methods could separate those respondents from genuine participants.
Pew fielded an online opt-in survey of 11,114 U.S. adults in November 2024. It tested trap questions about impossible or invented activities, CloudResearch’s Sentry automated prescreening system, and matching respondents’ names and addresses with a national voter file. Trap questions flagged 18% of respondents, including 10% who claimed to have visited the International Space Station. Sentry flagged nearly half, most often because free-text answers appeared unresponsive or copied. Voter-file matching validated only 36% of the full sample.
The chart compares trap questions, CloudResearch Sentry prescreening, and voter-file matching across demographic groups, supporting the article's discussion of how each method identifies questionable cases.
The methods improved some quality measures. Trap questions and Sentry reduced “yea-saying”—agreeing with yes-or-no questions regardless of their content—while voter-file matching slightly increased overall error by removing many valid respondents. A failed match does not prove dishonesty: someone may decline to share contact details, be unregistered to vote, or be missing from the commercial database.
This figure directly illustrates the quality-measure results discussed in the article, including the contrasting effects of the three cleanup methods on acquiescent responding.
The table supports the article's warning that failed voter-file matches can remove valid respondents and may make estimates less accurate rather than cleaner.
The political results show why screening cannot be treated as a simple fraud switch. All three methods modestly increased the sample’s overestimation of Democratic support in the 2024 presidential election. Pew attributes this to bogus respondents tending to claim they voted for the winner, Donald Trump; removing them therefore changed the reported vote choice. The study could not directly measure how many bogus cases remained or how many valid cases were wrongly removed, leaving no surefire cleanup method.