By using this site, you agree to our Privacy Policy and our Terms of Use. Close
sc94597 5 days ago
Edit Thread
Cerebralbore101 said:
sc94597 said:

I already explained why we should expect that they have fixed the old polling errors. They fundamentally changed the sampling and weighing methodology, and Trafalgar/Rasmussen are closer to the polls that had the Trump under-representation, this cycle. More evidence to that point is that in 2022 they overestimated Republican chances and the supposed "red wave" never happened. 

In response to your edit.

Imagine you have the following scenario. The true value that is being predicted/estimated turns out to be 51%. 

But let's say using methodology 1, you have pollsters with final predictions of: 49%, 50%, 50.5%, 48%, 52%. The error of each pollster is: -2%, -1%, -.5%, -3%, and +1% respectively. The average prediction is 49.9% (-1.1% error.) 

Now let's say using methodology 2, you have: 47%, 54%, 48%, 52%, 51% The error of each pollster is: -4%, +3%, -3%, +1%, 0% respectively. The average prediction is 50.4 (-.6% error.) 



You can see now how each pollster could be more accurate while their aggregation is less accurate?

Edit: 

To expand, the MAE (mean absolute error) for #1 is 1.5%. The MAE for #2 is 2.2%. But the error by taking the average of predictions is: -1.1% and -.6% respectively. This is a situation where each individual pollster has a closer value to the actual, but the aggregate is further off because the direction of the error matters, and can cancel out. 

Who overestimated Republican chances? I remember Biden being up in the polls nationally in 2020.

I do understand that polls can be wrong but basically come out to have a more correct aggregate if enough polls are wrong in both directions. And vice versa.

https://www.vox.com/policy-and-politics/2022/11/27/23475262/midterm-elections-2022-results-red-wave-democrats

If you were looking at polling averages that included Republican polls, “you were looking at a completely different election than we were looking at,” he added.

When Rosenberg stripped out the partisan polling, he foresaw an election in which New Hampshire, Arizona, Georgia, and Pennsylvania were leaning Democrat, Nevada was too close to call, and Ohio, North Carolina, and Wisconsin were leaning a little Republican. That’s consistent with what actually transpired.

 

As to the second sentence, there is no binary of "being wrong" and "being right" here. The point of polling isn't to get it exactly right at the individual poll level, but to create an aggregate picture that better reflects the population group that we want to estimate. If we cut-off the tails due to the individual poll being potentially biased (but without showing it), which is what pollsters seem to be doing currently (because of the rating system), we can possibly lose part of the picture and can have a worse aggregate because of it. This is called "herding" where the pollsters are refusing to publish results that show a large difference from their priors. It helps their individual rating and error rate, but hurts our over-all predictability. 

https://aapor.org/wp-content/uploads/2022/12/Herding-508.pdf