About · Sheaf Research

You see a poll and someone cites a number. Next to that important number is a " margin of error," like ±3 percent. That margin describes exactly one thing: How much this number would change if the random sample were changed. It is a real factor, but it's the smallest one that stands between that number and knowing the truth.

So what are some bigger ones? Who answers (or doesn't answer!) the questions, how the answers are adjusted, and how the questions themselves are asked. Changes in any of these can easily be large enough to decide an outcome.

You probably know this from San Francisco elections, which often use ranked-choice voting. The outcomes of these elections are sensitive to small changes in the orders in which people rank candidates. Topline numbers like first-choice preference can't show you that. If a poll misses voters with different ways of ranking, then the results can be far away from the truth.

You generally aren't given an analysis of poll sensitivity, because whoever sent you the poll didn't even think about the problem, couldn't analyze it, or worse: they're being cagey.

It isn't just polls. You get shown a chart, two lines going up, and someone tells you one caused the other. Nobody shows you what else could explain it. Rude.

Our standards

Every result comes with a sensitivity analysis

For polls, we show how far missing voters would have to differ to change the winner, alongside estimates of how sensitive the election outcomes are to changes in poll numbers. For other analyses, we show whether the conclusion survives plausible changes to the model and the data. Either way, we state the assumptions and how large a change would have to be to alter the conclusion .

The analysis is as sophisticated as the problem requires

Analyzing a ranked-choice election needs a full model of the ballot, because small shifts in rankings can change who gets eliminated. A two-candidate race doesn't, and we say so. Some charts can lead you to conflate correlations with real causes, and we only show you the ones the data supports. When the data can't give you an answer, we tell you.

Our objections come before yours

We make the strongest argument against each result , as someone who wants it to be wrong would.

We follow disclosure standards

We're not shady about what we've done, because we believe in our methods. We follow the industry-standard best practices for methodological disclosure in public opinion research.

Why & who

Why we're doing this

Polling and analysis for politics are expensive, and much of the expensive work isn't rigorous enough to guide political decisions. Professional societies that take statistical analysis seriously have made clear what good practice looks like, and pollsters and political consulting firms often don't follow it. We work outside that industry, hold ourselves to those standards, and publish the results so that anyone can check them. Rigor shouldn't be a luxury.

Who's behind this

For now, Sheaf Research is a small group. The founder: a mathematician with three degrees, two decades of applied experience, and a singular interest in getting things correct. He's got a PhD, has both done and taught statistics in academia and industry, and is a member of the American Association for Public Opinion Research.