• Both comments and trackbacks are currently closed.
• Trackback URI: http://hominidviews.com/wp-trackback.php?p=1561
• Comments RSS 2.0

11 Responses to “Poll Analysis: Obama Gains Very Slightly on McCain”

1. Steve Says:
   

   Tuesday, May 27th, 2008 at 7:09 am

   I’m forced to ask, why do you persist with your horribly flawed methodology when its flaws have been pointed out to you?

2. Steve Says:
   

   Tuesday, May 27th, 2008 at 8:35 am

   Let me expand, based on your replies in a previous thread:

   Your methodology is based on the idea that a 54% result for Clinton in a poll actually means that any individual in the state has a 54% chance of voting for her. This proposition is *absurd*. The number of factors that distort polling vs. electoral performance is staggering, as we have learned many times over in 2000, 2004, and the primaries this year. The polling for the state *must* be viewed as an approximation–not a fact that is reinforced repeatedly by your method of simulating the state’s internal results.

   You use the analogy of a coin with a 54% chance of turning up heads. Your mistake is in thinking because a poll results in a 54% result, each individual voter in the state can therefor be thought of as a coin with a 54% chance of turning up heads. That just isn’t so. The poll is the coin, not each individual respondent–the poll is your data point of 54%, the individuals in the state are not. The individuals in the state are too complex, too varied, too changeable, too susceptible to outside influence, too had to measure accurately, etc. etc. to base your analysis on. If your analysis is to be based on an aggregate figure, like a poll, then work in the aggregate. Trying to move backwards into individual behavior as you do falsely eliminates the uncertainty in the poll.

3. Darryl Says:
   

   Tuesday, May 27th, 2008 at 9:48 am

   Steve,

   “Your methodology is based on the idea that a 54% result for Clinton in a poll actually means that any individual in the state has a 54% chance of voting for her.”

   Umm…no. It is based on the idea that there is a binomial distribution of possible outcomes in a state, given the evidence offered by one or more polls. The distribution of outcomes is a function of the total sample size (only of those who selected the D or the R) and the number of individuals who selected each.

   “This proposition is *absurd*. The number of factors that distort polling vs. electoral performance is staggering, as we have learned many times over in 2000, 2004, and the primaries this year.”

   I haven’t examined the 2000 polling data, but I have used the 2004 data with these methods when I was testing them out. I used a narrower window of polls as the election approached. I’ll post on it sometime, but the results were consistent with the actual outcome of the election.

   “The polling for the state *must* be viewed as an approximation”

   Indeed…polls are simply one form of evidence. There are others, although polls tends to work better than most other methods (experimental “election markets” may be an exception—they may prove better given enough participation).

   “–not a fact that is reinforced repeatedly by your method of simulating the state’s internal results.”

   I’m not sure what you mean by this. A Monte Carlo analysis doesn’t treat data as “fact.” It treats the data as evidence.

   “You use the analogy of a coin with a 54% chance of turning up heads. Your mistake is in thinking because a poll results in a 54% result, each individual voter in the state can therefor be thought of as a coin with a 54% chance of turning up heads.”

   The entire purpose of polling is to find an unbiased estimate of those percentages. The methods pollsters use to assess whether a victory is outside the “margin of error” relies on these same assumptions

   Now…where we likely agree is that some pollsters may not do a very good job of selecting a random sample of future voters in the election.

   “The poll is the coin, not each individual respondent–the poll is your data point of 54%, the individuals in the state are not.”

   I am not sure what you are getting at here. In standard scientific practice, we would say that the poll is an “experiment” in which the surveyed individuals constitute a series of Bernoulli trials (i.e. coin tosses) that have an outcome of D (with some unknown probability p) and R (with some unknown probability [1 - p]). The result of a series of such experiments is some number of D responses and some number of R responses. The count of D and R responses are the data, observed from a sample, that are then used to estimate a probability distribution of possible values for p, which is a binomial distribution.

   “The individuals in the state are too complex, too varied, too changeable, too susceptible to outside influence, too had to measure accurately, etc. etc. to base your analysis on.”

   Your argument is that there is additional sources of variability (or heterogeneity) besides sampling error. And you are correct. There are other types of errors as well —systematic biases that change the expectation of the distribution of p and that may or may not also affect variability. One could try to control for such things, but that requires additional assumptions that I am not willing to make.

   “If your analysis is to be based on an aggregate figure, like a poll, then work in the aggregate.”

   Here is where you have a misunderstanding. I am working with individual-level data, not aggregate data. Given a sample size and an estimate of the mean of p, I know how each individual responded (i.e. the outcome of each Bernoulli trial). Granted…some pollsters make, largely undocumented, adjustments to their results for particular demographic characteristics. I cannot get inside that proprietary black box to do anything about it.

   “Trying to move backwards into individual behavior as you do falsely eliminates the uncertainty in the poll.”

   Incorrect. Uncertainty from sampling error is fully quantified. But the full uncertainty is not quantified because I do not control for the assumption of homogeneity, violations of random sampling, etc. Thus, you can properly argue that my methods underestimate the distribution of electoral outcomes, and I would completely agree with that.

   BTW: “Moving backwards” does not “falsely eliminate” or do anything to uncertainty in the polls. The result of my simulation analyses gives results that are the same as those found using standard methods of statistical inference. That is…the standard error computed using ordinary asymptotic methods for a series of Bernoulli trials will be the same as the standard deviation of results of the simulations.

4. Steve Says:
   

   Tuesday, May 27th, 2008 at 10:53 am

   “Given a sample size and an estimate of the mean of p, I know how each individual responded (i.e. the outcome of each Bernoulli trial). Granted…some pollsters make, largely undocumented, adjustments to their results for particular demographic characteristics. I cannot get inside that proprietary black box to do anything about it.”

   Which means that you’re dealing with an unknown variable by… ignoring it. Which should broaden the distribution of your calculated outcomes.

   Similarly:
   “Now…where we likely agree is that some pollsters may not do a very good job of selecting a random sample of future voters in the election.”

   Again, the range of possible outcomes should increase, your curve should flatten.

   You admit many times, particularly in your FAQ, that you often use “the best information available”, which is not always *good* information. I admit, I’m not a statistician, but I *do* know that it isn’t difficult to insert an x-factor (for example, the 2004 results, national polling, a 50/50 split, whatever you like) into your figures in order to attend to uncertainty. We have only one poll from Puerto Rico on the Democratic primary, for instance–by your methodology, it seems that that 52/37 result would end up with a 100% (or nearly) probability of victory for Clinton in the state. But that is a ludicrous assertion, given the age, small sample size, and un-tested source of the poll, even if the numbers themselves are outside the margin of error.

   In your FAQ, you state that once you’ve determined how many respondents (you believe) individually voted for each candidate in a poll, you then run a trial for that state–using a number of voters equal to the sample size of the poll.

   Again, forgive me ignorance, but why? How is this a valid way to simulate an election? Your simulation, essentially, just employs the law of average to achieve a result that is almost always going to be identical to the poll’s result. Why is the sample size of the poll a magic number that will give you a valid distribution of possible outcomes in an actual election? I understand that this means that polls with larger sample sizes will have less uncertainty to them, but I don’t know why you think that trend will actually correlate with the uncertainty in the state except in a very general sort of way. How can this simulation possibly reflect something like increased turnout, for instance? As far as I can tell it makes no attempt to, which is fine, because that’s an unknown–but you introduce few or no unknowns into your process, either.

   When I say this method falsely eliminates uncertainty in the polls, I mean that by, essentially, rolling the dice using the poll’s number hundreds of times, you’re using the numbers from the poll (which are, to an unknown degree, flawed) to try and determine the uncertainty in the state–which makes very little sense when the uncertainty in the state should be largely derived from our uncertainty with the poll. You’re using the poll as the test of the poll, because the only thing keeping the numbers you draw from the polls from being true 100% of the time in your electoral simulations is the sample size of the poll. It seems that, by your method, if you did a census of California, those results would be treated with 100% accuracy (if only through the law of averages, by running the numbers millions of times), despite the fact that a census and an election would probably actually return significantly different results.

   So, we are agreed in that:
   “Your argument is that there is additional sources of variability (or heterogeneity) besides sampling error. And you are correct.”

   and
   “Thus, you can properly argue that my methods underestimate the distribution of electoral outcomes, and I would completely agree with that.”

   And my argument, then, is that you are wrong when you say things like:
   “One could try to control for such things, but that requires additional assumptions that I am not willing to make.”

   I think it is the opposite. By *not* attempting to control these factors (or at least introduce a random or baseline element to your work) you are making the assumption that the polls you use are almost flawless information. I know full well that you realize they’re not–but your methods don’t. Your methods work with the numbers they’re given, and produce results for those numbers, regardless of what you may think. As such, I think your methods are deeply flawed, and only poorly approximate reality, which should be the goal.

5. Steve S. Says:
   

   Tuesday, May 27th, 2008 at 5:05 pm

   Just so you know, the Steve who seems to believe that your analyses are attempts at making fullproof predictions is not the same person as the Steve S. (me) who’s been commenting on the underpolling of African-Americans.

   -Steve S.

6. Steve Says:
   

   Tuesday, May 27th, 2008 at 5:14 pm

   Steve S–

   I don’t think the analysis is an attempt at “fool-proof predictions”, but it is an attempt at *some* kind of prediction. How useful is it to model how Senator Clinton is faring potentially against John McCain if the analysis is so narrow as to have her already at 100.00%, when, in fact, it is very possible that if an election were held tomorrow she would lose? How can we track her upward movement if her positions increase? How can we realistically compare her performance to Obama’s when this method shows her with three times the chance he does to be elected–when almost every other analysis (including ones which take into account the variables I discussed above) show a much closer comparison? What, in other words, is the point of putting all this obvious hard-work and intellect into an exercise that is, when all is said and done, such a rough, rudimentary guideline?

7. Darryl Says:
   

   Tuesday, May 27th, 2008 at 5:57 pm

   Steve,

   “Which means that you’re dealing with an unknown variable by… ignoring it. Which should broaden the distribution of your calculated outcomes.”

   Maybe it is broadening the distribution of outcomes…maybe not. I mean, the reason pollsters adjust their sample is to make-up for non-randomness in their sample based on demographic characteristics. Since the raw data are not typically released, there isn’t much I can do about their adjustments in any case.

   “Again, the range of possible outcomes should increase, your curve should flatten.”

   Could be…and I can imagine all kinds of ways polls might be biased or non-random. I could imagine pollster-specific effects, poll-specific effects, state-specific effects and secular trends in each of these over time—the possibilities are limitless. If I knew how non-random sampling affected the distribution of outcomes, I could incorporate that into the analysis, but I don’t know that. Instead I focus on the error that is quantifiable from the data (sampling error), and doesn’t require me to assign assumptions about effects and the magnitudes of such effects (i.e doesn’t require me to make shit up and throw it into the analysis).

   ‘You admit many times, particularly in your FAQ, that you often use “the best information available”…’

   Ummm…I use that phrase twice: (1) When there is no polls available (like D.C. now) I use the 2004 outcome as a deterministic best available information. (2) When there is no “current” poll available I use the most recent “expired poll” as the best information available.

   “…which is not always *good* information.”

   Perhaps. But whose to say it is *bad* information??? I mean…what is the gold standard here? It is the general election. So only polls held concurrently with the election can really be evaluated as bad or good.

   “I admit, I’m not a statistician, but I *do* know that it isn’t difficult to insert an x-factor (for example, the 2004 results, national polling, a 50/50 split, whatever you like) into your figures in order to attend to uncertainty.”

   But, as I state in my FAQ, I am only trying to quantify sampling error. I have an aversion to making shit up and throwing it in.

   However, given that I publish the distribution of electoral votes, it is a simple matter to inflate the variance of the distribution by XX percent. An easy way is to take the standard deviation I report and come up with an adjusted standard deviation as SE’ = SQRT(1.XX * SE^2) and then use a Z-test against the hypothesis that the results are greater than 269.

   Better yet, do your own simulations where you assume person-specific values of p that have a beta distribution. This modification is a trivial addition to what I do here…the only problem is finding and justifying the parameters of the beta distribution you pick. Of course, the moment you start posting results, someone will start harassing you about not controlling for non-independence among some states, not dealing with the undecided voters, not correcting for the proper demographic characteristics, and, of course, challenging your assumptions in picking the particular beta distribution you selected.

   Look…here is the thing about any kind of scientific modeling exercise: EVERY model ever created by a scientist has done violence to the underlying reality—they are all gross oversimplifications of reality. A good scientist tries to achieve parsimony in her models—the right trade-off between simplicity and realism. But different scientists have different aesthetics over how complicated a model should become.

   In my efforts, I begin with the assumption that all reputable pollsters attempt to achieve the ideal of an unbiased random sample. Even though I know the ideal is not strictly met, I take it as a given that they pretty much do. And then, as I point out in the FAQ, I find the distribution of sampling error that arises using the bare minimum of assumptions. That is an exercise that is meaningful to me and passes my parsimony test. (And given that I do this work as a hobby, meeting my expectations is what’s important).

   “We have only one poll from Puerto Rico on the Democratic primary, for instance–by your methodology, it seems that that 52/37 result would end up with a 100% (or nearly) probability of victory for Clinton in the state.”

   That would depend on the sample size of the poll.

   “But that is a ludicrous assertion, given the age, small sample size, and un-tested source of the poll, even if the numbers themselves are outside the margin of error.”

   Perhaps…but it seem even more ludicrous to simply make some shit up about the poll. I mean…you don’t even need a poll to do that! Here is a more serious response.

   Sample size: this is explicitly accommodated.

   Untested source: If I don’t believe a poll comes from a reputable pollster, I don’t include the poll.

   Age: the poll strictly only applies to when it was taken. So, what can you do about it? Well you can increase the variability of the poll with age (practically, one simply assumes a smaller and smaller sample of polled individuals with age). This has great intuitive appeal. I considered doing this for my analyses, but ultimately felt that was an unnecessary complication. One must make assumptions about a parametric distribution of aging (e.g. a negative exponential decay, linear decay, etc.), the values of the specific aging parameters (like a half-life), a cutoff beyond which to ignore polls (for practical/computational reasons), and finally, how each of these change as the general election approaches.

   My rules assume a uniform weighting for some unit of time (currently one month, but smaller as we approach the general election), unless there are no polls in the last month in which case only the most recent poll is used. When the rate of polling picks up (as the general election approaches) there will be minimal difference between my method and any reasonable decay-based method.

   “In your FAQ, you state that once you’ve determined how many respondents (you believe) individually voted for each candidate in a poll, you then run a trial for that state–using a number of voters equal to the sample size of the poll.”

   “Again, forgive me ignorance, but why? How is this a valid way to simulate an election?”

   That, my friend, is elementary statistical theory. If you don’t understand this very basic principle of sampling error then you ought to avoid criticizing the work of those who do. (Although your comments about ignoring other sources of heterogeneity are certainly a valid point to raise—even if you don’t have any specific solution).

   “Your simulation, essentially, just employs the law of average to achieve a result that is almost always going to be identical to the poll’s result.”

   Nope. This is only true if there is no sampling error…either there is an very large number of people polled or the entire population is polled. Otherwise, the results of a single election will rarely match the poll results. For example…this diagram comes from a single state poll in Washington state for the gubernatorial contest. The poll results suggested a 53% probability of voting for Gregoire and a 47% chance of voting for Rossi among the 328 people who had a preference:

   

   Clearly, after simulating a million elections, the exact poll result 54%/47% only happened about 5% of the time. The spread of the distribution arises from both the underlying probability (54%) and the sample size (328). If we had more people sampled, the distribution would be narrower. If there were fewer people polled, the distribution would be broader.

   “Why is the sample size of the poll a magic number that will give you a valid distribution of possible outcomes in an actual election?”

   Because the sample size, in part, determines the sampling error.

   “I understand that this means that polls with larger sample sizes will have less uncertainty to them, but I don’t know why you think that trend will actually correlate with the uncertainty in the state except in a very general sort of way.”

   I’m not sure what you are getting at here…the distribution of results of repeated simulations using the poll sample size and using the observed response percentages, generates a non-parametric distribution of p.

   “How can this simulation possibly reflect something like increased turnout, for instance?”

   “Increased turnout” is not something that polls address, except that by (ideally) drawing from a random sample of the poll of voters, poll results should be insensitive to increased turnout.

   “As far as I can tell it makes no attempt to, which is fine, because that’s an unknown–but you introduce few or no unknowns into your process, either.”

   Right…I am only (and intentionally only) dealing with the distributions of sampling error which, aggregated according to the rules of the Electoral College, generate a distribution of electoral votes for each candidates—based on known responses from known sample sizes and the standard assumptions behind a binomial experiment.

   “When I say this method falsely eliminates uncertainty in the polls, I mean that by, essentially, rolling the dice using the poll’s number hundreds of times, you’re using the numbers from the poll (which are, to an unknown degree, flawed) to try and determine the uncertainty in the state–which makes very little sense when the uncertainty in the state should be largely derived from our uncertainty with the poll.”

   I’m not sure what you are getting at here. I’m doing both…I am using the number in the polls to generate a distribution of outcomes. That distribution reflects the uncertainty of having polled only a small sample.

   “You’re using the poll as the test of the poll, because the only thing keeping the numbers you draw from the polls from being true 100% of the time in your electoral simulations is the sample size of the poll.”

   I have no idea what you are getting at here.

   “It seems that, by your method, if you did a census of California, those results would be treated with 100% accuracy (if only through the law of averages, by running the numbers millions of times), despite the fact that a census and an election would probably actually return significantly different results.”

   Still…no idea what you are talking about. If a census were done (i.e. complete enumeration of every voter) then there is no sampling error.

   “So, we are agreed in that…And my argument, then, is that you are wrong when you say things like:”
   “One could try to control for such things, but that requires additional assumptions that I am not willing to make.”

   I think it is the opposite. By *not* attempting to control these factors (or at least introduce a random or baseline element to your work) you are making the assumption that the polls you use are almost flawless information. I know full well that you realize they’re not–but your methods don’t.”

   This statement either reflects a misunderstanding of how the simulations work or ignorance of elementary sampling theory.

   “Your methods work with the numbers they’re given, and produce results for those numbers, regardless of what you may think.”

   Ummmm…yeah. Except for the method of “makin’ shit up,” don’t all methods work with the numbers (i.e. data) and produce results for those numbers?

   “As such, I think your methods are deeply flawed, and only poorly approximate reality, which should be the goal.”

   Uh-huh.

8. Steve Says:
   

   Tuesday, May 27th, 2008 at 6:59 pm

   All right, let me try and simplify this:

   You had no idea what I meant when I used my example of a census in California, so let me explain.

   If you took a census of California asking every likely voter who they would like to vote for, and then actually held an election and allowed people to vote that same day, the results would be almost certainly be different. We can agree there, right? Sampling error isn’t the issue, it’s simply the issue of human behavior polluting the poll. Inevitably, some non-likely voters will vote, some likely voters will not, some voters will change their minds within minutes, undecideds will break one way or another, some people will simply lie, etc. upon etc.

   Therefore, we have uncertainty, despite our lack of a sampling error. Your argument seems to be that, well, you don’t know how to introduce that uncertainty without just “making things up”, which is unscientific. This, to me, seems disingenuous. It seems rather like saying “I don’t know which direction I should be driving, so I’m just going to go ahead and drive in a direction I know is wrong, but not *that* wrong.”

   Polling itself is a guess–there is no inherent reason that a response to a poll question should necessarily correlate to behavior when voting. Now we can look at the evidence and say, yes, historically that is usually the case–But, all it was based on initially was someone’s gut instinct that response to polling *should* match behavior. (And even that is clearly not always the case.)

   Why not introduce other elements that strongly correlate to results? Past election results, for instance. Or consistent pollster-introduced error. Or demographic information. Or primary turnout. Any number of things.

   My problem is, essentially, that you’re doing an analysis of polling and relating that explicitly to electoral probabilities. When I say that you’re using a poll to prove a poll, that’s what I mean–the only uncertainty you seem to be introducing is the uncertainty inherent in the polls themselves, and polls are not elections. It seems to me essentially that you are producing probabilities not of who will win an election, but who will win another round of polls.

9. Darryl Says:
   

   Tuesday, May 27th, 2008 at 7:46 pm

   Hi Steve,

   “If you took a census of California asking every likely voter who they would like to vote for, and then actually held an election and allowed people to vote that same day, the results would be almost certainly be different. We can agree there, right?”

   Yep.

   “Sampling error isn’t the issue, it’s simply the issue of human behavior polluting the poll.”

   …or human behavior polluting the vote.

   “Therefore, we have uncertainty, despite our lack of a sampling error.”

   Indeed!

   “Your argument seems to be that, well, you don’t know how to introduce that uncertainty without just “making things up”, which is unscientific.”

   Yep. But, more importantly, I believe the difference between the actual vote and the census “vote” would be very similar.

   “This, to me, seems disingenuous. It seems rather like saying “I don’t know which direction I should be driving, so I’m just going to go ahead and drive in a direction I know is wrong, but not *that* wrong.””

   It doesn’t seem at all like that to me. The census is certainly a valid way to estimate the election outcome. It would certainly point in the right compass direction down to a couple of millidegrees!

   “Polling itself is a guess–there is no inherent reason that a response to a poll question should necessarily correlate to behavior when voting.”

   Ahhh…no. Polling yields valid estimates of election outcomes. I mean…do you really believe organizations spend hundreds of millions of dollars on polling (whether for elections or product preferences) if the results weren’t generally accurate down to the sampling error?

   “Now we can look at the evidence and say, yes, historically that is usually the case–But, all it was based on initially was someone’s gut instinct that response to polling *should* match behavior. (And even that is clearly not always the case.)”

   Huh?

   “Why not introduce other elements that strongly correlate to results? Past election results, for instance. Or consistent pollster-introduced error. Or demographic information. Or primary turnout. Any number of things.”

   Sure…all these things could be introduced by someone who wished to do so. Some of them might actually improve the results—some might make things worse because of the assumptions needed. (E.g. using 2004 pollster results to “correct” for pollster might not apply if the the pollster has changed or improved the methods used).

   “My problem is, essentially, that you’re doing an analysis of polling and relating that explicitly to electoral probabilities. When I say that you’re using a poll to prove a poll, that’s what I mean–the only uncertainty you seem to be introducing is the uncertainty inherent in the polls themselves, and polls are not elections.”

   Polls are a type of sample survey that is used as a tool to “score” and predict election outcomes. Proper polls quantify sampling error because they are a poll on a relatively small sample, and not an election. I’m using polls in a way that is completely consistent with their design and purpose.

   “It seems to me essentially that you are producing probabilities not of who will win an election, but who will win another round of polls.”

   Well…polls could certainly be used to predict future poll results. But, organizations don’t spend money (as in, many millions of dollars) to predict the results of another poll. They pay money to make inferences about election outcomes.

   Your only arguments seem to be: (1) polls potentially have other sources of error therefore they are invalid. And (2) other information can be used to estimate the results of a hypothetical election (i.e. an election held now).

   One is wrong…the fact that they have other sources of error doesn’t invalidate them. And (2) is correct…other information can be included. But I am analyzing polls…not “other information.”

10. Steve Says:
    

    Wednesday, May 28th, 2008 at 6:57 am

    My point is not that polls are invalid, but that they provide only rough estimations–and an analysis of them should reflect that. If you conducted a census of California the day before the vote and found that 50.5% were going to vote Obama and 49.5% were going to vote McCain, I would not put the odds on McCain as particularly long.

    Yes, people spend hundreds of millions of dollars on polls, but that isn’t proof of anything but their popularity. All a poll does, in of itself, is report who *claims* they are going to vote for a particular candidate. That is what it directly measures. Its usefulness in predicting electoral outcomes is in how that behavior translates to voting behavior. My point is not that the evidence from a poll is invalid, it’s that it is insufficient to predict voting behavior with confidence. You’re not measuring the behavior you’re attempting to predict, you’re measuring a behavior correlated (strongly, but with obvious differences) to that behavior. Your method would have almost undoubtedly produced a near-100% probability of Obama winning New Hampshire in the Primary this year. Now, the polls were wrong–very wrong. And I’m not going to be one of the people who claims polls are useless, but how can any method which relies 100% on polls and produces a 100% probability based on them be useful when reality often bucks polling?

11. Darryl Says:
    

    Wednesday, May 28th, 2008 at 10:35 am

    “My point is not that polls are invalid, but that they provide only rough estimations–and an analysis of them should reflect that.”

    Rough? Hardly. Polls are the single most commonly used tool for evaluating and predicting elections. That is because they tend to work well.

    “If you conducted a census of California the day before the vote and found that 50.5% were going to vote Obama and 49.5% were going to vote McCain, I would not put the odds on McCain as particularly long.”

    Really? I would. My guess is the the census election and the poll election would differ at two decimal points to the right–at most.

    “Yes, people spend hundreds of millions of dollars on polls, but that isn’t proof of anything but their popularity.”

    No…it is proof of nothing. But it is strong evidence that polls provide a useful tool—there is a lot of money spent on them. By contrast, not much money is spent on election prediction via the tea leaf reading, crystal ball reading, interpretation of ancient religious texts, etc.

    “All a poll does, in of itself, is report who *claims* they are going to vote for a particular candidate. That is what it directly measures.”

    Yep…even so…it is the method of choice for election evaluation and prediction. That’s because that proxy measure tends to reflect election outcomes.

    “My point is not that the evidence from a poll is invalid, it’s that it is insufficient to predict voting behavior with confidence.”

    Yet…somehow poll aggregation methods work pretty well.

    “Your method would have almost undoubtedly produced a near-100% probability of Obama winning New Hampshire in the Primary this year.”

    Indeed…but that election is considered a spectacular outlier by the entire polling community. In contrast to your anecdote are hundreds or thousands of polls in which the bias falls comfortably within the margin of error.

    “…but how can any method which relies 100% on polls and produces a 100% probability based on them be useful when reality often bucks polling?”

    Anytime someone says that a poll result is “outside the margin of error” they are simply saying that the poll leader has a greater than 95% chance of truly being ahead of their opponent.

    When I state “100% probability” it is, essentially, the same thing. It simply means that the result is far enough outside of the sampling error that the lead is not attributable to chance. This is ordinary, every day, statistical inference.