Amanda, they were nice enough to share the data (I’ll email it) and it looks like Alaska’s fairly well represented, so I’m unsure there’s really a blowout factor to worry about given the measurement Josh employed.

Amanda, I’m not sure how much the blowout factor matters since the statistic at focus is the difference between a given race’s predicted margin and its actual margin. (Irrespective, I assume, of whether it was a blowout or not.) But given your question I reached out to The Upshot to ask how many Alaska races were in the data set, which may clarify things … stay tuned!

@Chris. Point taken. I was just surprised that he didn’t think to take those things into account. Analyzing polls and investigating limitations of such work is supposed to be something that he’s good enough at that the New York Times would see fit to put him on its site. You also don’t mention one of the other main points: how easy it was to call the Senate races in Alaska pre-2004. I haven’t gone through the other states, but I’d bet few state races were so consistently predictable.

Yes, the scope of the data—889 polls since 1992 spread across 50 states—doesn’t allow for a very big per-state sample size, particularly for smaller states such as Alaska. Good point.
And yes, the absence of strong, fully representative voter samples does make it sound (per Hellenthal’s comments) as though sampling bias presents a problem in Alaska polling.

So perhaps Katz subscribed too quickly to his own interpretation without effectively investigating or explaining the limitations of his work. I don’t think either of those two points—that there are too few polls for comfort, and that the presence of sampling issues makes polling difficult—justify scrapping the results. The data offers insight, despite those two limitations. Besides, after this year’s Senate race, we (The Times) can rerun the numbers and, given the attention being given to Alaska this year, help overcome the first limitation.