Bbc_website_needs_a_better_editor_1

Click image for a popup window with full-size image.

And click below for the answer!

Read the rest of this entry »

Advertisements

Edison/Mitofsky today publicly released a 77 page report on what went wrong in their Election 2004 exit polls. 

(CNN story on the Edison/Mitofsky Report available here.)

(PDF file of report currently available here from CNN.)

(Also, the PDF is archived on my Typepad account here.)

[Sidenote: For a full background on what went wrong, please see my post here.  Or, for an even fuller background please see the many exit poll posts of Mark Blumenthal of MysteryPollster.com.   But for the purposes of this post, the key thing that went wrong was that the exit polls—as they appeared on various news websites (e.g., CNN) up to the wee early morning hours after Election Day (i.e., until about 2 or 3 AM on Wednesday, Nov 3)—overwhelmingly predicted better showings by Senator Kerry than the official tallies did in virtually all states: safe Democrat states, safe Republican states, and battleground states.  The mean and median discrepancies between the official (Bush % – Kerry %) vote margin and the exit poll prediction of the (Bush % – Kerry %) vote margin were 3.7% and 3.9% in Kerry’s favor, respectively.  Moreover, in four key battleground states eventually won officially by President Bush—Ohio, New Mexico, Nevada, and Iowa—the exit polls on CNN’s website up until the wee Wednesday early morning hours after Election Day had Senator Kerry ahead of President Bush.]

I’ll likely have much more to write about the Edison/Mitofsky report later.  Their main claim is that the overestimate of Kerry support was predominantly due to Kerry supporters being more willing than Bush supporters to fill out exit poll questionaires, thus skewing the sample.   The overall design of the poll was judged to be sound.  Most pertinently for our post, the precincts that were chosen to be exit polled were won by Bush and Kerry in proportions that statistically were not significantly different from the official tallies.  Thus, the focus of the Edison/Mitofsky report is on "within precinct error (WPE)" and not on error from choice of precinct to be exit polled.

And now here’s the key point of this post.  The key evidence Edison/Mitofsky offer in support of this "differential nonresponse" hypothesis is the following from pages 36 and 37 of the PDF file:

2. Precinct Partisanship:

When the precincts were grouped based on their vote (high Kerry through high Bush), the high Bush precincts have the greatest statistical bias. The average signed WPE increases sharply with the increase in the Bush vote.  A small Bush overstatement exists in the highest Kerry precincts. The analysis is more meaningful if the precincts where Kerry and Bush received more than 80% of the vote are ignored. In the highest Kerry precincts there is little room for overstatement of his vote. Similarly the highest Bush precincts have more freedom to only overstate Kerry rather than Bush. The three middle groups of precincts show a relatively consistent overstatement of Kerry.

Partisanship in Exit Polled Precincts versus Within Precinct Error (WPE) of Exit Poll

[NB: WPE is defined as the Official Bush-Kerry Vote Margin minus the Exit Poll Bush-Kerry Vote Margin.  Positive WPEs mean Kerry did better in official tally than exit poll.  Negative WPEs mean Kerry did worse in official tally than in exit poll.]

Precinct Partisanship Mean WPE Median WPE Mean Absolute Value of WPE Number of Precincts

Highly Democratic

(Official Kerry Vote > 80%)

0.3% -0.4% 8.8% 90

Moderately Democratic

(Official Kerry Vote 60-80%)

-5.9% -5.5% 13.4% 165

Even

(Official Kerry Vote 40-60%)

-8.5% -8.3% 15.2% 540

Moderately Republican

(Official Kerry Vote 20-40%)

-6.1% -6.1% 13.2% 415

Highly Republican

(Official Kerry Vote < 20%)

-10.0% -5.8% 12.4% 40

There was no significant difference between the completion rates and the precinct partisanship:

Partisanship in Exit Polled Precincts versus Completion, Refusal, and Miss Rates of Exit Poll

Precinct Partisanship Completion Rate Refusal Rate Miss Rate

Highly Democratic

(Official Kerry Vote > 80%)

53% 35% 12%

Moderately Democratic

(Official Kerry Vote 60-80%)

55% 33% 12%

Even

(Official Kerry Vote 40-60%)

52% 37% 11%

Moderately Republican

(Official Kerry Vote 20-40%)

55% 35% 10%

Highly Republican

(Official Kerry Vote < 20%)

56% 33% 11%

I have to say that, at first glance, this seems like pretty convincing evidence of a differential nonresponse by party.

UPDATE (Jan 19, 2005 — 10 PM):   But on second thought, I see I made a logical error.   When I first glanced at the data and wrote the post above, I believed the data was "pretty convincing evidence of a differential nonresponse by party" because, as the report emphasized, the precincts with the highest victory margins for Bush had the highest WPEs and those with the highest victory margins for Kerry had the lowest WPEs.

But then I remembered that the fundamental and infuriating aspect of the survey nonresponse problem is that if some group of people refuse to take part in your exit poll, you cannot have any definitive knowledge about them beyond that available at first glance (gender, approximate age, probable ethnicity, and that they–by walking out of the polling place—in all likelihood just voted). Bush versus Kerry voting preference, alas, isn’t one of those things.  [Sidenote: I see nothing in the report—and I must admit I’ve still more skimmed it than read it—that such first glance data on refusers was recorded, let alone analyzed.]

Thus, I now see that the presence of such a trend in the data in and of itself cannot offer any actual corroboration for the hypothesis that differential nonresponse by party caused most of the exit poll discrepancies, though seeing such a trend may suggest a mechanism for the differential response phenomenon, assuming the hypothesis is true.   For example, the trend might suggest that the more Republican that the environs of the Kerry-voter were, the more pissed off he or she was and the more enthusiastic to tell an exit pollster of his or her vote for Kerry. (Though on that note, realize that if there were a constant difference in the nonresponse rate of Bush voters versus the nonresponse rate of Kerry voters across all precincts, it’d be the even precincts and not the strongly Republican ones that should have the greatest WPE due to differential nonresponse.)

In fact, this data cannot do anything more than offer a way to approximate quantitatively the size of the differential nonresponse effect under the assumption that it indeed accounts essentially for the entirety of the exit poll discrepancies.   Again, the fundamental and infuriating aspect of the survey nonresponse problem is that if some group of people refuse to take part in your exit poll, you cannot have any definitive knowledge about them beyond that available at a glance, and Bush versus Kerry voting preference isn’t one of those things.   

So what is the size of the mean differential nonresponse under the assumption that constitutes the entirety of the exit poll discrepancies?   The Edison/Mitofsky report does not provide an explicit estimate.  However, the data it provides that is quoted above suffices to answer this question under the mild additional assumption that third party voting was negligible.   The simple algebra is as follows:

Let K = Kerry’s vote percentage.  Thus, 1-K is assumed to be Bush’s vote percentage.  Let c = the mean completion rate for all voters, and W = the mean within precinct error.  We seek to solve for b = the completion rate for Bush voters and k = the completion rate for Kerry voters.   Thus, we solve this pair of equations

    c  = b(1-K) + kK

    W = [b(1-K) – kK]/c – (1 – 2K)

[The first equation says the mean total completion rate c has to be the average of the mean Bush and Kerry completion rates, b and k, weighted by the respective Bush and Kerry votes, 1-K and K.   The second equation says the mean within precinct error W is the exit poll (Kerry-Bush) vote margin [b(1-K) – kK]/c minus the official (Kerry-Bush) vote margin (1 – 2K).]

which after some simple algebra yields

      b = c + cW / [2(1-K)]

      k = c – cW/(2K)

Thus, using the above quoted data from the Edison/Mitofsky Report and taking K to be the midpoint of the Kerry vote intervals defining the precinct partisanship categories, I thus produce the following estimates:

Estimated Response Rates versus Precinct Partisanship

Precinct Partisanship Mean WPE (W) Measured Total Response Rate (C) Estimated Kerry Response Rate (k) Estimated Bush Response Rate (b)

Highly Democratic

(Assumes K = 90%)

0.3% 53% 53% 54%

Moderately Democratic

(Assumes K = 70%)

-5.9% 55% 57% 50%

Even

(Assumes K = 50%)

-8.5% 52% 56% 48%

Moderately Republican

(Assumes K = 30%)

-6.1% 55% 61% 53%

Highly Republican

(Assumes K = 10%)

-10.0% 56% 84% 53%

The differential response in the moderately Democratic, even, and moderately Republican precincts is 7 or 8 percentage points more for Kerry voters than Bush voters, which prima facie does not seem very large at all, and thus for these precincts the hypothesis of differential response seems like a pretty plausible one to account for the entirety of the exit poll discrepancies. (Honestly, though, what actual data does anyone really have to assess whether a differential response rate is in fact unreasonably large or believeably small?  Again, this data can’t prove the hypothesis beyond reasonable doubt.  It just doesn’t add any further reasonable doubts for this vast majority of the precincts.)  On the other hand, the differential response in highly Republican precincts does seem, prima facie, to be implausibly huge: 31 percentage points if we assume K = 10% or 16 percentage points if we make the most conservative possible assumption that K = 20% (the very edge of the Edison/Mitofsky definition of "highly Republican").   Thus, for these precincts, the hypothesis of differential response seems, prima facie, pretty dubious as a complete explanation of the exit poll discrepancies.

I don’t know what to say about this, but as it’s getting late, whatever I shall say will wait for another day.

If liberals are going to start importing religious language into their lexicon, I’d commend to them the following theological reading from Brother Curtis Mayfield, a reading that quite strangely appears impossible to find in full on the internet, and so I was forced to transcribe it myself.


"(Don’t Worry) If There’s A Hell Below, We’re All Going To Go"

by Curtis Mayfield

As performed on Curtis (Rhino Records, 1970)

(Spoken by a women)

Last night, I was so depressed.  And I broke out the Bible. And I turned to the Book of Revelations.  And if people would just get and read the Bible and read the Book of Revelations, they’d really turn around and straighten up.   This is all we need to do: is just get the Good Book and read it and put it into everyday life.

(Declaimed by a preacher)

Sisters!

Niggas!

Whiteys!

Jews!

Crackers!

Don’t worry.

If there’s hell below,

we’re all

gonna go.

(Musical screams)

Aaaaahhhh!

(Sung by Mr. Mayfield)

Sisters,

brothers and the whiteys,

blacks and the crackers,

police and their backers,

they’re all political actors.

Hurry,

people running from their worries

while the judge and his juries

dictate the law that’s partly flawed.

Catcalling,

loveballing, fussing, and a-cussing.

Top-billing now is killing.

For peace no one is willing.

Kinda make you get that feeling.

Everybody smoke.

Use the pill and the dope.

Educated fools,

from uneducated schools.

Pimpin’ people is the rule.

Polluted water in the pool.

And Nixon talking about, “Don’t worry.”

He say, “Don’t worry.”

He say, “Don’t worry.”

He say, “Don’t worry.”

But they don’t know.

There can be no show.

And if there’s hell below,

we’re all gonna go.

Everybody praying.

And everybody saying.

But when come time to do,

everybody’s laying.

Just talking about, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

(Instrumental)

Sisters,

brothers and the whiteys,

blacks and the crackers,

police and their backers,

they’re all political actors.

Smoke.

The pill and the dope.

Educated fools,

from uneducated schools.

Pimpin’ people is the rule.

Polluted water in the pool.

And everybody’s saying, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

But they don’t know.

There can be no show.

And if there’s hell below,

we’re all gonna go.

Lord, what we gonna do?

(Instrumental)

Tell me what we’re gonna do

if everything I say is true.

This ain’t no way it oughta be.

If only all the masses could see.

But they keep talking about, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

They say, “Don’t worry.”

(Feedback, with drums and bass, fading out)

Not to pick on Al Franken, but whatever he’s paying his co-host Katherine Lanpher, it ain’t enough.

Witness this excerpt from Friday’s show where Mr. Franken succumbs to Democratic Forensic Folly #728: Rebutting an intrinsically disingenuous argument mainly by pointing out its logical inconsistencies:

FRANKEN: The President is going around saying the Social Security system is in crisis, which it is not. It just is not.  The Social Security system is on sounder footing now than it has been for most of the 70 year history of it.   Without altering any part of it, its Trustees say, it can pay full benefits straight through 2042.  Over the next 42 years, its shortfall will amount to just 0.7% of national income according to the Trustees or 0.4% according to the Congressional Budget Office.   That still amounts to—that’s real money—but compared to Bush’s Medicare drug benefit—that’s more than twice the size.   Bush’s tax cuts, if permanently extended, would be nearly 4 times as much, the cost.

Now, part of the way they’re trying to sell the crisis is—Bush said this the other day:

“In the year 2018, in order to take care of baby boomers like me, the money going out is going to exceed the money coming in.”

Now that’s not a crisis, because we’ve been building, building, building surpluses in the Trust Fund for years.   So that’s—and that’s—-because baby boomers will at that time in 2018, demographically, at that point there’ll be more money going out than coming in, but it’ll still be very close.

But guess what?  If you start privatizing it like Bush wants to do, when do you think the Social Security system will start taking in less than it pays out?

LANPHER: I don’t know.

FRANKEN: 2006.  (Starts laughing.)

LANPHER: That’s according to who?

FRANKEN: To Peter Orszag at Brookings, who is the expert on this stuff.  Because the plan is…

LANPHER: That’s in a year!

FRANKEN: Yes, that’s soon as it starts.   Because the plan is that “young workers”—whatever that means—2/3’s of the money they pay into FICA (that’s the withholding tax you pay—you pay—for Social Security) will go into your private account and will not be going into the Social Security Trust Fund.   But the Trust Fund will be paying out the exact same amount.   Because as you remember Bush promising seniors, “Your benefits will not go down.”  So as soon as this program goes into effect.

So if that’s their judge, it’ll take in less than it sends out.   So if that’s their judge of what a crisis is—and that isn’t what a crisis is—the crisis is greatly accelerated on their terms.

I think that’s our ad.   Show Bush saying “Social Security is going to start taking in less than it pays out in 2018.”   Then you have the announcer,

“President Bush says the Social Security Trust Fund will start taking in less money than it pays out in 2018.  But under Bush’s plan, the Trust Fund starts taking in less than it pays out in 2006.   It’s making something that isn’t a problem, a problem—and making it worse!  He gave you a giant deficit.  He’s created no new jobs, and now he wants to make Social Security go broke?!”

You see the benefit of this is that they’d then have to explain it.  I mean this is like their ads, their misleading ads, except this is actually, like, honest.   And then they’d have to do a real complicated explanation.

LANPHER:  No they wouldn’t.  They wouldn’t.

FRANKEN: I don’t know what they would do.   I don’t know how you would explain that.

LANPHER:  You wouldn’t have to, if you’re going to rely on a misleading ad to answer.

Bingo. 

Yes it’s true that, as that old saying of solicitors goes,

"When the facts are on your side, bang on the facts.

When the law is on your side, bang on the law.

When neither the facts nor the law are on your side, bang on the table,"

but that doesn’t mean that once your opponent starts banging on the table that you can just keep calmly arguing the facts and the law and be sure of winning over the proverbial jury.   Banging on the table is done precisely to distract people from the facts and the law.  For remembering this basic fact, Ms. Lanpher deserves a raise.

Today, January 15th, is the actual birthday of Dr. Martin Luther King, Jr.

He was born 76 years ago today. 

More to the point, if it weren’t for that tragic night of April 4, 1968 in Memphis, then Dr. King well might have been with us today.

Somehow, despite hearing every Martin Luther King Day the question "What would Dr. King say about (fill in the blank) if he were alive today?", it never struck me until now that there never has been any fundamental reason this question had to be merely a rhetorical one for my entire life.

Please excuse me now.  I must weep inconsolably.

Looking back on the year just past, I see that I have all too often neglected both my body and my blog.   I thus resolve to eat less and blog more.  Indeed, I intend to combine these two activities and add fat-burning fuel to my diet by blogging.   Despite the fact that four years of PhD work in physics here at MIT has amply proven that major mental effort doesn’t really burn all that many calories, I’m not yet ready to return simply to the cardiovascular training, weightlifting, and martial arts that kept me svelte in my youth.   Instead, I’m hoping that combining major mental effort with intense righteous indignation in that way that only blogging provides will kickstart my moribund metabolism into a lean, mean, calorie-burning machine.    

I’ll let y’all know how this works, and if it does work, expect to see Blog Yourself Thin!  — The Self-Righteous Way to Shed Unwanted Pounds on bookshelves by Christmas 2005.