MSNBC's Hardball falsely suggested McCain holds statistically significant lead over Obama among white suburban women
SUMMARY: On-screen graphics based on an NBC News/Wall Street Journal poll that aired during MSNBC's Hardball falsely suggested that Sen. John McCain's lead over Sen. Barack Obama among white suburban women is statistically significant because it provided only the poll's margin of error for the overall poll -- not the higher margin of error for the crosstab of white suburban women.
On June 12, Hardball host Chris Matthews aired two separate on-screen graphics, based on an NBC News/Wall Street Journal poll conducted June 6-9, that falsely suggested that Sen. John McCain's lead of 44 percent to 38 percent over Sen. Barack Obama among white suburban women is statistically significant. Specifically, the charts provided only the margin of error for the survey as a whole -- 3.1 percentage points -- and not the margin of error of 9.34 percentage points for the crosstab of white suburban women.
Huffington Post reporter Seth Colter Walls wrote on June 12 that after Huffington Post emailed "NBC's political unit asking for a numeric breakdown of 'suburban women' in their poll," "MSNBC has now provided The Huffington Post with more information on its 'suburban women' finding showing a 44-38 McCain lead over Obama. 'This is within the margin of error of 9.34 percent based on a sample size of 110 within the larger poll,' an MSNBC source wrote over email. (That's three times the margin of error for the entire poll.) This means McCain's 44 percent figure of support among suburban women could actually be as low as 35 percent, while Obama's 38 percent figure could rise as high as 47 percent -- assuming a 95 percent confidence interval (for the stat wonks in the house). Alternatively, McCain could be leading Obama 53-29. While those distant outcomes are less likely true than NBC-WSJ's 44-38 finding, that broad variance raises questions about the statistical usefulness of this one particular crosstab, as opposed to the rest of the NBC-WSJ poll on the whole."
From Walls' June 12 Huffington Post article:
Since that poll was released Wednesday, MSNBC has been reporting the "suburban women" finding often, sometimes hour-by-hour -- perhaps because it reinforces residual doubts about Obama's viability in the 'burbs. The question led Chris Matthews' "Hardball" program at 5pm on Wednesday with a graphic that read "Woman Trouble?"
But how solid was the NBC-WSJ poll's conclusion about those voters? With only 1,000 total respondents in the poll, and no guarantee that the sub-group of "suburban women" was balanced nationally -- meaning that these suburban women polled were drawn from a balanced cross section from America's vast suburbia -- a group of polling experts from across the ideological spectrum told The Huffington Post they viewed the findings with some suspicion.
"I am skeptical about results for smaller subgroups like 'suburban white women,'" said Emory University Professor of Political Science Alan Abramowitz. "There is more random 'noise' with smaller subgroups. How many of these 'suburban white women' were there in the NBC poll out of the 1,000 total registered voters? Figure about 750-800 whites, close to 400 white women, so maybe 150-200 suburban white women. A shift of a small number of voters would change the outcome."
Republican pollster Tony Fabrizio, who you might expect to celebrate the poll, also cast doubt on its findings regarding a preference among suburban women for McCain -- if for no other reason than that any one poll's margin of error increases as the reduced sample size of a "subgroup" becomes smaller and smaller. "If you have a thousand samples, maybe your margin of error for your overall sample is 3.1 percent [the margin cited in the NBC-WSJ poll]." Assuming approximately half of the poll's respondents were women, he said, "your margin of error [when considering them alone] goes up to 4.5 percent. Then take females and segment them among rural, urban and suburban [subgroups]. ... You've doubled your margin of error in that group." [See update below: MSNBC revealed the margin of error tripled for its "suburban women" subgroup.] Fabrizio also said that disrupting the national distribution of a sample by looking at subgroups can throw any analysis out of whack by the same proportion, adding, "there are other vagaries that can go on to influence the sub-sample, too."
Democratic pollster Anna Greenberg agreed with Fabrizio, saying "I'm not sure I believe" the NBC-WSJ numbers on suburban women. She also said it was "bizarre" to single out the suburban women numbers as a statistically significant finding, given the more robust evidence that Obama was competitive with McCain among white women overall. "They missed the big story among women, in my view," she said. "It's not like white women are a 'gimme' for Democrats. Obama is doing significantly better among college-educated white women [than Sen. John Kerry in 2004]."
[...]
MSNBC has now provided The Huffington Post with more information on its "suburban women" finding showing a 44-38 McCain lead over Obama. "This is within the margin of error of 9.34 percent based on a sample size of 110 within the larger poll," an MSNBC source wrote over email. (That's three times the margin of error for the entire poll.) This means McCain's 44 percent figure of support among suburban women could actually be as low as 35 percent, while Obama's 38 percent figure could rise as high as 47 percent -- assuming a 95 percent confidence interval (for the stat wonks in the house). Alternatively, McCain could be leading Obama 53-29. While those distant outcomes are less likely true than NBC-WSJ's 44-38 finding, that broad variance raises questions about the statistical usefulness of this one particular crosstab, as opposed to the rest of the NBC-WSJ poll on the whole.* [Added later]
From the June 12 edition of MSNBC's Hardball with Chris Matthews:
MATTHEWS: And the latest NBC News/Wall Street Journal poll shows that Obama -- Barack Obama -- trails McCain among suburban women voters by 6 points and among white women -- or rather white men, by 20 points. How does Obama connect with white men and keep suburban women from going for McCain? We've got an expert to join us who's been with him in the trenches, Illinois Senator Dick Durbin, his national co-chair.
[...]
MATTHEWS: We're back with the round table and the politics fix. Let's take a look at a new poll here, the NBC/Wall Street Journal poll. It shows that John McCain is leading Barack Obama among white suburban women by six points. But among all white women, Obama's ahead. That six-point spread there is a situation some people, [Financial Times managing editor] Chrystia Freeland, think can be easily reconciled by a simple exposition of the difference between him and McCain on abortion rights.
FREELAND: I think that is a very, very important point, and that's going be a real strength for Barack Obama. You know, the other thing, Chris, that I think is really important to remember is it's less than a week since Hillary Clinton suspended her campaign. And while they were in the heat of the nomination battle, it was very hard for Barack Obama directly to go out for the female vote. He was, after all, up against the first credible female contender for the White House.
So, I think we'll need to wait a few more weeks, post-Hillary, to see how well he does on a level playing field, as it were, with women.
MATTHEWS: Jonathan, it's an interesting thing, we're here talking about ethnic differences and politics. But it is a cutting edge story, and it's fair enough we've got to do it. There is a resistance out there among white voters to the Democratic Party. Let's face it, they have resisted McCain -- I'm sorry, they resisted Kerry. They resisted Gore. White men as a group tend to be Republicans now. They're sportsmen, gun owners, conservative on issues of crime and foreign policy. But doesn't he have to do better than losing by 20 points on that score?
JONATHAN CAPEHART (Washington Post editorial writer): Yeah, he has to. And how he -- exactly how he does that, at this point, is a little beyond me. I mean, he was having trouble with sort of overall blue-collar white voters when he was going toe-to-toe with Senator Clinton. And I think that's why it's going to be very important, you know, once Senator Clinton comes back from vacation and recharges and rejuvenates, that she and her allies -- [Pennsylvania Gov. Ed] Rendell, [Ohio Gov. Ted] Strickland, and all those guys -- get out there and, you know, sing the gospel, if you will, that Senator Obama is the one they should be looking at and not Senator McCain.
MATTHEWS: OK. Cory [sic], I only have 10, 20 seconds.
JIM WARREN (Chicago Tribune managing editor): Chris --
MATTHEWS: Jim, what does he do? Can he do it?
WARREN: Well, yeah, on a couple things. First of all, early data suggests that 60, 70 percent of the Clinton women are going to go his way. And if you're talking about the suburban females, educated, affluent -- they tend to be anti-war. They tend to be pro-environmentalism. They tend to be pro-health care --
MATTHEWS: Right.
WARREN: -- a lot of aging baby boomers taking care of aging moms and dads. They're going to be inclined to Obama.
How could you expect Chris Matthews to know what 44 minus 38 equals? I mean, do we really expect our news anchors and pundits to know 2nd grade math?
I guess the statistics stuff doesn't come until high school, but I'm sure Mr. Matthews has some sort of high school diploma... right?
Also - as a white male, I conducted a poll of myself that concludes that 100% of white male voters will be supporting Obama in November.
This poll has a margin of error of -100%, but I feel that this shows statistically significant support for Obama among white males.
I put as much stock in your poll as I do in most these days, and now they are subdividing white women into white suburban women? What's next, white suburban women who drive three or fewer kids to soccer games every other Saturday?
Good catch MMFA, the margin of error, even though these polls are just topic for silly cable chatter, should have been detailed correctly by MSNBC.
The next poll will be of: left-handed suburban women born on Tuesdays between 3:43 p.m. and 4:19 p.m. with exactly 2.4 children.
Thanks, just so you are aware there is one poster who gets very upset if you are critical of MSNBC. She basically accuses you of supporting FOX.
Of course that is just plain wrong and silly.
As for MSNBC, its conduct has been suspect throughout the primary process, lets just see how much they begin to pull for McCain. With the exception of KO who pulls for Obama the network is pulling for McCain. It is a very partisan network.
Nothing like pre-emptively sliming another poster here, even when they haven't done what you're accusing them of in this thread.
Good job Sue. Maybe you should stick to the topic of the tread, rather than just using empty insults.
Also, i am not targeting a specific poster, I have not named her.
Your post would be accepted by me if you acted the same way when I have been slimed on threads that i do not appear.
Thanks, just so you are aware there is one poster who gets very upset if you are critical of MSNBC. She basically accuses you of supporting FOX
That's targeting a specific poster, even if the posted isn't named.
Your post would be accepted by me if you acted the same way when I have been slimed on threads that i do not appear.
That's immaterial to your own conduct. Grow up.
Not only are you right that she is wholly responsible for her own behavior, and it's not an excuse to cite another's bad behavior, but that other bad behavior she describes exists only in her mind.
Topping that off, here's some of what she's said recently about this "infighting".
I have been asking JLyons to stop with this, so please do not make me apart(sic) of your infighting.- - SueEld / Monday June 9, 2008 4:56:20 PM EDT
J, thanks, but as I said earlier I do not want to get into this discussion. You may have an issue with Bottleblonde, I really just want to stick to issues.I have always tried to be respectful to others.
I do not want to upset Bottleblonde so here goes my post on KO.
- SueEld / Monday June 9, 2008 5:10:21 PM EDT
So she doesn't want to be part of the "infighting", and she has always tried to be respectful and she really just wants to stick to the issues, but she starts off her post last Monday with a smear of me, and then she starts off her post here with another smear.
I have provided evidence of her hypocrisy. She won't be able to provide a smigden of proof of her allegations. Quoting her own words is not a smear, just like Bill O'Reilly is wrong when he calls Media Matters a smear site for quoting his own words, in context.
"Also, i am not targeting a specific poster, I have not named her."
That sentence in itself shows she's talking about a specific person. "I have not named her". Apparently Sue thinks she can talk about a specific person but she's not targeting her if she hasn't named her. Of course the poster Sue's talking about knows exactly who she's referring to, because she's targeted her specifically.
Everyone has their lapses here, but that takes the cake. I don't know if it's incredibly disingenuous or incredibly stupid, but it's remarkable either way.
My blog post at Daily Nightly, the NBC Nightly News blog:
Why is it that NBC Nightly News reported that white male voters are a problem for Obama but didn't report the rest of NBC's poll that shows McCain having problems with (A)women 52/33; (B)Hispanics 62/28; (C)African Americans 83/7; (D)Catholics 47/40; (E)independents 41/36; (F)blue collar workers 47/42 G)white women 46/39?
That means McCain is having trouble with everybody but white, protestant, white collar, Republican males. Shouldn't your newscast have been about McCain's demographic troubles?
Your report also singled out white suburban females as a group Obama is having trouble with. He's down 44/38. That's only six points. Your report characterized the Obama's lead in the overall poll of all voters, six points, as a "very tight race." You characterized the six point lead among the suburban white women group that only makes up 10% of the electorate as a "problem" for Obama. How can 6% of 10% be a big deal when 6% of 100% isn't? What's more, the subgroup still has 18% undecided for Obama to work with.
There are two obvious conclusions. From those numbers, your report shows NBC is still openly supporting McCain and failing its responsibility to provide fair and balanced coverage. And, NBC feels the votes of establishment white folks matter more than anybody else's, particularly when those voters are male.
MSNBC, Matthews et al are leading the poll for Misinformers of the Year.
Look out FOX. You are about to be replaced.
"Suburban White Women": what, like the ones on Desperate Housewives? The ones so desperate, they'd jump on the mailman, or the UPS or FedEx guy, even the garbage man and the milkman, if it wasn't for the fact that they pass by too early in the morning... and the paper-boy? Alright now, cool off! Don't jump on the paper-boy for cripe's sake... he's likely to be all thumbs, thinking you're a video game or something... you're not that desperate... yet!
I couldn't find the "suburban white women" subgroup in that Hart/Newhouse MSNBC/WSJ poll either... I looked, but poorly written confused studies wear me out, and give me a pain behind my right eye, and slightly toward the ear... but it didn't matter: whatever was the number of women who constituted the "suburban white women" subgroup, it was still a part of the inclusive encompassing "women" group, who in that study, made up for 51% of the 1,000 interview subjects (510), and they preferred Mr. Obama to John McCain, 52 to 33 percent...
BANG.
Whatever number was worn by the horse name "suburban white women" is inconsequential, seeing as the much bigger horse named "women", won the race by 19 lengths... preferred Mr. Obama by 19 percentage points.
BANG, for Mr. Obama.
As for Mr. McCain, and whatever bang he gets out of "suburban white women", or whatever bang "suburban white women" get out of John McCain?
Come on girls... you cant't be that desperate!
Even the paper-boy, all thumbs from playing video games late into the night, would be a better bang, than John McCain...
Desperate Houswives indeed.
Where does the traveling band of women castraters stand?
These polls are getting ridiculous. Who gets paid for the nonsense? Why don't we collectively establish a poll and forward it to every news organization. How scary would that poll look? 10% here, 25% here, the most telling would be 99% for WITH.
Tim Russert has passed away.
Very sad Jeter.
Whatever anyone thought of Tim Russert, he was Meet The Press & an institution.
The man had a real passion for politics & it's especially sad that he will not get to see the outcome of this particular election.
The man had a real passion for politics & it's especially sad that he will not get to see the outcome of this particular election.
I agree, Jeter, well said. I always enjoyed hearing his take on the candidates during the primaries. I've watched him cover many presidential elections, but this one in particular one could tell he was excited about. I don' t know how NBC will replace him on Meet the Press when he's so irreplaceable in who fills that seat.
It will be interesting to see who fills that seat now, given Mr. R's agenda.
It seems the New York Times is leading the way in breaking this story on the Internet Wire.
Katharine Seelye's blog states "Mr. Russert was a towering figure in American journalism", and in citing Tom Brokaw's announcement of Mr. Russert's death, notes "a grave Mr. Brokaw called him, one of the premier journalists of our time"
To my knowledge, Mr. Russert was never a Journalist at any time in his life, nor did he ever have anything whatsoever to do with Journalism... it's strange how people can confuse occupations like that: I understand why Tom Brokaw would do it, because it's a great conceit of television personalities, that they think themselves to be Journalists... I don't know why the New York Times though, a place where there are actual Journalists at work, would refer to any television personality as a Journalist.
If "writing and editing" the television show that he appeared on, makes Tim Russert a Journalist, then that means Jon Stewart is a Journalist too... and Stephen Colbert, and his writers, they're Journalists too I guess...
If "writing and editing" a television show makes somebody a Journalist, then there are Journalists at Captain Kangeroo and Bozo The Clown too I guess.
Journalism is news, but the television show Meet The Press was never news, so it was never Journalism.
If Tim Russert was a Journalist, then so isn't bill o'reilly.
Bill O'Reilly is a commentator, not a journalist as I would define the word. Tim Russert is not commenting or opining, he is writing, editing and hosting a news and informational program with guests who are offering analysis and opinion.
If you still can't see it, whatever.
"Tim Russert is not commenting or opining"
That made me laugh, thanks. You don't by chance want to revise that statement, do you?
If you want to pick out little bits and pieces where he may have stepped over that line, fine.....but that doesn't disprove that he's a journalist, that is absurd.
We disagree.
Well, he was overweight--though nothing like morbidly obese. However, that would seem to be a causal factor.
58 does seem awfully young for someone who wasn't in obvious risk. However, that kind of bad luck can hit.
I'm reminded of Pete Maravich, who passed at 40. He turned out to have a rare congenital defect, but nobody knew it at the time. There may be an autopsy of Russert, which might shed some light.
Condolences to his family and friends.
"any man's death diminishes me, because I am involved in mankind, and therefore never send to know for whom the bell tolls; it tolls for thee." --John Donne Meditation 17
Actually, the distortion could be worse than the raw numbers suggest. In any poll with internal breakdowns, you'd expect the breakdowns to vary somewhat. Some subsets would have errors that favor Obama and some would favor McCain, and if you dug through the results looking for results that lean in a particular direction, you're as likely to find the polling errors as you are real data. I believe the technical term for this is selection bias.
As an example, suppose a poll reports 20 subsets, each with 95% confidence. There's a 65% chance that one of the subset results in the poll has an error exceeding its confidence range. If you go through such a list looking for anomalies, you'll find one more than half the time, even if there is no underlying anomaly in the sample space. Cherrypicking from randomized data sets is just a matter of ignoring the rest of the tree. Sometimes the process creates cherries where none actually exist.