Siegel uses “not scientific” IBD/TIPP poll to prop up unscientific Medicus Firm survey

From the March 17 edition of Fox News' America Live:

MEGYN KELLY (ANCHOR): Well, the health care survey conducted back in December is getting some new attention today. It shed some light on how some medical professionals feel about the president's plan to reform the health care system. According to the survey which was conducted by The Medicus Firm, which is a national physician search firm, almost half of doctors who participated say they will either be forced to leave or will want to quit the medical profession if this bill actually passes. Joining us now for a fair debate, Dr. Kathleen London, a family practician, and Dr. Marc Siegel, a member of our Fox news medical a-team. Good afternoon, doctors.

[...]

So this, we're just being told that actually the New England Journal of Medicine, which was originally responsible for posting, not publishing, not conducting the survey, but for posting it on its web site later removed it. It's not a scientific poll, it's a survey, but does it bear any truth in your experience?

[...]

KELLY: Dr. Siegel, do you accept -- do you agree that this survey probably doesn't represent how most doctors feel about this overhaul?

DR. MARC SIEGEL: No, Megyn I don't agree. First of all, there were 1,200 physicians that were surveyed here, and it also reflected what was found in an IBS/TIPP poll that was done back in September where Investors Business Daily also surveyed over a thousand physicians.

Siegel was referring to a September 2009 IBD/TIPP poll which found that 45 percent of practicing physicians would consider leaving their practice if health care reform were passed.

But IBD/TIPP poll's credibility previously refuted by Fox News, Silver

Nate Silver: Poll is “simply not credible.” In a September 16 post to his blog FiveThirtyEight.com, Silver listed five reasons why the IBD poll should be “completely ignore[d]”:

1. The survey was conducted by mail, which is unusual. The only other mail-based poll that I'm aware of is that conducted by the Columbus Dispatch, which was associated with an average error of about 7 percentage points -- the highest of any pollster that we tested.

2. At least one of the questions is blatantly biased: “Do you believe the government can cover 47 million more people and it will cost less money and th quality of care will be better?”. Holy run-on-sentence, Batman? A pollster who asks a question like this one is not intending to be objective.

3. As we learned during the Presidntial campaign -- when, among other things, they had John McCain winning the youth vote 74-22 -- the IBD/TIPP polling operation has literally no idea what they're doing. I mean, literally none. For example, I don't trust IBD/TIPP to have competently selected anything resembling a random panel, which is harder to do than you'd think.

4. They say, somewhat ambiguously: “Responses are still coming in.” This is also highly unorthodox. Professional pollsters generally do not report results before the survey period is compete.

5. There is virtually no disclosure about methodology. For example, IBD doesn't bother to define the term “practicing physician”, which could mean almost anything. Nor do they explain how their randomization procedure worked, provide the entire question battery, or anything like that.

Silver added: “There are pollsters out there that have an agenda but are highly competent, and there are pollsters that are nonpartisan but not particularly skilled. Rarely, however, do you find the whole package: that special pollster which is both biased and inept. IBD/TIPP is one of the few exceptions.”

Fox News itself acknowledged that the poll is “not scientific.” During Neil Cavuto's discussion of the IBD/TIPP poll on the September 16 edition of Fox News' Your World, the on-screen graphic indicated that the poll was “not scientific”:

The Medicus Firm's survey was a promotional document for firm, used unscientific methodology

The Medicus Firm - a medical recruiting firm -- conducted the survey in December 2009. The Medicus Firm, a Dallas- and Atlanta-based firm that recruits and places physicians in jobs, was responsible for conducting the survey. It issued a press release about the results on December 17, 2009. A report written by the Medicus Firm subsequently appeared in Recruiting Physicians Today, an employment newsletter produced by Massachusetts Medical Society, “the publishers of the New England Journal of Medicine.” The report also appeared on the NEJM “CareerCenter” website, but was taken down on March 17.

Methodology consisted of emailing doctors in the Medicus Firm's database. The NEJM CareerCenter article indicated that "[t]he survey sample was randomly selected from a physician database of thousands. The database has been built over the past eight years by The Medicus Firm (formerly Medicus Partners and The MD Firm) from a variety of sources including, but not limited to, public directories, purchased lists, practice inquiries, training programs, and direct mail responses. The survey was conducted via emails sent directly to physicians."

Survey write-up was essentially a promotional document for the firm. After discussing the results of its survey, Medicus touted the importance of physician recruitment firms "[a]fter health reform is passed and implemented":

What does this mean for physician recruiting? It's difficult to predict with absolute certainty, but one consequence is inevitable. After health reform is passed and implemented, physicians will be more in demand than ever before. Shortages could be exacerbated further beyond the predictions of industry analysts. Therefore, the strongest physician recruiters and firms will be in demand. Additionally, hospitals and practices may be forced to rely on unprecedented recruitment methods to attract and retain physicians. “Health reform, even if it's passed in a most diluted form, could be a game-changer for physician recruitment,” said Bob Collins, managing partner of The Medicus Firm in Texas. “As competitive as the market is now, we may not even be able to comprehend how challenging it will become after health reform takes effect.”

Fox News pushed both dubious survey and poll

Fox pushed Medicus survey, falsely attributed it to New England Journal of Medicine. Several Fox News personalities highlighted the dubious survey and falsely attributed it to the credible New England Journal of Medicine.

Fox ran wild with “not scientific” IBD poll. Numerous Fox News media figures highlighted the IBD/TIPP poll, even after Fox News had described it as “not scientific.”