Who’s Better at Playing Doctor, Boys Or Girls?

Keeping with Fadeback Fridays, which is soon to be an international success, this classic post, which originally ran on 23 December 2016. It’s yet another in an endless supply of Research Shows what scientist desperately want to be true: that men and women are Equal, by which they mean women are more Equal. Our lesson is if you screw with the data long enough, your P will be wee.

The NBC News story “Female Doctors Outperform Male Doctors, According to Study” makes these bold claims.

Patients treated by women are less likely to die of what ails them and less likely to have to come back to the hospital for more treatment, researchers reported Monday.

If all doctors performed as well as the female physicians in the study, it would save 32,000 lives every year, the team at the Harvard School of Public Health estimated.

Yet women doctors are paid less than men, on average, and less likely to be promoted.

“The data out there says that women physicians tend to be a little bit better at sticking to the evidence and doing the things that we know work better,” [Harvard’s Dr. Ashish Jha, who oversaw the study] told NBC News.

The ordinary reader would assume female doctors are always much better than male doctors, and the reason is (partly) because male doctors practice medicine regardless of what the evidence dictates. Worse, they receive greater rewards for their foolish and dangerous behavior.

The NBC story drew from paper “Comparison of Hospital Mortality and Readmission Rates for Medicare Patients Treated by Male vs Female Physicians” in the journal JAMA Internal Medicine by Tsugawa, Jena, and Figueroa. Its main claim is this:

Using a national sample of hospitalized Medicare beneficiaries, we found that patients who receive care from female general internists have lower 30-day mortality and readmission rates than do patients cared for by male internists. These findings suggest that the differences in practice patterns between male and female physicians, as suggested in previous studies, may have important clinical implications for patient outcomes.

Now those “suggests” in the second sentence should set alarm bells ringing. And, indeed, Tsugawa and his co-authors did not measure how doctors practiced, and so even if it were true that male and female physicians had different 30-day mortality and readmission rates, the researchers would have no way of knowing why the differences existed. And neither would NBC.

Let’s Examine the Numbers

What happened was this. The authors collected a sample of about a million-and-a-half “Medicare fee-for-service beneficiaries 65 years or older who were hospitalized in acute care hospitals.” Mean age of patients was about 80. The NBC summary misleads by saying just “patients,” which implies the research applies to everybody and not just elderly Medicare patients.

Here’s the conclusion (my emphasis):

Patients treated by female physicians had lower 30-day mortality (adjusted mortality, 11.07% vs 11.49% …) and lower 30-day readmissions (adjusted readmissions, 15.02% vs 15.57% …) than patients cared for by male physicians, after accounting for potential confounders.

First note the differences are small: 11.1% versus 11.5%. And then realize these are the “adjusted” and not actual numbers. Adjusted?

They mean adjusted using the statistical technique of regression modeling. It’s complicated, but everybody forgets that regression is an equation that describes how the guts inside a model vary as “covariates” or “confounders” do (these covariates are other possible explanatory variables). Those guts (which are called parameters) are not observable and do not make direct statements of what can be observed, like 30-day mortality. Hence, the researchers should only have spoken indirectly about 30-day mortality while also acknowledging the uncertainty that accompanies their statistical models.

The researchers did not do this; hence their results are stated in terms which are too sure. In their favor, theirs is a common error (discussed in this book).

An Overdose of Covariates and a Much More Plausible Explanation

Here’s another problem. According to the supplemental information to the paper, they crammed greater than 1,000 covariates into their models.

Greater than 1,000 covariates!

Any statistician will tell you that over-stuffed regression models like this are bound to lead to an uninterpretable morass. Nobody can have a clear idea what is going on with the actual 30-day mortality after all that adjusting. But because of the huge sample size and all those covariates, the model will look like it’s performing well (that is, it will evince wee p-values).

Are there other possible explanations to account for the small differences noted by the models? Yes. Female docs were about 5 years younger on average, and female docs also treated many fewer patients on average than men. This implies women docs had more time per patient.

Even more intriguing, we also know “female physicians treated slightly higher proportions of female patients than male physicians did.” And since women live longer than men, particularly at those advanced ages, maybe — just maybe! — any slight change in mortality or readmission rates between male and female docs could be explained by women doctors treating more longer-lived patients.

That explanation is surely as or more plausible than results from an unnecessarily complicated statistical model. It also eliminates the unwarranted theorizing about how women physicians are “better at sticking to the evidence” and are thus “underpaid.”

Subscribe or donate to support this site and its wholly independent host using credit card click here. Or use the paid subscription at Substack. Cash App: \$WilliamMBriggs. For Zelle, use my email: matt@wmbriggs.com, and please include yours so I know who to thank. BUY ME A COFFEE