Busy Saturday, so a plea from reader Aman Rastogi:
I am a student from Lucknow University, INDIA, pursuing my Masters in Public Health there.
Few days back I have watched two of your videos on youtube about statistical fallacies and crisis of evidence in public health and it has changed my perception of viewing the collected data, thank you for that.
In those videos you were saying that without observing each and every individual we cannot come on a correct conclusion about the causation or even the association of the problem with the disease because it may give a lot of statistical junk that we would believe.
So, what would you prefer for a country like INDIA where even the data collection is a big problem because of so many reasons like the weak health information system, low salaries and huge population covering burden on the local data collector, a not much interest of people itself, etc. here are so many uncountable problems to face for a health professional. So, what can be a one solution to counter this problem and engaging all of the needed population with a reliable statistical data.
Because the data suffers here when it goes from one level to another as either professionals don’t want them to be reported or some other reason. And the policy makers get a very manipulated data that arises the big problem. Because at first the data was not collected with keen observation and then it got manipulated.
Even the big shot organizations at state level or organization like UNICEF have to rely on the collected data, whatever it is, and then they make up the policy for it.
So please give me some sort of solution to counter this problem or please publish the solution in one of your paper or book ASAP.
I put this up for reader discussion since I know little about Indian health care and almost nothing about how the Indian government collects data.
But I do know about “anecdotal” data, which has been given a bad name. “Observational” or “anecdotal” data have different senses. The first are the daily living “data” that comes to us unbidden via regular experience, “data” which is responsible for tradition, commonsense, stereotypes, street knowledge, and so on. This is usually great data, and there is little wrong with the judgments we make using it.
Nobody bats 1.000, of course. For instance, Steven Goldberg in When Wish Replaces Thought and Fads and Fallacies in the Social Sciences shows us that our stereotypes are usually correct in their form. But they aren’t always right in their theory, i.e. what caused the stereotypes to be true.
The second sense is what we usually think of as observational data, collected ad hoc, say, from health ministries, and not gathered from controlled experiment. I use control in the same sense an engineer or physicist does, actual material control of a thing, and not in the statistical sense, which isn’t control at all but a way of seeing how uncertainty might change as a thing changes. That people mix these uses up accounts for much over-certainty.
Anyway, there isn’t anything inherently wrong with this second sense of observational data, except that it’s far, far too often input into statistical routines which guarantee over-certainty, like hypothesis testing and parameter estimation. People will claim causation has been found merely because they were able to quantify the analysis of observational data. Quantification is seen universally as superior to the conclusions reached by observational data of the first kind, when usually the reverse is true. This is because observational data of the second kind is often of a much more limited nature than the first kind, which reflects the broad experience of many.
Now WHO is one of those organizations, like all modern bureaucracies, that insist on quantification. This insistence is why so much is wrongheaded in government, because the insistence drives over-certainty. And the same would hold for true with the Indian medical system if it were to embrace rapid data collection. Again, it’s not that collecting data is bad per se, but that it’s collected for the sake of collection and then quantified because that’s what turns it into Science™ is a problem.
Therefore, it would be best to advocate discussions of elders, those who have had the longest experience in medicine as she is actually practiced.
Obviously there is much more that can be said, but you get the idea, I hope.