We have two kinds of medical treatments: those that have been tested and found to work and those that have yet to be tested, and in spite of people swearing by them- don’t work and will be relegated to the history pile of witchcraft, chiropractic, homeopathy, and bleeding

What Works Isn’t Always What We Think – the killing of a President
When George Washington died of pneumonia he was hastened to his death because of a common practice at that time: bleeding or phlebotomy. The theory of the practice was that there are bad humors in the blood causing bad things to happen, and by bleeding out some of that blood you’d rid the person of those bad humors and save them. It is a plausible theory, but bleeding left Washington anemic and with less ability for his tissues to oxygenate, so he died from what was standard practice at the time.

An anemic, dying George Washington

James Garfield, the 20th President, didn’t die of an assassin’s bullet – he died of medical malpractice. In the days of the wild west, the belief was that getting the bullet out, would allow the patient to recover. After the President was shot, a number of attempts were made to remove the bullet that was lodged in his spine. Each time with instruments or fingers that were not cleaned (his doctor didn’t believe in the germ theory). Over the course of 80 days, he developed a number of infections, and ultimately died in Long Branch, New Jersey weighing 100 pounds less than when he was shot. Garfield was also given enemas to feed him – they didn’t work, of course, and he clearly did worse than if he hadn’t been under the care of his doctor (more about this in my upcoming book).

Confirmation Bias

Our own bias is the hardest to overcome. If a particular medicine or treatment seemed to work for us, then we assume it should work for everyone. This is often the case with alternative treatments. Be they chiropractic, acupuncture, or Chinese medicine. Humans get better from some event, correlate it with some treatment, and it is difficult to change a mind. But it is also difficult for physicians: we get use to a treatment, see some result, and often have to overcome our own bias when studies don’t show that it works.

Correlation is not Causation

We take something- we have a result. We make a correlation. It doesn’t mean there is a causation, it could be random, it could be from something else. This also is a difficult to overcome. But to test this we must be willing to try it when we are blinded as to what we give a patient, and the patient must be blinded as to what is given to them.

We Have To Test It
In medicine we have to test new theories, and old theories to see if they work. Here is another example:
We used to think that after surgery, a patient should be sent home on antibiotics. It turns out that if you do that, you don’t reduce the incidence of surgical infection, but increase it- and increase it with bacteria that are resistant to those drugs. So now, the tested method is to give patients (if they need it) antibiotics just before surgery, and for no longer than 24 hours after surgery.

How Valid Is the Test?
People will often quote some authority in the field of medicine, and the opinions of we gray haired people isn’t good evidence. In fact, it is a logical fallacy called “appeal to authority.” In medicine we rank the level of evidence this way:

Lowest to Highest Levels of Evidence
Level III: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.

Level II-3: Evidence obtained from multiple time series designs with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.

Level II-2: Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group.

Level II-1: Evidence obtained from well-designed controlled trials without randomization.

Level I: Evidence obtained from at least one properly designed randomized controlled trial.

A randomized controlled trial means that one group of patients will get one treatment and the other another treatment. It may be a placebo, it may be a standard therapy. This is how new chemotherapies are evaluated (once they have been determined to be reasonably safe, and toxic levels determined). They will assign one group the therapy that is new, and the other group the standard therapy. Neither the patient nor the physician knows which patient or therapy they are getting (both sides are blinded, thus it is a double blinded and randomized trial). Once the results are analyzed the data is opened to see who did the best.  This prevents bias from the patient, as well as bias from the physician.

But not everything is so clear
There are a number of treatments out there that we don’t know if the benefits outweigh the risks. Part of medical ethics is to have the discussion with the patient so they make an informed decision about what is going on with them. Based on this there are levels of scrutiny determining how a clinician should approach patients with a potential treatment:

Level A: Good scientific evidence suggests that the benefits of the clinical service substantially outweigh the potential risks. Clinicians should discuss the service with eligible patients.
Level B: At least fair scientific evidence suggests that the benefits of the clinical service outweighs the potential risks. Clinicians should discuss the service with eligible patients.
Level C: At least fair scientific evidence suggests that there are benefits provided by the clinical service, but the balance between benefits and risks are too close for making general recommendations. Clinicians need not offer it unless there are individual considerations.
Level D: At least fair scientific evidence suggests that the risks of the clinical service outweighs potential benefits. Clinicians should not routinely offer the service to asymptomatic patients.
Level I: Scientific evidence is lacking, of poor quality, or conflicting, such that the risk versus benefit balance cannot be assessed. Clinicians should help patients understand the uncertainty surrounding the clinical service.

How Does This Help?
Insurance companies in the United States and public insurers in other countries usually wait for studies showing that the drug, device, or treatment has approval based on evidence-based guidelines before covering it. Where approval for a drug, device, or treatment has been given, and subsequent evidence-based findings indicate that the treatment may be less safe than originally anticipated, some insurers in the U.S. have stop providing that as a benefit.

There are many people who will believe in an untested product, and spend money on it but insurance companies will not approve such treatments. Some of these are called “complimentary medicine,” with “natural” therapies. The idea that something has “been around for a thousand years” does not mean that it works- in fact, often it does not.

Science Based Medicine
Some of our colleagues have gone further than evidence based medicine, and apply the rigors of science to it. Such is the work of a group of skeptics.  It is not an approach to evaluate what works, as much as it is an approach to show why a number of complimentary medicines do not work. Here is it in their own words:

Good science is the best and only way to determine which treatments and products are truly safe and effective. That idea is already formalized in a movement known as evidence-based medicine (EBM). EBM is a vital and positive influence on the practice of medicine, but it has limitations and problems in practice: it often overemphasizes the value of evidence from clinical trials alone, with some unintended consequences, such as taxpayer dollars spent on “more research” of questionable value. The idea of SBM is not to compete with EBM, but a call to enhance it with a broader view: to answer the question “what works?” we must give more importance to our cumulative scientific knowledge from all relevant disciplines.

Antiquity

An appeal to antiquity – a common logical fallacy. This assumes that the people who lived before us either had some secret knowledge that kept them healthy and vital, or that some treatment has “always worked” and therefore is beyond testing. Homeopathy lives by this – their world began and ended with their founder. Acupuncture- unproven. Chiropractic – which has the same effective nature as physical therapy, without the basic knowledge. There is no treatment that should not be tested, whose limits of treatments we should not be aware of. Prayer doesn’t work – its been tested, but it won’t stop people from praying, but preying on a fellowship is just as wrong as burning a witch (although perhaps those pastors should be submitted to the same dark age ritual for their hubris)