The Unreliability of Testimony

Believe nothing of what you hear

and only half of what you see.

-Edgar Allen Poe

     Should we believe what we hear? Why is it so easy for us to believe what people have to say? We like gossip, we like hearing stories, it makes us feel good. The problem is that the human mind is so easily fooled. Now, I don't claim to be above being fooled, I will admit that I have been fooled many times, and fallen prey to the shortcuts of our mind. I believe this is one of the most important aspects to learn in being able to make good decisions based on reliable evidence. Because of the many different ways our mind can be fooled, we should never take eyewitness testimony, or anecdotal evidence as truth, or reality. I'm not saying that everyone is a liar, but that we are easily fooled into believing something that is false.

     Think back to elementary school. I'm sure many of us remember playing the telephone game, where the teacher would whisper a phrase into the fist persons ear, and then it had to be passed on down the line. If you never played this game, I encourage you to look it up and try it. As I'm sure was the case, the last person to have this phrase passed to them would then say it out loud, and I'm sure it was never the same as the original. This is a great example of how information gets corrupted being passed on orally, or through eyewitness testimony.

     There are many ways we can be fooled into thinking something false. Our being prone to believe in anecdotal information is one of them. If anyone has facebook, this is a great example, if you're anything like me, I see weekly someone sharing some information that is false, but they believe to be true. There is a few websites dedicated to debunking a lot of these Internet “urban myths”. So lets talk about all the different ways our brains can be fooled, and why we shouldn't believe what we hear.

     Lets start with confirmation bias. This happens when we want to believe one way, we tend to find information that confirms our beliefs, and discredit information that disagrees with our beliefs. This is a well documented phenomenon. We are all prone to confirmation bias, even myself. This is something we should all work to be aware of. So if you're talking to someone who believes a certain way, chances are they have looked and trusted all the information that agrees with the way they think, and won't tell you any contradictory information.

     Lets talk about some logical fallacies. These are ways our brains get tricked into thinking something sounds logical, when really it isn't. I've posted a list of these in an earlier post, but I will cover some more here.

     Argument from ignorance. This is when one goes from a statement of uncertainty to a statement of certainty. For example, I don't know what those lights are in the sky, they must be aliens from another planet. Or I don't know what that noise was in the house, therefore it must be a ghost. You can see where the logic is failing in this. You cannot go from an abject statement of uncertainty, to an abject statement of certainty.

     Argument from authority. Just because someone claims they are an expert, or has initials after their name, doesn't always make them right. One of the times we see this is when a PhD in one field, makes statements or assertions about a field they know little about. Just because someone has a PhD in physics, it doesn't make them an expert in biology.

     Argument from popular belief. Just because a lot of people believe it's true, it must be true. It doesn't matter how many people believe in something, it doesn't make it true. Also it doesn't matter how few people believe in something, it doesn't make it false. The Earth will continue to travel around the Sun, regardless of how many people believe it's true or not.

     These are common arguments that can lead to false beliefs. They can trick people into believing something that might not be true. We have had an entire culture of UFOs spring up because of people making an argument from ignorance. Now with the advent of everyone having a cell phone with a camera and the Internet, these sightings are being recorded and quickly identified.

     I'm going to go quickly into the placebo effect, as I covered it in my last post. People can think something makes them better, when the reality is that it did nothing. The placebo effect can work both ways though. If someone thinks something is bad for them, they will get sick from it. This is referred to as the “nocebo” effect. The nocebo effect is one of the common arguments against electromagnetic radiation, and people getting sick from it.

Cum hoc ergo propter hoc. I've discussed this before as well. It states that a correlation between two variables implies causation. There are a lot of people, always falling for this in many situations. We have evolved great pattern recognition ability. However this great ability sometimes sees patterns and connections that aren't there.

This brings us to patterning. Because we are so good at seeing patterns, we tend to see patterns in true randomness. Many conspiracy theories come from us noticing patterns in random events. Or connecting things that have no relation. Most people have trouble recognizing a random pattern as random. The reason for this is that randomness naturally has clumping. We tend to see this clumping as a pattern.

In conclusion, there are lots of ways we can be fooled into believing something false. This doesn't make us dumb, or liars. This technique of using anecdotal evidence to get you to believe something is used extensively by the people peddling pseudoscience and pyramid scams. Beware, if someone is trying to convince you of something and all they're presenting you is success stories, chances are they are scamming you. I know we've all seen this sales pitch before, don't fall for it, demand real evidence for the efficacy of a product or system. Before we believe what someone tells us, we should look at the evidence backing this up. Now, we can get into a philosophical discussion of what needs evidence, and when we should trust someone at their word, but we won't.

Remember, 

extraordinary claims require extraordinary evidence – Carl Sagan

Alternative Medicine vs. Modern Medicine

Introduction

     This is a question I'm sure comes up often. What is the best way to treat an illness? We hear a lot about how science and modern medicine have made astounding breakthroughs in curing potentially fatal disease. We also hear about all these great things about Alternative Medicine, how it's “All natural”, how it's “Ancient Chinese Medicine”, or how it's so much safer than modern medicine. With so much information around on both of these it might seem like an enormous task to figure what will work the best. Hopefully in this article I can simplify things and hopefully make the choice seem a little bit easier.

The Placebo Effect

     While I'm sure most of the people reading this have heard of the placebo effect, this is an important issue to understand in making choices for treatment. The placebo effect was discovered in the 18^th century and is essentially a pill or procedure that does nothing, but tricks your mind into relieving the symptoms of a disease or illness. The most common form of a placebo used is a sugar pill, which is just a pill made of sugar. While modern science still doesn't understand a lot about how it works, they have discovered a lot about the effect. Research shows that 2 sugar pills are more effective than 1, and that an injection is more effective than a pill. The more invasive a procedure the more of a placebo effect it has. There are other things, such as blue sugar pills work better as a sleeping pill placebo than red sugar pills. Or a doctor in a lab coat gives out more effective sugar pills than a doctor in jeans and a T-shirt.

     So why is understanding the placebo effect important in making medical decisions? It's because sometimes, a treatment can seem like it's working even though all it's doing is creating this placebo effect.

Modern Medicine

     Modern Medicine, or western medicine as it's sometimes called is more properly known as Evidence Based Medicine. This is the medicine your doctor practices. Any drug your doctor prescribes, or recommends has been tested against a placebo and shown to work better. A lot of procedures and treatments come from a good understanding of human physiology. All of the treatments and procedures are based on scientifically sound evidence. A doctor should be able to tell you the benefits and risks of a particular treatment and what options are available.

     So how is medicine tested? A good test to see if a particular treatment is better than a placebo begins with a double blinded, randomized, placebo controlled trial. So what does all that mean? In a properly conducted trial, both the patients and the doctors are blinded, that means that the doctors giving out the medication, and the patients receiving the medication have no idea whether it's real or a placebo. The patients should be randomized using a computer algorithm or some other blinded method. A researcher selecting patients for the groups could unconsciously bias the trail in one direction. So during the trial, no one knows who is getting a placebo, and who is getting real medication, this should even extend to the statisticians doing the final tally on the numbers. Only at the end of the trial, after all the results are in should the blinding be taken off, and which group is which be revealed. In a well conducted trial, with lots of participants, we get a really good idea of the effectiveness of a treatment.

Alternative Medicine

     If modern medicine is evidence based, where does that leave alternative medicine? Well, alternative medicine is either unproven, or dis proven So either it hasn't been tested, or is difficult to test properly, or it has been tested and shown to work no better than a placebo. There are a lot of alternative therapies out there that have been tested and shown no better results than a placebo. I realize at this point my view of alternative medicine is looking pretty grim, as I am making the claim that none of it has passed a test and shown to be effective. So what happened to the alternative medicine that was tested and shown to work? Modern medicine, being evidence based, adopts any treatment shown to work, and as it's adopted, it becomes mainstream, and no longer “Alternative”.

     I'm not going to get into a description of every alternative medicine out there. That might be a topic for a later post. However be wary of any medical claims that rely on an abundance of anecdotal evidence. If I haven't made it clear yet, anecdotal evidence is completely worthless. How can you be sure that the anecdotal evidence in these claims, is simply not the placebo effect? The other effect that dis proven medicine relies on is called “Return to the mean”. This effect is best shown by your bodies natural ability to heal itself. How do you know if the anecdotal claims for these alternative therapies are not just the person getting better on their own? Truth is, we can't know, that is why we have such controlled and rigorous testing, to control for variables like this.

     One of the other claims of alternative medicine followers I hear quite often is “Big Pharma doesn't want you to know, because it's not profitable”. This claim is simply wrong. “Big Pharma” is mandated and regulated to prove that their drugs work better than a placebo, in order to bring them to market. The other part of this is that the alternative medicine industry is now making over 100 billion dollars a year. I don't know about you, but last time I checked, that seemed like a decent profit. Please don't take this paragraph as my endorsement of the pharmaceutical companies, they are definitely guilty of some shady practices for profit. Dr. Ben Goldacre is working hard to bring this shady practices to light and stop them.

Conclusion

While there are several topics I didn't discuss in this article like the ethics of selling a placebo, or particular alternative treatments, I hope there was enough information to help you make a better decision. My thoughts are this, why would you spend your hard earned money on a treatment that is either dis proven or unproven when we have a good system of evidence based medicine available. For further reading if you're interested I recommend Trick or Treatment by Edzard Ernst and Simon Singh and also Bad Science by Ben Goldacre.

Amber Teething Necklace

What is it

      The amber teething necklace is claimed to help infants with the pain of teething. Now the first thing I noticed about this, is that you're putting a string of choking hazards around your child's neck. To me, this doesn't seem like a safe alternative. As a parent I can sympathize with wanting to relieve the pain in your children, or even to have a few moments of quiet. Like most alternative medicine, I was not able to find a lot of science, or trials testing the efficacy of this, well actually none at all. As usual I found a lot of anecdotal evidence and post hoc fallacies. That being said, that alone does not mean it's entirely bunk.

WARNING: THIS PRODUCT POSES A CHOKING RISK. THIS ALSO POSES A STRANGULATION RISK WHEN WORN AROUND THE NECK

      Let's explore how this is said to work. The best explanation I can find is that the amber releases succinic acid and that is absorbed into the skin. There are a few other more wild explanations like it's bio-interactive, or that it aligns chakras, or that it has homeopathic qualities. So lets ignore the explanations that we know aren't possible and look at the one plausible explanation. There are three assumptions being made in the claims for this product. All of these assumptions need to be true for this product to work the way it is described.

1. Amber releases succinic acid
2. Succinic acid is an effective pain reliever
3. Succinic acid can be absorbed through the skin

Does it work

      Lets look at assumption 1. Amber does contain succinic acid, however getting it out of the amber requires a lot of processing. Amber is fossilized tree resin, that has been compressed, heated and cooled over hundreds of thousands to millions of years. The idea that it can release anything on contact with skin, after it has been pushed and polished for so long does not seem logical to me. The process of getting succinic acid out of the amber involves crushing it into powder and then distilling it. Looking at all the information available, while amber does contain succinic acid, I don't believe it's going to give it up just from skin contact.

      On to assumption 2. Succinic acid is used as an additive in the food on industry to control the acidity of products. There are reports that it has been used in the past (before modern testing) as a topical pain reliever. There is no evidence today that I could find that shows it is an effective pain reliever.

      Now assumption 3. Succinic acid, once extracted from the amber, is a solid salt like substance. While it could be possible to absorb it through the skin, this substance is known as a skin irritant. It is not considered a dangerous substance. When we look at it being a skin irritant, it would seem likely that if assumption 1 was correct we would see a skin irritation where the beads were worn.

Conclusion

      After looking at all the evidence and claims for this product, I would highly recommend not using it. This product poses a serious choking and strangulation hazard to infants. There is also no evidence to support that this relieves the pain in any way. As there has been no scientific trials conducted on this, most likely due to the high risk and no plausible method of it working, I can not say 100% that this will not work. All logical and reasonable examination of this points to this as being ineffective.

Standards of Evidence

Introduction

      So this post might be a little long winded, but I want to establish a foundation of what we consider rational evidence. This will do allow us to be honest in our critique of certain issues. At any time anyone can refer back to this post if we allow ourselves to be biased by non ration evidence.

Why having a standard of evidence is important in making decisions

      There is good evidence, bad evidence, and useless evidence. By having a standard on which to base what is good, and why, we can sort through all the bad evidence and get ourselves a truer picture of reality. We can only make decisions based on how good the evidence presented to us is. I don't know about any of the readers, but I myself want to be able to make the best possible decisions for both myself, and my family. Having a higher standard of what you consider evidence leads to better decisions and allows yourself to get a more realistic picture of the world around you. In this article I hope to explain how I came by this standard, and some of the “bad” evidence you should watch out for.

Standards in science

      Modern science has done a great job in realizing how flawed and biased our perception of reality can be. The scientific method is a way in which hypothesis' can be tested, and a true result be obtained in spite of these biases and flaws. You don't need to be a scientist to use the scientific method, just a little bit of rational thought behind how you come to a conclusion is needed. I will get into how our perception of reality is flawed later in this article. The other thing modern science has given us is called the “peer review”. In this process, scientists publish their test, their methodology, and their findings in a journal that is read by other scientists in their field. This way they can be critqued by their peers who have no bias or stake in the results of a certain experiment or trial.

     Understanding how peer review works is an important part of knowing if a result is worth accepting. A lot of articles get published and get media attention, but then once the peer review process finds flaws with the study or trial, often it's not covered again by the media. In this way, we need to be careful that the studies we're looking at for our evidence are not only published, but have also been thoroughly reviewed. A study that is published, but then peer review has shown many flaws, is not a study or trial I would accept as evidence.

Why anecdotal evidence is useless

      Anecdotal evidence, or eyewitness testimony, in matters of finding truth, or reality is not really evidence at all. There are many ways our brains can be tricked into believing in something that isn't true. Let me give you an example. I'm sure most of us have been to see a magic show, or seen one on T.V. Did any of us believe that David Copperfield actually made the statue of liberty disappear? Or that Chris Angel actually levitated? Or that any magician actually sawed a woman in half? Of course we don't believe they actually did this, we understand that they are simply taking advantage of the shortcuts in our brain that have been hardwired by evolution.

      I have great respect for magicians, they freely admit what they do is an illusion. However there are those out there that essentially do the same thing, and try to convince us it's real. Psychics, Alternative medicine practitioners, and Ghost Hunters are just a few of those that take advantage of similar shortcuts in our brains, but unlike magicians, they try to convince us they're powers are real.

      In this respect, if you tell me that you went to a psychic, and they were so good that they knew everything about you, so their powers must be real, should I believe you? Would you believe me if I told you David Copperfield really made the statue of liberty disappear? Of course you wouldn't, you know it's just a trick. In this same way, we can't take the eyewitness account of anyone, unless they can produce some more reliable evidence for their claim. There is two really good videos explaining this process more fully here  and here.

The logical fallacy

      The logical fallacy is when a statement follows what seems to be a logical order, but in reality is not logical at all. A brief explanation of how logic works is: if p, then q, p therefore q. A logical fallacy would be: if p, then q, q therefore p. While this follows the same form, it is not logic. I won't get into any more, but instead I'll take a look at some of the more common logical fallacies that are used.

      Post hoc ergo proper hoc. While I don't know the exact translation, this fallacy is one in which it's believed A happened, then B happened, therefore A caused B. While there is some contributing casue for B, unless it can be shown that A caused B, we can't assume that it is the case. An example of this in real life is the vaccines cause autism scare. Children aren't able to be diagnosed with autism until they are about three years old, around the same time the normal course of vaccinations are done. This link has been thoroughly debunked. This logical fallacy is especially prevalent in the alternative medicine industry. I drink this magic potion, then I got better, it must be the magic potion that made me better. I guess it's not possible for your bodies natural process to make you feel better.

      The natural fallacy. In this fallacy it's believed that everything produced in a lab is bad for you, and anything that come from nature is good for you. Not everything in nature is good for you, in fact, if you ever take a walk through the wilderness, most of the plants that you see will kill you if you eat them. Now I'm not saying everything produced in a lab is good for you, but some things can be better for you than their natural counterpart. Take asprin for example, the natural for of it found in the willow tree, while effective, is also toxic. A little tweaking in the lab and we come out with a safe version, that is still very effective.

      The ad hominem. This fallacy is when the person making a claim, or presenting evidence is attacked. It doesn't matter what the person, or company has done, or what they do, all that matters is the quality of the evidence they're presenting. An example of this is that most people tend to disbelieve any report Monsanto puts out, simply because they have gotten the reputation for being an evil, greedy corporation. Simply being evil and greedy doesn't mean you can't do good science.

      There are plenty of other logical fallacies. While I don't have the space to discuss all of them here, there is a list of them here.

Disproving established science

      So this last section I'll look at what established science is. There are certain things in science that are taken as true, such as the laws of thermodynamics, the theory of relativity, the theory of evolution, mathematics, and a few others. While there is always new things being discovered in these fields, most of the basics are pretty much established. So, if there is a claim that violates any of these laws, or is in contradiction to any of these established sciences, and you want me to believe you, there is one simple way. Publish your findings in a peer review journal, get through the peer review process, then claim your Nobel prize. At that time I will happily believe your claim.

      Hopefully this wasn't too long. Understanding evidence we can make better decisions for us and our children. After all, don't we all want to do whats best.

Introduction

      I currently work as a carpenter. I love working with my hands to create. I have a family, I enjoy spending time with them. I enjoy learning new things. I'm your average guy. You don't need a degree to learn how to make good decisions based on sound, solid evidence. All it takes is a little time to learn how to spot bad arguments, or useless evidence.

      
      Critical thinking is becoming one of the most important skills to learn. It's going to be even more important to be able to pass on those skills to the next generation. We're living in a time where we have unprecedented access to information. Good and bad, we can find all kinds of information on the Internet. Critical thinking gives you the skills to separate the good information, facts based on evidence and logic, from the nonsense and magical beliefs so prevalent on the Web. One good example is the flat earth society, if there wasn't so many pictures, and other evidence to refute this, would they be making a good argument?

      Critical thinking is not difficult to learn. All you need is a few simple skills. Hopefully through this blog, and by demonstrating how some of these arguments fall down under closer scrutiny, we can make the best decisions for our children.