Listener Feedback XX

Today we're going to open the mailbag and respond to some listener feedback. Usually the feedback springs out of the bag like a honey badger, hits me in the face and sprays blood on the wall like a slasher movie, but today I wanted to select some more rational comments and reply in a more rational way. These emails we're going to read today illustrate pretty common pushback that I get from people who are miffed by my takes, not on specific subjects so much, but on the scientific method in general.

I should be clear on one point: I usually only include an excerpt from these feedback emails. Most feedback raises a number of different issues, and for the sake of brevity and keeping on point, I generally include only the part that I'm responding to. What I don't do is take anything out of context or otherwise change the author's meaning.

The Piri Reis Map

Kent from Illinois sent in the following feedback to the episode about the Antikythera Mechanism, an exquisite archaeological find that greatly improved our knowledge of ancient Greek technology. Some say it had to be alien in origin or is otherwise unexplainable, but my analysis was unable to reach the same conclusion. Wrote Kent:

It's funny to see the similarities between the skeptics and the ancient-alieners. Both need an answer. One falls into the unchanging rational gestalt, the other hops on the popular bandwagon. You both MUST answer the question, and stop WONDERING. Some mysteries can not be answered because of limited information. Explain the Piri Reis map Mr. Dunning. To consider yourself a skeptic or a believer is to close your eyes.

It appears that Kent and I are defining skepticism differently. I think it's accurate to rephrase what he's saying as "You should neither always believe everything nor always reject everything", which is of course quite true. If so, he's defining skepticism as the rejection of everything, which is not at all the definition I'd use. The process of critical analysis separates good evidence from bad evidence and results in an informed provisional conclusion. If Kent has truly listened to any number of Skeptoid episodes and determined that I always reject everything without responsible critical analysis, then either I'm not doing my job very well or he hasn't listened very carefully.

Incidentally, the Piri Reis map that he mentions is a world map compiled in 1513 by Ottoman cartographer Piri Reis by piecing together fifteen other maps. Although popular sources frequently credit it with inexplicable accuracy, or a completeness impossible for its time, neither is true. It's so complete because it was sourced from so many different maps from different countries covering different continents, so its completeness is not unexpected; and its accuracy is no greater than that of those smaller maps. Some land masses are rotated or out of scale, because Reis didn't always have complete context. It is a valuable and excellent example of ancient cartography, but there's nothing unexplained or mysterious about it. We might look at it in deeper detail in a future episode.

Ghost Hunting Tools

Rory from White Bear Lake commented on my episode about ghost hunting tools of the trade, an evaluation of the validity of using simple electronic gadgets when trying to find ghosts. Ghosts have no known properties, so it's hardly logical to look for them using devices that sense very specific properties. Rory's focus is on EVPs, electronic voice phenomena, mysterious voices that appear on audio recordings even though researchers heard nothing at the time:

My group has done over 40 investigations. We have picked up some very clear EVPs including ones that have answered questions directly. My group has tried the cell phone experiment and our recorders didn't pick up our call. I am a musician and have trained ears. I would also know if we were picking up radio or TV signals. EMF meters are normally used to see if an area is worth staying at for further investigation. I don't how Brian would explain an EMF reading in the middle of an open field that is in the middle of nowhere and you are 99% sure there isn't any source of electricity near. We can't always explain some of the audio or visual items we find but that doesn't mean we didn't find anything of value.

I'd go even further, and say that it doesn't mean anything at all. Unexplained anomalies are just that; unexplained. That's not the same thing as valuable data. It's interesting, and it's useful as far as suggesting where to set up your microphone and hope to get something better, but by itself it's meaningless.

Rory mentions his experience as a musician with trained ears. Although I appreciate that, it's not a relevant skill set for what he's trying to do. Understanding of research methodology would be far more useful. Understanding of the ways human brains can fool themselves, and the fallibility of perception, would be great as well. My encouragement to Rory's group would be to leave the musicians at home and bring along experimental psychologists instead.

Rory didn't specify what he suspects the EVPs might be, but most EVP hunters usually have ghosts high on their list of suspects. How did they rule out Martians? Why aren't they looking equally hard for Russian psychics projecting mental images at them? Why not the souls of the surrounding trees? All of these possible culprits are as equally undefined as ghosts; yet the ghost hunters have dodged around this fundamental step and are trying to match their observations with their preferred explanation of ghosts. This isn't because they're not good enough musicians; it's because they're not good enough at defining research methodology.

Personal Anecdotal Experience

Darren from England wrote in regarding my episode on wheatgrass juice, which was a major food fad about five years ago but has declined in popularity since then. Its supporters claimed all sorts of amazing medical benefits from drinking it, none of which were ever supported by reliable testing; and its detractors pointed out that it was merely an absurdly expensive fad and that, as grass, it contained virtually nothing of nutritional benefit to the human body. Nevertheless, Darren said:

There is a fair bit of science supporting some of the benefits of wheatgrass juice, but let us suppose for a moment there isn't.

What if there was a sceptic, one who was dismissive of the benefits of wheatgrass juice?
What if that sceptic were to carry out an experiment?
What if that experiment was to involve drinking wheatgrass daily for a period of time?
What if that sceptic were to drink 3 fl oz. of wheatgrass juice every day for a period of 3 months for example?
What if before carrying out the experiment the sceptic got a variety of health checks carried out, bloodwork etc?
What if the sceptic got the same health checks carried out after carrying out the experiment?
What if the sceptic made no other changes to his lifestyle during that period?

What if that sceptic was Brian Dunning?

Up for the challenge Brian?

I'm about to carry out the same experiment myself, having never tried wheatgrass juice,but unlike Brian I am willing to be proved wrong based on my own experience.

It probably won't surprise Darren to hear this, but he's right, I'm not willing to be proven wrong based on my own experience; and furthermore, I would also encourage Darren not to let his own experience change his mind either. The reason is that personal experiences are a terrible way to learn something. As human beings, we are all subject to preconceived notions, personal biases, and differing expectations; and of course any one person's personal sampling of something is, by definition, an uncontrolled, unblinded test subject to external influences and all manner of unknown variables. This is why we all know people who have reached opposite conclusions based on their experiences. They can't all be right, and that's proof enough that personal experience is an unreliable way to learn practically anything.

Instead, we turn to well-designed, blinded trials that employ controls to eliminate bad data caused by all of these weaknesses. Running such a trial takes time and money. This has already been done with wheatgrass juice, and the results of such trials are always going to be more reliable than my own personal experience or Darren's. If I were to try wheatgrass juice, and I were to experience whatever health benefit Darren believes I would, I'm smart enough not to let my personal experience interfere with my professional assessment. This isn't always easy to do; but that's one nuisance of maintaining a truly open mind. You have to be willing to accept what we can prove even when it contradicts your personal experience.

Skepticism a Disease?

Finally, Nathaniel from New York replied to the episode on Mao's "Barefoot Doctors" who practiced what came to be recognized in the West as traditional Chinese medicine; a recognition based at least in part on a popular abridgement of the Barefoot Doctor's Manual that omitted all the modern medical information, leaving only the last-ditch herbal equivalent. This resulted in a widespread belief in the West that traditional Chinese medicine excludes modern medicine. Nathaniel was unhappy with this analysis, and wrote:

I wonder if skepticism is curable by acupuncture. It is such a vicious disease; or rather, it is more like a poison- beneficial when used appropriately, yet hopelessly harmful when overdosed. I only glanced at this material, but it was obvious that the author does not know enough about the subject. I therefore agree that it is best to read articles by experts. Expert skeptics seem to be mostly expert at skepticism, and willing to apply it at all cost. Who scientifically verifies this nonsense?

Skeptoid is not a peer-reviewed publication, rather it's a survey of existing previous research and, as closely as possible, represents the best information available on each subject. Of course mistakes always slip through the cracks, and these are corrected whenever they're found in periodic episodes of corrections. If you found an error or misleading omission in the Barefoot Doctors episode — as implied by your comment that I don't know enough about the subject — then by all means email me the correction.

I certainly agree that too much critical analysis of a given subject often reaches a point of diminishing returns, but I would not agree that its use is like a disease. On the contrary, I'd argue that it's more like a cure for incomplete or misleading thought processes, that all too many people lazily embrace. Simply accepting whatever you're told, whether that's to believe something or to reject it, is hardly a way to improve your knowledge and make the best life decisions.

By Brian Dunning
Follow @BrianDunning