In Part 3, I tried to explain the second pillar of Truth finding and look at what Data is and what it is not. We also looked at the difficulties with collecting objective and valid Data.
In Part 4, I want to discuss the role of the third pillar (Evidence) in Truth finding. Let us start with a standard definition of Evidence from Dictionary.com.
- That which tends to prove or disprove something; ground for belief; proof.
- Something that makes plain or clear; an indication or sign: His flushed look was visible Evidence of his fever.
- Data presented to a court or jury in proof of the Facts in issue and which may include the testimony of witnesses, records, documents, or objects.
If your look at the third definition, you might be excused for finding it somewhat circular. Evidence is data in support of facts? I don’t think I have a clue what this means. The first definition can be easily mistaken for what we called Data in Part 2 and possibly even hard to distinguish from a Fact. The second definition is so subjective that I am amazed they even listed it. So what is Evidence then? Here is my definition.
Evidence is relevant Facts and Data. There are lots of Facts and Data out there but not all are relevant to our proposition, case, theory, hypothesis or concepts. Evidence must have relevance to the issue we are studying. What do I mean by relevance? Let me give you an example.
I am working to prepare for a chess match with my neighbor. I happen to note in the paper the Fact that tomorrow will be a quarter moon. Does this Fact have any relevance to my playing chess? I don’t think so. Thus, I don’t really care that there will be a quarter moon. As far as my limited cognition or perception, I can see no relevance between the Fact of a quarter moon and my preparing for my chess match. I could be wrong. We can always mis-perceive the relevance of some information to an issue. This is often done in science and in police work. We don’t see the connection between two issues and we misjudge the outcomes. This provides one good reason for diversity and numbers in problem solving. You have less chance of being blindsided if you have a variety of opinions rather than just your own.
Let us look at another example where the issue of relevance is more salient. I am planning to go on a trip to England in 2017. I want to plan my trip for the best possible time of the year. I hypothesize that two Facts or Data points are very important to my planning. The first is the temperatures at various times of the year in England. The second is the rain fall. I found the ranges for this data on a weather site and used the information to plan my trip. Of course, some of the decisions anyone makes will depend on their own weather preferences. I wanted to minimize rainfall and also keep the temperature in a moderate range. What I call sweater weather. Thus, both these set of factors were relevant and important to my planning. I would call them Evidence to support the time of year that I decided to go.
On the other hand, if you like rain, you might have picked a different time of the year than I did. There were other mitigating factors which played a role in my decision making. These factors included costs for lodging during the year and transportation costs during the year. In general, off season times have better rates but are somewhat the worse for weather. Another factor was the value of the pound to the dollar. I considered the value of the dollar to the pound post Brexit but concluded that I did not have enough information to effectively evaluate the impact of this data on my decision. I am assuming that with the volatility involved in the situation, the value of the dollar might go either way against the pound. My best guess is that I will benefit if I go as soon as possible. The news has recently noted that after Brexit the value of the pound fell 14 percent against the dollar. This would mean I could get a significant cost advantage if I purchase anything in England. I am hoping this situation will continue until after my trip but there are too many variables at play here for me to use this information. I can only hope.
A more common example of relevance can be found by looking at police work. We are all familiar these days with what is called Forensic science. I am sure most of you reading this have watched some police show. As soon as a crime is discovered, the Crime Scene Unit (CSI) is brought in to collect Evidence. Keep in mind that everything at a crime scene is not Evidence. Only what may have a possible relationship to the crime. This can be a real problem. The CSI unit is going to be limited by their assumptions concerning what might be relevant. For instance, I doubt any Crime Scene Investigator will care whether or not the light bulbs are “bright” or “soft white” in the kitchen or bathroom. It is impossible to collect all the “Evidence” of stuff that might be related to the crime. Thus, relying on experience and training, the police investigators do their best to collect Facts and Data that appear to be relevant to the crime. The relevant Data and Facts are not just interesting, they are Evidence. The more they relate to the crime, the stronger the Evidence will be.
An eyewitness can provide Evidence via his/her testimony as to the events of a crime. The relevance of any eyewitness is high but the reliability of an eyewitness can be much lower. Second hand testimony is not as relevant as first hand testimony and is thus weaker Evidence. Testimony that might be compromised by some factors such as police record, bias, discrimination, physical disabilities might be relevant but will be weaker Evidence because the validity of the Evidence is suspect. That is why lab procedures and chain of custody is so important to police work. They may have the most relevant Evidence imaginable but if the validity of the Evidence can be comprimised because of sloppy police work, the Evidence will be useless.
The same is true of scientific Evidence. It must be valid and reliable. One example of how a Fact was exposed as a lie was in the work on so called “cold fusion.” Here is an excerpt from a paper on the dubious development of cold fusion in a laboratory:
“One year after the press conference that had garnered Pons and Fleischmann so much attention, the scientific process had finally been able to sort through the evidence regarding cold fusion. Few groups had found support for the hypothesis, and those few had inconsistent results and could not reliably reproduce their findings. This lack of replicable evidence was a major blow for cold fusion. The laws of nature don’t play favorites. If cold fusion works in one laboratory under a certain set of conditions, we’d expect it to work in other laboratories at other times under the same conditions. Hence, lack of reproducibility is a serious problem for any scientific finding, casting doubt on the validity of the original result and suggesting that there’s been a misinterpretation of what’s going on.” — http://undsci.berkeley.edu/lessons/pdfs/cold_fusion.pdf
It is seldom that findings of Evidence in police work or business are subjected to as much scrutiny as occurred in the so called development of cold fusion. Perhaps, since this was a finding of great scientific importance, it was held to a more rigorous standard than would occur in many other scientific studies. I am thinking in particular of findings in the health field, nutrition field and drug field. In each of these fields we often have much less rigor before results are posted or accepted. Business is even worse with advertisers spouting outright lies and fabrications. Little known phenomenon are routinely heralded as being highly reliable Evidence of the benefit of some product or service that someone wants to sell you. All kinds of spurious Facts and Data are then marshaled as Evidence to support the phony claims by Madison Avenue advertisers.
Next week in Part 5, the final part of this series on Truth, we will look at how one can put the three pillars of Facts, Data and Evidence together to find the Truth.
Time for Questions:
Can you tell me how you know a true Fact from a false Fact? How do you decide what to believe? How much credibility do you put in the news that you hear? How do you choose the news that you want to hear? How do you decide who is telling the Truth?
Life is just beginning.
“I am a firm believer in the people. If given the Truth, they can be depended upon to meet any national crisis. The great point is to bring them the real Facts.” — Abraham Lincoln
When I started working with Process Management International in 1986 after completing my doctorate degree at the University of Minnesota, I met the famous quality improvement expert and renowned statistician, Dr. W. E. Deming. Over the next seven years, he had the most profound influence on my life in terms of helping me to understand process improvement, statistics, quality and the use of Data to improve everything from widgets to health care. Under the influence of Dr. Deming, our company adopted his motto “In God we trust, all others bring Data.” Dr. Deming also said “Without Data, you’re just another person with an opinion.” So what is Data? 
If we understand what Data is, you have now entered the deep forest. However, we have a long way to go before we can get out of the forest. There are numerous obstacles along the way. Referring again to the concepts of validity and reliability, we must ask ourselves the same questions we asked about our Facts. Is our Data reliable and valid? How did we collect the Data? What method did we use to collect the Data? Are we taking a few samples each day for several weeks or are we taking a few samples for only a few days? Are we using a random sample or a stratified random sample? Different methods of collecting Data will lead to different results. And we are not even talking about interpreting the Data yet. For instance, when I worked at W.T. Grants cutting shades back in the late 60’s, I was told to make sure I took my measurements with a metal tape measure and not a cloth or plastic measure. The reason given was that it was easier to stretch a cloth tape measure and get a false result. This would lead to cutting a shade that was too large and would not fit.
Unfortunately, the scientific method is not infallible. It is subject to bias and disagreement over Data and interpretations. Even more problematic is that the scientific method is not a strong method when it comes to testing subjective theories that cannot be verified by Fact. For instance, “Is the Mona Lisa beautiful?” As stated, this is a subjective question that each individual will hold a different opinion on. However, if I asked: “Is the Mona Lisa the most beautiful painting in the world?” I could attempt to answer that question with a bit more objectivity. I could conduct a survey to see what percentage of people think it is the most beautiful. Subjective studies are not as strong as objective studies since they usually lead to results that follow a bell shaped curve. Thus, if we conducted the above survey, we would probably find that a certain percentage of people thought it was the most beautiful painting and a certain percentage did not. As in politics, opinions of beauty would be all over the place. This is why politics is so much more difficult to “Fact check” than issues like the atomic mass of hydrogen. Politics is a very subjective field that resists efforts to test and Fact check. Some examples that would be difficult to test with the scientific method would include:
Finally, if I have left you with some understanding of the difficulty with interpreting Data, I will have felt successful. The first step to knowledge is awareness of our cognitive limitations. We also need to be more skeptical when people present us with Facts and Data. My father used to say “Believe nothing of what you hear and half of what you see.” I still consider this good advice. There are too many fools and charlatans out there trying to convince us of things for a multitude of reasons that will benefit them and not us. Just as we would not walk down a dark alley in an unknown city by ourselves, we need to exercise caution when presented with Data and Facts. The more we understand the limits of Data and Facts, the more prepared we will be to make decisions based on Data and Facts that have a higher degree of validity and reliability. If the Data, Facts and Evidence that you base your knowledge on are not accurate than everything you think you know will be at best a half truth and at worst a total lie.
“The prosecution had expert witnesses that testified that the Evidence was often mishandled. Photos were taken of critical Evidence without scales in them to aid in measurement taking; items were photographed without being labeled and logged, making it difficult, if not impossible, to link the photos to any specific area of the scene. Separate pieces of Evidence were bagged together instead of separately causing cross-contamination; and wet items were packaged before allowing them to dry, causing critical changes in Evidence.” 
A validity error is when we are not measuring the right thing. IQ tests have been repeatedly criticized for not really measuring the intelligence of a human being or for being biased by many cultural Factors. Thus opponents of IQ tests argue that they are not valid measures of intelligence. A reliability error is when our measures are not consistent. The scale example given above illustrates the problem with reliability. Most people use a scale to weight themselves and most scales have problems with reliability. However, if you tried to equate your weight with your health, you would be assuming that the scale could also measure health and this would be a problem with validity. Scales cannot measure health although health might be correlated to some degree with appropriate height and weight.
Before we move on to looking at the concept of Data, we will look at two more problems with the concept of Facts. These are distortion and bias. Distortion relates to twisting the meaning of something. This can happen by taking something that someone has said out of context. For instance, I might be talking at a conference and say something in sarcasm such as “Yeah, I will definitely vote for Trump.” My words could be repeated verbatim and it would sound like I was endorsing Trump. It is difficult to detect sarcasm. To most people reading or hearing my words second hand, it will sound like I am a strong Trump supporter. Slick politicians and advertisers will often distort a Fact to make it sound like the Fact is supporting their position.
Aging Capriciously 