Types of Evidence
STAT 218 - Week 1, Lecture 2
January 10th, 2024
Today’s Menu
In this lecture, I will be briefly talking about
- FAQ From the Last Lecture
- How (not) to Design a Study/Research?
- Types of Evidence in Science
From the Last Lecture - I
Install R and RStudio
Install R: Yes, you do need to download and install R even if you have downloaded before. There is a newer version.
Install RStudio: Yes, you do need to download and install RStudio even if you have downloaded before. There is a newer version. Download the free Desktop version.
After installation, try the following test and come to my office hours if you need it.
Is my RStudio working?
Let’s Move Forward Today’s Topic
- As a scientist specializing in environmental science, you were assigned a task which is designing your own research study. Your area of interest centers around examining the impact of climate change on local ecosystems.
- What might be the steps?
Mind the Gap!
Before jumping straight to the data analysis
- Explore Existing Data:
- Examine the data closely.
- Pose questions about the data.
- Set the Stage for Data Collection:
- Review the relevant literature.
- Select an appropriate research design.
- Implement an appropriate sampling method.
- Choose suitable data collection techniques.
- PS: I acknowledge that there is no one single way to do science, but I do also believe that to be able to analyze,understand and interpret data as a statistician/data scientist, research design and data collection procedures must meet rigorous standards and adhere to the principles of sound scientific inquiry.
Types of Evidence
Question of the Day:
- Ever wondered what kinds of evidence scientists collect?
Discuss These Cases
- On a chilly day, I didn’t dress warmly, and soon after, I caught the flu.
- My friend’s dad had a heart attack and died after they gave him a new heart disease drug, so the drug must not work. (from Diez et al., 2022)
- I met two students who took STAT 218 more than once, so this course must be very difficult.
Possible questions to be considered:
- Is there any compelling evidence?
- Could those events be coincidences?
- Are there any other explanations?
Anectodal Evidence
- The evidences in those cases are labeled as anecdotal evidences.
- An anecdote is a concise story or illustration of a captivating event, as illustrated in our cases.
Important
Be cautious when handling data collected in a haphazard manner.
While such evidence may be authentic and verifiable, it often represents exceptional cases rather than forming a reliable basis for general conclusions.
Discuss This Case
(From Samuels et al. (2016, p.8))
Observational Studies
Example 1.2.2 illustrates an observational study.
- In an observational study, the researcher systematically collects data from subjects as an observer, without manipulating conditions.
A systematic review of all data serves as a barrier to selectively observing and reporting information that aligns with a predetermined perspective.
Important
The Presence Confounding Variables: Observational studies may lead to misinterpretations due to the presence of confounding variables.
In this study, having AIDS may influence the size of the anterior commissure. The effect of AIDS is confounded with the effect of sexual orientation.
The context in which data collected is crucial in statistics. It alerts us to potential effects of other factors.
Data analysis without reference to context is considered meaningless.
Another Type of Evidence
Light and exam performance. A study was designed to test the effect of light levels on the exam performance of students. The treatments included fluorescent overhead lighting, yellow overhead lighting, and no overhead lighting (only desk lamps). The researchers randomly assigned students to each light level and found a discernible difference in exam performance based on the varying light levels.
Possible questions to be considered:
- What are the variables in this study?
- Is there any compelling evidence?
- Could this event be coincidence?
- Are there any other explanations?
Experimental Studies
The design of this experiment allows for the investigation of the interaction between two factors:
light level and exam performance.
In this scenario, researchers applied the conditions—specifically, different light levels to the subjects, which were Homo sapiens.
By randomly allocating treatments to the subjects, we can address the issue of confounding that complicates observational studies, thereby expanding the scope of conclusions we can draw from the research.
Randomized Experiments as the Pinnacle in Scientific Inquiry: Randomized experiments are often regarded as the pinnacle in scientific investigation due to their ability to overcome confounding.
- However, it’s crucial to acknowledge that they are not without their own set of challenges.
Experimental Studies
Randomized experiments are generally built on four principles.
- Controlling
- Randomization
- Replication
- Blocking
Reducing Bias in Experimental Studies
We can reduce bias in experimental studies by employing:
- Treatment/Control Group
- Placebo Group
- Blinding / Double Blinding
- Placebo Group
Placebo
Placebos are commonly administered to human subjects in experiments, often in the form of an inert substance like a sugar pill.
The well-documented placebo response illustrates that individuals frequently exhibit positive reactions to any treatment, even when it lacks active ingredients.
In many cases, a placebo leads to a subtle yet genuine improvement in patients, a phenomenon known as the placebo effect.
- However, when implementing a placebo control, it is crucial for subjects to remain unaware of their group assignment—whether they are receiving the active treatment or the inert placebo.
Blinding/Double-Blinding
If researchers keep patients unaware of their treatment, the study is termed blind.
When both researchers and patients remain unaware of the individuals in the treatment groups, it is referred to as double-blind.
Question: What could be potential reasons for implementing blinding?