Once you're finished, continue on to the test to check your understanding of p-values. Unlike most other
"tests" in this class, this test actually has specific right and wrong answers. If you don't answer all 6 questions correctly the
first time around, you should not proceed to the rest of the module until you can understand the rationale for each answer (which
may require reviewing the previous activity or the reading from this activity).
For this test, imagine the following scenario:
I did a research project in which I taught Chinese tones in two different ways to two groups
of students. One group (the "control" group) was taught using a traditional method. Another group (the "experimental" group) was
taught using a new method I created based on some learning theory. At the beginning and end of the semester I tested each group in a
test of their tone recognition. I found that the control group improved 27 points on their tone recognition from the beginning to the
end of the semester, whereas the experimental group improved 40.7 points. In other words, the experimental group's improvement was
13.7 points higher than the control group. I used a statistical test to compare the improvement between the two groups, and found
that the difference in improvement was statistically significant, p=.022.
When you proceed to the test below, you will see six statements about the results of this study. For each statement,
indicate whether the statement is true or false, based on the description above.
*Other useful (but longer) readings about p-values, and statistical testing logic in general, include
Gigerenzer (2004) and
Wagenmakers (2007).
I proved that the null hypothesis (no difference in learning outcomes between the two teaching methods) is
false.
I showed that the null hypothesis (no difference in learning outcomes between the two teaching methods) is
unlikely.
I proved that there is a difference in effectiveness between the two teaching methods.
It is likely that there is a difference in effectiveness between the two teaching methods.
The null hypothesis is that both teaching methods lead to the same learning outcomes; if I reject this null
hypothesis (i.e., if I conclude that the teaching methods lead to different learning outcomes), there is a
2.2% chance that I will have made a mistake.
The null hypothesis is that both teaching methods lead to the same learning outcomes; based on the results of my
study, there is only a 2.2% chance that this hypothesis is correct. (In other words, there is a 97.8% chance
that the teaching methods lead to different learning outcomes.)
The answers are shown at the bottom of the page. If you got all the answers correct, continue to the next section of the module:
"T-test by hand". Otherwise, review the readings to make sure you understand
why your answer was incorrect.
Answers: All six statements are false. If you marked any as "true", review the previous readings and make sure you understand why the statement is false before you proceed to the next section of the module.