In this module we have discussed t-tests. You probably have noticed that there are other modules about other
kinds of tests (such as regression and analysis of variance), and you may have other of still other kinds of tests that are not
addressed in this subject (such as chi-squared tests). How do you decide what test to do? Why did we use t-tests, instead of
other ones, for the examples in this module?
To understand how to choose a test, you have to understand the variables in your research design. A variable
is some thing that you measure or manipulate in your study. Things that you measure might be stuff like people's proficiency, test
scores, brain responses, reaction times, age, etc. Things that you manipulate might be things like which kind of training each person
gets, which kind of sentence they read in an experiment, etc. There are two important distinctions to consider with your variables:
whether a variable is independent or dependent, and what kind of scale the variable is measured on.
Independent vs. dependent variables
In most research you are interested in causes and effects: e.g., you might predict that giving people a different
kind of training will cause them to learn grammar better, or showing people a different kind of sentence will cause their brain to
react a certain way, or having higher motivation will cause people to learn tone better, or whatever. (Note that statistical methods
usually cannot prove cause and effect in observational studies—see the "Types of research designs" module. Here I'm
oversimplifying.) Generally, the independent variable (自變量) is the thing that you think causes something, and
the dependent variable (因變量) is the effect that you want to measure. Another way of looking at it is that, in
experiments, the independent variable is the thing you manipulate (either directly, by, e.g., assigning people to different training
groups; or indirectly, by, e.g., making sure that your student sample has a good mix of high-proficiency, low-proficiency, and
medium-proficiency students), whereas the dependent variable is the thing that you measure. The independent variable is "independent"
because you choose it (you decide what kinds of groups you will look at, what kinds of words you will put in the experiment, etc.),
and the dependent variable is "dependent" because it is determined by the other variables (e.g., in a reaction time experiment, you
don't choose how fast a word will be responded to; it gets responded to faster or slower based on whether it's a noun or a verb,
whether it's preceded by a related or an unrelated prime, or whatever other independent variable you have been messing with). In the
example we discussed previously about comparing the age of PolyU students and UST students, "school" is the independent variable and
"age" is the dependent variable—we "manipulated" the school variable by intentionally measuring people from two different
schools, whereas we just passively observe what their ages are.
Measurement scales
Variables roughly fall into three types of measurement scale. Some variables are lumped into categories, with no
inherent order. For example, in a study that compares "experimental group" vs "control group", or a study that compares "French
speakers" vs. "Korean speakers", there is no inherent order; there's no particular reason to call French speakers "Group 1" and
Korean speakers "Group 2" or vice versa. These are nominal variables. (Sometimes they are also called categorical
variables; in R, they are called factors.)
Some variables do have an inherent order, but the distances between each level are not the same. For example, imagine
you have people take a survey (such as a evaluation you all take at the end of every class) and rate how satisfied they were, with
choices such as "Very dissatisfied", "Somewhat dissatisfied", "Neutral", "Somewhat satisfied", and "Very satisfied". Or imagine you
give participants a language background questionnaire and ask how often they use English, with choices like "Never", "Sometimes",
"Often", and "Always". These have a clear order ("never" is less than "sometimes", "sometimes" is less than "often", etc.). But the
distance between levels may not be the same: maybe "often" is a lot more than "sometimes", but "always" is only a little bit more
than "often". Or imagine people run a race, and you record who was first place, who was second place, who was third place, etc. There
is an inherent order there, but the distances between each person may be very different; maybe the first-place runner just barely
beat the second-place runner in an exciting last-minute sprint to the finish, but the third-place runner was very far behind both of
them. This kind of variable is called an ordinal variable.
Finally, some variables work like true numbers. They have an order, and the distances between each number are
consistent. For example, if you measure how tall someone is, or if you measure how quickly someone can finish reading a sentence, the
results you get are actual meaningful numbers. This is called an interval variable (sometimes called a numeric or
continuous variable). (As you learn more about statistics you may also sometimes hear about a distinction between
interval and ratio variables: interval variables are numeric but don't have a natural zero [this would be things
like TOEFL scores] whereas ratio variables do have a natural zero [things like length or reaction time]. In practice, this
this distinction rarely has any practical consequences on what I need to do for a statistical analysis, so I usually just ignore
it.)
Putting it all together
The reason this stuff is important is because it determines what kinds of test you use. Different tests are suited
for different kinds of variables. For example, a t-test works great when you have a nominal independent variable (with only
two levels) and a numeric dependent variable; you can review the examples from throughout this module and see that they all fit this
criteria. If you have a continuous independent variable and a continuous dependent variable (e.g., if you want to see if people
with higher language proficiency have faster reaction times—proficiency and reaction times are both usually measured as
continuous/numeric/interval variables), a t-test will be no good.
Likewise if you have a nominal independent variable and a nominal dependent variable. Likewise if you have a continuous dependent
variable and a nominal independent variable with more than two levels (e.g., if your research project has three groups instead of
two). For situations like that, you'll need different tests. As you've seen in previous tasks, the nature of the relationships
between groups (e.g., paired vs. independent) will also influence what kind of test you need.
https://stats.idre.ucla.edu/other/mult-pkg/whatstat/ is a great
resource for seeing what kind of test you need in any situation.
You can find further reading on variables types, in any intro statistics textbook or online tutorial, but what was
presented here should be enough to at least give you the foundation you'll need to do the remaining data analysis modules in this
class.
Describe one kind of research situation that an independent samples t-test would be appropriate for, one kind of research
situation that a paired samples (or one-sample) t-test would be appropriate for, and one kind of research situation that
would need some other test instead of a t-test.
When you finish this activity, you are done with the module (assuming all your work on this and the
previous tasks has been satisfactory). However, you may still continue on to the advanced-level
task for this module if you wish to complete this module at the advanced level (if you're aiming for a higher
grade or if you are just particularly interested in this topic). Otherwise, you can return to the
module homepage to review this module, or return to the class homepage to select a different module
or assignment to do now.