Previously we looked at external validity in quantitative
and mixed methods studies, today we look at internal validity. Internal
validity refers sto whether an experimental treatment/condition makes a
difference or not, and whether there is sufficient evidence to support the
claim. Some threats to internal validity include:
•History--the
specific events which occur between the first and second measurement. People
are affected by elements outside of the study. Let's use an extreme example of
this, say you were interested in fear of flying and gave people a survey
examining this variable on Sept 9, 2001. The participants then went through a
de-sensitization training for a week and came back on Sept 16, 2001 and were
retested. They are also going to be affected by an historical event outside of
the study- the traumatic events of 9/11, and you would need to account for
this.
•Maturation--the
processes within subjects, which act as a function of the passage of time. i.e.
if the project lasts a few years, most participants may improve their
performance regardless of treatment. As people age they change, so if you were
doing a study that lasted any period of time, you need to realize that they
will change without your intervention. This is often why a control group is
used, so the normal changes that occur can be compared with those of the
treatment.
•Testing--the
effects of taking a test on the outcomes of taking a second test. Simply taking
a test can change how people think, they also cannot forget what they read in
the first test. So if you give a second test people will have thought about
their first answers and may change them in the second test because of that
thinking process.
•Instrumentation--the
changes in the instrument, observers, or scorers, which may produce changes in
outcomes. Many things can affect the results, minor changes in wording, having
additional people in the testing area, having different people score the test
all may change the results.
•Statistical
regression--It is also known as regression to the mean. This threat is caused
by the selection of subjects on the basis of extreme scores or characteristics.
Give me forty worst students and I guarantee that they will show immediate
improvement right after my treatment, not because of my great treatment, but
because they expect to do better.
•Selection
of subjects--the biases which may result in selection of comparison groups.
Randomization (Random assignment) of group membership is a counter-attack
against this threat. However, keep in mind that randomization is only effective
with large samples.
•Experimental
mortality--the loss of subjects. For example, if you require people to
participate in multiple training sessions, some will drop out. Those who stay
in the project all the way to end may be more motivated to learn and thus
achieved higher performance.
•Selection-maturation
interaction--the selection of comparison groups and maturation interacting
which may lead to confounding outcomes, and erroneous interpretation that the
treatment caused the effect. A great example is if you had girls in a class
assigned to one treatment and the boys assigned to another treatment. You
compare them and discover there is a treatment difference. However, you do not
know if it is the treatment that caused the differences or was it any
differences in development between the girls and the boys.
In this section of your paper, you need to think through the
various internal validity issues and how you will address them.
Next time I will post an updated index. Do you have an issue
or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me
your ideas! leann.stadtlander@waldenu.edu
No comments:
Post a Comment