Blog Index

2017, Current to 10/30

Topic	Dates of Posts
Dissertation, general	1/20, 2/1, 5/5, 5/24, 6/5, 7/17, 8/18
Dissertation calculator	8/21
Selecting a Topic	1/27
Committee Members	5/1, 5/19, 5/26
Mentor Interviews
URR
Center for Research Quality
Overview of Process
Premise
Prospectus
Proposal	6/7, 6/9, 6/14, 6/16, 6/19, 6/21, 6/23, 6/29, 6/30
Research questions	2/10
Research design	2/13, 2/17, 7/9
Theory	2/6, 2/8, 8/25
C. 1	8/23, 8/28, 8/30, 9/1
C. 2 (literature related)	1/11, 7/5, 9/8, 9/11, 9/13, 9/15, 9/18
C. 3	9/20, 9/22, 9/25, 9/27, 10/2, 10/4, 10/6, 10/9, 10/11, 10/13, 10/18, 10/20, 10/23, 10/25, 10/27
Defense	1/9
IRB	2/15
Data Collection	2/20, 2/22, 2/24, 5/3, 7/19, 7/21, 7/26, 7/28, 7/31
Quantitative	3/28, 4/3, 4/7, 4/10, 4/12, 4/14, 4/17, 4/19, 4/21, 4/24
Qualitative	3/8, 3/10, 3/17, 3/20, 3/22, 3/24 3/27
Mixed Methods
C. 4	1/2
C. 5	1/4
Appendixes
Final Defense	4/26
Career	1/23
Goal Form
Motivation	6/2
Organizing
Secondary Data	1/13
Support, Getting
Resilience
Writing	1/16, 1/18
Other	1/6, 1/25, 2/3, 3/1, 3/3, 3/13, 4/5, 5/8, 5/10, 5/12, 5/15, 5/15, 5/22, 5/29, 6/12, 7/10, 7/12, 7/14, 7/24, 8/2, 8/7, 8/9, 8/11, 8/14, 9-6

Previous Years

Topic	Dates of Posts - 2014	Dates of Posts - 2015	Dates of Posts - 2016
Dissertation, general	6/25, 12/5	1/7, 3/6, 11/20, 12/16	4/8, 4/11, 5/11, 6/20, 8/29, 9/2
Dissertation calculator	9/5	3/9	4/15
Selecting a Topic	4/28, 5/9, 8/25		9/7
Committee Members	5/7, 10/8, 12/22	9/14, 10/14, 12/18	8/10, 9/19
Mentor Interviews			9/21, 9/23, 9/26
URR
Center for Research Quality
Overview of Process
Prospectus	4/4, 4/7, 4/9, 4/11, 4/14, 4/18, 4/21, 4/23, 4/25, 4/28, 5/2, 5/5, 8/27		2/3
Proposal	9/8, 11/3, 11/5		2/5
Research questions	4/18	6/19	4/13
Research Design		6/15, 6/22, 6/26
Theory		6/15, 6/17	2/1, 4/27, 5/2, 5/4
C. 1	11/7	3/11, 3/13, 3/16, 3/18, 3/20, 9/18, 9/21	3/23, 10/1, 10/3, 10/5, 10/7, 10/10, 10/12
C. 2 (literature related)	6/9, 6/11, 6/16, 9/10, 9/15, 9/17, 9/19, 9/26, 9/29, 11/10, 12/26	1/9, 3/23, 3/25, 3/27, 4/3, 4/8, 9/23, 10/21	1/4, 1/6, 1/8, 1/11, 1/13, 1/15, 1/20, 1/22,1/25, 2/10, 3/7, 3/9, 7/11, 7/13, 8/22, 10/14, 10/17, 10/19, 10/21, 10/24
C. 3	1/3, 1/6, 1/13, 11/12	4/10, 4/13, 4/15, 4/17, 4/20, 4/22, 4/24, 4/27, 4/29, 5/4, 5/6, 5/8, 5/11, 5/13, 5/15, 5/18, 9/25	3/21, 3/25, 10/26, 10/28, 11/7, 11/9, 11/11, 11/14, 11/16, 11/18, 11/21, 11/25, 12/2, 12/3, 12/5, 12/7, 12/8
Defense		9/28
IRB	1/10, 1/15, 1/17, 1/20, 1/22, 1/24, 1/27, 1/29, 2/3, 2/5, 2/7, 2/10, 2/12, 2/17, 2/19, 2/21, 2/24, 10/13, 10/15, 10/17, 10/20, 10/22, 10/24, 10/27	6/24 10/5	3/28, 4/4
Data Collection		6/24, 11/23, 11/27, 12/4, 12/7, 12/9	6/3, 6/6, 6/8, 6/10, 6/13, 6/15, 6/17
Quantitative	1/3, 2/26, 3/12, 7/9, 7/14, 7/16, 7/18, 7/21, 7/23, 7/25, 7/28	5/4, 5/8, 5/11, 5/13, 5/15, 7/1, 7/6, 7/8, 7/13, 7/15, 7/17, 7/20, 7/22, 7/24	11/9, 11/11, 11/18, 11/25, 12/21
Qualitative	1/6, 3/14, 10/29, 12/12, 12/15, 12/17, 12/19	1/5, 1/12, 10/19	5/16, 5/18, 5/20, 5/23, 8/1, 11/2, 11/4, 11/16, 12/3, 12/5, 12/7, 12/19, 12/26
Mixed Methods	1/3, 1/6	4/10, 5/6, 5/8, 5/11, 5/13, 5/15	11/2, 11/4, 11/18, 11/25, 12/5, 12/7, 12/16, 12/19
C. 4	3/5, 3/10, 3/12, 3/14, 3/17, 3/19, 3/21, 11/14	5/20, 5/22, 5/25, 5/27, 6/3, 6/5, 10/7	9/16, 12/12, 12/14, 12/16, 12/19, 12/21, 12/26
C. 5	3/21, 3/24, 3/26, 3/28, 11/19	6/5, 6/8, 6/10, 6/12, 10/9
Appendixes
Final Defense		10/12
Career	8/18, 8/20		9/9
Goal Form	5/23, 5/26, 6/2, 8/15, 11/24
Motivation	1/1, 6/4, 6/6, 7/4, 7/11, 9/22, 10/10, 12/3	1/21, 3/2, 7/3, 7/10, 9/2, 9/4, 9/7, 9/9 9/11	5/9, 7/1, 7/4, 7/6, 7/8, 7/15, 7/20, 7/22, 7/25, 8/8, 8/15
Organizing	8/1, 8/4, 8/6, 8/8, 8/11, 8/13, 12/8	7/27, 7/29, 8/3, 8/5, 8/10	2/22, 2/24, 2/26, 3/4
Secondary Data	2/24
Support, Getting	5/21	1/26	8/17
Resilience		2/6, 2/16, 2/18, 2/20, 2/23, 2/25, 3/4	5/25, 5/27
Writing	5/16, 5/19, 6/16, 6/18, 6/20, 7/2, 7/7, 8/27, 12/10	1/14, 1/23, 2/9, 12/11, 12/14	3/11, 3/14, 3/16, 6/24, 7/18, 8/3, 8/5, 8/12, 8/26
Other	2/14, 3/3, 3/7, 4/16, 5/12, 5/14, 5/28, 6/2, 6/13, 6/23, 6/27, 8/22, 9/1, 9/3, 9/12, 10/3, 10/6, 11/17, 11/26, 12/1, 12/24, 12/29, 12/31	1/16, 1/19, 1/28, 1/30, 2/4, 2/11, 2/13, 4/1, 4/6, 6/1, 8/12, 8/14, 8/19, 8/21, 8/24, 8/26, 8/31, 9/16, 10/2, 10/16, 10/23, 10/26, 10/28, 11/2, 11/4, 11/6, 11/9, 11/11, 11/13, 11/16, 11/18, 11/25, 12/2, 12/21, 12/23, 12/25, 12/28	1/1, 1/17, 1/29, 2/12, 2/15, 2/17, 3/2, 4/1, 4/6, 4/18, 4/20, 4/22, 4/25, 5/6, 5/13, 6/1, 6/22, 6/27, 7/27, 8/17, 8/19, 8/24, 9/12, 9/14, 9/19, 9/28, 11/23, 11/28, 12/23, 12/30

Next time we will continue our review - Chapter 3: Ethical Procedures. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Issues of Trustworthiness in qualitative and mixed method studies

Rather than issues of validity that we saw in quantitative studies, qualitative (and mixed methods) have trustworthiness issues. The first issue is credibility, which is comparable to internal validity. This is getting at the credibility of your data, common methods used are triangulation, prolonged contact, member checks, and saturation. You want to show that your data are as accurate as possible.

The second issue is transferability, which is comparable to external validity. This is getting at the generabilizability of your data to other groups. Common methods used are thick description and a variation in participant selection.

The third issue is dependability, comparable to reliability. You want to show the accuracy of your data methods, common methods are audit trails and triangulation. Triangulation is accomplished by asking the same research questions of different study participants and by collecting data from different sources and by using different methods to answer those research questions. Member checks occur when the researcher asks participants to review both the data collected by the interviewer and the researchers' interpretation of that interview data. Participants are generally appreciative of the member check process, and knowing that they will have a chance to verify their statements tends to cause study participants to fill in any gaps from earlier interviews.

The fourth issue is confirmability, comparable to objectivity. This is the degree to which the findings are the product of the focus of the study and not of the biases of the researcher One way to do this is through an audit trail. An adequate trail (or records) should be left to enable the auditor to determine if the conclusions, interpretations, and recommendations can be traced to their sources and if they are supported by the inquiry.

If you are using another coder(s), you must show how you will demonstrate intercoder reliability. Interrater or intercoder reliability is used to reduce bias by having multiple people code the data. How you go about this and how you resolve any discrepancies needs to be detailed. 

Next time I will post an updated blog index. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Threats to Validity- construct and statistical validity

The final section under Threats to Validity for quantitative and mixed method studies asks you to describe any threats to construct or statistical conclusion validity. Let's start with a definition of construct. A construct is an attribute, proficiency, ability, or skill that happens in the human brain and is defined by established theories. For example, "resilience" is a construct. It exists in theory and has been observed to exist in practice.

Construct validity has traditionally been defined as the experimental demonstration that a test is measuring the construct it claims to be measuring. Such an experiment could take the form of a differential-groups study, wherein the performances on the test are compared for two groups: one that has the construct and one that does not have the construct. If the group with the construct performs better than the group without the construct, that result is said to provide evidence of the construct validity of the test. An alternative strategy is called an intervention study, wherein a group that is weak in the construct is measured using the test, then taught the construct, and measured again. If a non-trivial difference is found between the pretest and posttest, that difference can be said to support the construct validity of the test.

There are a large number of threats to construct validity- too many to discuss here, but I do suggest that you take a look at http://www.socialresearchmethods.net/kb/consthre.php when you are ready to write this section. The author does a very nice job laying out the many types of possible threats.

The last issue is statistical conclusion validity. Statistical conclusion validity is the degree to which our conclusions about the relationship between your variables based on the data are correct or ‘reasonable’. This is getting at the two types of statistical errors that can occur: type I (finding a difference or correlation when none exists) and type II (finding no difference when one exists). Statistical conclusion validity concerns the qualities of the study that make these types of errors more likely. Statistical conclusion validity involves ensuring the use of adequate sampling procedures, appropriate statistical tests, and reliable measurement procedures. If you would like a more in depth discussion of this topic, please see http://www.socialresearchmethods.net/kb/concthre.php 

Next time we will continue our review - Chapter 3: Issues of Trustworthiness in qualitative and mixed method studies. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Threats to Validity- internal validity quant and mixed methods

Previously we looked at external validity in quantitative and mixed methods studies, today we look at internal validity. Internal validity refers to whether an experimental treatment/condition makes a difference or not, and whether there is sufficient evidence to support the claim. Some threats to internal validity include:

•History--the specific events which occur between the first and second measurement. People are affected by elements outside of the study. Let's use an extreme example of this, say you were interested in fear of flying and gave people a survey examining this variable on Sept 9, 2001. The participants then went through a de-sensitization training for a week and came back on Sept 16, 2001 and were retested. They are also going to be affected by an historical event outside of the study- the traumatic events of 9/11, and you would need to account for this.

•Maturation--the processes within subjects, which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment. As people age they change, so if you were doing a study that lasted any period of time, you need to realize that they will change without your intervention. This is often why a control group is used, so the normal changes that occur can be compared with those of the treatment.

•Testing--the effects of taking a test on the outcomes of taking a second test. Simply taking a test can change how people think, they also cannot forget what they read in the first test. So if you give a second test people will have thought about their first answers and may change them in the second test because of that thinking process.

•Instrumentation--the changes in the instrument, observers, or scorers, which may produce changes in outcomes. Many things can affect the results, minor changes in wording, having additional people in the testing area, having different people score the test all may change the results.

•Statistical regression--It is also known as regression to the mean. This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment, not because of my great treatment, but because they expect to do better.

•Selection of subjects--the biases which may result in selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, keep in mind that randomization is only effective with large samples.

•Experimental mortality--the loss of subjects. For example, if you require people to participate in multiple training sessions, some will drop out. Those who stay in the project all the way to end may be more motivated to learn and thus achieved higher performance.

•Selection-maturation interaction--the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect. A great example is if you had girls in a class assigned to one treatment and the boys assigned to another treatment. You compare them and discover there is a treatment difference. However, you do not know if it is the treatment that caused the differences or was it any differences in development between the girls and the boys.

In this section of your paper, you need to think through the various internal validity issues and how you will address them.

Next time we will examine threats to validity- construct and statistical validity. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas!

Chapter 3: Threats to Validity: external validity – quantitative and mixed methods studies

The threats to validity section in the quantitative and mixed methods checklist is one of the trickiest to understand. Let's see if I can make some sense of it for you. It begins with threats to external validity. External validity is the extent to which the researcher can conclude that results apply to a larger population (generalizability). Here are some common threats to external validity.

•Reactive or interaction effect of testing--a pretest might increase or decrease a subject's sensitivity or responsiveness to the experimental variable. Therefore, giving a pretest changes your participants, they will respond differently later because they took the pretest.

•Interaction effects of selection biases and the experimental variable. You may unintentionally choose people that have particular biases. For example if you are doing an online survey about use of the internet- you will only have people participate who are already comfortable enough with the computer and internet to choose to participate in an online survey. You will be missing people who are not comfortable with computers.

•Reactive effects of experimental arrangements--it is difficult to generalize to non-experimental settings if the effect was attributable to the experimental arrangement of the research. So, if you are doing some type of experiment in a controlled setting (picture a quiet psychology lab room), there is no way to know what will happen when a similar occasion occurs in the real world.

•Multiple treatment interference--as multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments. People cannot "unlearn" something, so whatever has happened to them previously will affect future learning/ experiences.

In this section of chapter 3, you need to think through what are the threats to external validity in your study. Keep in mind that no study is perfect, it is ok, in fact, it is expected that there will be issues. The important thing is that you recognize them.

Next time, we will look at threats to validity- internal validity quant and mixed methods. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Instrumentation in mixed methods studies

For mixed methods, the instrument section has to accommodate both qual and quant instruments. For the qualitative components, identify each instrument (observation sheet, interview protocol, focus group protocol, videotape, audiotape, artifacts, archival data, and other kinds of data collection instruments). State the source of each item and provide the permission for it in your appendix.

For published instruments, identify who developed it, where and with what populations it has been used. State why you think it is appropriate for your study, and any cultural or context issues that might be present with your population.

If you are designing qualitative instruments, explain how you developed them – what was the basis for them? How will you establish content validity?

Similarly, for the quantitative components, explain the background of each instrument. Discuss validity and reliability in previous studies and where it has been used before.

The next section is how you will recruit participants for each component (qual and quant). Go into detail on how and where the data will be collected for each component.

Finally, you need to lay out your data analysis plan for each component. For the quantitative aspects, indicate your hypotheses and what statistical tests will be used for each. How will you interpret the results? For the qualitative portion, indicate how you will code the transcripts and how you will handle discrepant cases. Then you need to integrate the two types of data and compare their results. How will you do this? 

Next time we will continue our review - Chapter 3: Threats to Validity. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu