Chapter 3: Instrumentation in qualitative

In qualitative studies, if you are talking to people and not using archival data, you will design most of your instruments.  You need to identify each data collection instrument and provide the source of it, if you did not design it (some examples: observation sheet, interview protocol, focus group protocol) There are also archival data which would need to be identified and the source (e.g., video-tape, audio-tape, artifacts, archived data).

If you are using historical or legal documents are used as a source of data (unusual to use), demonstrate the reputability of the sources and justify why they represent the best source of data. Then you want to clearly demonstrate the link between the data collection instruments and your research questions.

For published data collection instruments.

Explain who developed the instrument and provide the date of publication. Detail where and with which participant group it been used previously. You then need to justify its use in the current study (that is, context and cultural specificity of protocols/instrumentation) and whether modifications will be or were needed.

Describe how content validity will be or was established (how do you know it is looking at what you think it is?) A common way to do this is to use an expert panel. Discuss any context- and culture-specific issues specific to the population while developing the instrument. An example might be that if you are using an interview protocol that was designed for adults, and you want to use it with adolescents, you would need to change some of the language.

For researcher-developed instruments

What did you use as the basis for instrument development (some examples might be from the literature or from doing a pilot study)? Again you need to describe how content validity will be or was established (how do you know it is looking at what you think it is?). Finally, you want to describe how your instruments will answer the research questions. 

Next time I will look at instrumentation in quantitative studies. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Participant Selection- qualitative and mixed methods

The section on participant selection for qualitative and mixed methods is similar to the quantitative, except you need to think of it in terms of the methodology. So again, identify your population – to whom will the results generalize? In qualitative/ mixed method studies the population is generally smaller than quantitative, so if you are interested in women who have been in domestic violence relationships, think about what age range will be included? What geographic area? All of these issues limit the generalizability.

Identify and explain your sampling strategy. For example, will you use snowball sampling? How will that happen? How will you do your initial recruitment? Why is this the best method for your specific study? How will you know participants meet your inclusion criteria? Using the previous example, how will you know that they have been in abusive relationships?

How many participants do you need? Why did you decide on that number (support it with literature)? For mixed methods studies, you will also need to explain how many participants you need for each aspect, and you will need a power analysis for the quantitative portion. Talk to your methodologist about this.

Carefully describe how you will identify, contact, and recruit your participants. Be very detailed. Remember if you plan to rely on any other people/ organizations for referrals or help in any way, you will need a letter of agreement from them, spelling out exactly what they will do and provide.

In qualitative studies, you need to consider the idea of saturation, meaning when you are not getting any new information from participants. How does this fit with your sample size? If you have not reached saturation by the time you have talked to all of your required sample, what will you do? (hint, keep doing interviews).

Next time we will continue our review - Chapter 3: Instrumentation in qualitative studies. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Additional Information, quantitative

There are three special cases that are listed in the quantitative checklist that we will examine today: a pilot study, conducting an intervention, and archival data. I am a fan of pilot studies, but you do need to explain why you are doing one and who will be participating in it. Pilot studies are typically done to give you an opportunity to practice your procedures, check the clarity of the measure, and to determine how long the full study will take (so you can report that in your consent form).

Doing an intervention, which means introducing any new training, treatment, or information to your participants is a very tricky proposition at Walden. Even something as simple as having a control group may come under the intervention rules. So if you have been thinking along these lines, I caution you to write to the IRB today (irb@waldenu.edu) and talk to them about it. You, as the researcher, will not be allowed to do the intervention. So, some other group or institution will have to sponsor it. You are only allowed to collect the data on it. After you talk to the IRB about this, if it still makes sense to do an intervention, you will need to clearly describe what will be done and who will do it. Also, clearly indicate your role and what you will and will not be doing with it.

Archival (or secondary) data, using already collected data in your dissertation, is a great method to use. You will need to relate all the information about how the original study was done, including recruitment and data collection. Then you will need to describe the procedure used to access the data, including any permission letters you needed (include them in the appendix). If you are using historical or legal documents (this is uncommon), describe how you know they are accurate and why they are the best possible sources to use.

Next time we will switch to qualitative and mixed methods studies and look at Chapter 3: Participant Selection. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Recruitment- quantitative

In this section, you need to think details! It must be written in enough detail that someone else could replicate the study based on your description. This is always more difficult than you think it will be, you might find it helpful to talk through the section with someone, such as a classmate or spouse. Have them push you for as many details about it as possible.

The checklist is pretty vague in the specifics of this section. Let's start with how you will recruit- will you put up flyers? Send out emails? If you are sending out emails- where will you get the addresses? Make a note that you will need copies of all communications (flyers, emails) for the IRB and they will need to be in your paper's appendix. If you are planning to get emails from a company, institution, etc. you will need a letter of agreement from them spelling this out.

Once people contact you, what will you say to them? This needs to be written out. If you are sending them to a survey on a website, what will it include? What demographic information will you collect? Typically, you will want to know their sex, age, and maybe socio-economic status. This is collected so you will be able to describe who your sample is. There may be other information that you will want to know to confirm their eligibility to be in the study, perhaps marital status, part or full time work, etc.

Each participant will need to read and agree to a consent form (see the IRB website for a template). If they are completing an online survey, this is usually the first page of the survey. A copy will need to be sent to the IRB (when you do your IRB application) and be included in your paper's appendix.

Next, you need to describe how the data will be collected. Again, be very specific on how this will occur. Then you need to discuss if you will debrief your participants in some way (often done in in-person studies). This is where you would thank them for participating and give them an overview of the study, if it wouldn’t have been clear before. Again, a copy will need to be sent to the IRB of the debriefing and added to the appendix.

Finally describe any follow up procedures, such as their being contacted again in the future or having to return for any reason. 

Next time we will continue with quantitative studies and look at Chapter 3: Additional Information. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Population and Sampling Procedures from quantitative studies

In quantitative studies, it is particularly important to understand who your population and who it is not. Therefore, the first thing to address in this section is who is your target population. Again, the best way to think about this is who does the study generalize to? So if you were testing undergraduates, that is who your population really is (contrary to many of the old studies who make the case that they are similar to the population in general). Next, you want to state approproximately how large the population is that you are using. This may take some detective work to get at.

The next section is Sampling. You rarely can test the entire population; instead, you must strategically sample to make sure you get a representative group. How will you do that? There are many ways that have been suggested, the important thing is to realize the cost and benefit of each method and how they affect your study. Let's say that you decide that you will post flyers and have people contact you if they want to participate. There are a number of costs of doing it this way: you will only get people who frequent where you are posting flyers, you will only have people who volunteer, and only people who have a phone to contact you. None of these issues are fatal flaws, but you need to be aware of them.

What are your inclusion criteria (who can participate) and exclusion criteria (who cannot participate)? Think details. If you are doing a an internet survey on survey monkey- inclusion criteria include: people who can read English, have access to a computer, as well as any issues related to your population (e.g., divorced for a year).

You will need to do a power analysis to determine how large your sample should be. Check with your methodologist, as to whether they have a preference as to how to do it. I generalize recommend using one of the power analysis calculators available online (do a search for power analysis calculator). You need to give the website that you used for the calculation and why you used the parameters that you entered. You may need to talk to your methodologist about this aspect.

Next time we will continue with quantitative studies and look at Chapter 3: Recruitment. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Participant Selection (qual and mixed methods)

Today's section, participant selection, is only used for qualitative and mixed methods. First, you need to identify who your population is, it is often a good idea to think about it as who does the study generalize to? So if you are interviewing victims of domestic violence in your small rural town, your population may be victims in rural areas.

Next, you will want to identify and justify your sampling strategy. There are a number of ways to recruit your participants, a couple of examples are placing flyers in areas that you suspect they frequent (don't forget hair salons, churches), another example is snowball sampling where you ask each participant to suggest others that might be interested. You need to explain why this is a good method for your study, perhaps you have a hard to reach group and this will ensure a sufficient sample size.

Now you need to state your inclusion and exclusion criteria. Be very detailed, think about who can participate – people who are native English speakers? Able to read and speak clearly? Only people who have been divorced for over a year? Clearly articulate who can be in your study and also who cannot be in the study. Then you want to discuss how you will know if they meet the inclusion criteria- will you ask them? Do you assume if they can read your consent form they can read English?

How many people will you have participate, why did you select that number? For mixed methods, you will need to go further and indicate how many people will be in the quantitative portion and do a power analysis (more on this next time).

Then you need explain how you will be recruiting people and how you will identify they are appropriate for your study. This is similar to the section above, but here you are more specific in how it will be done.

Part of all qualitative studies is the concept of saturation, whereby you continue to sample until you are consistently getting responses you have heard before. You will need to discuss this and how it relates your sample size. 

Next time we will look at Chapter 3: Population and Sampling Procedures. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Chapter 3: Role of the Researcher (qual, mixed methods)

In qualitative and mixed methods you need to discuss your role as the researcher. Why? Because you are going to be doing the interviews, therefore, who you are and how you know and interact with the participant is essential to understanding the study.

You will want to first define and explain your role as an observer of the behaviors, a participant in the interview, and or an observer-participant. Next, you will discuss any relationships that you have with the participants, particularly if you have a supervisory relation (this will be a big issue for the IRB, so think if there is a way around using your supervisees).

Next, you will discuss how your own biases will be managed. Every researcher comes into a study with biases, if you didn’t have ideas as to what you thought would happen, you would not do the study! You may have been (or are) a member of the population group, how will you keep that from influencing your interview? You will need to be impartial and unbiased, how can you do that? One common way is to approach subjects/ participants neutrally and keep a journal where you relate your feelings that come up for you. It is common for new interviewers to want to engage in a social conversation with the participants and share their own experiences. Do not do this! Your story may influence how they respond to questions; you want to approach the topic as someone new to the topic. Do not assume that you know what they are going to say, let them explain the details.

Finally, you need to think through any other ethical issues, such as, doing a study within one’s own work environment, conflict of interest or power differentials, and justification for use of incentives. Then lay out a plan for addressing any issues that are present.

Next time we will look at Chapter 3: Participant Selection. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu