Secondary Data

What is secondary data? It is data that has already been collected by someone else. It may be data that is collected as a standard practice in an organization (e.g., medical charts, pre-employment testing). Some things to know about this type of secondary data, particularly if it is from an organization where you are employed. First, you will only be able to use information that is available to any outside researcher. Second, the data will need to be "de-identified" before you get access to it. This means someone else will have to remove all names of individuals, so the data is anonymous when you receive it. Check with IRB early in the process, if you are interested in this option. 

Another source of secondary data is large datasets that have been available by the original researchers. Walden provides students with the opportunity to access hundreds of datasets. To see what is available – go to http://www.icpsr.umich.edu/icpsrweb/ICPSR/ The researchers provide information on the background, variables, and any coding they used for data input. The use of such a dataset is perfectly acceptable as a dissertation, however a few cautions. Make sure that the study you are considering has not been previously published with the data. Typically, the researchers give a bibliography of all articles published by them. However, it does not mean that other people have not published. Search the library carefully, and include dissertations in the search. Use the name of the dataset as a search term. Another very important caution is do not try to access the data until you have IRB approval, you must have Walden's ok to proceed. Also, be aware that it can be tricky to convert the data to SPSS, and to understand the logic of their data entry. You probably will not be able to contact the researchers, so you will need to figure it out with the help of your committee. 

Secondary data analysis is a great way to get access to difficult populations' data; an example might be sex offenders. However, you are stuck with the variables that the original researchers used, and there is no possibility of getting additional information from the participants. If you decide to go this route, you can reduce your time to completion considerably, but be prepared to work through the issues I outlined above. 

Next time, I invite you to look over my shoulder as to I do the literature search for an article I am writing.

 

Alternative Sources for Articles

Students frequently ask me how to get articles that they cannot find in the Walden library. Today I will talk about a few alternative sources. First, make sure that you have checked all of the Walden databases. Some that are frequently missed- on the psychology topic page of databases (psycINFO is the first one listed); go down the page to "Psychology: A SAGE Full-Text Collection." This database has many psychology articles that, for some reason, are typically not picked up by PsycINFO. If you are interested in a health related topic, make sure you check in CINAHL (a nursing database) and Medline, both are available under Health Sciences. 

On the Topic page in the library (where all of the different topic areas are listed), look in the left column for "Thoreau: Search Multiple Databases." This often pulls up articles I can't find in individual databases. In the same column on the Topic page is "Database by Name". In here is Academic Search Complete. It brings up some info outside the regular databases. 

So, let's say you still cannot find a particular paper you are looking for; if you know the authors and title you can request a Document Delivery, in which the librarian will track it down for you. As a dissertation student, you are allowed 30 free ones, after that they may charge you. 

Let's say you have reached your limit on Document Deliveries or the library can't find it. An alternative, check Google Scholar – there is a link on the library website. 

Those are the easy sources -there are some other alternatives. If you have a college or university near you, you might check if they have it. If all else fails, you can write to the author, although this will probably require some detective work. If you know what affiliation (college, university, etc.) they listed on the article (it is usually listed in the abstract), go to the institution's website and see if they are still employed there. They usually provide email addresses. You can try the email they listed in the article (also often listed on the abstract page), however, be aware that faculty tend to move around a lot, and it may no longer work. The problem is that you usually do not receive any notice that the person no longer works there, so if you do not get a reply you don’t know if they are just rude or never got it. It is worth checking the institution's website.  

If you still can’t find the author, do a Google search on him or her; you may pick up a clue where they work now and can try to track down an email. 

What do you say when you write them? Tell them you are very interested in their work on X and ask if they could send you a copy of their article (give the citation for it). Also, ask them if they have any other papers in this area. Be sure to include your mailing address in case they want to send it that way. Here is a sample letter: 

Dear Dr. Jones,

I am very interested in your work on fear of pencils. I am a doctoral student in psychology at Walden University and my dissertation is in this area. I have been unable to locate your paper Jones and Smith (2010) "Fear and Pencils: How do I function now?" I would greatly appreciate it if you could send me a copy of this article. In addition, if you have any subsequent papers in this area, I would love to get copies of those too.

Thank you so much for any help that you can provide, 

Suzy Student

17 Hopeful Lane

Johnston, NY 98765 

Unfortunately, some people are rude, and you may not get a response. The final step would be to try to track down other authors on the paper. Next time, I will talk about the issues related to using secondary data and where to get it.

Final steps

You have written all five chapters, your dissertation is probably between 100-200 pages, you have put in many hours of work, thought, and sweat into this document. So what is next? First, your chair must approve it, and then it goes to your committee member. Expect changes, these are just part of the process. Once both of your committee members are happy with it, it goes to the University Research Reviewer (URR) who will read through the entire paper and probably want more changes. I strongly urge patience at this point. Again, as I have mentioned many times, each person is responsible for their small piece of this paper and will be held accountable for the quality of the final product. Rest assured that everyone wants you to be done with the best quality paper possible.  

These final steps often are very stressful for students; they are often nearly at the end of a quarter and feel a great deal of pressure. Keep reminding yourself, it will happen… the end is in sight. Patience will keep you from sleepless nights. 

Once the URR signs off on your paper, the research center sends it on for Form and Style review. In this review, an editor goes through your paper indicating what will need to be done to be compliant with the dissertation publishing company. The editor has 14 days for this, and in my experience, they generally take the full time. Meanwhile, you will be told that you can set the date for your defense to be after the 14 days for Form and Style review. Contact your committee members and come up two or three times that will work for everyone. Your chair will submit the necessary form to reserve the conference call line. Use this time to prepare your PowerPoint for your defense. 

As soon as you get the Form and Style review back begin working on the changes. In advance of the defense call, you will want to prepare a PowerPoint for your committee. You will be given 20 minutes to talk about your project, so keep the slides to less than 25. It is better to have just an outline on the slides, you can always write out everything you want to say and read it during your presentation. Concentrate on the analyses and results with just enough from your proposal to put it into context. Check with your chair how they like it done. Send your committee the PowerPoint prior to the defense call. 

On the day of the defense, have your computer on with your paper open for reference. Have any other papers available that you might need, as well as your PowerPoint. It is also a good idea to have some water available and make sure you have an environment where you won’t be distracted or disturbed. Once you call in, your committee will greet you and try to put you at ease. Your chair will leave the conference briefly to turn on the recorder; there will be announcement when it starts. Your chair will probably indicate the date and who is on the call for the recorder and then it will be your turn to go through the presentation. 

A few reminders, this is not a confrontational situation. I know it is called a defense, but it is really just a chance for the committee to hear about the project one more time and celebrate with you have completed it. It is typically very collegial, with everyone trying to make it the best project possible. After your 20-minute talk, your committee will ask you questions. They are just trying to clarify any remaining issues, there may be some minor changes needed in your paper. Some very common questions that you may be asked at the end of the call: what are your plans now? Are you planning to follow up on your results in the future? Will you attend graduation? 

After the defense, finish all of your Form and Style changes. Any final changes that came out of your defense should be highlighted for the URR. It is a good idea to also have a cleaned copy – with all highlighting, comments, and track changes removed also sent to the URR. Your chair will do a double check of the paper and send it on to the URR who will confirm that the Form and Style changes were done. When the URR approves it, just the abstract is sent to the Chief Academic Officer (CAO), who double-checks it. Often minor changes will need to be made (this step tends to be very short – a couple of days). And then… you are done! 

We have done an overview of the entire dissertation process over the last few weeks. Next time, I will begin discussing some of the questions and issues about which students have asked me. Next time I will look at some alternative sources for articles. 

Do you have an issue or a question that you would like me to discuss in a future post? Send me an email with your ideas. leann.stadtlander@waldenu.edu

Chapter 5 Summary

 

You have completed your data analysis and are ready to try to make sense of it all. Let's return to the checklist and see what needs to be included: Introduction, Interpretation of Findings, Limitations, Recommendations, Implications, and Conclusion. Chapter 5 is dealing with the big picture, what do your results mean in light of previous research? What is new, what have you clarified? What still needs more research? 

For the interpretation section, you will want to go back to your research questions and see what your results add to our knowledge. Relate your study to previous literature and back to your theories; what findings are unexpected based on previous work? Why do you think they came out this way? Which findings are consistent with previous studies? This point is important; because it validates that your study is consistent with previous research. How does your study fit the theoretical models you discussed earlier in your paper? If it is not consistent, it is important that you discuss why this may be. 

The limitations section asks you to consider your sample and its similarities and differences from the population. Unless you carefully sampled thousands of people, your sample will not be representative of the population, which is fine- you just need to understand how it is differs. Consider racial and cultural differences between your sample and the general population. Other things to consider are that people who volunteer to be in a research study are probably different from those who do not volunteer, so that is a limitation to mention. Think carefully about your recruitment and research methods, how did you recruit? Who would not have been included? Did you use a computer survey? Then people without one would have been excluded. Again, all of this fine, it is just part of research, but it is important to understand the study's limitations. 

In the recommendations section you propose future research ideas based on your study. What needs to be examined further? What new areas have you discovered that need to be addressed? 

The implications section asks you to think about social change and your work. How could future researchers, practitioners, or the public use your results? The biggest problem students have in this area is that they overstate the implications. Be realistic, your study is not going to change the world as we know it. You are just adding one little piece to the puzzle. In the conclusion section, you need to wrap up the study by summarizing the key issues. 

One difficult issue is when you have non-significant results; your study does not come out as hypothesized. This requires you to carefully rethink all of the assumptions and the thought process that led you to make the hypotheses that you did. Could there have been other factors involved that you missed earlier? Consider your sample; were they a good representation of your population? Perhaps, the results might have been different if you had a larger sample. Take a look again at your data, were any of the findings going in the right direction, but did not reach significance? This is evidence that a larger sample may have been needed.  

Next time, we will take the final steps in the dissertation process and examine the approvals needed and the final defense.

Chapter 4 Analyses: Mixed Method

By definition, a mixed method study incorporates both quantitative and qualitative methods. There are several ways to analyze the data; I am going to discuss two. In the first method, the quantitative data is analyzed and the qualitative simply adds to an understanding of that data through quotes.  

In the second method, the data are analyzed together. Typically, the quantitative data is analyzed first, using more sophisticated analyses, such as multiple regression. Then the qualitative data is transformed into quantitative data (see Creswell and Plano Clark, 2011). To do this you will need to develop a scoring rubric for the qualitative themes and codes that you want to incorporate. Let's use the qualitative data from last time (see May 20 post), in which we had 3 sample responses. I developed coding as follows: 

101- grandmother, death, visual imagery, with mother, crying (negative emotion)

102- grandmother, visual imagery, auditory, empathy (recognizing other's emotions)

103- mom, eating, auditory imagery, feelings of comfort 

I need to decide what is important to include in my data analysis. I want to include any person mentioned; if a female relative, it will be designated as 1, a male relative will be 2, nonrelative female 3, etc. I am also interested in the use of imagery; if they mention visual images it will be 1, no visual will be 0. Auditory imagery will be 1, no auditory will be 0. Emotions in the self will be 1, and emotions mentioned in others will be 2. I would then enter this into SPSS along with the participants' scores on the quantitative measures that were used. The resulting SPSS entry would look something like this: 

Subnum	Survey1Tot	Survey2Tot	Person	Vismage	AudImage	Emotions
101	56	24	1	1	0	1
102	32	23	1	1	1	2
103	86	26	1	0	0	1

 

Then I will analyze the data using probably correlations and chi square to see if the scores on the surveys are related at all to the memories that the participants reported. Presumably, I would have some theoretical rationale for doing this. 

If you would like to see what a mixed method analysis looks like in a journal, please see:  

Stadtlander, L., Giles, M., Sickel, A., Brooks, E., Brown, C., Cormell, M., Ewing, L., Hart, D., Koons, D., Olson, C., Parker, P., Semenova, V., & Stoneking, S. (2013). Independent Living Oldest-Old and Their Primary Health Provider: A Mixed Method Examination of the Influence of Patient Personality Characteristics. Journal of Applied Gerontology. (Available in Walden Library's Psychology/ Sage database) 

Next time we will look at the final dissertation chapter – Chapter 5, Summary. 

 

Creswell, J. W. & Plano Clark, V.L. (2011) Designing and conducting mixed methods research, 3rd ed. Thousand Oaks, CA: Sage.

 

Chapter 4 Analyses: Qualitative

You have completed your interviews and maintained your field notes, the next step is to do the transcriptions. If you can do the transcriptions as you go, all the better! You should include every utterance of the interviewee, including umms, laughter, etc. Incorporate your field notes and observations into the transcription (e.g., subject head down, begins to cry). The transcription is the most time consuming aspect, in my opinion.  

You now have a ton of information from all of your interviews- what do you do with it? First, I suggest reading through all of your interviews several times. Think about them as you read them, what are the people saying? What are they NOT saying? Sometimes, what they avoid discussing is as interesting as what they do say. Try to put yourself in their place; you want to understand what their life is like. 

Once you have a good feel for the data, the next step is to begin the analysis. There are a number of software programs available, such as Nvivo and Qualrus, but I am going to recommend a low tech method that I generally use. In this method a matrix or table is used in word. Picture a table as shown below for each interview question. Each person’s response is copied into the table. You then carefully read each response and develop the coding for each one. As you do this, questions to ask yourself: What are the key points that are mentioned? What is the underlying emotions associated with the response? You will want to go through this process several times. The first time, use the participants' words in the codes. Then go through again and think about larger concepts that are present in different people's responses- can you find an overarching code?

 

Sub#	What is your earliest memory?	coding
101	I remember my grandmother being sick, she had cancer. She passed away. I remember crying. It was summer time. I remember seeing her in her bed, not moving, with her eyes closed. I was with my mom.
102	I remember my grandmother holding me in her arms with a polka dot cap. I remember her holding me in her arms, rocking me, trying to get me to go to sleep. The look in her eyes of happiness and content, of having me there as she rocked me to sleep with her polka dot night cap. I remember vaguely maybe the TV being on, maybe the news. I remember an air conditioner in the wall going.
103	I was sitting at a table eating a sandwich and my mom was singing a song to me. It was a song I enjoyed her singing to me – I enjoyed the song and it made me feel comforted. I was focused on my mom’s voice and hearing the story in the song.

 

What coding would you give for each of the examples above? Try it and then see if yours are similar to mine (at the end of this post). It is always a good idea to have someone else independently code the responses and compare yours to theirs. For any that you differ on, try to reach a compromise. You might see if anyone in your dissertation class is at similar place in his or her dissertation process and would be willing to trade coding checks. If not, check with your committee member- perhaps they know another student who might work. Note that if you go this route, the person should sign a confidentiality agreement which will need to be submitted to the IRB along with a change of procedure including them.

Once the coding is complete, the next step is to look for similarities across your interviews based up on the coding; these similarities are called themes. Were there any themes in the examples presented above (of course your data will have more people, 3 people do not make up a theme)? I have listed two possible ones at the end of this post. 

Once themes are determined, the next step is to begin writing up c. 4, data analysis. You will want to give your themes and provide examples of quotes illustrating them. Follow the qualitative checklist for how the chapter is to be arranged. 

Next time we will look at the mixed method analyses. 

My coding and themes

101- grandmother, death, visual imagery, with mother, crying (negative emotion)

102- grandmother, visual imagery, auditory, empathy (recognizing other's emotions)

103- mom, eating, auditory imagery, feelings of comfort

Themes: female relative, visual and auditory imagery

Chapter 4: Quanitative Analysis

A request. As we are nearing the completion of the first dissertation review in the blog, what else would you like me to address in future posts? Please send me an email with your ideas and questions! leann.stadtlander@waldenu.edu 

You have collected all of your quantitative data, entered it into SPSS (or downloaded it), and double checked your data entry. Great… now what?? It can feel intimidating to view your first real data set and know you need to figure out what to do with it. I like to start with getting an overall feel for what is going on by calculating means and frequencies. Let’s take a moment and review what to use when. 

If your variable is continuous, meaning there are no categories that you set up previous to the study you can calculate means and standard deviations. An example of a continuous variable would be where you asked people to enter their age today (e.g., 32, 56, etc.). Examples of categories would be gender: 1 = female, 2 = male or age: 1 = 20-30, 2 = 30-40, etc.). Hopefully, these terms are sounding familiar, if not go back to your stats book and review. A good reference for SPSS is

Pallant, J. (2010). SPSS survival guide, 4th ed. Berkshire England: McGraw Hill 

For categorical variables (e.g., gender, education level) you can do a frequency table and get a feel for how your data looks. Make sure you don’t have any data entry errors- they will show up as a weird number, e.g., you have gender coded as 1 & 2 and a 21 shows up in the frequency table. Go back to the original data and double check it.  

One issue you may need to consider is what to do with missing data, for example from people skipping questions. There are a number of ways to deal with this issue. Check with your methodologist to see what they prefer. They will probably ask you how many cells (individual data points) are missing, for which variables are they missing, and what is the largest number missing per individual, so be prepared for those questions. 

Once missing data issues are resolved, my usual next step is run correlations between my variables just to get a feel for what is going on. I then do scatterplots for any that show up as significant. Again, I am just trying to get a feel for the data. My old undergrad stats professor used to say that you need to “take a bath in your data.” I like that idea, you need to understand the relationships before getting immersed in the formal data analysis. 

You should have developed an analysis plan in your proposal, so now is the time to go to that. What happens if you just can’t figure things out? Contact your committee members and ask for help. As a committee member I sometimes have students send me their data set and I play with it a little, then I can talk them through issues.

Sometimes students decide to hire statistical consultants. Personally, I am not a fan of this. I prefer that the student figure out the stats with the help of his or her committee. The problem with a consultant is that you don’t really understand what they did and why. Even if they explain it, you really don’t have the level of understanding that you should. It misrepresents your knowledge level. People reading your dissertation will assume that you did the analyses and are capable of doing it (and perhaps teaching it!) again. If you must use a stats consultant, my advice is to rerun all of the analyses that they do, so you understand them too. 

Another aspect to consider is keeping track of all the analyses that you run and what they show. There are several ways to do this. You can simply save all of your SPSS outputs in a separate file on your computer (my least favorite, because then you have to reopen each to see it). Another way is to print out all of your data outputs and save them in a file or binder. My own favorite way to keep my data output is to copy it or rewrite it as I go into a word file. The advantage of this is that I am keeping everything together that is relevant (you will generate a lot of irrelevant info as you go). Do keep in mind that SPSS tables are not in APA format, so any that you want to use in your paper will need to rewritten.  

I also find it helpful to think through what I am finding with each analysis (even though this technically goes in c. 5, I find it helpful to think about it at the analysis stage). Let’s work through an example, I find that my variable education level is correlated with my dependent variable, emotional intelligence (EI) total score. So my first question is which education level has a higher EI score? I could do a scatterplot or could just calculate the means for each education level (use analyze/ descriptive stats/explore). I then find that people with a graduate degree have a higher emotional intelligence score than people with a high school diploma in the sample. Is that what previous research has found? What if this is not the relationship other researchers have reported? I need to consider why my sample may be different. I check what else is correlated with education- perhaps I find that for this sample, gender is highly related to emotional intelligence. Do another scatterplot between gender and education. Whoa- all of the graduate level participants are female. Could that be the cause of the education and emotional intelligence score correlation?  

Remember your methodologist can help with any issues you may have. There are also statistical consultants available for tricky issues, a faculty member can ask questions on your behalf. Next time, we will look at qualitative analyses.

Data Collection Problems and Resolutions

Things happen in the real world when you are conducting a study, as the researcher it is your responsibility to both recognize the issue and resolve it. I am going to talk today about some of the most common issues that may arise. 

Confidentiality breach. Imagine that you are doing a program evaluation. You will survey participants before the program begins and again after the program. In order to match each person’s pre and post program survey, you will have them write their names on each form. Somehow during the data collection a few surveys get left behind. This is a breach of confidentiality. Someone else could see the responses and know who wrote them. In a program setting, that may not seem a serious offense, but consider if it was one’s supervisor that saw a derogatory comment about themselves. 

What to do? First, as soon as the loss is realized, get the surveys back in your possession. Second, report the breach to your committee and the IRB. They may want you to notify the individuals involved, but let them make that decision. You also may want to consider not using names in studies, instead ask people to pick a number or phrase that they will enter on the pre survey and remember for the post survey. 

Questionable data. When you are collecting in-person data, you may have someone act strangely, making you suspect drugs or alcohol use. Another scenario is that someone hands in a “completed” survey in a few minutes, as compared to the 20 minutes others took.  

What to do? First, make sure you are safe in the case of strange behavior (I once had a participant break the chair in which he was sitting from rocking and bouncing the chair due to amphetamine use). If there are any concerns, thank them for coming and get them out the door quietly. Second, mark their data forms, with the behavior that concerns you. Later when you analyze data, check whether their data deviates from the norm. Discuss the issue with your methodologist and make a joint decision about whether to include the data in the analyses. 

Participant Withdrawal. Occasionally you may have participants indicate that they want to stop and do not want to continue in your study. This is most common with special populations, such as the elderly and ill.  

What to do? First, I suggest asking if a break might help (and if you can recognize the need for one early, all the better). Second, as frustrating as it may be, you should just thank them for coming and let them go. 

Incomplete data. Thus far I have been addressing in-person research concerns, but one that is very common with online data collection is participants skipping questions. Some standardized surveys are invalid if questions are skipped.  

What to do? In person, you can do a quick check to see if any data is missing. Online is trickier. Most survey programs (like survey monkey) allow the option for requiring answers to the questions. There are some positives and negatives to consider with this. The positive is that your data will be complete, with no missing responses. The negative is some people will quit the survey if they can’t skip responses.  

Inappropriate disclosure. In interviews, you may have someone tell you things that are inappropriate, such scenarios include child or elder abuse, and disclosing medical or education information. 

What to do? As soon as you realize where they are going, stop them. Say that this is information you should not be told and move on with the study. The exception, is if you are a mandated reporter (check the laws in your state, some states say that everyone is a mandated reporter for child or elder abuse), in this case you are ethically required to report the incident to the appropriate authorities. I suggest first consulting with your committee and the IRB. Your status as a mandated reporter must be disclosed in the consent form. Such disclosures are unusual, but you need to be prepared in case they do happen. 

Whenever you run into something unexpected, know that you can contact your methodologist and/ or the IRB for help. Next time, we will consider issues in quantitative data analyses.

 

Collecting Data

Finally you have all the approvals and get to collect your data! A few pointers for this phase. Make sure you follow the procedures that were listed in your IRB application exactly, this is also called your research protocol. The IRB does random audits of students’ data, so be sure you are scrupulous about maintaining your protocol. Any changes (even minor ones) must be approved by the IRB through the “Change of Procedure” form. If you make any major changes, also be sure you notify your committee member and URR, and get their okay on it. 

One detail that is rarely mentioned, is the need for organization of your data. Let’s consider a couple of different scenarios. If you are using any type of paper collection methods (surveys, questionnaires, etc.) keep everything in one place, such as a file folder or large envelope. Don’t throw away or destroy anything until your total project is complete (even if you enter the data into software), you may need to double check your originals and if you are audited by the IRB, you will need the originals. 

If you are downloading your data directly from an outside source, e.g., survey monkey, secondary data, be sure that you double check that the data makes sense. Sometimes things get messed up in the transfer. Save your data in a couple of places, just in case of computer issues. 

Always double check your data entry, it is worth having someone else check you. Your results are only as accurate as your data entry. 

One question that students doing qualitative often have is whether they can deviate from their IRB approved interview questions. You are allowed to ask follow up questions to clarify the participants’ responses, but you may not go into new topics. If you believe that you need an additional interview question(s), you must complete a “change of procedure” form with the IRB, you will also need to show how the new interview questions relate to your research questions. 

If you are interacting directly with your participants through interviews or other methods, keep in the back of your mind that these people are doing you a great favor by being in your study. You could not get your degree without them. So be accommodating to their schedule, be polite, and thank them for participating. You have an obligation to all future researchers to show your participants that involvement in research is important and enjoyable. 

As you wait for the data to come in, use this time to go back through your proposal and change everything related to your study from future tense (“the study will explore…”) to past tense (“the study explored…”). Do not rely on a search and replace, I guarantee you will have to later go back and fix errors. Instead, read through the paper and correct them. You can also save yourself time in the future by double checking the Dissertation Guidebook as to form and style issues. Make sure you have followed the rules of grammar and writing as closely as possible. 

Sometimes, things don’t go as planned during data collection. Next time we will explore data collection problems and their resolution.

IRB: Ethics Review

Once your proposal has been accepted, the next step is the Institutional Review Board review (ethics board). The first step is to go to the research center, find the IRB website, and download the IRB application. One of the first items listed is that you need to take a short ethics course online (it is free), I recommend the NIH one. When you are done with the course, save the certificate to your computer, you will need to submit it to the IRB. Double check the IRB website for any relevant information sheets about your population. 

What is the IRB looking for? They want to know that you have thought out every detail of your study. The most common error students make is not providing enough detail, or not being clear in the information provided. As you begin to complete the IRB application, think about each item and consider: will someone unfamiliar with your study understand what you have written? Keep in mind that the board typically does not read your proposal, they rely on the information you provide in your application. 

The IRB is most concerned with your interactions with your participants, so go into great detail in this area. Think through each aspect of your study, some examples: Who will be approaching possible participants? What will they say? What happens to participants at each step of the process? Go into excruciating detail. If you find yourself assuming things (e.g., “I’ll use a room at XX for the interview”) stop and rethink it- do you have permission to use the room? Will it be private? Can people easily get there and find it? Will using the building give the impression that the organization is sponsoring the study? An example, of the last question would be using a church, you would want to be clear that the church is not affiliated in any way with your study. 

Let’s work through a fake study and think through some of the issues in the research methods. Let’s say we want to look at car shopping decisions and the relationship with self-efficacy (self-confidence). The plan is to do a mixed method study, we’ll use a standardized, brief self-efficacy survey and briefly interview the customer (What was your decision at the dealership regarding buying a car? What led you to make the decision that you did?). We decide that we want to use a local Nissan dealership, so first we approach the general manager, John Smith, and see if there is some way to make this work. John is open to the dealership participating but does not want us to interfere with customers in their decision making process. We initially discuss having everyone that speaks with a salesperson complete the 5 question survey. A brief consent form will be included and ask whether a researcher can contact them by phone and asks for their name and phone number. We will then call each customer that agrees and ask our questions. 

What are issues we need to consider? First we need a letter from John Smith stating everything that his people will and won’t do, and any information that the dealership will provide to us (so let’s wait on getting the letter until we have finalized the details). Second, we want to make it very clear that the researchers are not affiliated in any way with the dealership, that the person is not required to participate, and no one will treat them differently regardless of their choice in participating. One issue the IRB is going to have is with the salesperson giving out the survey, which gives the appearance of the dealership sponsoring the study. Is there a way around this? Perhaps at the end of the salesperson interaction he or she could give them a flyer with a link to an online survey (and the consent form) on survey monkey? As a researcher we need to consider that this is going to lower our participant rate, but will make it more acceptable to the IRB. We will probably have to continue the study longer and will need to mention this issue in the limitation section of the paper. 

Let’s consider our study’s inclusion criteria. People may participate if they are 18 years and older and if they are able to read English, which is the language of the survey. We should ask their age at the beginning of the survey. The exclusion criteria are being under 18 and unable to read English.

What other demographics might be a factor in purchasing a car? Perhaps marital status, having children, sharing decision making, employment, income per year? We would want to check the literature, make a decision, and add these into the survey. How will we determine who is interviewed? We will need to preset a rule for this. It could be the first 10 people who agree to be interviewed or if we think marital status is important we could say the first 5 with a partner/ spouse and the first 5 single/ divorced people who volunteer will be selected. The important thing is to select the criteria in advance and stick to it. 

Here is our current method:

1.      Get permission letter from John Smith

2.      Prepare flyer for salespeople with a link in survey monkey

3.      Prepare survey monkey with a consent form, the survey, and ask if person can be contacted for an interview, and have them give their email or phone number.

4.      Have a brief meeting with salespeople explaining the study and that we would like them to give a flyer to every customer, whether they buy a car or not.

5.      Wait to see if we get participants

6.      Contact those willing to be interviewed

7.      Call for interviews, record interviews

8.      Transcribe interviews

9.      Analyze qualitative data, developing coding

10.   Download survey data from survey monkey

11.   Analyze survey results compared to coding

12.   Send results to participants and dealership 

After a month, we only have three participants! We need to rethink our method, some possible options: we could go to the dealership and hand out the flyer personally as the person is leaving the dealership, sometimes the personal contact will help. We could increase the number of dealerships, each will require that we go through the first 4 steps listed above. Whatever, our final decision we will need to complete a “Change of Procedure” form for the IRB (available on their web site).  

A few final thoughts on the IRB process, you may need to go through several revisions of your methods. Work through each of the IRB’s comments and think them through. If you can’t figure out a solution, ask for a conference call with the IRB and your chair and they will brainstorm with you. Again, always be professional in your interactions, they are not trying to delay your progress. They just want your study to be the best one possible that protects you and your participants.  

Next time we will take a look at data collection. 

Moving On: The URR and proposal defense

Once your committee has approved your proposal (Chapters 1-3), each of your committee members completes a rubric. Your chair will submit your proposal to turnitin and assuming it is acceptable, the proposal, rubrics, and turnitin report will all be sent to your University Research Reviewer (URR). The URR is a faculty member that is assigned by the research center, this person provides an independent review of the paper and double checks that all is well. Why this step? The problem is that your committee is primarily looking at specific aspects of your paper, the content expert is concentrating on the literature. The methods expert is concentrating on the research methods. Unfortunately, they are focused on the trees, and need someone else to see the forest. Does everything hang together? Are the individual parts consistent? You will probably need to make some additional revisions for the URR. 

Once the URR approves your paper, he or she will complete a rubric and submit it to the research center. You will then receive an email indicating that you may schedule your one-hour proposal defense conference call. You will need to select several possible days and times that will work for you and send those to your committee with your time zone (check with your chair first, he or she may like to follow a different procedure). Once there are a couple of times that work for everyone, your chair will submit a conference call request form to the research center. You will be notified of the time selected and how to call in. 

In advance of the call you will want to prepare a power point for your committee. You will be given 20 minutes to talk about your project, so keep the slides to less than 25. It is better to have just an outline on the slides, you can always write out everything you want to say and read it during your presentation. Concentrate on the methods and analyses with just enough literature to put it into context. Check with your chair how they like it done. Send your committee the power point prior to the defense call. 

On the day of the defense, have your computer on with your paper open for reference. Have any other papers available that you might need, as well as your power point. It is also a good idea to have some water available and make sure you have an environment where you won’t be distracted or disturbed. Once you call in, your committee will greet you and try to put you at ease. Your chair will leave the conference briefly to turn on the recorder, there will be announcement when it starts. Your chair will probably indicate the date and who is on the call for the recorder and then it will be your turn to go through the presentation. 

A few things to know, first this is not a confrontational situation. I know it is called a defense, but it is really just a chance for the committee to hear about the project one more time and catch anything that may have been missed. It is typically very collegial, with everyone trying to make it the best project possible. After your 20 minute talk, your committee will ask you questions. They are just trying to clarify any issues that might arise during the data collection. It is okay to say that you don’t know something (but say that you will check it out, find the answer, and report back). 

Take a few moments at this stage of the process to reflect on your progress, celebrate! 

After the call, your chair will submit a form to the research center indicating that you passed the defense. You will then receive a notice that your next step is the IRB (Institutional Review Board) this is the ethics review and takes place before you collect any data. We’ll take a look at the IRB process next time.