Data Collection Problems and Resolutions

Things happen in the real world when you are conducting a study, as the researcher it is your responsibility to both recognize the issue and resolve it. I am going to talk today about some of the most common issues that may arise. 

Confidentiality breach. Imagine that you are doing a program evaluation. You will survey participants before the program begins and again after the program. In order to match each person’s pre and post program survey, you will have them write their names on each form. Somehow during the data collection a few surveys get left behind. This is a breach of confidentiality. Someone else could see the responses and know who wrote them. In a program setting, that may not seem a serious offense, but consider if it was one’s supervisor that saw a derogatory comment about themselves. 

What to do? First, as soon as the loss is realized, get the surveys back in your possession. Second, report the breach to your committee and the IRB. They may want you to notify the individuals involved, but let them make that decision. You also may want to consider not using names in studies, instead ask people to pick a number or phrase that they will enter on the pre survey and remember for the post survey. 

Questionable data. When you are collecting in-person data, you may have someone act strangely, making you suspect drugs or alcohol use. Another scenario is that someone hands in a “completed” survey in a few minutes, as compared to the 20 minutes others took.  

What to do? First, make sure you are safe in the case of strange behavior (I once had a participant break the chair in which he was sitting from rocking and bouncing the chair due to amphetamine use). If there are any concerns, thank them for coming and get them out the door quietly. Second, mark their data forms, with the behavior that concerns you. Later when you analyze data, check whether their data deviates from the norm. Discuss the issue with your methodologist and make a joint decision about whether to include the data in the analyses. 

Participant Withdrawal. Occasionally you may have participants indicate that they want to stop and do not want to continue in your study. This is most common with special populations, such as the elderly and ill.  

What to do? First, I suggest asking if a break might help (and if you can recognize the need for one early, all the better). Second, as frustrating as it may be, you should just thank them for coming and let them go. 

Incomplete data. Thus far I have been addressing in-person research concerns, but one that is very common with online data collection is participants skipping questions. Some standardized surveys are invalid if questions are skipped.  

What to do? In person, you can do a quick check to see if any data is missing. Online is trickier. Most survey programs (like survey monkey) allow the option for requiring answers to the questions. There are some positives and negatives to consider with this. The positive is that your data will be complete, with no missing responses. The negative is some people will quit the survey if they can’t skip responses.  

Inappropriate disclosure. In interviews, you may have someone tell you things that are inappropriate, such scenarios include child or elder abuse, and disclosing medical or education information. 

What to do? As soon as you realize where they are going, stop them. Say that this is information you should not be told and move on with the study. The exception, is if you are a mandated reporter (check the laws in your state, some states say that everyone is a mandated reporter for child or elder abuse), in this case you are ethically required to report the incident to the appropriate authorities. I suggest first consulting with your committee and the IRB. Your status as a mandated reporter must be disclosed in the consent form. Such disclosures are unusual, but you need to be prepared in case they do happen. 

Whenever you run into something unexpected, know that you can contact your methodologist and/ or the IRB for help. Next time, we will consider issues in quantitative data analyses.