Extreme Scores: Effects and Causes

Extreme scores can cause serious problems for statistical analyses. They generally increase error variance and reduce the power of statistical tests by altering the skew (making the data "lean" in one direction, rather than clustered in the middle) or kurtosis (when the distribution is squashed down or has a very high narrow peak) of a variable . This can be a problem with multivariate analyses. The more error variance in your analyses, the less likely you are to find a statistically significant result when you should find one (increasing the probability of a Type II error).

 

Extreme scores also bias estimates such as the mean and SD. Since extreme scores bias your results, you may be more likely to draw incorrect conclusions, and your results will not be replicable and generalizable.

 

Extreme scores can result from a number of factors. It is possible that the extreme score is correct- an example is although the average American male is around 5' 10" there are males that are 7' tall and some that are 4 foot tall. These are legitimate scores even though they are extreme.

 

Another cause of an extreme score is through data entry error, someone that was actually 5' 6" tall may be incorrectly entered as 6' 5". So the first step is to always double check that extreme scores were entered correctly. A third cause may be that participants purposefully report incorrect scores. It can also happen that a participant accidently reports an incorrect score. Thus, an extreme score that has been entered incorrectly may need to be removed.

 

The info in today's post comes from Osborne (2013).

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage.

 

Next time I will post an updated blog index. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Making Data Make Sense- Extreme Scores

What are extreme scores? They are scores far outside the norm for a variable or population, leading to the conclusion that they are not part of your true population and probably do not belong in your analyses. A common operationalizing definition for extreme scores is +/-3 standard deviations (SDs) from the mean.

Recall that standard normal distribution of a population has 68.26% of the population between +1 and -1 SD of the mean (see attached diagram: 34.13% between 0 to +1 SD +  34.13% between 0 and -1 SD = 68.26%).

So 95.44% of the population should fall between 2 SD from the mean (34.13% + 34.13% + 13.59% +13.59% = 95.44%), and 99.74% of the population should fall 3 SD of the mean. In other words, the probability of randomly sampling an individual more than 3 SD from the mean in a normally distributed is 0.26%, which gives good justification for considering scores outside 3 SD as suspect. Our concern is that these scores are not part of the population of interest in your study, but instead are different in some key way.


Next time we will consider how extreme scores affect statistical analyses. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Missing Data as a Variable/ Best Practices

You may wish to examine missing data an outcome itself, as there may be information in the missingness. The act of failing to respond vs. responding might be of interest. This can be examined through a "dummy variable," representing whether a person has missing data or not on a particular variable. You can then do some analyses to see if there any relationship that develop.

Osborne (2013) provides some best practices in dealing with missing data that are great to remember.

• First, do no harm.be careful in your  methodology to minimize  missing data. 
• Be transparent. Report any incidence of missing data (rates by variable, and reason for missing data if known). This can be important information for readers. 
• Explicitly discuss whether data are missing at random (i.e., if there are differences between individuals with complete and incomplete data).  
• Discuss how you, as the researcher, dealt with issue of incomplete data. 

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage.

Next time we will consider outliers or extreme scores. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Missing Data

Today we will take a look at methods of dealing with missing data. SPSS offers pairwise exclusion, which means only those cases with complete data are included in the analysis. It the missing data are few and a result of randomness, then such a plan may be acceptable. However, if they are not randomly missing, you could introduce biases.

A second commonly used method is substituting the overall sample's mean for the missing data. The logic of this is that in absence of any other information, the sample's mean is the best representation of an individual's score. If only a few scores are missing, then this may be an acceptable alternative. However, keep in mind that the more scores that are replaced, the more you are biasing the sample to the mean.

A third alternative is given by Osborne (2000, 2013) in which a prediction equation is developed through multiple regression. If you have quite a few missing scores, you may want to explore this alternative.

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage.

Osborn, J. W. (2000). Prediction in multiple regression. Practical Assessment, Research, & Evaluation, 7(2).

Next time we will consider missing data as a variable and best practices. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Categories of Missing Data

There are two categories of missing data, data that are missing at random (MAR) and data that are missing not at random (MNAR). If data are missing randomly, we can assume that they will not bias not the results. However, data missing not at random may be a strong biasing influence.

Let's use an example from Osborn (2013), of an employee satisfaction survey give to school teachers. The teachers are surveyed twice- once in September and once in June. Missing at random data would mean that data that were missing in June had no relationship to any variable from the September survey (such as satisfaction in Sept., age, years of teaching). An example, might be if we randomly selected 50% of the people who responded in September to again complete the survey in June- we would legitimately be missing half of the data in June (the 50% of people we did not ask). The missing data would be random and not related to a specific variable such as satisfaction, age, years teaching).

On the other hand, suppose only teachers that were satisfied responded to the survey in June (people who were dissatisfied were less likely to respond to the survey). Then the missing data are considered missing not at random (MNAR) and may substantially bias the results. Thus, the June survey would show a higher than expected satisfaction score (because unsatisfied people did not participate).

Next time we will consider how do deal with the missing data. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage.

Making Data Make Sense- missing data

In almost any research study, there will be missing or incomplete data. Missing data can happen for a number of reasons: participants fail to respond to questions, subjects withdraw (or quit) studies before they are completed, and data entry errors.

The problem with missing data is that nearly all statistical techniques assume or require complete data. There can be legitimately missing data; an example might be a survey in which one is asked if he or she married, and if so how long. If you are not married, than you would be correct in leaving the "how long" portion of the question blank.

It is also important to realize that legitimately missing data can be meaningful. The missing data allows a validity check and may inform the status of an individual. Osborn (2013) proves a great example. In cleaning the data from an adolescent health risk survey, he noticed that some individuals indicated on one question that they had never used illegal drugs, but later in the survey when asked how many times they used marijuana, indicated an answer greater than 0. Therefore, an answer they should have skipped (or missing), showed an unexpected number. The author suggests several possible explanations, such as the subject was not paying attention and answered in error. However, a more intriguing possibility is that some subjects did not view marijuana as an illegal drug, which is an interesting possibility that could be examined in future search.

One way of dealing with legitimately missing data is making the missing and present data two separate groups. Using the marriage survey example, we could eliminate non-married individuals from a specific analysis when looking at issues related to being married vs. not married. So instead of asking the silly research question- "How long, on average, do all people, even unmarried people, stay married- we can ask two more refined questions: "What are the predictors of whether someone is currently married?" and Of those who are currently married, how long on average have they been married?

Next time we will consider categories of missing data. Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage.

Making Data Make Sense- data cleaning

What do I mean by data cleaning? There are many definitions, but I am talking about a two-step process. First, double-checking that all cells are filled, you will probably discover some are not and decisions will need to be on this.  The second step is carefully checking the statistical assumptions of your variables and looking for extreme scores.  

Why are these steps necessary? Because the results of your study will only be as accurate as the data you analyze. Therefore, it is very important to take the time to check your data carefully, so that you know that your results are valid and accurate.  

I want to refer you to a great book that much of my advice over the next few posts will be based:  

Osborn, J. W. (2013). Best practices in data cleaning. DC: Sage. 

Next time we will look at the issue of missing data.  Do you have an issue or a question that you would like me to discuss in a future post? Would you like to be a guest writer? Send me your ideas! leann.stadtlander@waldenu.edu