An investigation of skewness, sample size, and test standardisation

An investigation of skewness, sample size, and test standardisation

In psychological assessment, a raw score transformation is the first step in the clinical decision-making process. During this process, clinicians will transform a raw score typically using linear standardised scores and a normative sample with characteristics similar to their client. While the literature stresses the importance of using adequate normative data, little research has evaluated the effect skewness has on sample size. Currently the consensus is that a sample size of 50 is deemed adequate for normative data. However, an alarming number of studies that present normative data have much smaller sample sizes particularly when the data is stratified by age, gender, and/or education. Additionally, the use of linear transformations onto a normal distribution introduces further problems the more positively or negatively skewed the normative raw score distribution is. Skewed distributions are commonly encountered in neuropsychology and accordingly their deviation from a normal distribution should be considered during the clinical decision-making process. The primary goal of the current thesis was therefore to evaluate the psychometric issues related to the standardisation process. In particular to investigate the current understanding of sample sizes in neuropsychological samples, assess how this is influenced by different skewed distributions, and evaluate the potential errors involved in the decision-making process. Three studies were conducted. The first study explored the minimum sample size needed to produce stable measures of central tendency and variance for a range of distributions. Results indicated that the optimal sample size required was dependent on the level of skewness of the distribution and was not the often cited N = 50. For normally distributed data, a sample size of 70 is required in each cell in order to produce stable means and standard deviations. Negatively or positively skewed distributions required sample sizes that ranged from 30 to 80 in each cell. This study highlighted the inadequacy of currently available normative data and called for further normative research to be conducted. The second study evaluated the errors introduced when using three different linear transformations on different skewed distributions with adequate sample sizes. Seven tests with differing skewness coefficients were evaluated using the z score transformation, a t-test method developed by Crawford and Howell (1998), and a median z score transformation developed for this research. Results indicated that the traditional z score transformation produced the least errors of the three methods. However, for highly positively skewed distributions, the use of this transformation introduces considerable error in the clinical decision-making process. A regression equation was derived as a tool for clinicians to help correct adequate data for the effect of skewness. The final study evaluated whether using different linear transformations created substantial errors when using normative data that ranged in skewness and that had sample sizes less than those recommended from Study One. This study is particularly important given the common practice in neuropsychology for at least some measures to be derived from the clinical research literature utilising inadequate sample sizes. Results indicated that the error in judgement when using the preferred z score transformation is nearly doubled in positively skewed distributions. It was recommended that normative data with sample sizes less than 30 should not be used in clinical practice and guidelines were proposed for incorporating issues of sample size and skewness into their testing practices. It is hoped that clinicians will adopt the findings and subsequent recommendations of these studies in order to improve the current standards of clinical decision-making in neuropsychology.