A reliable approach to psychological assessment using cognitive testing batteries

A reliable approach to psychological assessment using cognitive testing batteries

[Abstract]Cognitive tests are rarely used in isolation. Instead the collection of tests into batteries is common place in clinical assessment. Clinical batteries range from fixed collections of tests administered unchanged to each client, to batteries flexibly constructed according to a process of hypothesis testing whic varies between clients. Reviews of clinical practice indicate that clinicians predominantly employ a semi-flexibly constructed battery, comprised of a core group of measures with the addition of others drawn as needed from an available pool. While this accommodates for clinical concerns, the psychometric characteristics of such a battery tend to be unevaluated and clinicians draw inferences without reference to the resultant associated measurement error. This has been duly noted in the research literature which increasingly cites the need for psychometric evaluation at the battery level. The current investigation was undertaken to address this difficulty of clinical practice and aimed to develop a psychometrically and practically driven actuarial model with practicing clinicians could structure and analyse cognitive batteries with due reference to reliability, validity and clinical utility. To this end, a review of psychometric literature was undertaken to dermine theorical guidelines for the control and measurement of error at the individual test and battery level. Reviews indicated that to successfully accommodate for the impact of random measurement error, clinicians must apply reliability theory to evaluation of the error associated with domain-based combinations of tests. Additionally, to ensure the validity of test-based inferences and avoid errors in decision-making, clinicians must apply empirically vaidated structures of cognitive function to guide test selection and combination. Given the pressing necessity of battery flexibility, it was concluded that clinicians could best accommodate psychometrid and clinical factors by the use of flexibly constructed composite scores. A reliable approach to psychological testing (RAPT) was proposed which applied psychometric theory and clinical factors to the development of a robust battery structure. The RAPT method focussed on the use of composite scores of domain-specific tests, grouped according to empirically validated domains and moderated by direct estimation of composite reliability. The RAPT was developed with the aim of facilitating the application of psychometric, actuarial methodology to a flexible collection of cognitive tests. In explicating the RAPT model, fifteen primary algorithms were derived from psychometric literature and outlined according to 3 stages of battery usage: test selection; test analysis; and, test interpretation. The utility of the RAPT was examined in terms of its capacity to improve psychometric robustness within a flexible battery. Specifically, using simulated demonstration, RAPT was demonstrated to provide a means of formalising empirically validated structure within a battery of tests, of controlling and improving the reliability of domain-based composite scores, of reducing the impact of artifactual errors on domain-based inferences and of applying actuarial methods typically associated with fixed batteries to a flexible collection of measures. Following this, RAPT was demonstrated to replicate existing psychometrically valid and stable interpretative structures. Specifically, RAPT algorithms were used to re-create the normative information provided for the Wide Ranbe Achievement Test, Fourth Edition (WRAT-4)Reading Compoiste. Norms calculated using RAPT were compared with those provided in the WRAT-4 interpretative manual with minimal differences found. RAPT algorithms were then used to re-create normative and ipsative tables, summary scores intercorrelations, and reliability coefficients for the Wechsler Adult Intelligence Scale, Third Edition (WAIS-III) summary scores: Full Scale IQ (FSIQ); Verbal IQ (VIQ); Performance IQ (PIQ); Verbal Comprehensive Index (VCI); Perceptual Organisation Index (POI); Processing Speed Index (PSI); and, Working Memory Index (WMI). Again, RAPT, was demonstrated to successfully replicate these data. Finally, the clinical utility of RAPT was demonstrated with the illustration of case examples which outlined the capacity of RAPT to apply psychometrically sound methodology to the tasks of modifying existing composites, modifying existing battery structure and developing battery structure and analyses according to a factor analysis investigation of an Australian normative sample (n=1045). These investigations provided comprehensive evidence of the capacity of the RAPT to enable the direct application of psychometric theory to semi-flexible battery construction in a manner suitable for use in current clinical practice.