Towards quality in multiple-choice assessment

Towards quality in multiple-choice assessment

To many researchers in the area of teaching and learning, the above title may be an oxymoron. However, the use of multiple-choice in assessment programs has been and continues to be a popular choice amongst academics in the tertiary sector. Anecdotal evidence suggests that, while academics using multiple-choice tests may check the overall associated mark distribution for a test, few check its psychometric properties (for example, test reliability) and fewer still check the psychometric properties of the items within the test. The building blocks of quality multiple-choice tests are quality multiple-choice items. The psychometric quality of an item can be determined in part through an assessment of its ability to discriminate adequately between students of high and low ability. In a pilot study of three Faculty of Business courses at the University of Southern Queensland, statistical tests associated with the application of Rasch models to multiple-choice test results were used to identify items that failed to discriminate between students of high and low ability. This article reports on the findings of this study, discusses the consequences of using items with low discrimination and examines simple precautions that academics can take in order to avoid using such items.