Benchmarking of final year engineering projects

Benchmarking of final year engineering projects

In countries that are signatories to the Washington Accord, the final year (often honours) projects in four-year engineering degree programs are considered the culmination of the students' undergraduate work and are used as one of the key sources of evidence of students' design ability. Not surprisingly there is a lot of variation in how these projects are conducted and assessed. What academia and
accrediting bodies need is evidence of fundamental discipline-specific technical knowledge as well as
professional skills. These competencies need to be demonstrated on an authentic realistic open-ended capstone project, providing useful feedback to the student and facilitating significant reflection. Also, this grading had to be carried out in a consistent, repeatable, and reliable manner.
PURPOSE
The aim of this research is to critically analyse the project assessment practice at an Engineering Faculty at a regional Australian university and assess this practice against the criteria above.
DESIGN/METHOD
Twelve mechanical engineering projects were selected from the 2011 cohort for analysis. Each of these projects was assessed independently by two senior academics of the discipline. A full rigorous assessment of the dissertation was made, with the assessment instrument completed and a final grade awarded. A senior member of the mechanical engineering discipline from another university assessed four of these projects. This assessment followed the USQ assessment process using the USQ assessment instrument so that the same elements were assessed and final grades could be compared. This provided cross-institutional benchmarking and feedback on the USQ process. Finally, an academic from another discipline at USQ, who was already familiar with the general assessment process, carried out full grading as well as order ranking of the 12 projects. This assessment did not focus on technical details, but was designed to provide valuable insights into the value of the assessment instrument. Each project had the original and several independently determined grades.
RESULTS
Use of the assessment instrument:
• Within the mechanical engineering discipline at USQ produced repeatable assessment grades.
• Within the mechanical engineering discipline across institutions produced acceptable results (within
half a grade of the formal assessment).
• Across discipline boundaries at USQ produced assessment grades that followed the general trend of the formal assessments but with potential variation of one grade about this trend.
CONCLUSIONS
The several phases of the moderation practice make this process robust and significant outliers which may be presented from a single assessor are identified early and addressed at that time. The assessment instrument provided repeatable assessment grades across independent assessors within the same discipline. External benchmarking did not indicate any large discrepancy in assessment, although greater clarity and exemplars may be required in the assessment instrument and the expected level of achievement in individual elements in the rubric and also the overall grade. This paper highlighted the need to use project assessors who are well versed in the technical discipline
area. Overall it can be concluded that the USQ holistic method of assessment of competencies and grading of the final year capstone project can be considered consistent, repeatable, and reliable.

© 2013 Australasian Association for Engineering Education. This publication is copyright. It may be reproduced in whole or in part for the purposes of study, research, or review, but is subject to the inclusion of an acknowledgment of the source. Outside of these uses, no part may be reproduced by any process without the written permission of the publisher.