Assessment in First Year University: A Model to Manage TransitionJanet A. TaylorUniversity of Southern Queensland
IntroductionAdjusting to studying at university can be a challenge for students whether they are studying on-campus or by distance education (Lawrence, 2005, McInnis, 2001; Byrne & Flood, 2005); in Australia one in four commencing undergraduate students will not persist with their studies (Krause, Hartley, James & McInnis, 2005). The attrition of students studying at a distance can be even higher, with Simpson (2004) reporting in a UK university that 38% of distance students had withdrawn before submitting their first assignment. The factors contributing to such figures are complex and revolve around personal and contextual factors (see review by Pascarella and Terenzini, 2005). Lawrence (2002, 2005) in her framework for successful transition to university believes that three major groups of factors are pivotal to success in the first year of undergraduate studies:
Lawrence's views are supported by the work of Clegg, Bradley and Smith (2006) on help seeking behaviours; and Wingate (2006) on learning and study skills. Universities address these issues of transition in various ways, but most commonly through orientation programs which incorporate learning skill development. But is this enough? Hattie, Briggs and Purdie (1996) assert that the evidence suggests that learning skills are most effectively developed within a specific context rather than as generic initiatives. Wingate (2006) further argues that 'bolt-on' practices for study skills are 'remedial, not inclusive and divorced from subject knowledge'. In response to this research, many universities are now turning to embedding transition skills in curriculum design (Cluett and Skene, 2006). The question is what role should assessment play to support transition to first year? It is generally believed and widely stated that assessment drives the student academic experience and hence student learning: For most students, assessment requirement literally defines the curriculum. Assessment is a potent strategic tool for educators with which to spell out the learning that will be rewarded and to guide students into effective approaches to study. (James, McInnis & Devlin, 2002, p 7). The central location of assessment within students' perceptions of learning and studying means that it is a powerful tool that can assist their transition to university studies and thence their performance and willingness to persist. The core of any assessment plan involves three principles: one associated with development and learning; one associated with measurement of outcomes (validity and reliability); and one associated with academic standards (James, McInnis & Devlin, 2002). Yet today issues associated with measurement and standards are more commonly addressed at the expense of the need to develop and engage student learning (Gibbs, 2003). Yorke (2003) is concerned about the proliferation of end-of-unit summative assessments and argues that the theory and practice of formative assessment are poorly understood in higher education. In 2002 Yorke reported a move by some universities to redesign their first year of study to include only formative assessment. However, this does not match the above suggestion that assessment drives student learning. Unsworth & Kauter (2008) in a trial of a formative early bird scheme, found that although it was seen by students to be useful, few took it up. Trotter (2006) found that students welcome summative assessment as an incentive and motivator to study and with Roberts found that courses with higher retention rates had utilised formative assessment that also counted towards a final grade (Trotter & Roberts, 2006). So although formative assessment is well supported in principle as good practice (Nicol & Macfarlane-Dick, 2006), students will not necessarily value and thence undertake it unless it is worth something more concrete in their eyes. The issue of balance between formative and summative assessment has been taken up by Hounsell, Xu & Tai (2007) in the suite of resources associated with integrative assessment. The authors indicate that striking a balance between assessment of learning (summative) and assessment 'for learning' is 'especially tricky because what aids the former may be deleterious to the latter, and vice versa' (p.1.). To add to the complexity are the tensions between desired assessment practice, university policies for assessment, and cost and time effectiveness. In some institutions course designers are rigidly tied to a few summative assessments or are using multiple-choice assessments to save time and money (Nicol, 2007). In times of increasing student numbers and diversity in first year courses, reduction in the number of assessment pieces or marking times are seen as ways to cut workload and costs. Within this climate this paper proposes a model for effective assessment in first year university, positioned within research findings on assessment in higher education and transition to university. The model has been synthesised from a range of effective practices offered in diverse first year courses offered at the University of Southern Queensland, a regional multi-modal Australian university, in which eighty two percent of its 26 000 students study by distance education. Model of assessment for students in the first year of studiesThe literature reviewed above underpins the development of the model and has lead to the consideration of six notions that should be considered when designing assessment in first year undergraduate courses. These include:
The model (Figure 1) has been synthesised from the practice of the author in a large first year mathematics course and from colleagues within engineering, surveying, nursing, communication and computing. All courses are core courses within their relevant programs of study, are offered in the first semester of first year and enrol large numbers of students, usually by both distance and on-campus education. The model for assessment proposed divides the semester into three overlapping assessment phases.
The model also allows for continuous assessment to regularly engage students and/or allow them to monitor their understanding and progress. Assessments for transitionFirst year undergraduate students are novices to university study. To successfully negotiate their first semester they need to encompass a wide range of literacies and competencies, but the first task is for them to engage with the course and thence manage themselves throughout its progress. Traditionally, on-campus students can be engaged easily through classes, but for the growing numbers of students who no longer attend lectures (Dolnicar, 2005) and for distance students, engagement can be slow. These early assessments can take a number of forms, however to be valued by the student they should contribute a small percentage to the final grade. In this sense they are both summative and formative. Assessments for transition invariably encourage students to look both backwards and forwards, by reflecting on past performance or behaviours, or by preparing a study plan for the semester. Often they will involve a pre-test or self-audit to refresh prerequisite skills, or a survey to assist students in understanding their learning skills. In some instances, they could be contracts to awaken students to specific needs of a course e.g. regular internet access, compulsory online discussions or to question students' understanding of what is required to complete a course. The early assessment should require very little marking time, but allow the staff to identify students who are uncertain or have poor skills or attitudes to learning. More importantly it allows the tutor to commence a working relationship with the student which encourages students to ask for help when needed. 1. MathematicsFoundation Mathematics (MAT1100) is a large first year service mathematics course which enrols 800 students (500 distance) studying science, engineering, surveying, and computing. The students in the course demonstrate a huge diversity of mathematical backgrounds and attitudes to studying mathematics. The first assignment asks students to reflect on their past mathematical experiences, to confirm vital information about how the course operates and to develop a study plan for the course. It is compulsory and is completed in week two, with a flexible submission time to allow for late enrolments. This assignment ensures that students do not procrastinate, while answers to reflective questions confirm (or otherwise) to the tutors that students have the skills and knowledge necessary for transition to the course, and allows for follow-up of students with concerns. In a course evaluation students had mixed feelings about this assessment and were often surprised by its reflective and discursive nature, especially in a mathematics subject. Yet 8 weeks into the semester, 69% of students indicated they were using their study plan to assist with their study requirements. One student indicated: Making us do a study plan. I thought it a bit stupid and irrelevant at first but [it] was in fact the most useful and helpful thing for maintaining the workload evenly throughout the semester The year in which this assignment was introduced saw an increase in students' completion of the next assessment activity and an increase of 10% in the overall pass rate for the course. 2. CommunicationCommunication and Scholarship (CMS1000) is offered on-campus, externally and internationally to over 500 students. The course aims for its students to engage, master and demonstrate key literacies, including students' learning and critical capabilities, academic and tertiary discourses, oral presentation skills, information literacies, research methodologies, communication and cultural awareness literacies. The assessment scheme is detailed by Kiernan, Lawrence and Sankey (2006). The early assessment is the Preliminary Essay Plan (PEP) is completed in week three. The assignment appears simple but is underpinned by important academic skills (planning, developing arguments, structure, referencing, and tone of writing). The weighting of the assignment is low (10%) but its importance is highlighted because of its link with a later essay (35%). There are two parts to the assignment:
Evaluations undertaken by Kiernan, Lawrence and Sankey (2006) indicate that this assignment has achieved change within their course. The authors argue: The PEP is a non-threatening assignment, because of the low weighting, but it has significance because of its application to a major assignment and because it reduces students' anxiety and provides them with an early gauge of their progress. At the same time it equips them with the skills and literacies they need to persist at university. 3. Engineering and SurveyingProblem Solving 1 (ENG1101) is studied by all engineering and spatial science students and is the first of a suite of problem solving courses students undertake. In the course all students (both on campus and distance) work in teams of eight and prepare nine assessment tasks (reports, portfolio and online postings) with group and individual components (Gibbings & Brodie, 2006a). The first suite of assessments, due in week 3, builds individual portfolio and team report writing skills. In the portfolio component students complete a skills audit (Gibbings & Brodie, 2006b), tasks associated with professional attributes and characteristics of teamwork. The report component asks student teams to formulate a team goal; a code of conduct and responsibility; guidelines for peer assessment; team meeting plans. These early assessments (20% of final grade) are set within the context of an engineering problem and aim to:
Considerable feedback is provided by team facilitators before and after these first tasks, in preparation for later more heavily weighted individual and team assessments. Gibbings and Brodie (2006a) report that grades and participation have improved with the introduction of the assessment scheme with its early assessments. One student says:
4. ComputingFoundation Computing (CSC1402) enrols 1000 students (650 distance) from diverse disciplines. Students submit eight assignments, seven of which are peer reviewed (de Raadt, Toleman & Watson, 2005). The first assignment is submitted week three and is the only assignment which is not peer reviewed. This assignment focuses on students' computer software, previous computing experiences and expectations of the course. They are asked to answer a series of questions on a bulletin board and in an email attachment. Questions include:
CSC1402 Course Home Page (2007) Although the large number of students in the course precludes tutors answering questions directly, the assignment allows students to reflect on what is required for the course and to see other students' experiences. By giving tutors specific knowledge of students' computing systems and software it speeds tutors' responses when students have technical difficulties. Assessments for developmentAssignments in this group are the core of any first year course. Once engagement is established by the early assessments, the task of the middle assessments is to maintain the engagement and develop and confirm students' skills and knowledge. These assignments aim to develop skills necessary for later success and have strong links with assignments designated 'assessment for achievement'; feeding forward into these assignments. To achieve this, these assignments should have significant marking time dedicated to provision of timely feedback. The closer to the commencement of the semester these assignments occur then the lower their contribution to the final grade. But in all cases the resources allocated to marking should be relatively high. The assignment(s) could take variety of forms: a draft for a later assignment, a reflective reading log, components of a portfolio, laboratory reports or online discussion group submissions. 1. MathematicsIn Foundation Mathematics, an assignment is submitted around week seven and assesses problem solving and mathematical communication skills. It is a mathematically simple assignment containing at most three mathematical concepts. Prior to this assignment students have practiced these skills in compulsory online discussions and on-campus workshops (Taylor & McDonald, 2007). At this stage students are also asked to reflect on the current progress and performance, linking this assignment with self-regularity behaviours set up in the early transitional assignment. This assignment (10%) prepares students for a later problem solving assignment in which more complex mathematical concepts are examined (20%). Tutors are allocated a large proportion of their making time to grade this assignment and participate in pre-marking workshops to ensure that useful and consistent feedback is provided to all students. 2. CommunicationWithin the Communication and Scholarship course, the Preliminary Essay Plan (PEP) described previously not only serves as an assignment for transition but also allows significant feedback to be given on course content and processes leading to assignment 3 (35% of final grade). To help students understand these concepts the course designers supplemented the item with a number of formative multimedia activities that provide key information in a variety of modes – textual (written), visual and aural (auditory). Students have responded to this multimodal approach in a very positive way (Sankey & Kiernan 2006). Overall this assignment in combination with the multimedia formative activities is said to have
(Sankey & Kiernan, 2006) 3. NursingCommunication Concepts in Nursing (CMS1007) is a first year on-campus course that operates in collaboration with two partner courses (Psychosocial Foundations of Nursing and Introductory Nursing). This assignment, conducted around week three, asks students to select a research article, based on research topic in partner courses, and using library data bases; summarise and critically evaluate it. It includes a reflection on their own learning practices. Significant feedback both in writing and in person is given to students. Lawrence (2006) reports comments from a variety of students indicating the success of this assignment in preparing them for the assessments in their partner courses as well as for the later essay in the parent course. Assessments for achievementThis type of assessment is best known in higher education. They include examinations, as well as major essays, final portfolios, reports or projects. In most instances, this type of assessment occurs towards the end of the course, usually with a relatively high weighting. Examinations provide very few opportunities for developmental feedback, with students only receiving feedback in terms of their final mark. The marking time of examinations will thus be relatively smaller than assignments which require extensive feedback. The assumption behind major assessments other than examinations is that they will however include significant feedback. But in reality the lateness of these assignments in a semester usually precludes the receipt of any feedback that could be meaningful within the course. For distance education courses operating through mail system the return of such assignments will usually occur after the end of the semester. It is essential then for specific links to be made between assessments for development and assessments for achievement, so that students have already received feedback, often called 'feed forward', on their skills and knowledge prior to attempting the final assessment tasks. Continuous self-assessmentIn most courses it is assumed that students will perform better if they engage and then maintain their engagement by regularly completing course activities. It is also desirable that students are given the tools to enable them to monitor the quality of their own work. Of course it would not be possible for tutors to be directly involved in such assessments, especially in large first year courses. With the advent of new technologies the implementation and maintenance of useful continuous assessments which includes timely feedback are now possible. Such assessments could be either formative alone, or simultaneously formative and summative. In the latter case depending of the objectives of the assessment the percentage contributed to the final grade may be very small. 1. MathematicsIn Foundation Mathematics to ensure that students continuously participate in a course, computer managed quizzes are incorporated in the course from week four. Students have a series of short quizzes associated with each mathematics module. They have up to four alternative quizzes to choose from, and can repeat quizzes if their performance is not considered high enough. These quizzes contribute 6% to the final grade. Feedback is provided instantly. This mastery approach allows students to achieve 100% on all quizzes if they so choose. Students, especially distance education students appreciate this type of quiz in which feedback is instant. 2. ComputingIn Foundation Computing, de Raadt, Toleman and Watson (2005) have revolutionised assignment submission and marking procedures to transfer time away from marking to teaching. All assignments are submitted online with seven of them marked using a peer marking system with instructor moderation. These assignments are simultaneously developmental and summative and contribute 56% of the final grade. The authors indicate that with electronic peer-reviewed feedback is almost instantaneous and has changed the course participation levels with students benefiting by:
de Raadt, Toleman & Watson (2005) ConclusionsThe increasing prevalence of assessment practices in which the number and nature of assessment pieces are restricted or regulated is a disturbing trend. It may be particularly detrimental to first year undergraduate students battling with a new culture of learning and studying. The model presented here is a way forward for designers of first year courses. Its three assessment overlapping phases: assessment for transition, assessment for development and assessment for achievement, encompass six notions about first year assessment gleaned from the literature and clearly addresses the needs of first year students as they adjust to new teaching methods, independent learning and the vagaries of managing study, work and life. It presents a way that is pedagogically sound focusing strongly on the importance of transition and development to first year learners. The potential for the 'assessments for achievement' to dominate is addressed by the strong linkages between assessments for development and those for achievement, which occur late in the course. Simultaneously it moves away from practices in which resources and marking times are distributed evenly between assessments to one which front loads resources to the first half of the semester to allow for increased feedback at times when it will be most effective. First Year courses are often characterised by large numbers of students. On-campus these students are more likely to be young or straight from school, while distance education students are more likely to be mature and involved in significant amounts of paid work. In both cases they have difficulty managing their time and commencing their studies. On-campus students have the advantage of regular engagement through classes, but for distance education students this early engagement is more elusive and often only commences when an assessment task is due. In most of the examples provided from USQ, the needs of distance education students have driven the redesign of assessment schemes in which a priority is placed on an early assessment in an attempt to reduce high levels of drop out (Simpson, 2004). Some may argue that such assessments do not recognize students' maturity and learning skills. Yet as universities are challenged by increasing diversity of students, many of whom claim to be unprepared for university study (Krause, Hartley, James & McInnis, 2005), new strategies must be explored. This model presents such a strategy incorporating issues Lawrence (2002, 2005) raised in her model of transition to university alongside Gibbs (2003) steps of effective assessment practice. The pressure of resources, fears associated with quality and fears induced by plagiarism have meant that assessment change as a strategy to improve first year experience has been largely unexplored. This paper commences the exploration. AcknowledgementsI would like to acknowledge my colleagues at the University of Southern Queensland who kindly allowed me to include their work in this paper: Christine McDonald; Jill Lawrence; Eleanor Kiernan, Michael de Raadt, Lyn Brodie and Peter Gibbings. ReferencesByrne, M. & Flood, B. (2005). A study of accounting students' motives, expectations and preparedness for higher education. Journal of Further and Higher Education, 29(2), 111-124. Clegg, S.; Bradley, S. & Smith, K. (2006). 'I've had to swallow my pride': help seeking and self esteem. Higher Education Research & Development; 25(2), 101-113. Cluett, L. & Skene, J. (2006). More than the sum of its parts: Provision of Learning and Academic Skills services through a centralised model. University of Western Australia Teaching and Learning Forum. Available online at:, http://www.lsn.curtin.edu.au/tlf/tlf2006/refereed/cluett.html (accessed 14 January 2008). CSC1402 Course home pages. (2007). Available online at: http://www.sci.usq.edu.au/courses/csc1402/ (accessed 18 January 2008). de Raadt, M.; Toleman, M. & Watson, R. (2005). Electronic per review: A large cohort teaching themselves? Proceedings of the 22nd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE2005). Brisbane, 4-7 December, 2005. Dolnicar, S. (2005). Should we still lecture or just post examination questions on the web?: the nature of the shift towards pragmatism in undergraduate lecture attendance. Quality in Higher Education, 11(2), 103-115. Gibbings, P. & Brodie, L. (2006a). An assessment strategy for a first year engineering problem solving course. In:17th Annual Conference of the Australasian Association for Engineering Education, School of Engineering, Auckland University of Technology, Auckland, New Zealand. Available online at: http://eprints.usq.edu.au/archive/00001629/ (accessed 18 January 2008) Gibbings, P.D. & Brodie, L.M. (2006b). Skills audit and competency assessment for engineering problem solving courses. Paper presented at the International Conference on Engineering Education, Liverpool, England. Available online at: http://eprints.usq.edu.au/2252/ (accessed 18 January 2008) Gibbs, G. (2003). Implementing learning and teaching strategies. National Coordination Team Teaching Quality Enhancement Fund. Open University Press. Available online at: http://www.headacademy.ac.uk/documents/ilts.pdf (accessed 10 March 2006) Hattie, J.; Biggs, J. & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66(2), 99-136 Hounsell, D.; Xu, R. & Tai, C. (2007). Integrative assessment: Balancing assessment of and assessment for learning (Guide no 2). The Quality Assurance Agency for Higher Education (UK). Available online at: http://www.enhancementthemes.ac.uk/documents/IntegrativeAssessment/IABalancing.pdf (accessed April 2008). James, R.; McInnis, C. & Devlin, M. (2002). Assessing learning in Australian universities. Centre for Higher Education Studiers and Australian University Teaching Committee, Australian. Available online at: http://www.cshe.unimelb.edu.au/assessinglearning/ (accessed March 2006). Kiernan, E.; Lawrence, J. & Sankey, M.D. (2006). Preliminary essay plans: assisting students to engage academic literacy in a first year communication course. In: Proceedings of the 9th Pacific Rim in Higher Education Conference: Engaging students. Queensland University of Technology, Brisbane. Available online at: http://eprints.usq.edu.au/1145/ (accessed 18 January 2008) Krause, K.; Hartley, R.; James, R. & McInnis, C. (2005). The first year experience in Australian universities: Findings from a decade of national studies. DEST, Canberra. Available online at: http://www.dest.gov.au/sectors/higher_education/publications_resources/profiles/first_year_experience.htm (accessed 3 September 2005). Lawrence, J. (2002). The 'deficit-discourse' shift: University teachers and their role in helping first year students persevere and succeed in the new university culture. 6th Pacific Rim, First Year In Higher Education Conference 2002: Changing Agendas - Te Ao Hurihuri, Available online at: http://www.fyhe.qut.edu.au/FYHE_Previous/abstracts02/LawrenceAbstract.htm (accessed 3 September 2005). Lawrence, J. (2005). Reconceptualising attrition and retention: integrating theoretical, research and student perspectives, Studies in learning, evaluation and development, 2 (3), 16- 33. Available online at: http://sleid.cqu.edu.au (accessed 3 October, 2005). Lawrence, J. (2006). 'Engaging first year students: a collaborative approach implemented in a first year nursing course.' In:( 9th Pacific Rim Conference - First Year in Higher Education: Engaging Students, 12-14 July 2006, Gold Coast, Australia) Available online at: http://eprints.usq.edu.au/archive/00000983/ (accessed 18 January 2007). McInnis, C. (2001). Signs of disengagement? The changing undergraduate experience in Australian universities (13 August) Inaugural Professorial Lecture, University of Melbourne, Available online at: http://www.cshe.unimelb.edu.au/downloads/inauglec23_8_01.pdf, (accessed 3 September 2005). Nicol, D. (2007). E-assessment by design: using multiple-choice tests to good effect. Journal of Further and Higher Education, 31(1), 53-64. Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. Pascarella, E.T. & Terenzini, P.T. (2005). How college affects students (Vol. 2: A third decade of research). San Francisco: John Wiley & Sons. Sankey, M.D. & Kiernan, E. (2006). Demystifying assessment with 'bells and whistles' in a first year communication course. In:Evaluations and Assessment Conference: Enhancing student learning, Curtin University of Technology, pp. 115-122. Available online at: http://eprints.usq.edu.au/archive/00001250/ (accessed 18 January 2007). Simpson, O. (2004). The impact on retention of interventions to support distance learning students. Open Learning, 19(1), 79-95. Taylor, J.A. & McDonald, C. (2007). Writing in groups as tool for non-routine problem solving in first year university mathematics. International Journal of Mathematics Education in Science and Technology 38(5), 639-655. Trotter, E. (2006). Student perceptions of continuous assessment. Assessment and Evaluation in Higher Education, 31(5), 505-521. Trotter, E. & Roberts, C. (2006). Enhancing the early student experience. Higher Education Research and Development, 25(4), 371-386. Unworth, K. & Kauter, K. (2008). Evaluating an early bird scheme: encouraging early assignment writing and revising. Higher Education Research and Development, 27(1), 69-76. Wingate, U. (2006). Doing away with 'study skills'. Teaching in Higher Education, 11(4), 457-469. Yorke, M. (2002). Formative assessment: the key to a richer learning experience in semester one. Exchange, 1, 12-13. Yorke, M. (2003). Formative assessment in higher education: Moves towards theory and the enhancement of pedagogic practice. Higher Education, 45, 477-501. 1 Formative assessment is defined here as one that provides feedback to the students on their learning and usually does not contribute to their final grade. Summative assessment is one that contributes towards the final grade for the course. |