Reframing evaluation of learning and teaching: An approach to change

Article

Alderman, Lyn, Towers, Stephen, Bannah, Sylvia and Phan, Le Hoa. 2014. "Reframing evaluation of learning and teaching: An approach to change." Evaluation Journal of Australasia. 14 (1), pp. 24-34. https://doi.org/10.1177/1035719X1401400104
Article Title

Reframing evaluation of learning and teaching: An approach to change

ERA Journal ID39946
Article CategoryArticle
AuthorsAlderman, Lyn (Author), Towers, Stephen (Author), Bannah, Sylvia (Author) and Phan, Le Hoa (Author)
Journal TitleEvaluation Journal of Australasia
Journal Citation14 (1), pp. 24-34
Number of Pages11
Year2014
PublisherSAGE Publications Ltd
Place of PublicationUnited Kingdom
ISSN1035-719X
Digital Object Identifier (DOI)https://doi.org/10.1177/1035719X1401400104
Web Address (URL)https://journals.sagepub.com/doi/10.1177/1035719X1401400104
Abstract

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Keywordsevaluation approach; evaluation methodology; higher education; pedagogy; teaching
ANZSRC Field of Research 2020390201. Education policy
Byline AffiliationsQueensland University of Technology
Institution of OriginUniversity of Southern Queensland
Permalink -

https://research.usq.edu.au/item/q69w1/reframing-evaluation-of-learning-and-teaching-an-approach-to-change

Log in to edit
  • 71
    total views
  • 3
    total downloads
  • 1
    views this month
  • 0
    downloads this month

Export as

Related outputs

Examining the re-territorialisation of beneficiary accountability: Digitising nonprofit services in response to COVID-19
Kingston, Kylie L., Luke, Belinda, Furneaux, Craig and Alderman, Lyn. 2023. "Examining the re-territorialisation of beneficiary accountability: Digitising nonprofit services in response to COVID-19." British Accounting Review. 55 (5). https://doi.org/10.1016/j.bar.2023.101199
Evaluator Perspective
Alderman, Lyn. 2022. "Evaluator Perspective." Evaluation Journal of Australasia. 22 (4), pp. 282-284. https://doi.org/10.1177/1035719X221121400
Performance through the lens of evaluation: How to stretch evaluative thinking with strategic decision-making tools
Harris, Benjamin and Alderman, Lyn. 2022. "Performance through the lens of evaluation: How to stretch evaluative thinking with strategic decision-making tools." Evaluation Journal of Australasia. 22 (4), pp. 237-253. https://doi.org/10.1177/1035719X221120293
The continuous learning framework: Applying accountability for widespread organisational change
Alderman, Lyn. 2022. "The continuous learning framework: Applying accountability for widespread organisational change." Evaluation Journal of Australasia. 22 (4), pp. 206-220. https://doi.org/10.1177/1035719X221120294
Scoping recent investment in evaluation education in Australian universities
Quadrelli, Carol, Staheli, Jessica, Cook-Watkins, Clarissa and Alderman, Lyn. 2022. "Scoping recent investment in evaluation education in Australian universities." Evaluation Journal of Australasia. 22 (4), pp. 254-281. https://doi.org/10.1177/1035719X221120292
Course enhancement conversations: A holistic and collaborative evaluation approach to quality improvement in higher education
Forbes, Melissa, Alderman, Lyn and Murphy, Angela. 2022. "Course enhancement conversations: A holistic and collaborative evaluation approach to quality improvement in higher education." Evaluation Journal of Australasia. 22 (4), pp. 221-236. https://doi.org/10.1177/1035719X221120295
A Reflection on Critical Methodology: Accountability and Beneficiary Participative Evaluation in Third Sector Research
Kingston, Kylie L., Luke, Belinda, Furneaux, Craig and Alderman, Lyn. 2021. "A Reflection on Critical Methodology: Accountability and Beneficiary Participative Evaluation in Third Sector Research." Voluntas: International Journal of Voluntary and Nonprofit Organizations. 33 (6), pp. 1148 - 1155. https://doi.org/10.1007/s11266-021-00395-x
Foundation handwriting plus: computer zone activities - year 6
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 6. St. Leonards, Australia. Horwitz Martin Education.
A short primer on innovative evaluation reporting by Hutchinson Kylie
Alderman, Lyn. 2018. "A short primer on innovative evaluation reporting by Hutchinson Kylie ." Evaluation Journal of Australasia. 18 (1), pp. 69-71. https://doi.org/10.1177/1035719x17753985
The Future of Evaluation in Society: A tribute to Michael Scriven by Stewart I. Donaldson
Alderman, Lyn. 2017. "The Future of Evaluation in Society: A tribute to Michael Scriven by Stewart I. Donaldson." Evaluation Journal of Australasia. 17 (2), pp. 42-43. https://doi.org/10.1177/1035719X1701700207
Finding your research voice: the application of bibliometrics as an analytic literature tool on the pedagogic field of higher education
Alderman, Lyn. 2010. "Finding your research voice: the application of bibliometrics as an analytic literature tool on the pedagogic field of higher education." Postgraduate and Newer Researchers Conference (SRHE 2010). Newport, Wales 13 - 14 Dec 2010 United Kingdom.
Employment opportunities in Australia for educational researchers: A review of recent advertisements
Alderman, Lyn. 2008. "Employment opportunities in Australia for educational researchers: A review of recent advertisements." Australian Association for Research in Education Focus Conference (AARE 2007). Canberra, Australia 13 - 14 Jun 2007 Australia.
Evaluation of the quality of diverse teaching practice in higher education research methods
Alderman, Lyn. 2007. "Evaluation of the quality of diverse teaching practice in higher education research methods." 30th Higher Education Research and Development Society of Australasia Annual Conference (HERDSA 2007). Adelaide, South Australia 08 - 11 Jul 2007 Sydney, Australia.
2005 pedagogical orientation program in the School of Architecture and Built Environment: a case study
Alderman, Lyn. 2005. "2005 pedagogical orientation program in the School of Architecture and Built Environment: a case study." 2005 Enhancing Student Success Conference. Newcastle, Australia 11 Apr 2005 Australia.
Reflection in Integrated Problem Based Learning : 20 years of continuing application to the teaching of architecture
Alderman, Lyn. 2004. "Reflection in Integrated Problem Based Learning : 20 years of continuing application to the teaching of architecture." Problem Based Learning Conference (PBL 2004): Pleasure By Learning. Cancun, Mexico 30 Apr 2004 Mexico.
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2016. "Editorial ." Evaluation Journal of Australasia. 16 (4), pp. 2-3. https://doi.org/10.1177/1035719X1601600401
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2016. "Editorial ." Evaluation Journal of Australasia. 16 (3), pp. 2-4. https://doi.org/10.1177/1035719X1601600301
How can evaluative thinking and practice inform performance models for organizations and personnel
Alderman, Lyn. 2016. "How can evaluative thinking and practice inform performance models for organizations and personnel." H-PEA Spring Workshop. Hawaii 22 Mar 2016 Hawaii, United States.
Developing Monitoring and Evaluation Frameworks
Alderman, Lyn. 2016. "Developing Monitoring and Evaluation Frameworks." Evaluation Journal of Australasia. 16 (3), pp. 42-43. https://doi.org/10.1177/1035719X1601600306
The application of learning analytics to improve learning: Questions from the academy
Alderman, Lyn. 2015. "The application of learning analytics to improve learning: Questions from the academy." Australian Learning Analytics Summer Institute (ALASI 2015). Sydney, Australia 26 - 27 Nov 2015
Reframe: QUT’s evaluation framework Encompasses a multifaceted approach to meet stakeholder needs
Alderman, Lyn. 2015. "Reframe: QUT’s evaluation framework Encompasses a multifaceted approach to meet stakeholder needs." Teaching Quality in Higher Education. Christchurch, New Zealand 10 Jul 2015 New Zealand.
Reframe QUT's Evaluation framework: A case study of planning, policy and positioning leading to educational best practice
Alderman, Lyn. 2014. "Reframe QUT's Evaluation framework: A case study of planning, policy and positioning leading to educational best practice." Australasian Association for Institutional Research (AAIR) 2014 Forum: Turning Silver into Gold. Melbourne, Australia 19 - 21 Nov 2014
Teacher evidence matrix: an evaluation using multiple lines of evidence
Alderman, Lyn. 2008. Teacher evidence matrix: an evaluation using multiple lines of evidence. Brisbane, Australia. Queensland University of Technology.
Quality Assessment Matrix: a collaborative evaluation using multiple lines of evidence
Alderman, Lyn. 2008. Quality Assessment Matrix: a collaborative evaluation using multiple lines of evidence. Brisbane, Australia. Queensland University of Technology.
Mechanisms and strategies to evaluate your teaching at university
Alderman, Lyn. 2007. Mechanisms and strategies to evaluate your teaching at university. Brisbane, Australia. Queensland University of Technology.
Course evidence matrix: a collaborative evaluation using multiple lines of evidence
Alderman, Lyn. 2007. Course evidence matrix: a collaborative evaluation using multiple lines of evidence. Brisbane, Australia. Queensland University of Technology.
Effective data visualization by Stephanie D. H. Evergreen
Alderman, Lyn. 2020. "Effective data visualization by Stephanie D. H. Evergreen." Evaluation Journal of Australasia. 20 (1), pp. 57-58. https://doi.org/10.1177/1035719X19879038
Avoiding the accountability 'sham-ritual': An agonistic approach to beneficiaries' participation in evaluation within nonprofit organisations
Kingston, Kylie L., Furneaux, Craig, de Zwaan, Laura and Alderman, Lyn. 2023. "Avoiding the accountability 'sham-ritual': An agonistic approach to beneficiaries' participation in evaluation within nonprofit organisations." Critical Perspectives on Accounting. 92. https://doi.org/10.1016/j.cpa.2020.102261
Future Directions in Course Quality Assurance
Duncan, Margot and Alderman, Lyn. 2010. "Future Directions in Course Quality Assurance." 2010 Tertiary Education Management Conference (TEMC). Melbourne, Australia 03 - 06 Oct 2010 Australia.
Reframe: QUT's evaluation framework survey business rules
Alderman, Lyn, Phan, Le Hoa and Bennett, Joanna. 2014. Reframe: QUT's evaluation framework survey business rules. Brisbane, Australia. Queensland University of Technology.
Kaleidoscope : shaping a better future through a stakeholder approach to widespread organisational change
Alderman, Lyn and Melanie, Larissa. 2013. "Kaleidoscope : shaping a better future through a stakeholder approach to widespread organisational change." 2013 AES International Conference: Evaluation Shaping a Better Future: Priorities, Pragmatics, Promise and Power. Brisbane, Australia.
Kaleidoscope: illuminating learning and teaching evaluation for a shared stakeholder lens
Alderman, Lyn and Melanie, Larissa. 2013. "Kaleidoscope: illuminating learning and teaching evaluation for a shared stakeholder lens." Australasian Higher Education Evaluation Forum (AHEEF 2013): Performance Indicators for Universities and Private Providers: A time of transition, challenge and collaboration. Launceston, Australia 28 - 30 Oct 2013 Hobart, Tasmania.
Kaleidoscope: Reframing learning and teaching evaluation through a shared stakeholder lens
Alderman, Lyn and Melanie, Larissa. 2013. "Kaleidoscope: Reframing learning and teaching evaluation through a shared stakeholder lens." Australasian Higher Education Evaluation Forum (AHEEF 2013): Performance Indicators for Universities and Private Providers: A time of transition, challenge and collaboration. Launceston, Australia 28 - 30 Oct 2013 Hobart, Tasmania.
REFRAME: a new approach to evaluation in higher education
Alderman, Lyn and Melanie, Larissa. 2012. "REFRAME: a new approach to evaluation in higher education." Australasian Higher Education Evaluation Forum (AHEEF 2012). Rockhampton, Australia 08 - 10 Oct 2012
How to meet stakeholders' expectations, not manage them?
Alderman, Lyn and Melanie, Larissa. 2012. "How to meet stakeholders' expectations, not manage them?" Australasian Higher Education Evaluation Forum (AHEEF 2012). Rockhampton, Australia 08 - 10 Oct 2012
Editorial
Alderman, Lyn and Clinton, Janet. 2016. "Editorial." Evaluation Journal of Australasia. 16 (2), pp. 2-3. https://doi.org/10.1177/1035719X1601600201
Editorial
Alderman, Lyn and Clinton, Janet. 2016. "Editorial ." Evaluation Journal of Australasia. 16 (1), pp. 2-3. https://doi.org/10.1177/1035719X1601600101
QUT’s Evaluation Framework Protocols: Protocols to accompany the policy for Evaluation of courses, units, teaching and student experience (Manual of Policy and Procedures C/4.7)
Alderman, Lyn, Bennett, Joanna and Phan, Le Hoa. 2018. QUT’s Evaluation Framework Protocols: Protocols to accompany the policy for Evaluation of courses, units, teaching and student experience (Manual of Policy and Procedures C/4.7). Brisbane, Australia. Queensland University of Technology.
Editorial
Alderman, Lyn. 2018. "Editorial." Evaluation Journal of Australasia. 18 (2), pp. 75-77. https://doi.org/10.1177/1035719X18785040
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2017. "Editorial ." Evaluation Journal of Australasia. 17 (4), pp. 2-3. https://doi.org/10.1177/1035719X1701700401
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2017. "Editorial ." Evaluation Journal of Australasia. 17 (3), pp. 2-3. https://doi.org/10.1177/1035719X1701700
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2017. "Editorial." Evaluation Journal of Australasia. 17 (2), pp. 2-3. https://doi.org/10.1177/1035719X1701700201
Editorial
Alderman, Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2017. "Editorial ." Evaluation Journal of Australasia. 17 (1), pp. 2-3. https://doi.org/10.1177/1035719X1701700101
Student feedback systems in higher education: a focused literature review and environmental scan
Alderman, Lyn, Towers, Stephen and Bannah, Sylvia. 2012. "Student feedback systems in higher education: a focused literature review and environmental scan." Quality in Higher Education. 18 (3), pp. 261-280. https://doi.org/10.1080/13538322.2012.730714
From monologic to dialogic: accountability of nonprofit organisations on beneficiaries’ terms
Kingston, Kylie L., Furneaux, Craig W., de Zwaan, Laura and Alderman, Lyn. 2019. "From monologic to dialogic: accountability of nonprofit organisations on beneficiaries’ terms." 9th Asia-Pacific Interdisciplinary Research in Accounting Conference (APIRA 2019). Auckland, New Zealand 01 - 03 Jul 2019
The multi-modality of downwards accountability to beneficiaries within nonprofit organisations
Kingston, Kylie L., Furneaux, Craig W., de Zwaan, Laura and Alderman, Lyn. 2018. "The multi-modality of downwards accountability to beneficiaries within nonprofit organisations." 14th Australian and New Zealand Third Sector Research Conference 2018. Sydney, Australia 29 - 30 Nov 2018 Australia.
REFRAME: a new approach to evaluation in higher education
Alderman, Lyn and Melanie, Larissa. 2012. "REFRAME: a new approach to evaluation in higher education." Studies in Learning Evaluation Innovation and Development. 9 (1), pp. 33-41.
Mapping government reforms in quality against higher education theory: is the relationship symbiotic?
Alderman, Lyn. 2016. "Mapping government reforms in quality against higher education theory: is the relationship symbiotic?" Quality in Higher Education. 22 (3), pp. 197-212. https://doi.org/10.1080/13538322.2016.1251047
From monologic to dialogic: accountability of nonprofit organisations on beneficiaries' terms
Kingston, Kylie L., Furneaux, Craig, de Zwaan, Laura and Alderman, Lyn. 2019. "From monologic to dialogic: accountability of nonprofit organisations on beneficiaries' terms." Accounting Auditing and Accountability Journal. 33 (2), pp. 447-471. https://doi.org/10.1108/aaaj-01-2019-3847
Beyond Average: Contemporary statistical techniques for analysing student evaluations of teaching
Kitto, Kirsty, Williams, Cameron and Alderman, Lyn. 2019. "Beyond Average: Contemporary statistical techniques for analysing student evaluations of teaching." Assessment and Evaluation in Higher Education. 44 (3), pp. 338-360. https://doi.org/10.1080/02602938.2018.1506909
Context-sensitive evaluation: Determining the context surrounding the implementation of a government policy
Alderman, Lyn. 2015. "Context-sensitive evaluation: Determining the context surrounding the implementation of a government policy." Evaluation Journal of Australasia. 15 (4), pp. 4-15. https://doi.org/10.1177/1035719X1501500402
Dumbing it down: where do standards fit?
Alderman, Lyn. 2004. "Dumbing it down: where do standards fit?" International Journal of Learning. 11, pp. 1007-1012.
From rhetoric to practice: issues in teaching and learning touch keyboarding
Alderman, Gwendolyn. 2004. From rhetoric to practice: issues in teaching and learning touch keyboarding. Masters Thesis Master of Education. University of Newcastle.
From policy borrowing to implementation: an illuminative evaluation of learning and teaching in higher education in Australia (2002 to 2008)
Alderman, Gwendolyn. 2014. From policy borrowing to implementation: an illuminative evaluation of learning and teaching in higher education in Australia (2002 to 2008). PhD Thesis Doctor of Philosophy. Queensland University of Technology.
Foundation handwriting plus: computer zone activities - year 5
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 5. St. Leonards, Australia. Horwitz Martin Education.
Foundation handwriting plus: computer zone activities - year 4
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 4. St. Leonards, NSW, Australia. Horwitz Martin Education.
Foundation handwriting plus: computer zone activities - year 3
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 3. St. Leonards, NSW, Australia. Horwitz Martin Education.
Foundation handwriting plus: computer zone activities - year 2
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 2. St. Leonards, NSW, Australia. Horwitz Martin Education.
Foundation handwriting plus: computer zone activities - year 1
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - year 1. St. Leonards, NSW, Australia. Horwitz Martin Education.
Foundation handwriting plus: computer zone activities - kindergarten
Woods, Andrew and Alderman, Lyn. 2002. Foundation handwriting plus: computer zone activities - kindergarten. St. Leonards, NSW, Australia. Horwitz Martin Education.
Keyboarding plus handbook
Alderman, Lyn. 2002. Keyboarding plus handbook. St. Leonards, NSW, Australia. Horwitz Martin Education.
Evaluation Perspectives
Alderman Lyn. 2019. "Evaluation Perspectives." Evaluation Journal of Australasia. 19 (1), pp. 49-53. https://doi.org/10.1177/1035719X19837025
Editorial
Alderman Lyn, Gould, Liz, Quadrelli, Carol and Rossingh, Bronwyn. 2017. "Editorial." Evaluation Journal of Australasia. 17 (3), pp. 2-3. https://doi.org/10.1177/1035719X1701700301
Editorial
Alderman, Lyn and Clinton, Janet. 2015. "Editorial." Evaluation Journal of Australasia. 15 (4), pp. 2-3. https://doi.org/10.1177/1035719X1501500401
Editorial
Alderman, Lyn and Clinton, Janet. 2015. "Editorial ." Evaluation Journal of Australasia. 15 (3), pp. 2-3. https://doi.org/10.1177/1035719X1501500301
Editorial
Alderman, Lyn and Clinton, Janet. 2015. "Editorial." Evaluation Journal of Australasia. 15 (2), pp. 2-3. https://doi.org/10.1177/1035719X1501500201
Editorial
Alderman, Lyn and Clinton, Janet. 2015. "Editorial ." Evaluation Journal of Australasia. 15 (1), pp. 2-3. https://doi.org/10.1177/1035719X1501500101
Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education
Alderman, Lyn. 2015. "Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education." Evaluation Journal of Australasia. 15 (1), pp. 4-14. https://doi.org/10.1177/1035719X1501500102
Editorial
Alderman, Lyn, Clinton, Janet and Harvey, Graeme. 2014. "Editorial ." Evaluation Journal of Australasia. 14 (2), pp. 2-3. https://doi.org/10.1177/1035719X1401400201
Editorial
Alderman, Lyn, Clinton, Janet and Harvey, Graeme. 2014. "Editorial ." Evaluation Journal of Australasia. 14 (1), pp. 2-3. https://doi.org/10.1177/1035719X1401400101
Purposeful Program Theory: Effective Use of Theories of Change and Logic Models by Sue C Funnell and Patricia J Rogers
Alderman, Lyn. 2014. "Purposeful Program Theory: Effective Use of Theories of Change and Logic Models by Sue C Funnell and Patricia J Rogers." Evaluation Journal of Australasia. 14 (1), pp. 46-47. https://doi.org/10.1177/1035719X1401400107
Editorial
Alderman, Lyn, Clinton, Janet and Harvey, Graeme. 2013. "Editorial ." Evaluation Journal of Australasia. 13 (2), pp. 2-2. https://doi.org/10.1177/1035719X1301300201
Editorial
Alderman, Lyn, Clinton, Janet and Harvey, Graeme. 2013. "Editorial ." Evaluation Journal of Australasia. 13 (1), pp. 2-3. https://doi.org/10.1177/1035719X1301300101
Context: A Framework for its Influence on Evaluation Practice (New Directions for Evaluation, No. 135) by Debra J Rog, Jody L Fitzpatrick and Ross F Conner
Alderman, Lyn. 2013. "Context: A Framework for its Influence on Evaluation Practice (New Directions for Evaluation, No. 135) by Debra J Rog, Jody L Fitzpatrick and Ross F Conner." Evaluation Journal of Australasia. 13 (1), pp. 49-51. https://doi.org/10.1177/1035719X1301300110
A risk-based approach to course quality assurance
Towers, Stephen J., Alderman, Lyn, Nielsen, Samuel and McLean, S. Vianne. 2010. "A risk-based approach to course quality assurance." Australian Quality Forum 2010 - Higher Education: Quality in Uncertain Times. Gold Coast, Australia 30 Jun - 02 Jul 2010 Melbourne, Australia.
Articulation in property programs: an international multi-campus tertiary model
Kirk, A. and Alderman, Lyn. 2005. "Articulation in property programs: an international multi-campus tertiary model." International Conference on Problem-Based Learning (2005). Lahti, Finland 09 - 11 Jun 2005 Finland.