Theory-led design of instruments and representations in learning analytics: developing a novel tool for orchestration of online collaborative learning

Article

Kelly, Nick, Thompson, Kate and Yeoman, Pippa. 2015. "Theory-led design of instruments and representations in learning analytics: developing a novel tool for orchestration of online collaborative learning." Journal of Learning Analytics. 2 (2), pp. 14-42. https://doi.org/10.18608/jla.2015.22.3
Article Title

Theory-led design of instruments and representations in learning analytics: developing a novel tool for orchestration of online collaborative learning

ERA Journal ID210923
Article CategoryArticle
AuthorsKelly, Nick (Author), Thompson, Kate (Author) and Yeoman, Pippa (Author)
Journal TitleJournal of Learning Analytics
Journal Citation2 (2), pp. 14-42
Number of Pages29
Year2015
ISSN1929-7750
Digital Object Identifier (DOI)https://doi.org/10.18608/jla.2015.22.3
Web Address (URL)https://epress.lib.uts.edu.au/journals/index.php/JLA/article/view/4265
Abstract

This paper describes theory-led design as a way of developing novel tools for learning analytics (LA). It focuses upon the domain of automated discourse analysis (ADA) of group learning activities to help an instructor to orchestrate online groups in real-time. The paper outlines the literature on the development of LA tools within the domain of ADA, and poses an argument for conducting tool development based upon first-principles. It describes first principles as being drawn from theory and that these principles can subsequently inform the structure and behaviour of tools. It presents a framework for this process of theory-led design. The framework is substantiated through the example of developing a new tool for assisting instructors with the orchestration of online groups. A description of the tool is given and examples of results from use with real-world data are presented. The paper concludes with a call for intent on the part of designers to connect the design process explicitly to theory on the basis that this has the potential to yield innovation when developing tools as well as the prospect of outcomes from tools connecting back to theory.

Keywordsautomated discourse analysis; collaborative learning; orchestration; learning analytics; topic detection
ANZSRC Field of Research 2020390409. Learning sciences
Byline AffiliationsAustralian Digital Futures Institute
University of Sydney
Institution of OriginUniversity of Southern Queensland
Permalink -

https://research.usq.edu.au/item/q33z0/theory-led-design-of-instruments-and-representations-in-learning-analytics-developing-a-novel-tool-for-orchestration-of-online-collaborative-learning

Log in to edit

Download files

Published Version
Kelly_Thompson_Yeoman_PV.pdf
License: CC BY-NC-ND
File access level: Anyone

  • 1708
    total views
  • 167
    total downloads
  • 1
    views this month
  • 1
    downloads this month

Export as

Related outputs

Australian teachers’ causal attributions along a motivational continuum in supporting their resilience
Peel, Karen, Kelly, Nick and Danaher, Patrick A.. 2024. "Australian teachers’ causal attributions along a motivational continuum in supporting their resilience." Issues in Educational Research. 34 (1), pp. 163-182.
Analysing teachers’ figurative language to shed new light on teacher resilience
Peel, K.L., Kelly, N., Danaher, P.A., Harreveld, B and Mulligan, D.. 2023. "Analysing teachers’ figurative language to shed new light on teacher resilience." Teaching and Teacher Education. 130. https://doi.org/10.1016/j.tate.2023.104175
Critical online learning networks of teachers: communality and collegiality as contingent elements
Kelly, Nick, Clara, Marc, Kehrwald, Benjamin A. and Danaher, P. A.. 2020. "Critical online learning networks of teachers: communality and collegiality as contingent elements." Simpson, Ashley and Dervin, Fred (ed.) The meaning of criticality in education research: reflecting on critical pedagogy. Cham, Switzerland. Palgrave Macmillan. pp. 101-126
Can massive communities of teachers facilitate collaborative reflection? Fractal design as a possible answer
Clara, Marc, Kelly, Nick, Mauri, Teresa and Danaher, P. A.. 2017. "Can massive communities of teachers facilitate collaborative reflection? Fractal design as a possible answer." Asia-Pacific Journal of Teacher Education. 45 (1), pp. 86-98. https://doi.org/10.1080/1359866X.2015.1095280
Early career teachers’ intentions to leave the profession: the complex relationships among preservice education, early career support, and job satisfaction
Kelly, Nick, Cespedes, Marcela, Clara, Marc and Danaher, Patrick A.. 2019. "Early career teachers’ intentions to leave the profession: the complex relationships among preservice education, early career support, and job satisfaction." Australian Journal of Teacher Education. 44 (3), pp. 93-113. https://doi.org/10.14221/ajte.2018v44n3.6
Slipping through the cracks: teachers who miss out on early career support
Kelly, Nick, Sim, Cheryl and Ireland, Michael. 2018. "Slipping through the cracks: teachers who miss out on early career support." Asia-Pacific Journal of Teacher Education. 46 (3), pp. 292-316. https://doi.org/10.1080/1359866X.2018.1441366
Networked learning for agricultural extension: a framework for analysis and two cases
Kelly, Nick, Bennett, John McLean and Starasts, Ann. 2017. "Networked learning for agricultural extension: a framework for analysis and two cases." Journal of Agricultural Education and Extension: international journal on changes in agricultural knowledge and action systems. 23 (5), pp. 399-414. https://doi.org/10.1080/1389224X.2017.1331173
A framework for self-determination in massive open online courses: design for autonomy, competence, and relatedness
Martin, Neil I., Kelly, Nick and Terry, Peter C.. 2018. "A framework for self-determination in massive open online courses: design for autonomy, competence, and relatedness." Australasian Journal of Educational Technology. 34 (2), pp. 35-55. https://doi.org/10.14742/ajet.3722
Early career teacher peer support through private groups in social media
Mercieca, Bernadette and Kelly, Nick. 2018. "Early career teacher peer support through private groups in social media." Asia-Pacific Journal of Teacher Education. 46 (1), pp. 61-77. https://doi.org/10.1080/1359866X.2017.1312282
Using process analysis techniques to understand students' learning strategies with computer models
Markauskaite, Lina, Jacobson, Michael J., Southavilay, Vilaythong and Kelly, Nick. 2012. "Using process analysis techniques to understand students' learning strategies with computer models." Tyson, Cynthia A. (ed.) 2012 American Educational Research Association Annual Meeting (AERA 2012). Vancouver, Canada 13 - 17 Apr 2012 Washington, DC. United States.
Interpretation as driver for psychological creativity
Kelly, Nick and Gero, John S.. 2011. "Interpretation as driver for psychological creativity." Goel, Ashok K. (ed.) 8th ACM conference on Creativity and Cognition (C&C 2011): Creativity and Technology. Atlanta, Georgia 03 - 06 Nov 2011 New York, NY. United States. https://doi.org/10.1145/2069618.2069693
Online learning networks for pre-service and early career teachers
Kelly, Nick, Clara, Marc, Kehrwald, Benjamin and Danaher, Patrick Alan. 2016. Online learning networks for pre-service and early career teachers. Basingstoke, United Kingdom. Palgrave Macmillan.
Model-based learning about climate change with productive failure: preliminary findings
Jacobson, Michael J., Markauskaite, Lina, Kelly, Nick and Stokes, Paul G.. 2012. "Model-based learning about climate change with productive failure: preliminary findings." Tyson, Cynthia A. (ed.) 2012 American Educational Research Association Annual Meeting (AERA 2012). Vancouver, Canada 13 - 17 Apr 2012 Washington, DC. United States.
Digital apprehension and first year university students
Smith, Heather, Quinn, Andrea and Kelly, Nick. 2015. "Digital apprehension and first year university students." Ho, Curtis and Lin, Grace (ed.) E-Learn 2015: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2015. Kona, United States 19 - 22 Oct 2015 Chesapeake, VA, United States .
Teacher peer support in social network sites
Kelly, Nick and Antonio, Amy. 2016. "Teacher peer support in social network sites." Teaching and Teacher Education. 56, pp. 138-149. https://doi.org/10.1016/j.tate.2016.02.007
Situated interpretation in computational creativity
Kelly, Nick and Gero, John S.. 2015. "Situated interpretation in computational creativity." Knowledge-Based Systems. 80 (1), pp. 48-57. https://doi.org/10.1016/j.knosys.2014.12.005
An integrated approach to attracting and retaining teachers in rural and remote parts of Australia
Kelly, Nick and Fogarty, Rod. 2015. "An integrated approach to attracting and retaining teachers in rural and remote parts of Australia." Journal of Economic and Social Policy. 17 (2).
Exploring relationships between undergraduate students' use of eLearning system, library, and learning outcomes
Jara, Magdalena, Kelly, Nick, Montenegro, Maximilliano, Gonzalez, Carlos, Alarcon, Rosa, Saurina, Elvira, Peralta, Montserrat and Cano, Felipe. 2014. "Exploring relationships between undergraduate students' use of eLearning system, library, and learning outcomes." Rensing, Christoph, de Freitas, Sara, Ley, Tobias and Munoz-Merino, Pedro J. (ed.) 9th European Conference on Technology-Enhanced Learning (ECTEL 2014): Open Learning and Teaching in Educational Communities. Graz, Austria 16 - 19 Sep 2014 Switzerland. https://doi.org/10.1007/978-3-319-11200-8
Pleasant encounters of the VM kind: leveraging on NeCTAR VMs to achieve research outcomes
Gacenga, Francis, Kelly, Nick and Rayment, Erin. 2014. "Pleasant encounters of the VM kind: leveraging on NeCTAR VMs to achieve research outcomes." eResearch Australasia 2014 Conference: Towards Unified Global Research. Melbourne, Australia 27 - 31 Oct 2014 Sydney, Australia.
Constructive interpretation in design thinking
Kelly, Nick and Gero, John. 2009. "Constructive interpretation in design thinking." Cagdas, G. and Colokoglu, B. (ed.) 27th Conference of the Association for Education and Research in Computer Aided Architectural Design in Europe (eCAADe 2009): Computation: The New Realm of Architectural Design. Istanbul, Turkey 16 - 19 Sep 2009 Vienna, Austria.
Constructive interpretation with examples from interpreting floor plans
Kelly, Nick, Gero, John S. and Saunders, Robert. 2011. "Constructive interpretation with examples from interpreting floor plans." Herr, C. M., Gu, N., Roudavsky, S. and Schnabel, M. A. (ed.) 16th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 2011): Circuit Bending, Breaking and Mending. Newcastle, Australia 27 - 29 Apr 2011 Hong Kong.
Beginning teacher support in Australia: towards an online community to augment current support
Kelly, Nick, Reushle, Shirley, Chakrabarty, Sayan and Kinnane, Anna. 2014. "Beginning teacher support in Australia: towards an online community to augment current support." Australian Journal of Teacher Education. 39 (4), pp. 68-82. https://doi.org/10.14221/ajte.2014v39n4.6
Computational modelling of the design conversation as a sequence of situated acts
Kelly, Nick and Gero, John S.. 2012. "Computational modelling of the design conversation as a sequence of situated acts." Kiang, Tan Beng (ed.) 17th International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA 2012): Beyond Codes and Pixels . Chennai, India 25 - 28 Apr 2012 Chennai, India.
Discovering indicators of successful collaboration using tense: automated extraction of patterns in discourse
Thompson, Kate, Kennedy-Clark, Shannon, Wheeler, Penny and Kelly, Nick. 2014. "Discovering indicators of successful collaboration using tense: automated extraction of patterns in discourse." British Journal of Educational Technology. 45 (3), pp. 461-470. https://doi.org/10.1111/bjet.12151
Combining collaboration spaces: identifying patterns of tool use for decision-making in a networked learning environment
Thompson, Kate and Kelly, Nick. 2012. "Combining collaboration spaces: identifying patterns of tool use for decision-making in a networked learning environment." Hodgson, V., Jones, C., De Laat, M., McConnell, D., Ryberg, T. and Sloep, P. (ed.) 8th International Conference on Networked Learning (NLC 2012). Maastricht, Netherlands 02 - 04 Apr 2012 Lancaster, United Kingdom.
How can CAAD tools be more useful at the early stages of designing? Towards situated agents that interpret
Gero, John S. and Kelly, Nick. 2008. "How can CAAD tools be more useful at the early stages of designing? Towards situated agents that interpret." Nakapan, Walaiporn, Mahaek, Ekkachai, Teeraparbwong, Komson and Nilkaew, Piyaboon (ed.) 13th International Conference on Computer Aided Architectural Design Research in Asia (CAADRIA 2008): Beyond Computer-Aided Design. Chiang Mai, Thailand 09 - 12 Apr 2008 Chiang Mai, Thailand .
Interpretation in design: modelling how the situation changes during design activity
Kelly, Nick and Gero, John S.. 2014. "Interpretation in design: modelling how the situation changes during design activity." Research in Engineering Design. 25 (2), pp. 109-124. https://doi.org/10.1007/s00163-013-0168-y
Integrating technology, pedagogy and content in mathematics education
Handal, Boris, Campbell, Chris, Cavanagh, Michael, Petocz, Peter and Kelly, Nick. 2012. "Integrating technology, pedagogy and content in mathematics education." Journal of Computers in Mathematics and Science Teaching. 31 (4), pp. 387-413.
Using automated and fine-grained analysis of pronoun use as indicators of progress in an online collaborative project
Thompson, Kate, Kennedy-Clark, Shannon, Kelly, Nick and Wheeler, Penny. 2013. "Using automated and fine-grained analysis of pronoun use as indicators of progress in an online collaborative project." Rummel, Nicol, Kapur, Manu, Nathan, Mitchell and Puntambekar, Sadhana (ed.) 10th International Conference on Computer-Supported Collaborative Learning (CSCL 2013): To See the World and a Grain of Sand: Learning across Levels of Space, Time, and Scale. Madison, WI. United States 15 - 19 Jun 2013 New York, NY. United States.
An opportunity to support beginning teachers in the transition from higher education into practice
Kelly, Nick. 2013. "An opportunity to support beginning teachers in the transition from higher education into practice ." Carter, Helen, Gosper, Maree and Hedberg, John (ed.) 30th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2013). Sydney, Australia 01 - 04 Dec 2013 Sydney, Australia.
Technological pedagogical content knowledge of secondary mathematics teachers
Handal, Boris, Campbell, Chris, Cavanagh, Michael, Petocz, Peter and Kelly, Nick. 2013. "Technological pedagogical content knowledge of secondary mathematics teachers." Contemporary Issues in Technology and Teacher Education. 13 (1), pp. 22-40.
Processes of decision-making with adaptive combinations of wiki and chat tools
Thompson, Kate and Kelly, Nick. 2012. "Processes of decision-making with adaptive combinations of wiki and chat tools." van Aalst, Jan, Thompson, Kate, Jacobsob, Michael J. and Reimann, Peter (ed.) 10th International Conference of the Learning Sciences (ICLS 2012): The Future of Learning. Sydney, Australia 02 - 06 Jul 2012 Sydney, Australia.
Agent-based computer models for learning about climate change and process analysis techniques
Kelly, Nick, Jacobson, Michael J., Markauskaite, Lina and Southavilay, Vilaythong. 2012. "Agent-based computer models for learning about climate change and process analysis techniques." van Aalst, Jan, Thompson, Kate, Jacobson, Michael J. and Reimann, Peter (ed.) 10th International Conference of the Learning Sciences (ICLS 2012): The Future of Learning. Sydney, Australia 02 - 06 Jul 2012 Sydney, Australia.
Processing and visualizing data in complex learning environments
Thompson, Kate, Ashe, David, Carvalho, Lucila, Goodyear, Peter, Kelly, Nick and Parisio, Martin. 2013. "Processing and visualizing data in complex learning environments." American Behavioral Scientist. 57 (10), pp. 1401-1420. https://doi.org/10.1177/0002764213479368