Learning Analytics: Shifting from theory to practice.

Abstract

As online and blended learning continue to increase in higher education, so does the amount of data that is housed within Learning Management Systems that can be analyzed and processed within the framework of Learning Analytics. Learning Analytics is a new and developing field. As with many new fields of study, a gap between theory and practice is evident. Some attribute this gap to the lack of situating learning analytics within learning theory. In order for Learning Analytics to find interest and usability among educators, a shift is needed from the technical use to practical application. In this theoretical paper a number of potential inhibitors and uses to full application of Learning Analytics are presented.

Introduction

With the increase in demand by students to participate in higher education courses there has also been a steady increase in the use of online or blended learning platforms to support student learning. As the increase in e-learning has risen, so has the need for managing the curriculum content. The use of such Learning Management Systems (LMS) or Content Management Systems (CMS) have appeared as a readily available means for housing course learning content. Recent studies have found that LMS have created a constructive method for acquiring knowledge and engaging student learning (Emelyanova & Voronia, 2014). As e-learning has grown in usage among higher education institutions, similarly has the number of LMS platforms and other tools that are incorporated to support the online student learning (Firat, 2015) within products such as Moodle, Blackboard, and Canvas. Together with the adoption of various digital technologies, a new chance to understand student learning better has arisen as LMS platforms can provide large amounts of “trace” (Gasevic et. al, 2016, p. 68) or log data about student interactions within the course. These digital footprints from students in online courses are collected and saved in digital archives of the LMS that can later be “mined and analyzed to identify patterns of learning behavior that can provide insights in to educational practice” (Gasevic, Dawson, & Seimens, 2015, p. 64). The practice of analyzing data produced by students as they interact with LMS, coupled with student information systems of the institution (eg. demographics, performance, and other data), has garnered interest by many teachers, managers, and researchers as a possible solution in addressing many issues faced in the field of education (Gasevic et al., 2016).

Gasevic and others (2016) described that the techniques used to analyze trace and archival data are often applied to discover patterns (Baker & Yacef, 2009) which can then be interpreted to inform more about the learning and teaching process, provide models for predicting achievement, and supply possible remediation and intervention support. Seimens and Gasevic (2012) have labeled this process as Learning Analytics. Learning Analytics (LA) is a fairly new and developing field, and as with most new fields of study, there are many authors providing definitions of what LA constitutes, where LA originates, what gaps exist between research and practice, and how to apply LA to established learning theory concepts.

Background

Defining Learning Analytics

Many authors have defined learning Analytics, yet the following definitions are used in framing the focus of this paper. The Society for Learning Analytics (SoLAR, n.d.) stated that LA, “is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs”. This definition emphasizes the focus on the learner and optimization of the learning process. It also highlights the potential use of techniques in modeling, generating profiles of learners, and possibility of personalized and adaptable learning as well as others (Seimens, 2012).

Johnson and colleagues in 2014 (as cited by Firat, 2016) defined LA as “an area which focuses on reaching patterns or tendencies via data sets related to student or via large sets of educational data to maintain the development of supplementary and personalized higher education systems.” (p.76) Similar to this definition, Agudo-Peregrina et al. (2014) have emphasized the focus of LA being on discovering the “unobservable patterns and the information underlying the learning process.” (as cited by Firat, 2016, p.76) These definitions provide a vision of the potential usability and application of LA in assisting educational institutions, teachers, and even learners in improving student learning.

Origins of Learning Analytics

Tracing the historical roots of LA, some authors (Gasevic et al, 2016) identify educational data mining (EDM) as the closest related field, while others (Ferguson, 2012; Seimens, 2013) suggested roots in various fields of business intelligence, machine learning, web analytics, and even artificial intelligence. Despite the lack of agreement of the origin of LA, Ferguson (2012) established that the development of LA through time reveals a movement away from a focus on technology to a focus on education. Seimens (2013) has suggested that many other fields have found success by shifting economies and increasing productivity with the use of analytics, but education at every level has not taken advantage of the opportunity to use the readily available data that could potentially improve teaching and learning. Seimens did note that even with the lag in education there is a recent “explosion of interest” (p.1381) in LA as a means of increasing retention and offering learner support. Others have found potential success of LA in assisting the learning process (Baker & Seimens, 2014), creating predictive models of academic success to increase retention (Seimens, Dawson, & Lynch, 2014).

Theory and Practice Gap

The most notable gap within LA research and practice, common among many fields of study, is translating research to inform practice. Siemens (2012) described that much of the research and contribution of LA has occurred within university laboratories and software companies, and has been shared and disseminated within scholarly realms. He continued to explain that practitioners are utilizing the tools and techniques and are acquiring knowledge through the development and application of corporate products in their teaching roles, which often involve a level of risk taking.

Despite the research that has been conducted, there is also a lack of empirical studies evaluating the transferability and impact in other domains (Dawson et al., 2014). Gasevic et al. (2015) added that the dearth in the literature has revealed a significant issue where LA tools are not developed within “theoretically established instructional strategies.” (p.65) The authors go on to claim that the field of LA needs to “ground data collection, measurement, analysis, reporting and interpretation processes within the existing research on learning.” (p.65) They describe how much of past LA research has focused on impacts of performed operations using representative trace data without focusing on elements of instructional conditions.

Learning Theory Application

The potential benefit in using LA within understanding internal and external conditions of the student learning can yield a more detailed view of how the student engages with the learning content, how they approach learning, and even how students create learning goals (Gasevic et al. 2015). One internal condition that multiple authors describe (Ferguson, 2012; Gasevic et al., 2014; Seimens 2012; Seimens, 2013) is the focus on the needs and personalization of the course content for the learner. Concepts such as student choice, personalization, self-directed, adaptive, and self-regulating have been connected with the benefits of LA. The student-centered foci can be approached within the LMS by a number of different methods and tools. Although studies about the use of such tools reveal differences in the number of tools and how they are utilized in facilitating learning (Winne, 2006), the simple use of such tools by the student have been categorized by researchers (Lust, Elen, & Clarebout, 2013) as personalized learning process of student choice using tools based on both internal conditions and personal goals in their learning.

Inhibitors to Learning Analytic Use

Another area that many of these studies have also neglected, and could potentially fill the gap of practice and research, is the inhibitors that teachers, managers, or even administrators face in adopting a LA approach to understanding student LMS interaction. In focusing on the practice and how teachers/managers/administrators may view data in general, there are a number of reasons they may not see LA as a potential method for understanding how students are learning within an LMS course. Although assessments and outcomes may be collected and measured and even student behaviors of enrollment and attendance may be gathered, in many e-learning and online student courses teachers/managers/administrators are not focusing on how students are engaging with and consuming the curriculum. As described earlier, understanding the internal and external conditions of students’ choice provide insights into the learning process and connecting it to pedagogical design. The following table describes possible inhibitors that prevent teachers from using LA as a method for understanding student learning.

Although many of the above listed inhibitors may transcend the use of LA and could be broadly applied to most pedagogical approaches, some are very specific to how LA barriers inhibit full usage. Although there is an organizational capacity that is not addressed here, Siemens (2013) described there are issues beyond the technical processes,

“The effective process and operation of learning analytics require intuitional change that does not just address the technical challenges linked to data mining, data models, server load, and computation, but also addresses the social complexities of application, sense making, privacy, and ethics alongside the development of a shared organizational culture framed in analytics.” (p. 1391)

Within Siemens work, he noted there are many challenges that face the use of LA in education that are not related to the technical aspect. He referred to the work of Slade and Prinsloo (2013) where they listed challenges as concerns of data quality, issues related to scope and reflecting accurately the learning experience, privacy, and ethics of analytics.

Benefits of Learning Analytics

In addition to shifting the culture of the organization to be able to focus more on analytics, institutions must promote the potential benefit and application from the knowledge gained through analytics. Although there is still a gap in the practice and theoretical literature of LA, there are a number of potential benefits for the practice. Table 2 lists potential benefits from the implementation of LA.

The potential benefits of LA data, although not limited to this list, can help shift organizations from speculative decision making within course instruction to a more data informed and evidenced based foundation of decision making and understanding of how students are learning. The benefits listed here also provide a shift from the theoretical practices of predicting student success and monitoring student profiles to understanding the internal conditions of how students are interacting with course content and how choice and personalization can contribute to the overall success.

As Siemens (2012) stated, “LA has potential to dramatically impact the existing models of education and to generate new insights into what works and what does not work in teaching and learning.” (p.4) The described shift is an essential change on what LA focuses on, where in the past the focus was on the institutional needs of an organization and now the focus is on the “perspectives of learners” (Ferguson, 2013, p.313). Where organizations worked within the realms of the technical information and orientation there needs to be a redirection to one that “emphasizes sense making, decision-making, and action required to increase interest among educators and administrators.” (Siemens, 2012, p. 4) Evidence of this shift or demonstrating the long-term influence on teaching practice and student learning will be the new measure of success in LA (Gasevic et al., 2015).

Conclusion

The benefits and potential educational application are a new and developing area of Learning Analytics that not only provide a method of analyzing student perspective data from LMS, but could also provide a framework for conducting research. Siemens (2013) noted that, “the future success of LA and EDM as research domains requires the development of academic programs to foster and develop new researchers as well as development of grant programs that target LA.” (p.1396) As with many new fields of study, they have the possibility of losing relevance and applicability if not utilized effectively to yield the greatest impact and understanding, “learning analytics that do not promote effective learning and teaching are susceptible to the use of trivial measures” (Gasevic et al., 2015, p. 69). Avoiding the “trivial”, researchers and practitioners can frame the use of LA by involving those that both create the data analyzed and those that use the information to make future decisions. Students, teachers, administrators, and designers need to be included in all levels of development and utilization to help yield the greatest information possible to inform learning within our institutions.

References

Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M. Á., & Hernández-García, Á. (2014). Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Computers in Human Behavior, 31 , 542-550.

Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences. Cambridge, UK: Cambridge University Press.

Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: Review and future visions. Journal of Educational Data Mining, 1(1), 3–17.

Dawson, S., Gašević, D., Siemens, G., & Joksimovic, S. (2014). Current state and future trends: A citation network analysis of the learning analytics field. In Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 231–240). New York, NY, USA.

Emelyanova, N., & Voronina, E. (2014). Introducing a learning management system at a Russian university: Students’ and teachers’ perceptions. The International Review of Research in Open and Distributed Learning, 15 (1).

Ferguson, Rebecca (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6) pp. 304–317.

Firat, M. (2016). Determining the effects of LMS learning behaviors on academic achievement in a learning analytic perspective. Journal of Information Technology Education: Research, 15, 75-87

Gasevic, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends. 59 (1) 64-71.

Gasevic, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education. 28(2016) 68-84.

Johnson, L., Adams Becker, S., Estrada, V. & Freeman, A. (2014). NMC Horizon Report: 2014 Higher Education Edition. Austin, Texas: The New Media Consortium.

Siemens, G., & Gašević, D. (2012). Special Issue on Learning and Knowledge Analytics. Educational Technology & Society, 15(3), 1–163.

Lust, G., Elen, J., & Clarebout, G. (2013). Students’ tool-use within a web enhanced course: Explanatory mechanisms of students’ tool-use pattern. Educational Technology & Society, 29(5) 2013-2021.

Siemens, G. (2012). Learning analytics: Envisioning a research discipline and a domain of practice. LAK12: 2nd International Conference on Learning Analytics & Knowledge, 29 April – 2 May 2012, Vancouver, BC, Canada

Siemens, G. (2013). Learning analytics: The emergences of a discipline. American Behavior Scientist. 57 (10) 1380-140.

Siemens, G., Dawson, S., & Lynch, G. (2014). Improving the quality and productivity of the higher education sector — Policy and strategy for systems-level deployment of learning analytics. Camberra, Australia: Office of Learning and Teaching, Australian Government (Retrieved from http://solaresearch.org/Policy_Strategy_Analytics.pdf).

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist.

Society for Learning Analytics Research. About, 2012. Available from http://www.solaresearch.org/about/ [accessed April 15, 2012]

Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41 (1) 5-17.

About the Author

Courtney Stewart is a former middle/high school science/math teacher and principal. He received his M.Ed. and Ph.D. in Educational Leadership and Foundations from Brigham Young University. Prior to joining Utah State University’s Instructional Leadership faculty, he held appointments as associate professor at University of Montana and Minnesota State University, Mankato. In addition to being a well-respected teacher, he was recently ranked as the third most influential person on campus. Dr. Stewart has previously directed principal academies and consulted for schools developing professional learning communities. He has published works within the areas of school reform, educational leadership, and rural education.