Led by the Ecampus Research Unit, the study reveals best practices for collecting and understanding data about online learners.
As universities work to increase their online offerings, the use of digital learning platforms and learning management systems is now widespread. With these powerful tools comes a proliferation of new data sources that universities can use to better understand how students learn. This is referred to as learning data, and it can be a powerful tool for universities to draw conclusions — and make decisions — about the student experience.
But how aware are students of the data that’s collected and the ways in which institutions use this data? And do universities have safeguards in place to combat bias in the collection and use of this data? Furthermore, are university faculty and staff prepared to utilize learning data for decision-making purposes?
An interdisciplinary team of researchers, led by Oregon State University’s Ecampus Research Unit, recently published a paper that shines a light on some of these issues.
The research team, referred to as the learning analytics cohort, included ten authors from nine higher education institutions (HEIs) across the United States. The team conducted an interview-based study across eight different HEIs. Their recently published paper includes the voices of three distinct stakeholder groups — students, diversity and inclusion leaders, and senior administrative leaders. The researchers asked each of these groups about their degree of concern regarding issues of equity and bias when it comes to online learner data.
“The stakeholders we chose to interview are at opposite ends of the power spectrum, which gives the study a unique perspective,” said Mary Ellen Dello Stritto, director of the Oregon State University Ecampus Research Unit. “Researchers are not typically studying these particular groups together.”
The study findings, recently published in the Journal of Learning Analytics, showed that all three stakeholder groups shared a significant concern about bias and equity in learner data and the potential for making biased decisions based on this data. All three groups also expressed a level of skepticism about the use of learning analytics by higher education institutions. Additionally, the study showed that stakeholders had varying degrees of data literacy, which may indeed contribute to biased decision-making.
“One value of this study is that this research represents diverse types of higher education institutions engaging in online education, providing a more national level of understanding.” said Rebecca E. Heiser, doctoral candidate in distance education at Athabasca University.
So knowing the potential for bias and mistrust exists — what’s to be done?
The research team offers the following recommendations for how universities can operate in a more transparent and ethical fashion.
- Build organizational processes that facilitate collaboration and feedback sessions with stakeholder groups — including student voices — which focus on the institutional collection and use of student data.
- Develop ongoing feedback loops to address existing learner data policies and practices. There should be organizational processes for addressing anonymous, individual or group reporting of concerns around bias and equity.
- Incorporate stakeholder perspectives — again, including students — when new digital platforms are being negotiated or acquired by the university.
To learn more about the details of this study, you can read the full publication in the Journal of Learning Analytics.
Learn more about the team who carried out this research and Oregon State’s Ecampus Research Unit who led the study and contributes other award-winning research to the field of online education.
