By Tay T. McEdwards
Proctoring manger
Oregon State University Ecampus
Until recently, when the COVID-19 pandemic caused the massive shift from on-campus to remote learning, there was little interest in online exam proctoring. Additionally, few resources were available on how to explore technology as a potential proctoring solution. However, online learning is here to stay (Capranos et al., 2022) and the needs of institutions will vary.
Over the past 10 years at Oregon State University, I have worked with students, faculty and administrators on exam proctoring in higher education. I have also negotiated with proctoring vendors, led Oregon State Ecampus’ transition to new proctoring practices and consistently prioritized accessibility. Most recently, my focus has been on developing equitable online exam proctoring for diverse learners located around the world, facilitating approximately 50,000 exams in academic year 2022-2023.
Based on these years of experience, I created a framework for the process of examining technology as a potential proctoring solution — a framework that is focused on meeting students’ needs. This article will help you identify the goals of your institution and provide a framework for analyzing proctoring tools.
Types of proctoring
Here are brief descriptions of the most common types of proctoring for online exams and how technology is often used with each of them.
Automated proctoring
- Students download a web browser extension to their device.
- The device’s camera and microphone records audio and video during an exam session.
- The instructor may review the recording after it is filtered through automated proctoring or algorithmic software to determine and flag anomalies.
Online proctoring
- Students may need to log into a specific site where a live proctor at a different location monitors the exam session through their device’s camera and microphone.
- The exam session is recorded and may be reviewed by the instructor.
In-person proctoring
- Students go to a testing center in a local city where a person at that location monitors the exam.
- A computer is provided at the testing center for the student to use.
Each type of proctoring could potentially use a lockdown browser, which is a technology that restricts a test-taker’s ability to access other programs or navigate outside of the exam window during an exam session by ending an exam if that occurs.
Know your goals
Before diving into the world of technology solutions for online exam proctoring, consider these questions that can help you remain focused on the needs of your institution:
- What is the goal of using technology for proctoring? (e.g., reduce costs, increase efficiency or testing capacity)
- Why is technology needed to accomplish this?
- How do you want your staff, students, and instructors to interact with the technology?
Your responses to these three questions can guide you as you begin to explore all of the technology available for proctoring online exams. For example, if you determine your major goal is to reduce costs for your students and increase affordability, you can quickly weed out options that either require students to pay or are over budget.
P.A.W.S.
When exploring technology as a proctoring solution, some of the most important things to consider can be summarized as P.A.W.S. (Pricing models, Accessibility, Workflow/integration, Support/customer service).
Pricing models
Pricing models are often separated into two main categories: institution-pay and student-pay. There is not one model that is guaranteed to work best for everyone, so determining what will work best for your students and your institution is key.
- Institution-pay is often categorized as institutions paying for proctoring per exam, per user, or through a site license. It is important to consider the number of students who will be using the technology and the number of exams it will be used for to determine what would make sense for your institution. For example, if you have a low number of students in courses with a high number of exams, comparing the cost per student versus per exam may be helpful. If you have many students taking a low number of exams, then a per-exam pricing model may work best. However, if you have many students taking a high number of exams, a site license might be the most economical solution.
- Student-pay is often categorized as students paying for proctoring per exam or with a flat fee. Consider the parameters for a flat fee rate to determine whether it is per term or yearly. This option is less likely to be beneficial for students unless you are certain they will remain enrolled during the entire time frame (e.g., the academic term). Additionally, you should consider the average number of exams in a course.
Keep in mind that using a student pay model increases the additional costs of education for students. If affordability is a top priority, I caution against using a student pay model without a thorough examination of how it could impact your students financially. It is important to consider the diverse needs of online learners who are typically older (Venable, 2022a), balancing work, their education and family responsibilities (Aslanian et al., 2022; Venable, 2022b).
One solution could be to provide two options: a free proctoring technology option and a student-pay option, so students can choose which approved method will work best for them.
Accessibility
Many vendors will indicate that a product is accessible; however, the definition of accessibility differs across companies, geographic locations and industry standards. Additionally, one company’s idea of an accessible product might not align with your institution’s commitment to accessibility for students.
The following questions can be helpful to evaluate the accessibility of technology beyond minimum expectations.
- What makes it accessible? Ask vendors how a technology meets the minimum guidelines and standards required to be considered accessible. Meeting these requirements should be considered the baseline. During a demonstration, it can be helpful to request the vendor show how a technology exceeds these standards. A good way to approach exploring the accessibility of a technology is understanding that even an accessible technology can become inaccessible in certain situations given the variability of technology, learner ability and instructor exam settings.
- Does it work with the assistive technology your students might use? It is extremely important to consider how proctoring technology will interact with assistive technology used by your students. Is the technology compatible or would there be barriers for use? Are there specific settings or methods of use that would determine whether it would be compatible? For example, does the technology allow a student to use their computer screen reader or does it depend on what settings are used for the exam?
- Does it create any barriers for your students? If the technology works with computer screen readers, ensure it works with multiple types of computer screen readers and those available at your institution. Otherwise, you may inadvertently be creating a barrier to an accessible product.
- Are there multiple ways to get support? Ensure support is available to students through multiple methods. Consider the various ways students communicate and prefer to receive assistance (e.g., email, 24/7 live chat, telephone). A greater number of support options might meet the needs of more students; however, be sure those options are accessible.
Workflow/integration
Workflow/integration for new technology can have both short-term and long-term effects on test-takers, faculty and staff, requiring a thorough examination when considering technology for proctoring solutions.
- How would integrating the technology impact workflows? Does the technology integrate with your current learning management system (LMS)? If it does, to what extent? Would it streamline workflows by using exam information already in the LMS or shift workloads by requiring an individual to input exam information into a separate platform? Identify the potential benefits and barriers of implementing any new technology and how all users would interact with it.
- What training would be needed? Be sure to consider the training needs of all who would use the technology, including students, faculty, and staff. Do you have the capacity to deliver this training? Does the technology provider offer trainings or resources that could help address these needs?
- Where are exam sessions stored? Privacy and security of data collection have become priority topics in the field of online proctoring. Technology providers should be able to tell you how they meet or exceed federal guidelines and industry standards for privacy and data security. Consider what data is collected, where exam sessions are stored, how long they are retained, and who can access them.
Support/customer service
Consider the type of support each technology provider offers test-takers, faculty members, and staff members. For example, if you have asynchronous exams, support may be needed at different times than it would be for synchronous exams. Your test-takers and faculty members may both need 24/7 access to support.
- Customer satisfaction ratings should be critically examined to understand how a proctoring vendor balances customer service with exam integrity. As mentioned in a forthcoming piece by McEdwards and Underhill, online proctoring vendors are not held to academic regulations and conventions for transparency in collecting, analyzing, and reporting data. For example, some vendors might collect data about student views only at the end of exams, missing valuable insight from students who did not complete their exam.
This framework is a starting point to assist you with analyzing the exam proctoring needs at your institution and assessing technology as a potential proctoring solution. I hope you will find these suggestions helpful as you consider how to explore technology as a proctoring solution for online exams.
References
Aslanian, C. B., Fischer, S., & Kitchell, R. (2022). Online college students report 2022. EducationDynamics.
https://insights.educationdynamics.com/2022OnlineCollegeStudentsReport.html
Capranos, D., Dyers, L., & Magda, A. J. (2022). Voice of the online learner 2022: Shifting preferences in post-pandemic online learning. Wiley University Services. https://universityservices.wiley.com/voice-of-the-online-learner-2022
McEdwards, T. & Underhill, G.R. (2024). Online students’ preferences for features and types of exam proctoring [Manuscript submitted for publication]. Ecampus, Oregon State University
Venable, M. A. (2022a). 2022 online education trends report. BestColleges.com.
https://www.bestcolleges.com/research/annual-trends-in-online-education
Venable, M. A. (2022b). 2022 trends in online student demographics. BestColleges.com.
https://www.bestcolleges.com/research/online-student-demographics
