Category Archives: Research

Valuable Skills for Online Students and Online Instructors

Posted on by | Leave a reply

By Cat Turk and Mary Ellen Dello Stritto

In this time of rapid change in online education, we can benefit from leveraging the expertise of faculty who have experienced the evolution of online education. At the Oregon State University (OSU) Ecampus Research Unit, we have been learning from a group of instructors who have taught online for ten years or more. A review of recent research uncovered that these instructors are an untapped resource. Their insights can provide valuable guidance for instructors who are just beginning their careers or instructors who may be preparing to teach online for the first time. Further, their perspectives can also be enlightening for online students.

In 2018-2019 we conducted interviews with 33 OSU faculty who had been teaching online for 10 years or more as a part of a larger study. Two of the questions we asked them were the following:

  1. What skills do you think are most valuable for online instructors to have?
  2. What skills do you think are most valuable for online students to have?

We will share some of the results of a qualitative analysis of these questions and highlight the similarities and differences.

When asked about the most valuable skills for online instructors, three key skills emerged: communication, organization, and time management. When asked about the most valuable skills for online students to have, the same skills were among the most frequently mentioned by these instructors.

As the table below shows, in the responses about skills for online instructors, communication emerged as the most prominent skill, with 85% of instructors in the study emphasizing its importance, while time management and organization were split evenly at 45%. In their response about skills for students, 64% of the instructors emphasized both communication and time management, while 42% discussed organization. When discussing communication for instructors, they indicated that effective communication is essential for building rapport with students, providing clear instructions, and facilitating meaningful interactions in the online environment. Organization (such as structuring course materials or their weekly work process) and time management skills (such as scheduling availability to connect with students) were also highly valued by these instructors. Read more about the analysis of instructor skills here.

 Skills for InstructorsSkills for Students
Communication    28 responses (85%)   21 responses (64%)
Time Management15 responses (45%)  21 responses (64%)
Organization15 responses (45%)   14 responses (42%)
Self-Motivation   —21 responses (64%)            
Frequency of responses of skills for instructors and students.

The responses to both questions emphasized the significance of communication skills in written assignments and in proactive connections within the scope of the online learning environment. Instructors articulated that online students needed to be proactive communicators themselves. Examples of this include contacting their instructors about questions and clarification in a timely way, interacting with their peers in a respectful manner, and turning in quality written assignments that demonstrate comprehension of their learning material. For students, clear and effective communication ensures understanding and engagement, while organization facilitates seamless navigation through course materials, and time management ensures that students are able to make the most of the asynchronous environment.

While time management and organization were both considered by instructors to be just as crucial for students, their responses demonstrated that these skills were needed for different reasons than would be the case for instructors. Instructors personally valued time management and organization due to the nature of facilitating courses online. When the online classroom can travel from place to place, setting blocks of intentional time and structuring hours accordingly were considered essential to instructors for maintaining a work-life balance and so tasks would not be missed.

On the other hand, according to these instructors, students need time management and organization due to the asynchronous and sometimes isolating nature of online courses. One instructor stressed that:

 “[Students] do need to be more organized than on-ground students, because there’s not that weekly meeting to keep students on track.”

These instructors indicated some online students may need to structure their study time to accommodate a different time zone, while others may need to structure their academic pursuits around careers or children. Another instructor emphasized that:

“A lot of our [online students] actually work full-time, so they have families and kids and have to be much more organized too.”

While there were overlaps with the responses to the two questions, a notable difference was the emergence of another skill for students: self-motivation. This concept of self-motivation emerged from the instructor responses about students’ capacity to persevere in online courses. This included their level of motivation, capacity to learn on their own, and comfort with self-paced learning.

One instructor said the following about students’ self-motivation,

“Some people would say it’s self-discipline, but I think it’s more of they have to have a purpose for that class.”

Self-motivation was not mentioned by the instructors as a skill for online instructors, suggesting that these instructors perceive this as more pertinent to students for success in managing their own learning process. It is worth noting that proactive communication was highlighted as an essential aspect of self-motivation, with instructors emphasizing that students who take the initiative in reaching out to them tend to be more successful. This observation suggests that self-motivated individuals are more likely to actively seek support and clarification, which can enhance their learning experience and overall success. 

Another noteworthy aspect was the need for students to be comfortable with learning in physical isolation. Instructors acknowledged that online learners must navigate the challenges of studying independently without the immediate presence of peers and instructors. For online students specifically,

“They need to be motivated because they’re not going to have peers sitting in a classroom with them, and they don’t have a place that they have to physically go every week.”

This finding underscores the importance of maintaining motivation and engagement, as students ideally possess an intrinsic drive to succeed despite the absence of a physical connection to the university and their classmates.

The findings from this study highlight the importance of certain similar skills for both online instructors and students. Effective communication, organization, and time management are vital for success in the online learning environment for both instructors and students. We found this to be an interesting connection that online students might benefit from understanding: these are key skills that students and instructors have in common.

Our findings about self-motivation may be useful for online instructors. Consider incorporating strategies that foster student self-motivation, such as goal-setting exercises, regular check-ins, and providing opportunities for self-reflection. By empowering students to take ownership of their learning, instructors might enhance student engagement and success in the online environment.

Further, students can learn from the instructors’ emphasis on communication, organization, and time management skills. They can intentionally work on improving their communication skills, seeking clarification when needed, and actively participating in online discussions. Developing effective organization and time management strategies, such as creating schedules, prioritizing tasks, and breaking them down into manageable chunks, may significantly enhance their online learning experience.

The field of online education is evolving rapidly, and here we can see how educators and students alike are adapting to these changes. The experiences of long-term online instructors provide valuable insights into the skills necessary for success in the online learning environment. In the future, what answers would we find if we asked students the same question: what do online students think are the skills needed to succeed in the online classroom? By understanding the shared and distinct perspectives of instructors and students, educators can design effective online courses and support systems that foster meaningful learning experiences and empower students to succeed.

Learning Outcomes: Do Students Even Read Them?

Posted on by | Leave a reply

Learning outcomes (LOs) are used in instructional design to describe the skills and knowledge that students should have at the end of a course or learning unit, and to design assessments and activities that support these goals. It is widely agreed that specific, measurable outcomes are essential for planning instruction; however, some educators question the benefits of explicitly presenting them to students. I have been asked (and wondered myself): “What is the point of listing learning outcomes in the course?” “How do they help learning? “Do students even read them?”

So, I went on a quest for research that attempted to answer such questions. I was particularly interested in unit/module-level outcomes, as those are the ones that directly steer the content, and students see them throughout the course. Here’s a brief summary of what I found.

Note: the studies use the terms “learning outcome”, “learning objective”, or “learning goal” – they all refer to the same concept: a specific and measurable description of the skills and knowledge that students are expected to have at the end of a learning unit/period of study. At OSU we use the term “outcomes”.

What Does the Research Say?

Armbruster et al. (2009) redesigned an Introductory Biology course at Georgetown University, Washington, DC, using active learning and student-centered pedagogies, leading to increased student performance and satisfaction. One of the strategies used was to include explicit learning goals in the lecture slides, and labeling exam and quiz questions with the related goals. Students’ attitudes towards the course were assessed via a questionnaire and comparison of university-administered student evaluations. Students were asked to rank lecture components in terms of helpfulness to learning, and the authors found that one of the highest-ranking elements was the inclusion of explicit learning goals.

Simon and Taylor (2009) surveyed 597 students from computer science and microbiology and immunology courses at the University of British Columbia, where instructors presented learning goals at the beginning of each lecture or topic area. The questions were open and the answers coded into a number of categories, which helped them identify several values of goals. The main value was “knowing what I need to know”: students reported that the goals showed them how to focus their efforts and felt that the goals “allowed them to organize the information more effectively and be more expertlike in their approach to the class” (Simon & Taylor, 2009, p.55). The authors did not find any difference between presenting the goals before each lecture versus at the beginning of the unit/topic area.

Brooks et al. (2014) examined students’ views of learning outcomes at the University of Leicester, UK. First, they surveyed 918 students taking Biological Sciences, English and Medicine courses. They found that 81% of participants agreed or strongly agreed that learning outcomes are useful learning aids. Additionally, 46% found LOs more useful as their courses progressed, and 49% reported that they engaged more with the LOs as the course progressed. The authors also investigated when LOs are most useful, and found that the most common answer (46%) was when reviewing the material. Moreover, 49% of students reported that LOs can only be fully understood at the end of a module. The researchers followed up on these results with a focus group, which confirmed that students use LOs in various ways and at various points during the course.

Osueke et al. (2018) looked into students’ use and perceptions of learning objectives at University of Georgia. 185 students in an undergraduate Introduction to Biochemistry and Molecular Biology course took part in the study. The instructors included instructions in the syllabus, which they also stated on the first day of class: “Focus on the learning objectives. The exams will assess your accomplishment of the learning objectives. Use the learning objectives as a guide for what to focus on when you are completing assignments and studying for exams.” Students completed two assignments requiring them to explain their use of the LOs. The researchers found that many students (33.8%) reported they had been instructed on how to use LOs to study – these instructions ranged from passively “look over” to using them as a study guide. The ways students used the LOs were: as questions to answer (47.4%), as a resource for studying (24.1%), as a self-assessment tool (14.3%), and passive use (13.5%). When asked why they find the LOs helpful, students said that they help them: narrow down the information (57.1%); organize their studying (23.3%); communicate information (5.3%); monitor their understanding (4.5%); forced them to study (1.5%).

Sana et al. (2020) conducted three experiments aiming to find to what extent presenting the LOs improve retention of information. Participants were asked to read five passages on a neuroscience topic, and then they were tested on comprehension and retention. The experiments took place at McMaster University, Ontario and employed different participants, methods, materials, and procedures. They found that: interpolating LOs throughout the lesson (as opposed to all LOs presented at the beginning) improved learning compared to not including LOs, especially when students’ attention was explicitly directed to them; converting LOs into pretest questions (that students attempted to answer) further enhanced performance; multiple-choice and short answer questions were equally effective; and withholding feedback on pretests was more effective than providing feedback – the explanation proposed by the authors for this last finding was that students may be more motivated to seek the correct answers themselves, which causes further processing of the material.

Barnard et al. (2021) investigated students’ and academics’ perspectives on the purpose of learning objectives and approaches to assessment preparation. They conducted focus groups with participants from an undergraduate Psychology course at the University of Nottingham, UK. The students reported that LOs are useful for guidance, as they “use them to create direction for some of the learning and revision strategies” (Barnard et al., 2021, p. 679).

Conclusions and Recommendations

Good news! The findings of these studies suggest that many students do appreciate clear LOs and use them to guide their learning. The LOs help them understand what they are expected to know – thus, students use them to focus their study, to review for an exam, and to self-check their knowledge.

As instructors and instructional designers, what can we do to help students take full advantage of LOs? Apart from having specific and measurable LOs, make sure that the LOs are well aligned with the activities, and make this alignment explicit. It may also be helpful to offer some guidance on how to use the LOs, for instance by prompting students to recap their learning at the end of a unit based on the LOs. Finally, we could turn the LOs into questions and use them as a pretest.

For more on creating and using LOs, check out the CBE—Life Sciences Education website, which has an informative guide, including a section on student use. 

Do you have any other ideas or resources on how to use learning outcomes to improve students’ experience and study habits? If so, we’d love to hear from you!

References

Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered pedagogy improve student attitudes and performance in Introductory Biology. CBE Life Sciences Education, 8(3), 203–213. https://doi.org/10.1187/cbe.09-03-0025

Barnard, M., Whitt, E., & McDonald, S. (2021). Learning objectives and their effects on learning and assessment preparation: Insights from an undergraduate psychology course. Assessment and Evaluation in Higher Education, 46(5), 673–684. https://doi.org/10.1080/02602938.2020.1822281

Brooks, S., Dobbins, K., Scott, J. J. A., Rawlinson, M., & Norman, R. I. (2014). Learning about learning outcomes: The student perspective. Teaching in Higher Education, 19(6), 721–733. https://doi.org/10.1080/13562517.2014.901964

Osueke, B., Mekonnen, B., & Stanton, J. D. (2018). How undergraduate science students use learning objectives to study. Journal of Microbiology & Biology Education, 19(2). https://doi.org/10.1128/jmbe.v19i2.1510

Sana, F., Forrin, N. D., Sharma, M., Dubljevic, T., Ho, P., Jalil, E., & Kim, J. A. (2020). Optimizing the efficacy of learning objectives through pretests. CBE Life Sciences Education, 19(3), ar43–ar43. https://doi.org/10.1187/cbe.19-11-0257

Simon, B., & Taylor, J. (2009). What is the value of course-specific learning goals? Journal of College Science Teaching, 39(2), 52–57. Retrieved from: https://www.colorado.edu/sei/sites/default/files/attached-files/what_is_the_value_of_course-specific_learning_goals.pdf

Assessment Design Considerations for Ungrading

Posted on by | Leave a reply

Over the last couple of terms, I joined a series of reading sessions with instructional design colleagues to read Alfie Khon and Susan Blum’s book Ungrading. Why Rating Students Undermines Learning (and What to Do Instead) and discuss the practices and implications of this approach to reconceptualize assessment design and the place of grading. This two-part blog aims to capture the takeaways from those discussions including the main concepts, approaches, types of activities, implications, and challenges of adopting ungrading practices. This first part of the blog covers a brief overview of the concept of ungrading, its major benefits, and design considerations; and the second blog will include a summary of the types of ungrading practices and challenges to implementation ––all derived from the authors’ extensive arguments and examples. For a detailed review and summary of the book chapters, you can also check the blog Assessment Design: Ideas from Ungrading Book.

Overview of Ungrading

The concept of ungrading is sparking widespread interest only recently even though educators have been studying and using ungrading approaches for quite some time. The foundational premise of ungrading is to move away from a focus on grades that judge, rank, sort, and quantify student learning to adopting an approach that focuses on using alternative and authentic means to assess learning such as self-evaluation, reflection, student-generated questions, peer feedback, to name a few. Along with that premise is the questionable ranking that comes with grading which makes students compete with one another in an artificial way. Sorensen-Unruh (chapter 9) sees ungrading as a conversational method that facilitates the communication between instructors and students about how students perform in the class. If, as underscored by the authors, grading and the fact of assigning point values to students’ performance makes more harm than good, then, why use grading? Considering that grading is rooted in our educational systems, many of these authors conclude that it becomes inevitable to grade student learning as it is currently done today.  

A clock and a checklist

Several scholars and instructors consider grading to be problematic. First, grades are not good indicators of learning. Blum (chapter 3) argues that grading assumes all students are the same, does not provide accurate information about student learning gains, is consequential, adds fear and avoidance of negative consequences, and is arbitrary and instructor-led. Second, the overemphasis on grades can lead to a decrease in intrinsic motivation, students’ excessive anxiety, and the complexity of quantifying how learning happens (Stommel, chapter 1). Third, it can also decrease interest in learning, students may feel inclined for easier tasks, and critical thinking is lacking (Alfie Khon, foreword). Fourth, grading makes students be fixated more on their grades than on the process of learning, leading them to believe that grades are all that matters in school (Khon & Blum; Talbert, 2020). And finally, too much focus on grades can be detrimental to students’ mental health (Eyler, 2022). However, ungrading does not mean dismissing grades altogether. Instead, Stommel proposes creating a learning space that fosters critical thinking, reflection, and metacognition– all skills that are valuable for 21st-century education. Likewise, Alfie Khon contends that grading can be participatory since it does not require a unilateral decision, and thus, students can also propose their own grades (with the instructor’s reservation to accept them). 

“Ultimately ungrading— eliminating the control-based function of grades, with all its attendant harms— means that, as long as the noxious institutional requirement to submit a final grade remains in place, whatever grade each student decides on is the grade we turn in, period.”

(Khon, 2020, p. xv)

While ungrading may be an innovative approach to assessments, it should be thought of carefully and adopted with a clear objective. Ungrading, as pointed out by Katopodis and Davison (chapter 7), needs a structure to be effective, allowing students to envision themselves as authoritative, creative, confident, and active, thus achieving a high impactful goal. As ungrading requires instructors to evolve in their approach to assessment, it does too for students who are expected to engage in a process of self-evaluation, self-assessment, and reflection. This requires engagement in metacognitive practices that many students might not be ready to embark on or don’t know how to do it. In addition, while ungrading is believed to be student-centered, it can deepen equity gaps if guideposts are entirely removed. Sorensen-Unruh (chapter 9) believes that ungrading is a matter of social justice –going beyond the expected student agency and aiming at having students exercise their voice and participate in assessment decisions. 

As a whole, Blum (introduction chapter) provokes us all to rethink the nature of grading considering that students’ learning conditions vary, with many enduring inequities at many levels. Blum wants us to keep focused on how “varying assessment and feedback methods contribute to the real learning of real individual learners, rather than imposing an arbitrary method of sorting.” (p. xxii); all for the sake of healthy learning. 

These are a few key points about the arguments for ungrading. While this assessment practice is taking force in higher education, there are also many critics and skeptics. The purpose of this blog is not to enter into the discussion and controversy of ungrading, but to share a few perspectives and takeaways after an intense and well-structured book club discussion. In the following section of this part-one blog, I will share considerations for designing for ungrading. 

Assessment Design Considerations

Tasks and activities from a laptop computer

The educational system requires all instructors to submit grades at the end of every term. There is dissatisfaction with the current grading practices among many instructors and students as explained at large in the book. Here is where the ungrading movement takes force to provide alternative ways to account for evidence of student learning. Riesbeck (chapter 8) argued that by implementing ungrading practices, students can focus more on the content and feedback than on the grades. The use of critique-driven learning allows for more easily quantifiable efforts, progress, and accomplishment. Each ungrading consideration is dependent on a myriad of factors that may or not apply to each instructor’s context. The bottom line in ungrading is re-envisioning the teaching and learning process, engaging students in active learning, and active self-assessment through feedback. The following are design considerations:

Decenter grading and communicate (un)grading practices

Instructors can encourage students to focus on the process of learning, instead of talking about grades, Blums says, we should talk about the purpose and goals of the activities with students. These conversations can help develop relationships with students to encourage them to own their learning and have a voice in that process. Decentering grades also involves having an ongoing conversation with students, colleagues, and administrators about assessment decisions. Although each instructor exercises their academic freedom, it is also essential to share assessment practices that work and possible changes to implement.  In these conversations, it is also important to carefully use language that conveys a clear understanding of the concept and practice of ungrading to avoid confusion, anxiety, misunderstandings, and reactions that prevent its implementation. Having these kinds of conversations can help shift the mind from a grade-focused to a learning-focused approach. A key element in these conversations is to ensure that the pedagogical reason behind the adoption of ungrading practices is not only clear but well understood (and this may take time).

Set a structure for ungrading

As with other elements of exemplary course design considerations, the structure of assessment practices is necessary. Adding a structure for ungrading assignments gives students a clear objective, steps, and flow that allow them to be consistent and accountable to their own learning goals and strategies. 

Reflect on pedagogical and assessment practices

Instructors are invited to examine more in-depth their grading policies, why they grade in the way they do, what they are grading, and how they grade. In many cases, the path to ungrading is a response to dissatisfaction with grading policies. Aaron Blackwelder (chapter 2) says that, over time, he turned into a gatekeeper; he lost focus and was more interested in meeting institutional and “rigor” requirements than building relationships with students. His students had turned into competitive grade seekers. He questioned what a grade really suggests and posits that grades fail to communicate learning. The fact that grading allocates a specific number or letter that can bring some negative feelings to students, can also negatively affect the potential for learning. Sackstein (chapter 4) calls for a change in mindset to identify the way in which learning can be communicated and understood beyond the traditional use of numbers and letters.  While Blum also argues for assessing the entire learning experience (with portfolios, for example), Sacksatein suggests considering changes in the language of grading which can provide students with an opportunity to shift the way they feel and think about their own learning. 

Teach students to view mistakes as a necessary step in the learning process

Instructors are invited to reflect on how traditional grading practices are punitive, dehumanizing, and demotivating. Gibbs (chapter 6) points out that a system that punishes students for making mistakes reinforces the notion that all learning is flawless and therefore mistakes need to be avoided. Ungrading, on the other hand, aims to implement and cement the idea that learning is a process that needs constant feedback for that learning to be consolidated. Therefore, students need to be given opportunities not only to learn from their mistakes but to act on them in an interactive way. This requires instructors to plan for assessments that include steps for review (e.g., self, peer) to help the student build their skills, and knowledge over time.

Care for students and their learning

Instructors are also invited to demonstrate more explicitly that they care and validate students’ work. Further, Gibbs (Chapter 6) argues that her teaching philosophy is better summarized by the word “freedom”, the freedom that learners have to learn and grow at their rhythm and the freedom to make mistakes and learn from them. The role of the instructor is then to be supportive in that process through feedback and empathy. Ungrading, as it is overall discussed throughout the book, does not mean that there are no assessments or grading at all. On the contrary, the assessments should focus on helping students build their knowledge and understanding in less stressful ways, allowing students to build learning habits, develop creativity, become better communicators, and connect to their lived experiences and contexts. Caring for students also involves valuing their identity as learners and what they bring into the learning environment.  

Be aware that ungrading can increase student anxiety and uncertainty

It is critical that instructors who are considering ungrading be cognizant that it involves a high level of anxiety and uncertainty on the part of students. Let’s recognize that students are so used and “conditioned” to grades that they will find it confusing not to have a grade associated with each assignment in the course. Many students consider being successful if they score a perfect grade which can be overwhelming and obscure the value of learning. Instructors who adopt ungrading should explain why and how ungrading will be done in certain classes. This will add transparency to the expectations and assumptions that instructors have about students.

Implement student voice and choice supported with personalized feedback

Instructors can help students take ownership of their learning through hands-on, real-life activities that allow students to use the content they are learning in projects of their interest, conduct research, and solve problems. Students can choose their topics and projects and the instructors can guide them to narrow topics and ensure the projects are feasible within the course timeframe. Consider feedback as a formative assessment approach that enables students to make choices about their learning strategies and needs to improve their learning tasks. Sackstein (chapter 4 ) suggests teaching students to collect feedback and identify the strategies that work for different kinds of assignment revisions. This way, students can develop better strategies that move them from lower-thinking to higher-thinking processes. 

Since ungrading promotes the use of student-self assessment and reflection practices, it implies that instructors will need to personalize and tailor feedback to meet students where they are. In addition, instructors can consider setting a culture of feedback (Gibbs, chapter 6) where instructors teach students to use feedback to improve their work, provide peer feedback effectively, and see the value of learning from their mistakes. 

Promote peer support

Authors of several chapters in this book have posited that students are more likely to give each other better feedback in the absence of grades. This kind of feedback can allow students to help each other, learn from one another, expand their awareness of their own understanding, and develop skills for life. Peer support will also help students build their confidence and autonomy to learn from each other. 

Trust students

One critical aspect of assessment is trust –trust that students do the work they are expected to do by themselves. Instructors have legitimate reasons to express their concerns and create course policies about academic integrity that lead them to adopt plagiarism systems and surveillance tools to monitor and proctor students’ work. In adopting ungrading, trust is fundamental to change the way learning and performances are assessed. It involves helping students think differently about what it means to learn. Instructors can help students evolve in their approach to learning to move away from grades to focus on their learning by including in assessments strategies for building capacity for metacognition, confidence in their skills, life-long learning goals, and owning their learning. 

Ungrading does not mean instructors don’t grade and students don’t receive grades on their work. Ungrading, as posited by the authors in the Ungrading book, is a mindset to approach student learning differently. In the second part of this blog, I will share the types of ungrading practices, implications, and challenges as presented in the book. 

References

What is the Future of Online Education? Perceptions of Long-term Online Instructors

Posted on by | Leave a reply

The changes in higher education precipitated by the COVID-19 pandemic have reignited questions and misconceptions about online education.  This is a time that we should draw on the insights and experience of online faculty. At Oregon State University (OSU) we have a significant number of faculty who have been teaching online for over a decade. In the 2018-2019 academic year, the Ecampus Research Unit interviewed 33 OSU instructors who had taught online for 10 years or more. In a series of interviews, the instructors were asked to reflect on their experiences as an online educator and how their perspectives have changed over time. More information about the broader study can be found on the study website. The final question asked of the instructors was, “What do you think is the future of online learning?” We conducted a qualitative analysis of their responses to this question. The findings were recently published in the Online Journal of Distance Learning Administration. Below, we discuss some of the key findings from this analysis.

Key finding #1: Online and blended learning will continue to grow

Two-thirds (22) of the instructors expected online learning to expand as higher education moves toward increased access and accessibility, and as employers show increased expectations of continuing education. They acknowledged that online learning would continue to be the choice of adult learners as they balance work and life responsibilities.

Key finding #2: Online learning will increase access and accessibility

More than half (18) of the instructors predicted that online learning would increase access to education. These instructors discussed how online learning increases accessibility because online courses can be taken anywhere (location flexibility) and online courses can be accessed anytime (time flexibility). While these instructors were interviewed before the COVID-19 pandemic, their responses are now particularly timely and relevant, as the pandemic shifted higher education’s focus to remote and online teaching.  

Key finding #3: Will online learning replace brick and mortar institutions?

One third (11) of the instructors discussed the possibility that online learning may grow to become the primary modality used in higher education, replacing face-to-face learning.  However, 13 instructors indicated that they did not think the face-to-face learning should be eliminated in the future. Many of these instructors hoped that online education could provide more options for students rather than replacing brick and mortar institutions.

Key finding #4: Technology development will increase

Nearly 40% of the instructors (13) discussed the role of technology development in the future of online education. Acknowledging that the development of technology has already made teaching online easier and more effective, many optimistically predicted this would continue to improve the teaching and learning experience. Others were more pessimistic about technology replacing elements like face-to-face communication.

Overall, instructors’ ideas of the future aligned with some themes in the broader field of higher education, such as diversity, opportunity, and access. These key findings have implications for the professional development of online instructors. As more faculty transition to online teaching, it is important that they be well prepared for the online learning landscape. As the population of students in online education continues to evolve, it is also important that instructors understand the diversity of their students and the needs of adult learners. As technology is rapidly changing, timely and accessible training that can be used across multiple modalities is needed for future faculty development. Enhancing instructors’ pedagogy and technology skills across a range of modalities will enhance the educational experience for online learners around the globe.

Assessment Design: Ideas from Ungrading Book

Posted on by | Leave a reply

A group of instructional designers at Ecampus participated in a book club reading “Ungrading” (Kohn & Blum, 2020). We learned many creative ways of designing assessments through participation in this book club. If you happen to be searching for ideas on designing or re-designing assessments in your teaching, we would highly recommend this book!

The idea of “Ungrading” may sound radical to many of us. Yet instructors at all types of educational institutions have tried ungrading in many different courses, ranging from humanity courses, to STEM courses, and from primary education to higher education. Starr Sackstein (author of Chapter 4 “Shifting the Grading Mindset” of the book) encourages educators to consider “ways to adjust small things in the classroom that will lead to important growth for students”. And this suggestion of starting small is coherent with what James Lang proposes in his book “Small Teaching” (Lang, 2016) and Thomas Tobin’s +1 strategy for implementing new teaching and learning strategies (Tobin & Behling, 2018). Sackstein provides a table comparing the grades vocabulary that focuses on judgement or criticism, with the non-grade vocabulary focusing on assessing and opportunity for improvement.

In chapter 5, Arthur Chiaravalli proposed a way for teaching without grades: Descriptive Grading Criteria, such as A for outstanding, B for Good, C for Satisfactory and I for Incomplete. Do you remember elementary school report cards that use E for Excellent, S for Satisfactory, and NI for Need Improvement type of categories? I think that is exactly what descriptive grading criteria represent. 

In chapter 7, Christina Katopodis and Cathy Davidson offer a new approach to start a new term/semester by asking students:” What is Success in this class for you? And How can I help you achieve it?” (p. 107) Katopodis and Davidson also remind us the importance of explaining why when you challenge your students to take their own learning seriously and give students opportunities for metacognitive reflections about the learning activities themselves. Katopodis and Davidson also offer a model of contract grading for Twenty-First Century Literacies and a model of collaborative peer evaluation. Students’ grades in the course come from self-and-peer evaluations using detailed evaluation forms. 

In chapter 8, Christopher Riesbeck described his critique-driven learning and assessment design of do-review-redo submission process for his intermediate-level programming course. I have used similar approach in my own teaching before and it works very well for any course with manageable number of students. The advantage for this approach is every one of your students can improve their first submissions based on feedback they receive from the instructor. The disadvantage for this approach is the potentially extended time instructors may spend on providing the feedback and reviewing the submissions and re-submissions. The key to this assessment method is making sure that the workload of providing feedback and reviewing revisions is manageable. In chapter 9, Clarissa Sorensen-Unruh provided her experience of using ungrading in her organic chemistry II course, giving students opportunities to practice evaluating their own work.

And that is only snippets of what I took away from a few chapters from this book. Many resources about ungrading outside the book were shared during our book club meetings, such as two-stage exams, group exams  and public exams. To answer a common question that ungrading practices may fit humanity courses more easily, Cyndie McCarley shared “Grading for Growth” blog written and maintained by two math instructors Robert Talbert and David Clark. To learn about all the creative assessment design methods introduced in this book, read it yourself either through library ebook or get a hard copy and enjoy reading, designing and experimenting! 

References

Kohn, A. and Blum, S. (2020). Ungrading. West Virginia University Press. 

Lang, J. (2016). Small Teaching. Jossey-Bass. 

Tobin, T.J. and Behling, K.T. (2018). Reach Everyone, Teach Everyone. West Virginia University Press. 

Social Annotation as a Learning Tool

Posted on by | Leave a reply

“Annotation provides information, making knowledge more accessible. Annotation shares commentary, making both expert opinion and everyday perspective more transparent. Annotation sparks conversation, making our dialogue – about art, religion, culture, politics, and research – more interactive. Annotation expresses power, making civic life more robust and participatory. And annotation aids learning, augmenting our intellect, cognition, and collaboration. This is why annotation matters.” 

                                          -Kalir and Garcia

Annotation 

When you think back to your early college years, you may remember your professor assigning a text to annotate. Annotating a text has long been a common task in higher education, one that ideally promotes deeper reading, interaction with, and comprehension of important texts. Annotation assignments vary widely but the traditional paper-based type of annotation asks readers to respond to a text as it is read, physically marking or highlighting the text itself and perhaps writing in the margins. This approach allows students to enter into a dialogue with the text by recording their responses to the text, adding reflections or critiques, and anchoring those reactions to a specific place in the text. When students annotate a text, they are working their way through skills that span the levels of Bloom’s Taxonomy, from remembering to predicting, connecting, analysing, and evaluating. Annotation, at its best, encourages active engagement with a text beyond the surface level, promoting deeper critical thinking and stronger retention of concepts. 

While this is, of course, fantastic individual practice, the nature of traditional annotation assignments is primarily solitary. Today’s classrooms place more of an emphasis on 21st century skills such as group and collaborative study, and new digital tools have been developed that have revolutionized what, how, and with whom we can annotate. So-called social annotation has picked up speed with the growing popularity of two major players, Perusall and Hypothes.is, bolstered by the sudden shift to remote learning in 2020. Online instructors seeking ways to replicate the back and forth, robust discussion of a face-to-face class have found these tools a fitting substitute, and the asynchronous format of the discussion means these tools have a place in all modalities. 

Equity, Inclusion, & Community

“As a teaching method, critical social annotation allows for equitable conversations to unfold in-line with the knowledge being presented in course texts. In this way, it can potentially subvert or even redress instances of inequity in course content.” 

                                                                                                         – Brown and Croft

Social annotation platforms increase equity and inclusion in a course in several ways. Digital annotation platforms offer students a variety of ways to connect with material, allowing students to post links, images, video, and more in response to the text, their peers, and other annotators. By putting students’ ideas front and center, social annotation can empower learners to take initiative and experience more feelings of control over their educational process. Unlike the fast paced back and forth of traditional face-to-face discussions, the nature of digital social annotation allows students more time to engage with the text and to take as long as they need to post and respond (within the assignment boundaries). Additionally, the major platforms discussed in this post feature easy-to-use controls that require little technical expertise to use. They also boast comprehensive accessibility features that combine to provide inclusivity to a wide range of student needs. 

Social Annotation as Collective Construction of Meaning

One major difference between today’s digital social annotation and traditional solitary practice is that when students in a particular class collectively annotate a text using one of the digital platforms available today, they are actively building knowledge and understanding as a group. By sharing the document for collective annotation, the act of annotating itself becomes a social activity and contributes to the interaction of individuals within the group.  Socially annotating a text is one of the best ways to encourage close and active reading skills, with many studies reporting higher levels of student comprehension of socially annotated materials. Students who collectively annotate a text are entering into an exchange of questions, opinions, perspectives, and reactions to a text, engaging in a discourse with their peers (and facilitator, usually) and by extension learning from and with them. The process of a social annotation assignment allows students to see knowledge creation in action and become co-creators. 

Use of social annotation in asynchronous online courses can also increase sociability within an otherwise geographically remote, disparate group of students. Asynchronous instructors sometimes struggle to provide opportunities for real social interaction and building of community given the limits of the modality. An often unstated goal of higher education is to socialize students to academic community norms, and social annotation allows students to experience and practice some of these. For example, students annotating collectively learn appropriate language for responding to peers’ ideas and criticisms, develop online academic social identities, and gain experience with navigating power dynamics within the higher education classroom. 

Ready to Try It Out?

Adding social annotation to a course involves matching the task to your learning outcomes, deciding which readings would be best suited to annotation, and choosing your online annotation platform. 

Assignments can be tailored to meet a variety of instructional purposes and goals:

  • Recognizing formatting of various documents
  • Providing background or contextual information 
  • Learning academic norms for responding to peers- supporting, agreeing, disagreeing
  • Drafting questions and responses that are rigorous and meaningful
  • Determining main points vs. supporting details
  • Distinguishing fact from opinion
  • Identifying themes, tone, biases, author’s purpose, rhetorical devices, etc.
  • Learning and practicing discipline-specific writing and reading conventions 
  • Connecting the material to the field of study, to their own practice, or to other course materials
  • Developing evaluative and analytical skills
  • Considering differing perspectives and viewpoints in constructing knowledge
  • Facilitating a deeper understanding of difficult passages

Some best practices to consider when using collective annotation online:

  • Remind students that they have already practiced annotation in their everyday lives (reading and making your own notes in your inherited cookbook, reading your teacher’s remarks on your essay, leaving comments on a colleague’s report)
  • Model annotation with a fun text first
  • Seed the reading with your own comments, questions, and notes to help guide students 
  • Situate the social annotation assignment within the context of the course and make clear the intentions you have for the activity
  • If the activity is to be graded, be sure students know the grading criteria, preferably by providing a rubric
  • Annotation assignments are ideal for small group activities, and some platforms automatically create groups
  • Be prepared to provide guidelines for behavior and etiquette among students and to need to enforce these guidelines if students step out of line
  • Monitor the discussion and provide nudges, likes, upvotes, or validations, and otherwise engage with the dialogue throughout the assignment
  • If the platform allows tagging, do so- students get notified when someone responds to their post or asks a question, a convenience which increases the likelihood of them returning to the assignment for further interaction 

Social Annotation Tools: The Major Players

Perusall (stand-alone site/integrated into various LMS, including canvas)

Perusall is a collective annotation platform developed by Harvard University following a major research initiative into online collective annotation. Perusall offers free accounts for teachers and students at the basic level with options for institutions to integrate the tool into their LMS. The platform allows educators to use Perusall directly for stand-alone courses and upload their own materials for annotation as well as partner with their textbook catalog to purchase and annotate textbooks directly. For integrated LMS users, Perusall offers seamless grade pass back and options for pass/no pass grading as well as a robust automatic AI grading system that saves time and effort. Some instructors have also used Perusall for peer review to great effect using student-uploaded documents. *recommended tool for Ecampus courses

Sample Perusall annotation assignment

Hypothes.is (Google Chrome browser extension)

Hypothes.is is a groundbreaking new tool that bypasses restrictions of the classroom and enables anyone anywhere to annotate any webpage via a unique delivery system- as a browser extension that creates a layer over any webpage. This open source, free tool can revolutionize how we view and interact with web pages as well as texts by allowing us to save our annotations privately as well as publicly, inviting the world at large to socially annotate with us. Hypothes.is is also available as an integrated tool in most LMSs. The company also hosts the AnnotatED community, a group of users that hosts events, studies, and conferences to learn best practices for the tool. *recommended tool for research with a wider audience.

Hypothes.is Introduction Video

Sources

Adapting to Disciplinary Literacy Conventions – Open English @ SLCC

Home : Hypothesis

Innovation | Cégep Vanier College

Kalir, Remi H., and Garcia, Antero. Annotation. United States, MIT Press, 2021.

https://www.google.com/books/edition/Annotation/ejoiEAAAQBAJ?hl=en&gbpv=0

Pedagogy | ANNOTATION TOOLS

Perusall

Social Annotation | Center for Teaching Innovation

Social Annotation – Pedagogical Support and Innovation Pedagogical Support and 

Social Annotation and an Inclusive Praxis for Open Pedagogy in the College Classroom

Tools for Social Annotation in the Digital Age

Impact the future of online education through actionable research

Posted on by | Leave a reply

Each year, the Oregon State University Ecampus Research Unit funds projects, up to $20,000 each, to support the research, development and scholarship efforts of faculty and/or departments in the area of online education through the OSU Ecampus Research Fellows program.

This program aims to:

  • Fund research that is actionable and impacts student online learning
  • Provide resources and support for research leading to external grant applications
  • Promote effective assessment of online learning
  • Encourage the development of a robust research pipeline on online teaching and learning at Oregon State

Fellows program applications are due Nov. 1 each year. If you are interested in submitting an application, reach out to Naomi Aguiar, the OSU Ecampus assistant director of research. Research Unit staff are available to help you design a quality research project and maximize your potential for funding.

Many Oregon State colleagues have had transformative experiences in this program.  A Fellows study funded in 2020 highlights the ways in which these projects have advanced research in online/hybrid education, as well as Fellows’ programs of research.

Fellows program highlight

Funding recipients expand the inclusivity mindset of computer science students

Lara Letaw, an experienced online instructor and lead researcher from Oregon State’s School of Electrical Engineering and Computer Science, partnered with Heather Garcia, an OSU Ecampus inclusive instructional designer on a research study called “Impacting the Inclusivity Mindset of Online Computer Science Students.”

Together with their team, Letaw and Garcia implemented an intervention that was designed to improve feelings of gender inclusivity among online computer science students and to train these students to develop more gender-inclusive software applications.

In this intervention, online computer science students experienced new curriculum developed by Letaw and Garcia’s team. The curriculum was based on GenderMag, a software inspection method for identifying and correcting gender biases in software. Curriculum for teaching GenderMag concepts can be found on the GenderMag Teach website. Students completed a set of assignments and, if they chose to participate in the research study, questionnaires about inclusivity climate, both in the course and in the computer science major. Students’ software design work was also evaluated for the use of gender-inclusive principles.

The image below shows examples of the cognitive facet values people (e.g., Letaw and Garcia) bring to their use of software, shown across the spectra of GenderMag facets (information processing style, learning style, motivations, attitude toward risk, and computer self-efficacy).

examples of the cognitive facet values people (e.g., Letaw and Garcia) bring to their use of software, shown across the spectra of GenderMag facets (information processing style, learning style, motivations, attitude toward risk, and computer self-efficacy).

Computer science students in the Ecampus courses Letaw and Garcia modified learned about their own cognitive styles and those of their teammates. They also built software that supports the cognitive diversity of software users. One student reflected, “Identifying my facet values was tremendously helpful [for articulating what had] been abstract… I feel much more confident.”

The results of their study showed that, overall, students felt included by the GenderMag curriculum (nobody felt excluded by it), it increased their interest in computer science, and it had positive effects on their team dynamics and self-acceptance. Students who completed the GenderMag intervention were also more effective in developing gender-inclusive software designs, and they reported greater recognition and respect for the diversity of software users.

The image below highlights what students considered when designing a software user interface before (left) and after (right) learning GenderMag concepts. As one student put it, “Now when I think of users using a piece of software I don’t picture them … just jumping in and tinkering … I am more aware that there are [people whose] interests in using a software … might not align with mine.”

what students considered when designing a software user interface before (left) and after (right) learning GenderMag concepts

As a result of this project, Letaw and Garcia published a paper in the ACM’s International Computing Education Research conference proceedings in 2021. This project contributed to a $300,000 National Science Foundation grant awarded to Oregon State’s Margaret Burnett, Letaw, and Kean University. With this funding from the NSF, they will partner on a project entitled, “Embedding Equitable Design through Undergraduate Computing Curricula.”

This Fellows project has also provided research opportunities for two female Ecampus computer science students (Rosalinda Garcia and Aishwarya Vellanki), a group that is typically underrepresented in STEM fields. Rosalinda Garcia successfully defended her honors thesis with these data in the spring of 2021, and Vellanki is currently working on her own.

Join the Ecampus Research Fellows Program

Learn more about the Fellows Program and what materials are needed to prepare your proposal.

Instructional Designers and Research

Posted on by | Leave a reply

Last fall, my colleague featured the Ecampus Research Fellows (ECRF) program in her blog post. The ECRF program, which began in 2016, funds OSU faculty-led research on online and hybrid education. Each year, approximately five projects are selected to receive funding. One unique aspect of the program is that, in the past few years, 1-2 members of the Ecampus Course Development and Training (CDT) team are paired with the faculty on funded research projects. The CDT team includes instructional designers and media developers. These professionals have expressed interest in conducting research, but in most cases, have had few opportunities to engage in formal research projects. Similar to faculty, CDT fellows have to apply to the ECRF program.

For this blog post, I’d like to share some takeaways from my experience as a CDT research fellow, as well as some takeaways my CDT colleagues have shared with me. I will also share some feedback from faculty fellows who have had CDT colleagues join their research teams. But before I dig into these valuable takeaways from past participants, let me first address the importance of this program for instructional designers and related disciplines.

In 2017, the Ecampus Research Unit published a report titled “Research Preparation and Engagement of Instructional Designers in U.S. Higher Education.” This report was the result of a national study of instructional designers working in higher education environments. Among the many findings of this study, one compelling finding was that more than half (55%) of respondents indicated that instructional designers need more training in research methods to fulfill their role. Instructional designers also indicated why they think it is important to gain more experience in research. Among the reasons, respondents indicated that research skill development would allow them to grow professionally, further their discipline, better understand the needs of students and faculty, and collaborate with faculty.

The Ecampus Research Unit (ECRU) answers this call through their CDT research fellows program.

In the summer of 2020 at the NWeLearn conference, three CDT fellows reflected upon their participation in the program, sharing valuable insights and experience. I, Heather Garcia, was one of them. The other participants were Susan Fein and Tianhong Shi. The full recording can be viewed on YouTube at this link, but I’ll summarize some highlights from the session in the following paragraphs.

The projects undertaken by CDT research fellows in partnership with faculty spanned disciplines from computer science to field-based courses. 

When asked why they were interested in being research fellows, all three participants indicated that they were pursuing additional graduate education at the time they applied. One participant also indicated that acquiring more knowledge and experience with research would allow faculty to see course design suggestions as “more convincing and easily accepted,” giving her additional credibility when recommending new design approaches to faculty.

The fellows also shared details about their contributions to the research projects they were working on. All of the instructional designers spoke to ways their existing expertise was valued by the researchers. They gave examples of the expertise they offered, which ranged from reviewing course design and educational technologies to designing surveys to offering a fresh perspective and a critical eye. In addition to contributing their design expertise to the research projects, CDT research fellows contributed to the research processes as well, through data analysis and research paper writing and reviewing.

All of the CDT research fellows indicated that they learned a lot from their experiences partnering with faculty on research. One particular highlight in this area is that fellows learned that they contribute diverse perspectives to the research process; they have different backgrounds, experiences, and areas of expertise, and everyone on the team contributes something valuable. CDT fellows also indicated that they learned about the IRB process and the importance of asking questions. Perhaps most importantly, they learned that their expertise is valuable to research teams.

Faculty fellows were also given the opportunity to share how having a CDT fellow on their research team enhanced the research experience, and their feedback was shared in the conference session. They expressed many positive sentiments about the experience including the following:

  • “Our research team started as a group of inspired but like-minded computer scientists wanting to make better online classrooms for diverse students. After she joined the team as an instructional design fellow, the work became credentialed, interdisciplinary, and stronger. She brings expertise and sees what we miss—she not only makes us better able to serve the students we hope to, she makes our team better by adding diversity of thought.”
  • “The combined knowledge and experience of teaching faculty and an instructional designer is incredibly powerful.”
  • “She viewed the scope of the research and content of the courses involved through a different lens than I did.”
  • “The instructional designer provided valuable input on areas of my project merging the instructional design with the research.”
  • “My work with the instructional designer let me explore very practical logistic issues that are often not included in the literature.”

Altogether, it becomes clear that many instructional designers are eager to participate on research projects and they are valuable contributors to the research process. The questions I have now are: How can we continue these partnerships into the future? And, how can we create more research partnership opportunities for other instructional designers and teaching and learning professionals, who aren’t traditionally involved in research?

References

Dello Stritto, M.E., Fein, S., Garcia, H., Shi, T. (2020). Instructional Designers and Faculty Partnerships in Online Teaching Research. NWeLearn 2020 Conference.

Linder, K. & Dello Stritto, M.E. (2017). Research Preparation and Engagement of Instructional Designers in U.S. Higher Education. Corvallis, OR: Oregon State University Ecampus Research Unit.

Loftin, D. (2020). Ecampus Research Fellows Program. Ecampus CDT Blog.