Category Archives: Course Design/Learning Design

Flexibility and structure in course design: You don’t have to choose

Posted on by

Flexibility is an inclusive practice. Structure is an inclusive practice. Both of these statements are true–yet, many people might wonder how to reconcile these seemingly opposite approaches in their course designs. How does one build a course that is both flexible enough to accommodate the diverse needs of their students, yet structured in a way that is clear and unambiguous? In a practical sense, what do these words really mean?

First, let’s define these terms and consider why each of these approaches are critical to student success. What do we mean by flexibility and structure and why are they both important features of course design and facilitation?

Flexibility

Flexibility is getting a lot of press right now, due to our global pandemic. We are all encouraged to be flexible and understanding of one another and to recognize that most of us, especially right now, are dealing with increased responsibilities. As a student myself, I recall how much relief it gave me to read in a note from my professor that this term is “all about flexibility” along with detail around what this means in the context of our course. 

For those of us familiar with online learning, accommodating students with full-time jobs and child or eldercare responsibilities, for example, is not new. However, even for our online students, these responsibilities are compounded by school closures and other distancing measures. Everyone needs additional flexibility, understanding, and support right now. Even you, reader! Let’s be explicit and honest about this in our communications with students and each other.

In the context of our online or remotely taught courses, how do we communicate this to students? Here are a few ideas and suggestions to get you started:

  1. Flexible policies: Saying you will be flexible is not enough. Build flexibility into your policies. For example, if students are required to do field observations for a report or lab, are the guidelines for these observations too restrictive? Might students with mobility challenges or high-risk health considerations be unable to spend extended periods of time outdoors? What alternatives can you provide to these students?
  2. Student choice: Providing your students options will increase their autonomy and engagement. Choice is especially important now because it will allow students to make decisions based, not only on their personal and professional interests, but also based on their individual circumstances, which may have drastically changed in recent months.
  3. Communication: Keeping the lines of communication open is essential. Frequent communication builds feelings of connection so that student needs are more likely to be articulated.

Structure

Building structure into your course means removing ambiguity and avoiding assumptions about your students. Structure does not mean being inflexible. You can be explicit and unambiguous without being rigid.

Two helpful tools for adding structure to your course are rubrics and models, or examples. Rubrics will help you to communicate with your students and will allow you to identify your expectations along with how each criterion will be evaluated. Model assignments will help students to interpret your expectations.

When you don’t have enough structure built into your course, when your expectations are ambiguous, your underrepresented students are disproportionately impacted. This level of ambiguity often results from assumptions about your students’ prior experiences. Assuming they know how to use an LMS or that they have reliable WiFi at home, for example, puts students who don’t have these resources at a disadvantage. 

When you don’t have enough structure built into your course, your students will be forced to make assumptions, correctly or incorrectly, about your expectations. Some students may ask questions, but others will do their work and hope for the best. This results in a clearly unequal playing field, exacerbating existing inequalities. 

Balance

Given that both flexibility and structure are needed in course design and teaching, whether online, remote, on-ground, or hybrid, how does one balance these competing elements?

Too much structure, and your students will lose agency and motivation. Too much flexibility, and your students may feel ungrounded and directionless.

Here are some tips for finding balance:

  • Give choice, but include clear parameters for evaluating student work.
  • Provide multiple lower stakes assessments and stage your course projects, so that students have multiple opportunities to get feedback, correct misconceptions, and earn course points.
  • Welcome student questions and concerns and share your feedback with the whole class. If one student is asking a question, many others are thinking about asking it and would benefit from the same communication. 
  • Don’t wait for students to request alternatives: odds are high that only your most privileged students will feel comfortable asking for accommodations such as more time or additional feedback. If one student requests an accommodation, others who need similar considerations, may not be asking for them. Why not proactively offer these options to all students?

As a final thought, both structure and flexibility are essential ingredients in the recipe for exemplary teaching. When you find the perfect blend of these elements, all your learners will benefit!

References

Parker, F., Novak, J, & Bartell, T. (2017). To engage students, give them meaningful choices in the classroom. Phi Delta Kappan 99 (2), 37-41.

Sathy, V. & Hogan, K.A. (2019). Want to reach all of your students? Here’s how to make your teaching more inclusive: Advice guide. Chronicle of Higher Education. Retrieved from https://www.chronicle.com/interactives/20190719_inclusive_teaching

Beyond intentions: Contextualizing learning outcomes

Posted on by

Beyond intentions: Contextualizing learning outcomes

“After completion of this module, you will be able to…” Does this sound familiar? Have you created statements like this before? If so, it is possible that you have come across Bloom’s taxonomy or the taxonomy for teaching, learning, and assessments (Anderson & Krathwohl, 2001). The taxonomy is a guide to identify the specific knowledge that students are expected to acquire and demonstrate at the end of an educational activity (i.e., a course, module, lesson). However, connecting the outcomes to the activities and assessments can be challenging. Before we look into a guide and examples of alignment, let’s take a brief refresher at outcomes.

Learning Outcomes Explained

Many of us have heard of several terms to refer to outcomes such as objectives, intended results, aims, and goals. I will use the term outcomes in this blog to avoid any confusion. Overall, educational outcomes are statements of what learners should achieve through their engagement in educational activities and processes that allow them to acquire or construct knowledge (Anderson & Krathwohl, 2001; Bloom, Engelhart, Furst, Hill, & Krathwohl, 1956). As disciplines differ, so do their outcomes. Instructors can make use of the Revised Bloom’s Taxonomy (Anderson & Krathwohl, 2001) to explicitly focus on the essential cognitive process and the knowledge required in each discipline. With the learning outcomes clearly stated, students know what is important to learn in a specific course or module within a timeframe. In addition, instructors can plan the curriculum and instruction more appropriately. 

Oftentimes, learning outcomes are confused with tasks. In other words, the means become the ends. In brief, outcomes refer to measurable intended results in the form of skills, knowledge, content that students are expected to demonstrate acquisition of. Whereas, tasks refer to class work that involves the students in completing, interacting, or producing something to achieve the outcomes. Anderson and Krathwohl proposed a formulaic phrase (explicit or implicit: [Verb phrase + verb phrase]) to convey the intention of a learning outcome and differentiate them from means:

“At the end of this blog post, you should be able to use the concept of alignment to contextualize learning outcomes.”

Learning Outcomes Alignment

The concept of “alignment” refers to connecting the outcomes to the learning activities and assessments. Therefore, alignment is an essential characteristic of high-quality online courses. Alignment ensures all course components work together and are mutually reinforced so that learners are able to accomplish the learning outcomes (Quality Matters Rubric, 2018). See the blog post Alignment by Karen Watte, out OSU Ecampus Director of Course Development and Training for more details.

Learning Outcomes in Action

In my meetings with instructors and faculty workshops, I often hear common concerns about student comments related to the lack of clarity in the activities and assignments. Instructors have realized that they need to make the course components more transparent, connected, or “aligned” to the learning outcomes. While the connection, or alignment, to learning outcomes is a fundamental piece to ensuring quality of the learning experience, this connection should be clearer. In addition, the visibility of learning outcomes can help learners be more intentional in their engagement and ways to integrate their knowledge in the course activities and personal life endeavors (LEAP National Leadership Council, 2007).

Oftentimes the learning outcomes are part of the course activities without making any connection (implicit or explicit) to the course activities. Making the learning outcomes more transparent can help students see why they need to complete the variety of course activities, which affects their motivation. Most importantly the learning outcomes play an anchor role that redefines the activities to engage learners in constructing meaning (Biggs, 2003). For example, Biggs (2003) posits that the learning outcomes refer to “sought-for qualities of performance, and it is these that need to be stated clearly” (p.3) throughout the course components. Above all, we should avoid mere completion of tasks in what Mintz (2020) refers to as “mechanical learning experience” when the task-based approach asks students to linearly complete tasks. In fact, understanding the purpose for learning helps motivate students to be more engaged and invested in the course. 

Rather than offering a set of formulaic steps to follow, I invite you to consider a practical strategy and examples as a guide to see the learning outcomes in action. Further, in this strategy learning outcomes are the compass to create learning activities and assessments where students see how the work they do matters beyond the grade it represents. Whether the learning activities you design require students to develop theoretical understandings or apply practical skills, the outcomes will help students see the meaning behind the activities.

Further, the alignment will help students understand the kinds of knowledge and processes involved that in many cases —as the instructors who have shared their concerns with me often note— are not sufficiently transparent. In fact, we learn best when we understand the reason for learning something new. Research supports this and our students understand this too. In what follows is the guide that suggests examining the learning outcomes more closely from the two-dimension approach proposed in the taxonomy (Anderson & Krathwohl, 2001). This two dimensions are: 

  • Knowledge dimension: factual, conceptual, procedural, metacognitive 
  • Cognitive processes (measurable and observable actions): remembering, understanding, applying, analyzing, evaluating, and creating.

By using this two-dimension approach, we can thread the learning outcomes in the purpose and/or instructions of the learning activities and assessments. If we understand that purpose, according to Merriam Webster dictionary, is “something set up as an object or end to be attained’, we can contextualize the outcomes in this “something”. Let’s take a look at the following examples that illustrate how instructors took the learning outcomes to action during the course design process.

Example 1 (Responsible Conduct of Research – GRAD 520)

In this example from a graduate-level class we can see the learning outcomes threaded from within the purpose statement of the Course Reflection Assignment to the instructions.  

Course learning outcomes:

  1. Analyze and defend positions related to responsible conduct of research.
  2. Apply a process for ethical decision-making and apply it to research situations where there are conflicting ethical values
  3. Identify and analyze the moral values and ethical principles, relevant facts, and affected stakeholders in scholarly research
Diagram that illustrates learning outcomes in the context of an assignment for course reflection that includes factual, procedural and metacognitive knowledge.
Learning outcomes treaded in the purpose and instructions

 

 

 

 

 

 

 

 

 

 

 

Example 2 (Business Spanish – SPAN 319)

This example from an undergraduate-level language class shows how both module outcomes become the essential part of the purpose statement in a discussion board.

Learning outcomes module 2 (translated):

  1. Describe the concept of enterprise and its components to a Spanish speaking audience
  2. Create dissemination material that facilitate promote an entrepreneurial project
Diagram illustrating how both learning outcomes were included in the purpose statement of a discussion board
Learning outcomes as essential part of the purpose statement

 

 

 

 

 

Example 3 (Human Development and Family Studies – HDFS 460)

This example from this undergraduate-level class shows the outcomes from multiple weeks aligned to a multi-stage assignment overview.

Weekly learning outcomes (multiple outcomes):

  1. Critically analyze who shapes policy (e.g. Who is excluded and why?) (week 3)
  2. Analyze different ways that families are marginalized in social policy (week 4)
  3. Compare differences in family policies in the US and other countries based on how they are formed through government and other social programs (week 5)
  4. Identify cultural, market economy, and social safety net factors that influence what families look like over time (week 6)
Diagram illustrating how multiple-week learning outcomes were threaded in the overview of a multi-stage assignment where factual and conceptual knowledge was essential
Multiple learning outcomes threaded in a multi-staged assignment overview

 

 

 

 

 

 

 

 

 

 

 

 

 

 

By contextualizing the outcomes we can help students understand better why they do what they do. Finally, it is worth mentioning that the learning outcomes guide the design of the course activities, while they also leave room for creativity and unintended learning to occur (Biggs & Tang, 2011). 

Special thanks to Sandi Phibbs, Ph.D., instructor of GRAD 520; Emily Malewitz-Davis, instructor of SPAN 319, and David Rothwell, Ph.D., and Kylee Probert, instructors of HDFS 460 who graciously agreed to share the examples from their online courses.  

References

Biggs, J. (2003). Aligning teaching for constructing learning. Higher Education Academy, 1(4).

Biggs, J., and Tang. C. (2011). Teaching for quality learning at university: What the student does. Berkshire, UK: McGraw-Hill Education.

Krathwohl, D. R., & Anderson, L. W. (2009). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.

National Leadership Council for Liberal Education and America’s Promise. (2007). College learning for the new global century. A Report from the National Leadership Council for the Lbetal Education and America’s Promise. Washington, DC: Association of American Colleges and Universities

Mintz, S (February 13, 2020). Online Course Design [Webpost]. 

Quality Matters. (2018). Course Design Rubric Standards for Higher Ed Sixth Edition. Retrieved from https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric, March 2020

Roach, A. T., Elliot, S. N., & Webb, N. L. (2005). Alignment of an alternate assessment with state academic standards: Evidence for the content validity of the Wisconsin alternate assessment. The Journal of Special education, 38(4), 218-231.

 

 

 

 

Self-Determination Theory and Online Education: Thinking About Grades

Posted on by

red lightbulb

Photo by Terry Vlisidis on Unsplash

Last Time on SDT & Online Education…

This post is a continuation of my previous blog posts on Self-Determination Theory (SDT) and Online Education and a companion post to Chris Lindberg’s series of posts, Games as a Model for Motivation and Engagement.

In my last post, I posed some difficult questions to consider as you start thinking about how you use grades and motivation in your courses. In case you missed that post, here are the questions—I invite you to spend a few minutes, hours, or days with these questions before moving on with the rest of this blog post:

  • Do you use grades to create external regulation of behavior in your course?
    • Are you grading a behavior or the demonstration of a skill?
  • Do you want to emphasize performance goals or mastery goals?
  • Are there ways to help students identify and integrate the activities and assessments in your course?
  • Do you need to grade this activity/assessment/task?

Why are you grading?

“The troubling truth is that rewards and punishments are not opposites at all; they are two sides of the same coin. And it is a coin that does not buy very much.” (Kohn, 1993, p. 50)

When I first started teaching, I remember asking a colleague if they would review my syllabus draft. They read the attendance policy and asked me something along the lines of, “Why are you grading this? Does it have anything to do with the outcomes of your course?” I probably spewed the usual talking points that students do better in class when they attend, I wanted them to come to class, etc. The reply: “So, do you want to grade learning, or behavior?” That question rocked my world as a young teacher. And made me question everything.

Why do we use grades? Numerous studies have shown that grades and rewards, especially for intrinsically motivating activities such as learning, have detrimental effects. And what do these grades communicate? In terms of SDT, “grading in educational contexts has two functions. One is providing competence-relevant feedback to students, presumably as an aid to enhancing subsequent performance … A second is gatekeeping. Grades can be used to make sure that only students who have mastered material and are thus qualified are eligible for higher training…” (Ryan & Deci, 2017, p. 371). What does an A, B or F communicate to you? For Ryan & Deci, grades by themselves provide little in the way of relevant feedback. (For more on effective feedback, see Wiggins, 2012.)

So can a “grade” be useful? In the context of SDT, there needs to be important information communicated to students about how they’re doing and where they need to spend more attention and effort. In order for this to be autonomy-supportive, it should be informative for the learning process and not judgmental, pressure-inducing, or a social comparison, both in terms of being a reward or a punishment. For example, “Great work. You might consider adding an example in your third paragraph.” is not informative toward the learning process; “You do a great job of defining the problem in your third paragraph, which is an improvement from your first draft. Adding an example would do even more to help your readers relate to this problem.” is an example of feedback that informs a student about where they have come from and where they still need to go.

Performance vs. Mastery

“There do, indeed, seem to be few empirical or theoretical supports for the motivational or competence-building advantages of classical grading schemes. Yet, in most school settings, grades and evaluations are employed as if they were the key to motivation, when, in fact, especially for those who need competence supports, they are likely to be undermining influences.” (Ryan & Deci, 2017, p. 371)

Is it possible to see a letter grade of your work in a course outside the context of social comparison? How often is a B simply viewed as “better than average” or “in the top half of the class”? Focusing on performing a certain way relative to others is a performance goal, while focusing on increasing competence or learning is a mastery goal. (For more on this topic, see Elliot, 2005.) Additionally, each category of goal has an approach type (seeking to achieve something) and an avoidance type (seeking to avoid something). Multiple studies have found that the performance-avoidance combination has the most detrimental outcomes for both learning and student well being, and that this combination is adopted most when students have expectations of being graded (Ryan & Deci, 2017, p. 373).

Ungrading

“…the grading system that higher education in the United States has relied on for many decades has serious problems. It does not work in anyone’s interests, and it genuinely hurts those most directly associated with it: the faculty and the students. In fact, the system is broken.” (Nilson, 2015, p. 23)

At this point you might be asking, “what am I supposed to do?” There are numerous tools, resources, alternatives, and considerations when thinking about using or eliminating grades in a course, such as self-assessment, student-created rubrics, or specifications or criterion-based grading. (Several more are listed in the Reference & Resources section below). One option is to eliminate grading all together, which some institutions have done. If you aren’t ready to eliminate all your grades, try focusing on providing students with feedback in an autonomy-supportive way and empowering them to learn the valuable skill of self-assessment. Without rewards and punishments, students will feel a greater sense of competence; self-assessment and valuable and informative feedback will give students a greater sense of autonomy—that they’re in the driver’s seat for their own learning; with an increased focus on communication about learning, students will also gain a greater sense of relatedness.

There is much more to the topic of grading and ungrading than can be covered in a single blog post, so I’d like to invite you to check out the references and resources below, add your comments, suggestions, and experiences in the comments, contact your instructional designer, or keep an eye out for other opportunities to continue the discussion at various upcoming Ecampus events!

References & Resources

Center for Self-Determination Theory (CSDT). (2019).

  • This website is a treasure-trove of resources on SDT and its application in numerous fields, including education.

Elliot, A. J. (2005). A conceptual history of the achievement goal construct. In Elliot, A. J., & Dweck, C. S. (Eds.) Handbook of competence and motivation (52–72). New York: Guilford Press.

Flaherty, C. (2019). When grading less is more. Inside Higher Ed.

Kohn, A. (2018). Rewards are still bad news (25 years later). New York Times.

Kohn, A. (2011). The case against grades. Educational Leadership.

Kohn, A. (1993). Punished by rewards: The trouble with gold stars, incentive plans, A’s, praise, and other bribes. Boston: Houghton Mifflin.

Nilson, L. (2015). Specifications grading: Restoring rigor, motivating students, and saving faculty time. Serling, VA: Stylus.

Ryan, R. M., & Deci, E. L. (2017). Self-Determination Theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford Press.

Stommel, J. (2018). How to ungrade.

Wiggins, G. (2012). Seven keys to effective feedback. Educational Leadership 70(1), 10–16.

Accessibility, Universal Design for Learning (UDL), and Inclusive Design: What Do They Really Mean?

Posted on by

The other day, my six-year-old asked me what the word “industrious” means, and I was overcome with pride and, moments later, mild panic as I tried to answer his question and couldn’t clearly articulate the meaning of the word.

This experience ended well (thanks, Alexa), but prompted me to think about how often we use words without fully understanding what they mean. We don’t question the meaning of these words when they are used in our work or daily interactions. We may use these words ourselves on occasion–or with regularity–but when we stop and try to define these words, the proper associations and descriptions don’t come immediately to mind.

In my work as an instructional designer, it’s common to talk about universal design or inclusive design, and in many cases, to use these descriptors interchangeably, when talking about design that is usable by a wide range of people. To a lesser extent, accessibility is used in a similar way, but, I think, our shared understanding of this term is more reliable.

For this blog post, I would like to spend some time defining and distinguishing these terms and grounding them in a historical context to more fully convey the nuances and layers of meaning ascribed to each term. I’ll wrap up with some strategies for designing courses to better meet the needs of all learners.

Accessibility

According to the Web Accessibility Initiative (WAI), “Web accessibility means that websites, tools, and technologies are designed and developed so that people with disabilities can use them.” It’s clear from this definition that accessibility is intended to address the needs of users with disabilities, so let’s consider disability. 

Prior to 2001, the World Health Organization (WHO) defined disability as a personal health condition. This definition placed emphasis on the individual. However, in 2001, the WHO redefined disability as a mismatched interaction between a person and their environment. This new definition places emphasis on the environment, rather than the individual. As a result, the onus is no longer only on the disabled individual to manage their health condition; rather, those who have influence over the environment need to make changes to the environment to better accommodate everyone who is interacting with it. In our case, the learning environment is the web, or more specifically, online courses. 

Unlike the other two design approaches we’ll consider, accessibility is intended to address the needs of users with disabilities. Another distinguishing feature of accessibility is that it describes an end goal. Our web content should be presented in such a way that the end result is an accessible website or technology. While this post will not go into the how of making web content accessible, here are some elements you may be familiar with: alternative text (alt tags), headings (H1, H2, H3, etc.), color contrast, captions and/or transcripts, reading order, keyboard navigation, and descriptive URLs are all examples of accessibility elements. All of these elements define what our design should look like, not how to get there.

Another distinguishing feature is that accessibility is required by law. We won’t delve into the specifics here, but it’s important to recognize that accessibility is a legal compliance issue.

Universal Design for Learning (UDL)

While accessibility addresses specific features of a website or online learning environment, Universal Design for Learning (UDL) takes a broader approach. UDL guidelines still emphasize accessibility, but the emphasis is not solely on making disability accommodations or complying with the law. The goal of UDL is to provide the greatest degree of access and usability for the widest range of individuals.

UDL includes a framework with three general principles, each of which includes multiple guidelines and checkpoints for actual practice. A UDL approach is structured and practical and, similar to accessibility, UDL defines an end goal: a product that is usable by the widest range of individuals possible. The framework, however, emphasizes the design, which is only one aspect of creating an online course.

To broaden our understanding of UDL, it’s important to understand that UDL emerged from universal design, which is an architectural concept. Architecture, unlike the web, is physically fixed, and as such, the emphasis is on a single design that works for everyone. 

Inclusive Design

While UDL emerged from architecture, inclusive design was “born out of digital environments,” and, while architecture is fixed, the web is flexible and ever-changing. As such, inclusive design emphasizes flexibility and process. Inclusive design is iterative. With an emphasis on iteration and process, inclusive design cannot be separated from the lived experience of actual users. In other words, if the users (in our case, students) are contributing to and evaluating the design, then we can no longer separate the design and delivery–the creation and facilitation activities.

With a focus on process, inclusive design emphasizes co-creation and frequent feedback from multiple developers as well as end users. In particular, seeking contributions from excluded communities during the entire design and evaluation process is critical to an inclusive process.

Unlike accessibility and UDL, inclusive design is focused on process and iteration. To help illustrate how we see these three design approaches working together, my colleague, Elisabeth McBrien and I created the figure below (figure 1).

Three circles. The outer circle represents inclusive design. The middle circle represents UDL. And the smallest circle represents accessibility.
Figure 1. An inclusive design process will always include UDL and accessibility as end goals.

We see accessibility compliance as core to any design. UDL goes beyond the requirements of accessibility to meet the needs of all users. In an inclusive design process, UDL and accessibility are always the end goal, but inclusive design emphasizes the importance of feedback and iteration. We can always improve and we always have more work to do.

In Practice

Now that we have a better understanding how accessibility, UDL, and inclusive design work together to contribute to a learning environment that meets the needs of all learners, how do we apply them and improve? Ecampus has many guidelines and templates that help us to meet the goals of accessibility and UDL, but how can we be more inclusive throughout this process? 

Here are some inclusive approaches that you might consider integrating into your course facilitation and teaching:

  • Build rapport with students. This is accomplished by infusing instructor presence whenever possible. Respond to Q&A questions and emails within 24-48 hours. Share resources. Deliver feedback promptly. An important element of rapport and presence is showing your personality, so consider using video to welcome students and to encourage them throughout the course.
  • Solicit feedback. One of the easiest ways to solicit feedback from your students is to use a survey. Keep surveys short and consider asking students to share in a few words how the course is going or what they find most challenging.
  • Establish clear criteria and structure. Rubrics, templates, examples, and consistent naming and organization of course materials are just a few ways to provide clarity and structure.
  • Acknowledge student contributions. Praise is an instant confidence booster. Do you have a student–particularly, an underrepresented student–who did an exceptional job on one of your assignments? Let them know. Consider sharing their work as an example–with their permission, of course.
  • Feature counter-stereotypical examples of people in your field. One common barrier to success for underrepresented students is that they don’t see themselves reflected in a particular discipline. Make sure your readings, examples, and other course materials represent a variety of identities. If there’s a lack of diversity in your field, find a way to acknowledge this to your students.
  • Promote student agency and autonomy by giving them choice, whenever possible. Providing choice and promoting agency allow students to connect your course to their own experiences and values.
  • Emphasize real world applications of course work. Often, we assume that our students understand the purpose of course activities, but this is not always the case. Sharing real world applications will help students to see the value and greater purpose of their studies.

Final Thoughts

We’ve covered a lot in this post, and I hope that we’ve come away with a better understanding of disability, accessibility, Universal Design for Learning (UDL), and inclusive design. One of the most important takeaways is that inclusive design is an ongoing process of feedback and iteration. As our student body changes, so do their needs. In an upcoming blog post, Elisabeth McBrien will share more details about student needs and how you might use student personas to design more inclusively.  

As we continue the challenging–yet meaningful–work of creating welcoming online learning environments, it’s important that we have a shared understanding of what that work entails, what work we have done, and what work we have yet to do.

Resources

  1. Appert, L. et al. (2018) Guide for Inclusive Teaching at Columbia. Columbia University: Center for Teaching and Learning.
  2. Gannon, Kevin. (2018) The case for inclusive teaching. Chronicle of Higher Education.
  3. Hogan, Kelly A. and Sathy, Viji. (2019, July 22) “Want to Reach All of Your Students? Here’s How to Make Your Teaching More Inclusive: Advice Guide.” Chronicle of Higher Education.
  4. The inclusive design guide. Inclusive Design Research Centre at OCAD University. CC-BY 3.0.
  5. Inclusive Teaching: Supporting All Students in the College Classroom. EdX course from Columbia University.

Peer Review: Providing a Rationale for the Process

Posted on by

As online educators, we strive for a balance of learning activities that incorporate three types of engagement: learner-to-content, learner-to-instructor, and learner-to-learner.  The learner-to-learner component is often filled through discussion boards or group projects, but an underutilized and undervalued option is peer review.

The Rationale

There are many ways peer review benefits students, among them Cornell University Center for Teaching Innovation lists:

  • Empower students to take responsibility for and manage their own learning.
  • Enable students to learn to assess and give others constructive feedback to develop lifelong assessment skills.
  • Enhance students’ learning through knowledge diffusion and exchange of ideas.
  • Motivate students to engage with course material more deeply.

More broadly, the authors of The Knowledge Illusion argue that our individual capacity for knowledge is often much more limited than we realize and that our true depth of knowledge is held collectively.  They remind us that, “when you put it all together, human thought is incredibly impressive.  But it is a product of a community, not of any individual alone” (page 5).  In our increasingly complex world, some evidence of a shift towards building knowledge collectively can be seen in research. For example, in the MEDLINE database, “the average number of authors per article has nearly quadrupled from about 1.5 in 1950 to 5.5 in 2014” (page 226).  This is just one of many examples the authors use to illustrate how essential collaboration and relationship skills have become.  In nearly every field, students need to be prepared to be more than individual achievers, but rather to contribute effectively to a group.  Peer review provides students an opportunity to give and receive feedback with the goal of creating a better end product, but it is also an opportunity for students to practice and build their teamwork skills.

Moreover, International Society for Technology in Education (ISTE) Standard 3b emphasizes the need for students to, “evaluate the accuracy, perspective, credibility and relevance of information, media, data or other resources.”  Peer review is a great way for us to meet this standard and to combat against misinformation, by teaching students to evaluate and challenge claims.  In Weaponized Lies: How to Think Critically in the Post-Truth Era author Daniel J. Levitin shares strategies for how we can think more critically and evaluate the trustworthiness of what we are being told.  He notes that, “sometimes the people giving you the facts are hoping you’ll draw the wrong conclusion; sometimes they don’t know the difference themselves” (page xx).  If your students are in either of these groups, it benefits them to have an attentive reader review their work and provide respectful suggestions for improvement prior to a final assignment submission.  This may help you as the instructor to avoid catching errors too late in the process when students cannot revise their work.

The Explanation

However, students may not see the value of peer review on their own.  The Teaching Center at Washington University in St. Louis describes many reasons students may express uncertainty around peer review as, “Many students do not perceive feedback from peers as relevant to the process… students are likely to assume that it is only the instructor’s feedback that ‘counts.’”  Therefore, it is important that we explain to students why we are asking them to engage in peer review explicitly.

It can be helpful to explain specifically how this will relate to industry or field of study requirements as a student advances as a professional and scholar – it looks different for a researcher than it does for a project manager, so motivate students by sharing with them how they will engage in similar activities in the future as this gives them an opportunity to practice what Starting Point: Teaching Entry Level Geoscience describes as, “key skills such as abstracting, developing arguments, describing, assessing, criticizing, analyzing, and reviewing.”  As Faculty Focus advises, we can’t assume that students will implicitly understand the purpose of peer review.  When we craft a peer review assignment, we need to think carefully about how we will articulate the benefits of the process to students.  It can be helpful to answer questions like, “Why am I having students do this?” and “Why should students be excited about this process?”  Or, to take it a step further, we can anticipate the questions from our students’ perspective and proactively address the purpose and logistics in the assignment description, by answering questions like, “Why am I doing peer review?” and “How am I supposed to review my peer’s work?”  Make sure the technology needed and processes are clear and that resources are provided for students that need more guidance.

The Process

Remember, knowing why students are peer reviewing and being able to peer review are two totally different skills.  If you are an Ecampus instructor, talk with your instructional designer about strategies that can help your peer review process be more successful.  Some of the best practices suggested by Center for Instructional Technology & Training at the University of Florida include:

  • Clarify expectations in advance
  • Check your students have all the tools they will need
  • Provide enough time in the peer review process so that students can meaningfully engage – this may span more than one module
  • Model the type of feedback you want your students to use
  • Create a quality rubric as a guide

Your instructional designer can also talk to you about digital tools or strategies that can be used to introduce students to peer review. For example, you can discuss whether it makes more sense to use Canvas Peer Review or another tool, like Peerceptiv, which is research-validated peer assessment technology available for Ecampus courses.

Remember, students need opportunities to practice peer review, as they may never have done it before.  That means they have to get familiar with both the tools and the process.  It’s best if they can practice with the technology on a low stakes assignment before using it for a high stakes assignment, so that they can familiarize themselves with a peer review process without the added anxiety of a major grade on the line.  It will also take time for you as the instructor to get familiar with the process, but it is a completely worthwhile investment!

I invite you to consider some concluding thoughts from Levitin, “Information gathering and research that used to take anywhere from hours to weeks now takes just seconds… The implicit bargain that we all need to make explicit is that we will use just some of that time we saved in information acquisition to perform proper information verification” (page 253).  Let’s reinvest some of the time our students saved researching to engage them in verifying claims, evaluating evidence, offering commentary, and incorporating feedback – all of which support the development of a stronger student work and the building of a collective knowledge.

Self-Determination Theory and Online Education: Types of Motivation

Posted on by

image of several birds sitting and one is moved to fly Photo by Nathan Dumlao on Unsplash.

My interest in learning about motivation in education began many years ago when I started learning about motivation in game design. In order to better understand motivation, in a classroom, while playing a game, and in an online learning environment, I am turning to the body of research that has grown from Edward Deci and Richard Ryan’s Self-Determination Theory (SDT). This blogpost will be a continuation of my previous SDT Primer and an excellent companion to Chris Lindberg’s Games as a Model for Motivation and Engagement series of posts.

While I had intended to use this entry for discussing grades and assessment, an important piece of SDT and its application is understanding the different types of motivation explored by the SDT community of researchers. This post will define and expand on the numerous types of motivation in preparation for a discussion on grades and assessment.

Before we begin, take a brief minute to explore and reflect about what moves you to do something? As an example, what moved you to open this blog post and begin reading it?

The Autonomy-Control Continuum

The types of motivation you might be most familiar with are intrinsic and extrinsic motivation. Intrinsic motivation refers to doing something because it is inherently interesting or enjoyable, while extrinsic motivation refers to doing something because it leads to a separable outcome. I might be moved to read a chapter of a novel over lunch because it is inherently enjoyable (intrinsic), or I might be moved to run errands over lunch because of external factors, like visiting the bank or post office due to their limited open hours (extrinsic). While these opposites are often displayed and discussed as an either-or, they are really just two ends of a spectrum that contains more nuanced gradations.

(Gagné & Deci, 2005, p. 336)

The autonomy-control continuum (Ryan & Deci, 2017) is an outgrowth of the intrinsic-extrinsic spectrum, representing the spectrum between autonomous regulation, or a feeling of complete volition and controlled regulation, or a feeling of being externally or internally compelled to act. While intrinsic motivation would fall under the category of autonomous regulation, extrinsic motivation can sometimes come close to the autonomy end of the spectrum for personally important or valued tasks, or can swing all the way to the controlling side with external rewards or punishments for tasks. And on the extreme opposite end of the spectrum from intrinsic motivation is amotivation, or the complete absence of intentional regulation. Ideally, we hope that students will feel autonomous motivation, which has also been shown as optimal for learning.

Internalized Motivations: External vs. Internal

Now let’s explore some of the murky gradations between feeling autonomous and controlled. The first step is to compare two degrees of controlled regulations: external vs. internal regulations. External regulation is motivation that is controlled by external factors—a student might experience external regulation when they have to complete a group project in a course. External factors, the instructor in this case, dictates that students collaborate in groups for this project. Internal regulation (or introjection), occurs when internally controlling factors are at play, e.g. shame, guilt, or fear. Continuing with the group work as an example, a student might feel moved to complete a task for the group project by placing internal pressure on themselves, resulting in feeling guilty if they don’t perceive that they’re pulling their weight, or shame in being the last group member to respond to a discussion assignment, or fear that their lack of activity will punish everyone in the group with a lower grade. In both cases, the student feels controlled, either by an external factor or internal pressure.

Identified & Integrated Regulations

As we move closer to the autonomy end of the spectrum, we run into identified regulation, or the acceptance of extrinsic value. Our student from the example above might feel extrinsically motivated to complete the group project, but through the use of a rationale statement from the instructor, might accept the value of this group work, thus feeling more of a sense of autonomy than with external or internal regulation. Lastly, and moving even closer to autonomy, is integrated regulation, or adding the value of a task to one’s own beliefs or sense of self. Perhaps through reflection or a particularly well designed group project, a student comes around and now believes that group work is an essential part of their desired educational experience. While integrated regulation is not the same as feeling autonomous, you might be able to imagine a situation where an identified or integrated regulation would feel more motivating than an external or internal regulation.

How to Begin Thinking About Grades

In a recent Q&A with Richard Ryan, one of the authors and lead researchers of SDT, responded that “there has been no empirical justification for why we have grades in schools at all.” My next blog post will dive deeper into the role that grades and assessment play in SDT and motivation. In the meantime, I would like to pose some questions to get you started thinking about how you use grades in relation to motivation in your courses:

  • Do you use grades to create external regulation of behavior in your course?
    • Are you grading a behavior or the demonstration of a skill?
  • Do you want to emphasize performance goals or mastery goals?
  • Are there ways to help students identify and integrate the activities and assessments in your course?
  • Do you need to grade this activity/assessment/task?

These are big, difficult questions! And thinking about motivation in terms of a spectrum is complicated! If you find yourself wanting to continue the discussion of motivation in course design, check out the companion blog series mentioned in the introduction above, contact your instructional designer, or keep an eye out for other opportunities to continue the discussion at various upcoming Ecampus events!

References & Resources

Center for Self-Determination Theory (CSDT). (2019).

  • This website is a treasure-trove of resources on SDT and its application in numerous fields, including education.

Gagné, M., & Deci, E. (2005). Self-determination theory and work motivation. Journal Of Organizational Behavior, 26(4), 331–362.

Ryan, R. M., & Deci, E. L. (2017). Self-Determination Theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford Press.

The Unique Advantage of Teaching Assistants as Online Course Innovators

Posted on by

A new Ecampus project has underscored the potential of graduate teaching assistants (GTAs) to add immense value to the process of reviewing and improving courses by thinking both as an instructor and as a student. Ecampus analyzed data and collaborated with partners in academic departments to identify five courses in which students were experiencing barriers to success that were not being addressed in Ecampus’ rigorous course development and support process. The academic departments also identified five pedagogically-minded and innovative GTAs to analyze and begin addressing the barriers to student success. The result is a pilot that we believe will be beneficial for the students, the GTA Fellows, and the faculty, while also providing learning opportunities for all stakeholders about how to tackle the most challenging course design problems. While we’re still in the first term of the pilot, our collaborative investigation process and emerging creative solutions have already made us very excited about the findings to come. 

Determining first steps 

At Oregon State Ecampus, we have a strong framework to help support and maintain course quality.  Courses are carefully and thoughtfully designed through a collaborative effort between a faculty course developer who has received training in online course design best practices and an Ecampus instructional designer. The development process spans from the two term initial design and build period, where we ensure courses meet our set of Ecampus Essentials, to iterative first term adjustments, to support for continued lifetime maintenance, to formal course “refreshes” every 3-5 years.  Finally, many of our courses are also submitted for and earn Quality Matters certification, which is an important indicator of quality based on research-based standards. This rigorous and supportive development process aims to make sure the course continues to stay relevant, accessible, and effective for learners.

But, in spite of all this careful planning, development, review, and maintenance, what is the appropriate response when courses have recently come through this rigorous and comprehensive design process, faculty have been trained in best practices for teaching online, and students are still encountering barriers to success in the course?

We recently launched a new pilot project to tackle this question head-on.  As a starting point, and using a few basic indicators of student persistence, retention, and success in our courses — such as the DFWU rate, or a rate at which students receive grades of D, F, W (withdrawal), or U (unsatisfactory)  — we created an initial list of courses across our online offerings where students were least successful in passing or completing. From this list, we identified which courses had been redesigned within the last five years (to rule out our standard redevelopment process as a solution to increasing student success).  The latter group of courses underwent some additional review by our team to identify if there were any stand-out issues that could be easily resolved.

What we arrived at was a short-list of courses that had higher than usual DFWU rates and were redesigned recently.  In these courses, we knew that something else was going on beneath the surface; the underlying problem was neither an obvious issue with design or facilitation techniques.  Many of the courses on this short-list are problematic not for a high rate of D/F/U grades at the end of the term, but rather for a high rate of W (withdrawal) grades. Our Ecampus student population is largely comprised of non-traditional students who have a different set of needs than our more traditional on-campus students, namely in that they need flexibility to balance their busy out-of-school lives while also meeting their educational goals; so, through this pilot, we wanted to find an effective way to determine what could be changed to better support Ecampus students in staying in (and succeeding in) these courses that were particularly challenging for reasons we could not easily identify.

Designing the pilot

With these course profiles compiled, we reached out to five department partners to assess their interest in collaborating on a project to further review and revise a course (or, in some cases, a sequence of courses).  We proposed to fund a Graduate Teaching Assistant (GTA) for three consecutive quarters to evaluate and then to propose and implement innovative interventions in these targeted courses with an eye toward increasing online student success.  In general, pilots are following the below schedule:

  • Quarter 1: the GTA is an active observer of the course(s), and reviews previous sections’ data to look for patterns in obstacles that students might face; in collaboration with the faculty course lead and Ecampus staff, the GTA then proposes a first set of interventions for quarter 2; IRB approval of research is confirmed if necessary for design of interventions and/or for desire for possible future publication.
  • Quarter 2: the GTA continues to be an active observer in the course(s) and helps to implement the first set of interventions; in collaboration with the faculty course lead and Ecampus staff, the GTA then proposes new or refined interventions for quarter 3.
  • Quarter 3: the GTA continues to be an active observer in the course(s) and helps the instructor to implement the new or refined interventions; data reporting is wrapped up and a campus presentation is arranged.

Note that, across the three quarters, the GTA does not undertake the traditional tasks associated with a teaching assistant in an online course, such as grading assignments, responding to student questions, holding virtual office hours, etc., modeling our pilot on fellowship programs such as Duke University’s Bass Digital Education Fellowships.  Rather, all stakeholders agreed to allow the GTA to not be constrained by these time-consuming tasks and focus their efforts instead on observational work and then planning and implementing interventions.  The instructors assigned to these courses continue to take on their regular duties of interacting with and assessing students.

The unique advantage of GTAs

With our five unique pilots underway as of this summer, it has already become clear that the key to this pilot is the unique positioning of the GTA to tackle these student success problems from both the faculty and student perspectives.  At Oregon State, GTAs regularly serve as teaching assistants or instructors of record in on-campus, hybrid, and online courses, so our GTAs have come to these pilot projects with prior teaching experience (and, often, with some training in pedagogy and course design).  Yet, our pilot program GTAs are also still students themselves, so they are particularly attuned to the student experience as they follow and track current and upcoming groups of students working through these courses.

Our pilot will also benefit from the fact that these GTAs have a strong interest in pedagogy and in their own professional development as instructors.  With that in mind, we have worked to structure some of the individualized goals of each pilot to reflect how we can help the GTA get the most value out of this opportunity (such as through a campus presentation, a published paper when we have results, or connecting with Ecampus leaders as possible references for job applications).  The final name for our pilot – GTA Innovations for Student Success Fellowship – is crafted both to reflect the central goals of the pilot (student success) and to call out the important and unique work that GTAs are doing as fellows.

Looking forward (to sharing innovative interventions and results)

We are still in the very early stages of each of these pilots, so while we don’t yet have any results to share, the deep engagement of our stakeholders in this process has been heartening, and wonderful plans are in the works for the first sets of interventions to be implemented this fall.  We are also so pleased to see the support behind allowing this group of GTAs inspire innovative online teaching within their home departments, and the willingness of the faculty who teach the courses under review to think collaboratively and differently about the creative ways we can support their online students.

As part of their pilot work, we will encourage these GTAs to contribute to the blog and share their insights and takeaways along the way.  What they learn about how to support student needs in these particularly challenging courses and course sequences, learning design, teaching methods that better motivate disengaged learners, etc. will no doubt be useful to Ecampus stakeholders across the university and beyond.  Stay tuned for more!

On Good Design

Posted on by

Introduction

For those who work in higher education, it may not come as a surprise that the field of instructional design has grown in tandem with the expansion of online programs and courses. Evidence of this growth abounds. While the discipline of instructional design has expanded rapidly in recent years, the history of instructional design is not well known by those outside of the field.

This post will cover a brief history of instructional design with a particular emphasis on design: What influences design? How are design decisions made? How has the way we approached design changed over time? We’ll also consider how instructional designers actually design courses and the importance of course structure as an inclusive practice.

Instructional Design: Theory and History

Every instructional design curriculum teaches three general theories or theoretical frameworks for learning: behaviorism, cognitivism, and constructivism. While an instructional designer (ID) probably wouldn’t call herself a cognitivist or a behaviorist, for example, these theories influence instructional design and the way IDs approach the design process.

The field of instructional design is widely believed to have originated during World War II, when training videos like this one were created to prepare soldiers with the knowledge and skills they would need in battle. This form of audio-visual instruction, although embraced by the military, was not initially embraced by schools.

B.F. Skinner
“B.F. Skinner” Portrait Art Print
by Xiquid

In the 1950s, behaviorists, such as B.F. Skinner, dominated popular thought on how to teach and design instruction. For behaviorists, learning results in an observable change in behavior. The optimal design of a learning environment from a behaviorist perspective would be an environment that increases student motivation for learning, provides reinforcement for demonstrating learning, and removes distractions. Behaviorists are always designing for a specific response, and instruction is intended to teach discrete knowledge and skills. For behaviorists, motivation is critical, but only important to the extent that it elicits the desired behavior. 

Cognitivism was largely a response to behaviorism. Cognitivists emphasized the role of cognition and the mind; they acknowledged that, when designing learning environments, there is more to consider than the content to be learned. More than environmental factors and instructional components, the learners’ own readiness, or prior knowledge, along with their beliefs and attitudes, require consideration. Design, from a cognitivist approach, often emphasizes preparedness and self-awareness. Scaffolding learning and teaching study skills and time-management (metacognitive skills) are practices grounded in a cognitivist framework.

While cognitivists emphasize the learner experience, and in particular, acknowledge that learners’ existing knowledge and past histories influence their experience, the learner is still receiving information and acting on it–responding to carefully designed learning environments.

Constructivism, the most current of the three frameworks, on the other hand, emphasizes that the learner is constructing their own understanding of the world, not just responding to it. Learners are activity creating knowledge as they engage with the learning environment.

All–or nearly all–modern pedagogical approaches are influenced by these theoretical frameworks for learning.

Design Approaches

“A single course can be seen as a microcosm of theoretical frameworks, historical models, and value-laden judgements of pedagogical approaches“

Learning theories are important because they influence our design models, but by no means are learning theories the only factor guiding design decisions. In our daily work, IDs rely on many different tools and resources. Often, IDs will use multiple tools to make decisions and overcome design challenges. So, how do we accomplish this work in practice?

  1. We look to established learning outcomes. We talk about learning goals and activities with faculty. We ask questions to guide decision making about how to meet course learning outcomes through our course design.
  2. We look to research-based frameworks and pedagogical approaches such as universal design for learning (UDL), inclusive design, active learning, student-centered design, and many other models. These models may be influenced by learning theory, but they are more practical in nature.
  3. We look to human models. We often heed advice and follow the examples our more experienced peers.
  4. We look to our own past experiences and solutions that have worked in similar situations, and we apply what we learned to future situations.
  5. We make professional judgements; judgements rooted in our tacit knowledge of what we believe “good design” looks like. For better or for worse, we follow our intuition. Our gut.

Over time, one can see that instructional design has evolved from an emphasis on teaching discrete knowledge and skills that can be easily measured (behaviorism) to an emphasis on guiding unique learners to actively create their own understanding (constructivism). Design approaches, however, are not as straightforward as simply taking a theory and applying it to a learning situation or some course material. Instructional design is nuanced. It is art and science. A single course can be seen as a microcosm of theoretical frameworks, historical models, and value-laden judgements of pedagogical approaches–as well as value-laden judgements of disciplinary knowledge and its importance. But. That’s another blog post.

Design Structure to Meet Diverse Needs

“Meeting diverse needs, however, does not necessitate complexity in course design“

If learners are unique, if learning can’t be programmed, if learning environments must be adaptable, if learners are constructing their own knowledge, how is all of this accommodated in a course design?

Designing from a modern constructivist perspective, from the viewpoint that students have vastly different backgrounds, past experiences, and world-views, requires that many diverse needs be accommodated in a single course. Meeting diverse needs, however, does not necessitate complexity in course design. Meeting diverse needs means that we need to provide support, so that it is there for those who need it, but not distracting to those who don’t need it. Design needs to be intuitive and seamless for the user.

Recent research on inclusive practices in design and teaching identify structure as an inclusive practice. Design can be viewed as a way of applying, or ensuring, a course structure is present. In that way, working with an instructional designer will make your course more inclusive. But, I digress. Or, do I?

Sathy and Hogan contend, in their guide, that structure benefits all students, but some, particularly those from underrepresented groups, benefit disproportionately. Conversely, not enough structure, leaves too many students behind. Since many of the same students who benefit from additional course structure also succeed a lower rates, providing course structure may also help to close the achievement gap.

How are We Doing This?

The good news is that Ecampus is invested in creating courses that are designed–or structured–in a way that meets the needs of many different learners. Working with an Ecampus instructional designer will ensure that your course materials are clearly presented to your students. In fact, many of the resources we provide–course planning templates, rubrics, module outlines, consistent navigation in Canvas, course banners and other icons and visual cues–are intended to ensure that your students navigate your course materials and find what they need, when they need it.

References

Icons made by phatplus and Freepik from www.flaticon.com are licensed by CC 3.0 BY

Boling, E., Alangari, H., Hajdu, I. M., Guo, M., Gyabak, K., Khlaif, Z., . . . Techawitthayachinda, R. (2017). Core Judgments of Instructional Designers in Practice. Performance Improvement Quarterly, 30(3), 199-219. doi:10.1002/piq.21250

Eddy, S.L. and Hogan, K. A. (2017) “Getting Under the Hood: How and for Whom Does Increasing Course Structure Work?” CBE—Life Sciences Education. Retrieved from https://www.lifescied.org/doi/10.1187/cbe.14-03-0050

Sathy, V. and Hogan, K.A. (2019). “Want to Reach All of Your Students? Here’s How to Make Your Teaching More Inclusive: Advice Guide. Chronicle of Higher Education. Retrieved from https://www.chronicle.com/interactives/20190719_inclusive_teaching

Tanner, K.D. (2013) “Structure Matters: Twenty-One Teaching Strategies to Promote Student Engagement and Cultivate Classroom Equity,” CBE—Life Sciences Education. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3762997/

Rubrics Markers of Quality –Tips and Best Practices (Part 2)

Posted on by

There are many benefits to using rubrics for both instructors and students, as discussed in Rubrics Markers of Quality Part 1 – Unlock the Benefits. Effective rubrics serve as a tool to foster excellence in teaching and learning, so let’s take a look at some best practices and tips to get you started.

Best Practices

Alignment

Rubrics should articulate a clear connection between how students demonstrate learning and the (CLO) Course Learning Outcomes. Solely scoring gateway criteria, the minimum expectations for a task, (e.g., word count, number of discussion responses) can be alluring. Consider a rubric design to move past minimum expectations and assess what students should be able to do after completing a task.

Detailed, Measurable, and Observable

Clear and specific rubrics have the potential to communicate to how to demonstrate learning, how performance evaluation measures, and markers of excellence. The details provide students with a tool to self-assess their progress and level up their performance autonomously.

Language Use

Rubrics create the opportunity to foster an inclusive learning environment. Application of clear and consistent language takes into consideration a diverse student composition. Online students hail from around the world and speak various native languages. Learners may interpret the meaning of different words differently. Use simple terms with specific and detailed descriptions. Doing so creates space for students to focus on learning instead of decoding expectations. Additionally, consider the application of parallel language consistently. The use of similar language (e.g. demonstrates, mostly demonstrates, and doesn’t demonstrate) across each criterion can be helpful to differentiate between each performance level.

Tips of the Trade!

Suitability

Consider the instructional aim, learning outcomes, and the purpose of a task when choosing the best rubric for your course.

  • Analytic Rubrics: The hallmark design of an analytic rubric evaluates performance criteria separately. Characteristically this rubric’s structure is a grid, and evaluation of performance scores are on a continuum of levels. Analytic rubrics are detailed, specific, measurable, and observable. Therefore, this rubric type is an excellent tool for formative feedback and assessment of learning outcomes.
  • Holistic Rubrics: Holistic rubrics evaluate criteria together in one general description for each performance level. Ideally, this rubric design evaluates the overall quality of a task.  Consider the application of a holistic rubric can when an exact answer isn’t needed, when deviation or errors are allowed, and for interpretive/exploratory activities.
  • General Rubrics: Generalized rubrics can be leveraged to assess multiple tasks that have the same learning outcomes (e.g., reflection paper, journal). Performance dimensions focus solely on outcomes versus discrete task features.

Explicit Expectations

Demystifying expectations can be challenging.  Consider articulating performance expectations in the task description before deploying a learning task. Refrain from using rubrics as a standalone vehicle to communicate expectations. Unfortunately, students may miss the rubric all together and fail to meet expectations. Secondly, make the implicit explicit! Be transparent. Provide students with all the information and tools they need to be successful from the outset.

Iterate

A continuous improvement process is a key to developing high-quality assessment rubrics. Consider multiple tests and revisions of the rubric. There are several strategies for testing a rubric. 1) Consider asking students, teaching assistants, or professional colleagues to score a range of work samples with a rubric. 2) Integrate opportunities for students to conduct self-assessments. 3) Consider assessing a task with the same rubric between course sections and academic terms. Reflect on how effectively and accurately the rubric performed, after testing is complete. Revise and redeploy as needed.

Customize

Save some time, and don’t reinvent the wheel. Leverage existing samples and templates. Keep in mind that existing resources weren’t designed with your course in mind. Customization will be needed to ensure the accuracy and effectiveness of the rubric.

Are you interested in learning more about rubrics and how they can enrich your course? Your Instructional Designer can help you craft effective rubrics that will be the best fit for your unique course.

References

Additional Resources

The Basics
Best Practices
Creating and Designing Rubrics

Rubrics: Markers of Quality – Unlock the Benefits! (Part 1)

Posted on by

Would you like to save time grading, accurately assess student learning, provide timely feedback, track student progress, demonstrate teaching and learning excellence, foster communication, and much more? If you answered yes, then rubrics are for you! Let’s explore why the intentional use of rubrics can be a valuable tool for instructors and students.

Value for instructors

  • Time management: Have you ever found yourself drowning in a sea of student assignments that need to be graded ASAP (like last week)?  Grading with a rubric can quicken the process because each student is graded in the same way using the same criteria. Rubrics which are detailed, specific, organized and measurable clearly communicate expectations. As you become familiar with how students are commonly responding to an assessment, feedback can be easily personalized and readily deployed.
  • Timely and meaningful feedback: Research has shown that there are several factors that enhance student motivation. One factor is obtaining feedback that is shared often, detailed, timely, and useful. When students receive relevant, meaningful, and useful feedback quickly they have an opportunity to self-assess their progress, course correct (if necessary), and level up their performance.
  • Data! Data! Data! Not only can rubrics provide a panoramic view of student progress, but the tool can also help identify teaching and learning gaps. Instructors will be able to identify if students are improving, struggling, remaining consistent, or if they are missing the mark completely. The information gleaned from rubrics can be utilized to compare student performance within a course, between course sections, or even across time. As well as, the information can serve as feedback to the instructor regarding the effectiveness of the assessment.
  • Effectiveness: When a rubric is designed from the outset to measure the course learning outcomes the rubric can serve as a tool for effective, and accurate, assessment. Tip! Refrain from solely scoring gateway criteria (i.e. organization, mechanics, and grammar). Doing so is paramount because students will interpret meeting the criteria as a demonstration that they have met the learning outcomes even if they haven’t. If learning gaps are consistently identified consider evaluating the task and rubric to ensure instructions, expectations, and performance dimensions are clear and aligned.
  • Shareable: As academic programs begin to develop courses for various modalities (i.e. on campus, hybrid, online) consistently assessing student learning can be a challenge. The advantage of rubrics is they can be easily shared and applied between course sections and modalities. Doing so can be especially valuable when the same course is taught by multiple instructors and teaching assistants.
  • Fosters communication: Instructors can clearly articulate performance expectations and outcomes to key stakeholders such as teaching assistants, instructors, academic programs, and student service representatives (e.g. Ecampus Student Success Team, Writing Center). Rubrics provide additional context above and beyond what is outlined in the course syllabus. A rubric can communicate how students will be assessed, what students should attend to, and how institutional representatives can best help support students. Imagine a scenario where student contacts the Writing Center with the intent of reviewing a draft term paper, and the representative asks for the grading criteria or rubric. The grading criteria furnished by the instructor only outlines the requirements for word length, formatting, and citation conventions. None of the aforementioned criteria communicate the learning outcomes or make any reference to the quality of the work. In this example, the representative might find it challenging to effectively support the student without understanding the instructor’s implicit expectations.
  • Justification: Have you ever been tasked with justifying a contested grade? Rubrics can help you through the process! Rubrics which are detailed, specific, measurable, complete, and aligned can be used to explain why a grade was awarded. A rubric can quickly and accurately highlight where a student failed to meet specific performance dimensions and/ or the learning outcomes.
  • Evidence of teaching improvement: The values of continuous improvement, lifelong learning, and ongoing professional development are woven into the very fabric of academia. Curating effective assessment tools and methods can provide a means of demonstrating performance and providing evidence to support professional advancement.

Value for students

  • Equity: Using rubrics creates an opportunity for consistent and fair grading for all students. Each student is assessed on the same criteria and in the same way. If performance criteria are not clearly communicated from the outset then evaluations may be based on implicit expectations. Implicit expectations are not known or understood by students, and it can create an unfair assessment structure.
  • Clarity: Ambiguity is decreased by using student-centered language. Student composition is highly diverse, and many students speak different native languages. Therefore, students may have different interpretations as to what words mean (e.g. critical thinking). Using very clear and simplistic language can mitigate unintended barriers and decrease confusion.
  • Expectations: Students know exactly what they need to do to demonstrate learning, what instructors are looking for, how to meet the instructor’s expectations, and how to level up their performance. A challenge can be to ensure that all expectations (implicit and explicit) are clearly communicated to students. Tip! Consider explaining expectations in the description of the task as well.
  • Skill development: Rubrics can introduce new concepts/ terminology and help students develop authentic skills (e.g. critical thinking) which can be applied outside of their academic life.
  • Promotes metacognition and self-regulatory behavior: Guidance and feedback help students reflect on their thought processes, self-assess, and foster positive learning behaviors.

As an Ecampus course developer, you have a wide array of support services and experts available to you. Are you interested in learning more about rubric design, development, and implementation? Contact your Instructional Designer today to begin exploring best-fit options for your course. Stay tuned for Rubrics: Markers of Quality (Part 2) –Tips & Best Practices.

References:

  • Brookhart, S. M. (2013). How to create and use rubrics for formative assessment and grading. Alexandria, Va.: ASCD.
  • Richter, D., & Ehlers, Ulf-Daniel. (2013). Open Learning Cultures: A Guide to Quality, Evaluation, and Assessment for Future Learning. (1st ed.). Berlin, Heidelberg: Springer.
  • Stevens, D. D., & Levi, Antonia. (2013). Introduction to rubrics: an assessment tool to save grading time, convey effective feedback, and promote student learning (2nd ed.). Sterling, Va.: Stylus.
  • Walvoord, B. E. F., & Anderson, Virginia Johnson. (2010). Effective grading: a tool for learning and assessment in college (Second edition.). San Francisco, CA: Jossey-Bass.