In this post I’m returning to an important topic: accessibility. In a previous blog my colleague Susan Fein explained how everyone benefits from more accessible materials and that a large number of our students have some degree of disability.
Word documents are ubiquitous in our courses, as well as for other work-related activities. If a Word document is designed for digital consumption – such as posting in the Learning Management System or on a website – it needs to comply with accessibility standards. Fortunately, Word includes excellent tools for making your file accessible! I will first go over the main accessibility features, and then show you how to implement them in the video below.
Accessibility checker: Word includes a tool that helps you check your work. It is useful but it doesn’t catch all the errors.
Structure: headings, spacing, lists: Marking these properly will let screen reader users skim the content and understand its organization easily. Structure a document in a hierarchical manner: the title should be Heading 1 (NOT the “Title” style – that one just gets read as simple text). The next major sections should be Heading 2, subsections of a Heading 2 are Heading 3, and so on. Do not skip levels. You can change the appearance of all these styles to match your aesthetic. If you wish, you can also save style sets to have them ready to use.
Images: There are two main things to take care of here: adding alt text (so screen reader users can listen to the description) and making sure that the image is in line with the text (to keep the reading order clear).
Colors: If you use colors, make sure there is enough contrast between text and background. Even people with good eyesight can struggle to read something if the contrast is not strong. In addition, remember that many people are color blind, so do not rely on color to convey essential information. For example, avoid something like “The readings in blue are very important, make sure you read them carefully! The optional resources are in green”. Use other means of signaling instead, such as bold or italics.
Links: Links need to include meaningful text rather than the URL. A screen reader will read the URL one letter at a time, which is not very helpful. In addition, descriptive links help both screen reader users and sighted users skim the document to get an idea of the content or find specific information.
Tables: Tables can cause trouble to screen reader users – do not use them for layout! Only use them for actual tabulated information. When you use tables, the main rule is to keep them simple and avoid split cells, merged cells and nested tables. Then, make sure you have a designated header row, which helps screen reader users navigate the data.
Document properties: The document needs to have a title set in its properties. This title is helpful for blind users because the screen reader announces it as the document is loaded in the program.
Save to PDF – yay or nay? Avoid turning your document into a PDF file, if the document is meant for online reading. PDFs are hard to make accessible. If you must make a PDF, start with a fully accessible Word file. It is recommended to use PDFs only when the design includes complex or unusual elements (for example special/technical fonts, musical notes etc.). If you are using a PDF because you have a complex layout, consider posting both the PDF and a simplified Word file, in case someone needs the fully accessible version.
Watch this 10-minute video that walks you through an example of making a document accessible. I’m using Microsoft 365 on Windows – if you’re using another version of Word or platform, things may look slightly different. Timestamps:
Accessibility checker – 00:38
Headings – 01:46
Lists – 04:56
Spacing – 05:27
Images – 06:16
Colors – 07:29
Links – 08:09
Tables – 08:49
Title Property – 09:33
As you can see, the process of creating accessible Word documents is straightforward. Turning this into a standard practice will greatly help people who access information electronically, with or without assistive devices. Let’s make it happen!
According to the 2020 Brandon Hall Group Team Development Pulse Survey findings (Werder, 2021), at least half of work is currently done in teams in over seventy percent of companies. Global Human Capital Trends (2016) confirmed that this trend is continuing, with over 7000 organizations moving towards more team-based designs. However, the success of team collaboration is not a guarantee and requires diligent planning and hard work. Tannenbaum and Salas (2020) suggest that there are seven “Cs” (or drivers) of teamwork, namely: capability, cooperation, coordination, communication, cognition, coaching, and conditions.
To contextualize and apply each of these 7 “Cs”, I’ll use a recent team collaboration I participated in as an example. A team of four staff from Oregon State University Ecampus gave a virtual presentation on the role of instructional designers in research. Speaking of the first C – capacity, thanks to the selection of team members, this team had the perfect mix: the facilitator was in charge of setting up the stage and engaging the audience with an opening poll and scenario. A second team member was assigned to cover the institutional level, a third team member was assigned to cover the team level and the last team member was assigned to cover at the individual level. Capability: checked ✅!
Cooperation: During the preparation for the presentation, each of the four team members worked individually on our own parts. When we met again, we reviewed each other’s parts, felt comfortable voicing any concern or areas that could use improvement. We each revised our individual parts and met again to review. At this point, we felt we had the content nailed down. Laurie, Tianhong and Heather already know each other very well since we all work in the same instructional design team at Ecampus. Naomi opened herself up and welcomed us to give her feedback and ideas for improvement up front, which is very helpful for Laurie, Tianhong and Heather to connect with her, and built trust for working together on this project. Viola, Cooperation: checked ✅!
Coordination: During the two rounds of peer review sessions, we made many changes, based on feedback from team members. Naomi opened up with a poll of attendee roles and a scenario to illustrate why instructional designers need to be involved in research. Laurie demonstrated diligence and surveyed the entire instructional design team at Ecampus and was able to present some solid data on our team composition in terms of degree/education, and years of career in instructional design. Laurie also provided Tianhong with two prepared slides on areas to be covered as a suggestion. Tianhong conducted comprehensive research and her findings demonstrated that over 50% of instructional designers at Ecampus have participated in research activities with support from Ecampus. Heather’s storytelling of her research involvement was rich and fascinating. So she had the pleasant struggle of cutting down her content to fit within a nine minutes time frame. And we all put scripts of what we plan to say in the notes area of the google slide we were collaborating on, which help us to stay within the limited time and allow us to have discussion time with all participants. Since each of us diligently completed our individual work as planned, the whole presentation is full of data and stories. Coordination: accomplished✅!
Communication within the team of four presenters was relatively easy since we use slack as a communication tool internally and we used calendar invites and emails for scheduling purposes. Our slack messages were quite active throughout the preparation and on the day of the presentation and after the presentation with many suggestions, encouragement, and compliments! Communication: accomplished✅!
Cognition or shared understanding among the team members is vital. In my opinion, this should be the first C on the list! For our team project, Naomi hand-picked the three panelists to join her on this collaboration because she sensed that all three of us share a common understanding on the value of instructional designers being involved with educational research. This common understanding and vision is visible the entire time while we worked on this project. Cognition: checked✅!
Coaching: Does leader and/or team members demonstrate leadership behaviors? Yes, Naomi is a great leader in this project. It was a pleasure to work under her leadership since the role of each panelist is very clear, and we started the collaboration early enough so that we have plenty of time to review, revise, practice and practice again before the actual presentation. Laurie also demonstrated leadership by offering help to cohesively formatting and beautifying each of our slide decks into one presentation file. Coaching: accomplished✅!
Conditions: Does the team have favorable conditions such as resources and culture? Yes, each team member brought with them expertise in their own roles, we were also able to use existing tools such as slack and google slides, and ecampus presentation template for this collaborative presentation. Naomi could have done it all by herself. But she invited a panel of three instructional designers to collaborate with her on this presentation. Our combined effort makes our story stronger, richer and more impressive because we work as instructional designers and we have experience doing research as instructional designers. Conditions: checked✅!
On the day of the virtual presentation, Laurie and Tianhong were presenting from campus offices housed inside the campus library while Heather and Naomi were presenting from their remote offices. In the middle of the presentation, there was a 🔥fire alarm in the library which required everyone to evacuate from the library. Laurie and Tianhong moved to a nearby building and logged back online and re-joined the presentation within 10 minutes. We are so thankful that the four of us are presenting from different locations so that the fire alarm did not stop us from presenting. This is how virtual team collaboration saved our work during a fire alarm emergency. And this is how the 7 Cs led us to a great team collaboration. The next time you sit down to plan a team project or initiative, you might benefit from reflecting on these following questions:
Does the team have the right people with the right mix? (Capability)
Does each team member have constructive attitudes about their team? (Cooperation)
Does each team member demonstrate necessary teamwork behaviors? (Coordination)
Does each team member exchange information effectively with each other and outside? (Communication)
Does each team member possess a shared understanding? (Cognition)
Does leader and/or team members demonstrate leadership behaviors? (Coaching)
Does the team have favorable conditions such as resources and culture? (Conditions)
I hope I have encouraged and convinced you a tiny bit in your next decision for teamwork and have fun collaborating and doing effective teamwork!😊
References: Werder, C. (2021). How to develop a winning team. Brandon Hall Group. Retrieved from https://www.brandonhall.com/blogs/how-to-develop-a-winning-team/
Global Human Capital Trends. (2016). The new organization: Different by design. Deloitte University Press. Retrieved from https://www2.deloitte.com/content/dam/Deloitte/global/Documents/HumanCapital/gx-dup-global-human-capital-trends-2016.pdf
Tannenbaum,S.I. & Salas, E. (2020). Teams that work : the seven drivers of team effectiveness. Oxford University Press.
If you use slide presentations to deliver information and then provide a digital version of the slides to support learners, this post is for you!
Instructors teaching online or who use a companion LMS or website to accompany in-person classes often upload the slide file to aid students in notetaking. However, you may not be aware that digital files are not automatically accessible to those using assistive technologies, such as screen readers. Following a few simple and easy guidelines will improve accessibility of your materials for all students and demonstrate your thoughtful attention to inclusivity and equity.
Who Benefits from Accessibility?
Everyone, not only those with disabilities, benefit from accessible learning materials. The U.S. Census Bureau estimates that there are more than 40 million people in the U.S. with a disability, so odds are good that some of them will be in your courses.
Accessibility practices support all learners, not just those who require them. In 2016, the OSU Ecampus Research Unit conducted a nationwide survey about student use of video closed captions. In that study, 70% of respondents who did not self-identify as having a disability used captions at least some of the time.
I asked OSU’s disability access center how many online students request disability-related accommodations. So far this year, 23.9% of those served by their office are Ecampus students. Last year, nearly 40% of all Ecampus courses had at least one student with an accommodation, and nearly 15% of all online-only students used a disability-related accommodation.
To ensure equity, regardless of who does or does not depend on accessibility support, it is vital to make all learning materials compliant with accessibility standards. When educators intentionally create fully accessible materials, we more equitably serve all online learners.
What Can You Do?
Here are five easy-to-follow tips that elevate your commitment and ability to create accessible materials.
Tip #1. Use a template. Templates are important because basic formatting for accessibility is already built in. By inserting your content into designated sections, you preempt some accessibility issues without any extra effort. For example, when you insert the topic of each slide into the designated title field, the slide structure maintains the correct sequence in which a screen reader encounters the various elements on the slide. If you are concerned about being too constrained or predictable, these designated fields accommodate your creativity! It is okay to reshape, resize, or reposition a field if you do not like its default appearance or location.
Regardless of which end of the design spectrum you lean, always start with a template. If you are not fond of colorful designs or fancy formats, there is a basic, unadorned template you can use. If you are a fan of fun, frivolity, or fabulous, select one of many free template options found online to suit your theme or topic. Check out the different templates Ecampus has developed with college-specific themes. One of them might be a good fit for you.
Tip #2. Enter a unique title on each slide. Each slide in your presentation must have a unique title. This permits a screen reader to navigate easily from one slide to the next. What happens when you have segments of the presentation that require two or more slides to fully deliver the information? No problem! There are various ways to address this.
When several slides focus on a different aspect of the primary topic, use that in the title. For example, you are creating a presentation about Health and Wellness and have multiple slides on the topic of Cooking. You want to introduce the topic, describe meal preparation, and offer ideas for healthy snacks. Since these are three distinct subtopics, a good approach is to label the slides as Cooking: Overview, Cooking: Meal Preparation, and Cooking: Healthy Snacks. Repeating the main topic in the title helps the learner connect each segment but still delineates separate subtopics.
If the subject matter does not neatly break into clear subgroups, it is fine to use a sequential number, such as Cooking Part 1, Cooking Part 2, etc. Since most creators develop a presentation’s content, sequence, and flow thoughtfully and logically, if you take a moment to consider why you grouped together specific ideas, the unique titles will likely emerge.
Tip #3. Follow best practices. If you search online for guidance about how to create effective slide presentations, you will discover that many sources offer similar suggestions. Most of these include recommendations about text (contrast, font size, font style), use of images, page structure, and so on. Use this short list as a helpful reminder of these other accessible-friendly best practices.
Text should be easy to read, with good contrast. Black text on a white background is ideal and classic. Be cautious of templates with too subtle contrast. They might not meet accessibility guidance for visually disabled learners. Use 18-point (or larger) sans serif font for readability.
Use images judiciously. Pictures convey themes, present an idea, or evoke a mood. However, too many can detract from the message, be confusing, or appear unprofessional. Aim for a “less is more” approach. (Learn more about accessibility for images in the next tip.)
Include adequate white space to separate and group content. Bullets are optional. Keep slide structure simple. Use phrases or a few words rather than full sentences. Break up content into multiple slides to avoid crowding.
Tip #4. Create alt-tags for images. A screen reader recognizes the presence of an image but it cannot discern the content. To be accessible, that information is provided as a text description or alt-tag.
If you have images in your slide deck, each must have an alternate text description. The alt-tag describes and explains the content of an image. Usually it is not accessible or helpful to use the file name. And beware of tools that try to divine the content of an image and insert descriptions. These are usually wildly inaccurate and unhelpful.
The majority of images in an effective presentation should be essential to the learner’s experience; the image is required for accurate comprehension of the content. The are images such as charts, graphs, photos, maps, or data. Other images may be optional or decorative; nice to have but not essential to the learning and, if not seen by the student, do not impede the learner’s ability to grasp the material.
For essential images, write a brief (1-3 sentences) text description. No need to include lead-in words like “this is an image of…” Describe the key educational value of that image. What about it is important to the learner? What is the essence of the information you want the learner to know about that chart, graph, or photo?
Screen image from Office 365 PowerPoint
Decorative images have two options: enter a description or skip over the image. To skip, enter null text (“ ”) as the alt tag or, if available in your version of PowerPoint, select the “decorative” option. Both choices direct the screen reader to ignore the image. If you prefer to tag a non-essential image, use a simple description, such as “team logo” or “Professor Kumar.”
Understand that writing good alt tags is a challenging skill that takes time and practice to master, so do your best. You may want to confer with the Disability Access Center, an instructional designer, or other faculty support group if you need assistance.
For more information about how to write effective alt tags, refer to these or other resources.
Tip #5. Use meaningful text to format links. Please do not insert a full URL on your slide. Screen readers recognize a URL link and read aloud every individual letter and symbol, often in a monotone mechanical voice, depending on the specific assistive tool. Think about how frustrating, confusing, and unhelpful that is. Instead, format each link using meaningful text, as demonstrated in this post. For example, the two resources linked above use the article’s full title as the meaningful text. Also, avoid the over-used, too generic “Click here for more information,” with the word “here” formatted as the hyperlink. Instead, select text that specifically identifies the URL content, such as “Visit the Disability Access Services web page for more information.”
Accessibility Supports Equity
Demonstrate your commitment to equity! With just a few extra minutes, you can easily meet minimum accessibility standards by following these tips and using the accessibility checker tool built right into PowerPoint!
Reference
Linder, K. (2016). Student uses and perceptions of closed captions and transcripts: Results from a national study. Corvallis, OR: Oregon State University Ecampus Research Unit.
In your work as an instructor or instructional designer, what kinds of course design problems have you been trying to solve lately? Perhaps there’s a discussion board assignment that you’d like to make more engaging, or maybe there’s a lab activity that students struggle with.
According to recent research findings published in the journal Nature, “…people are more likely to consider solutions that add features than solutions that remove them, even when removing features is more efficient” (Meyvis, T. and Yoon, H., 2021). In a course design context, this means we will probably reach for something we can add to our course to fix a problem: a shiny new tool we learned about at a conference, an extra video lecture, or an additional step for that lab activity that students struggle with. We do need to include quality tools and fresh media to our courses, certainly. However, because there is so much to cover in any given course, we should strategically subtract all unnecessary elements. The aforementioned Nature article has brought to my attention that I may be regularly neglecting the powerful design move that subtraction offers. As a result of reflecting on that, I’ve collected a few examples of how instructors and instructional designers can use subtraction in course design.
Subtraction example #1: Remove formal requirements from informal practice activities
Let’s take a look at a discussion board assignment as an example. Sometimes discussion boards are treated more like formal writing opportunities than discussions. For example, do you require students to not only respond to the prompt but also use a formal style of discourse and cite evidence in accordance with APA style guidelines and do so in a certain number of words? We don’t require students to use APA style or converse with a word count in our in-person discussions, so why would we do this in an online course (Darby, 2020)? Consider subtracting those formal writing requirements for discussion boards, and reserve the APA citations and formal requirements for polished writing assignments. That way, online discussions can be a place to practice and informally grapple with new concepts and ideas the way that face-to-face discussions are. Then, after students have a chance to work through topics in an informal way, they will be ready to try more formal tasks on the topic, such as constructing an argument and appropriately citing evidence according to a particular style guide.
Subtraction example #2: Reduce the frequency of office hours
If you hold office hours at a set time each week, and you notice that few students attend, consider doing away with the set weekly “office hours” altogether. Instead, invite students to make an appointment with you at least once during the term, at a time and in a modality that works well for them, or reduce the frequency of office hours to two or three timely sessions (optional attendance) per term, such as before each exam. Take questions in advance so that students too busy to attend can still benefit from the sessions, and post a recording of the session afterward in an announcement. According to Lowenthal, Dunlap, and Snelson (2017), less frequent but more focused office hours increased participation from students. Lastly, consider renaming your new infrequent office hours to something warm and inviting, such as “Coffee Break” or “Consultations” (Darby, 2019), which students may find more welcoming. Building in plans for facilitation and instructor presence in the early stages of course development allows faculty to focus more on teaching while the course is running and less on reactive problem solving.
Subtraction example #3: Dethrone teaching “folklore”
Teaching “folklore,” which John Warner calls the ineffective “practices handed down instructor to instructor” (p. 207, 2020), shows up uninvited, particularly when you have been assigned a course to teach that you had no hand in developing. One example of teaching folklore is the stubborn assumption that serious scholars are gatekeepers for their fields. Evidence of this exclusionary approach may show up in the form of a stern, overly formal, or cold tone in a syllabus. Consider removing verbiage that conveys a cold tone, since we now know that warm-tone syllabi encourage students to reach out to their professors (Gurung and Galardi, 2021). Another example of this is inflexible class “policies” that reflect an individual instructor’s preferences and not university policies. Consider reviewing the policies stated in the syllabus and delete any that are not aligned with actual university policies. Further, the idea that students must achieve a level of eloquence and scholarly sophistication on par with faculty in order to be considered for a grade of “A” on an assignment is another example of this. Evidence of this type of folklore could be found in rubrics with benchmarks that are nearly impossible to achieve. The element that could be subtracted here is not the rubric, but rather the specific language in the rubric that makes it impossible for students to succeed in the assignment. Neuromyths, which are false beliefs about the brain and learning, could also be included in the category of teaching folklore. If you spot neuromyths in a course, remove them. In summary, if you spot one of these ineffective teaching folklore elements in your course, consider removing the “folklore” item.
For this design challenge, try subtracting ineffective design elements before adding new items to solve course design problems. If you are unsure if something should stay or go, ask yourself what purpose this element serves in the course. It should then become clear whether the item belongs or needs adjustment. Even small adjustments can transform learners’ experiences. What have you removed from your course? Share how the process was for you by leaving a comment.
References:
Blum, S. D. (Ed.). (2020). Ungrading : Why rating students undermines learning (and what to do instead). West Virginia University Press.
Darby, Flower, and James M. Lang. Small Teaching Online : Applying Learning Science in Online Classes, John Wiley & Sons, Incorporated, 2019. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/osu/detail.action?docID=5780349.
Created from osu on 2022-01-18 23:50:41.
Gasiewski, J.A., Eagan, M.K., Garcia, G.A. et al. From Gatekeeping to Engagement: A Multicontextual, Mixed Method Study of Student Academic Engagement in Introductory STEM Courses. Res High Educ 53, 229–261 (2012). https://doi.org/10.1007/s11162-011-9247-y
Gurung, R. A. R., & Galardi, N. R. (2021). Syllabus Tone, More Than Mental Health Statements, Influence Intentions to Seek Help. Teaching of Psychology. https://doi.org/10.1177/0098628321994632
Lowenthal, P. R., Dunlap, J. C., & Snelson, C. (2017). Live synchronous web meetings in asynchronous online courses: Reconceptualizing virtual office hours. Online Learning , 21 (4). https://doi.org/10
Relationships matter for successful collaborative work. Yet, when it comes to online/ blended/hybrid course design, development often begins with a focus on course content, assuming that the collaborative relationship between faculty and instructional designers is secondary to the design process (Tate, 2017). If we care about student success, we must turn our attention to the ways in which effective collaborative relationships among design partners contribute to the course feel — that is, the online learning environment as perceived by students. In our recent OLC presentation, we proposed an interactive strategy for developing collaborative relationships between faculty and instructional designers based on curiosity, enthusiasm, and mutual respect.
Faculty and instructional designers often find themselves at cross-purposes. The design process expects two groups of experts to bring their unique perspectives and skill sets together in course creation, without providing instruction or support for the kinds of intentional shared knowledge transmission and production necessary for collaboration (Richardson, J. C., et al., 2018). In fact, faculty buy-in to a collaborative working relationship ranks as the number one obstacle to instructional designers’ success (Intentional Futures, 2016, pp. 3, 15). Institutional frameworks must be in place that set up faculty-instructional design teams for success, so that together they in turn can set up students for success.
We suggest that fostering curiosity in each other’s disciplinary norms and approaches serves as an igniting spark for establishing effective instructional designer-faculty autonomy supports. In this way, relationships begin with trust, mutual respect for professionals’ expertise, and socioemotional growth.
ID-Instructor Cooperative Instrument
To aid in this process, we developed an ID-Instructor Cooperative Instrument for individual, flexible needs of both parties. Through a series of definitions and short prompts, users can see similarities and differences between their viewpoints on topics like student success, well-designed courses, and course feel. Feel free to use the spreadsheet linked above or this list of questions:
Course feel: Name up to 5 keywords that describe how you want the course to feel.
Student interactions: List the kinds of interactions your students will encounter in your course. Feel free to list them in order of importance, or to modify the categories.
Teacher ⟷ Student
Content ⟷ Student
Student ⟷ Student
Definitions:
How do you want students to define success in this course?
How do you define success in this course (for yourself and/or students)?
Working Together:
Name your best course — one you have designed, taught, taken, etc.
Name a course activity you are proud of, or one you would like to be proud(er) of.
Identify 3-5 keywords or phrases that describe your working style.
Anything else you would like to note?
Initial Feedback
In our OLC presentation, some session attendees were purely instructional faculty, while others worked full-time in instructional design. Many people served in joint roles, including some administrative responsibilities for facilitating course design, or had started their careers in one area and then switched roles. Despite these different backgrounds, almost everyone agreed with our assertion that lack of mutual respect between faculty and instructional designers negatively impacted their ability to create and deliver high-quality courses.
During the presentation, we enjoyed seeing the enthusiasm and excitement from participants wanting to share their own stories and experiences. Working through the instrument questions as a group for this session was engaging for both participants and for us as presenters. Participants were able to quickly and clearly pick up the instrument and begin to share their ideas right away. Feedback both during the interactive presentation and during the Q&A suggests that our instrument serves the purpose we created it for: to get faculty and IDs excited about talking with one another about teaching and course design.
Conclusion & Getting Started
We advise other faculty and ID teams to rethink their working relationships, starting with curiosity about each other’s experiences and hopes for the course and their future students. Administrators — who often need to increase faculty buy-in to course development programs, and help instructional designers meet faculty where they are in their pedagogical experience and comfort levels in online instruction and design — can play an important role in encouraging design teams to take the time to work through the kinds of questions and conversations outlined in this tool. The road to pedagogical expertise is often varied, windy, and complex, for everyone involved. When the course design process focuses on growth and learning for the faculty and ID, it leaves a positive mark on a course that reverberates for the students experiencing that online space.
We invite faculty and IDs to get curious about their counterparts. If there is not currently an intake meeting for faculty and IDs at your institution, you could share this instrument either for a one-on-one meeting, or with a dedicated pedagogy session through your Center for Teaching and Learning and discuss how it could be a starting point for a collaborative working relationship. If there is already an intake meeting that is part of the course design process, think about bringing in aspects of this instrument that might be missing from that session.
If you are an administrator, suggest that your faculty or IDs spend some time at the beginning of a project getting to know one another. You might even help spark some curiosity by including a professional introduction — what excites you about having a particular instructional designer or faculty member working on a specific course or program? What can you share about the unique experiences of your faculty or IDs that would help start that initial conversation?
We hope that this instrument will be a tool you can use as new course developments begin, whether you are working with a new collaborator or wish to get to know someone better that you have worked with previously. We would also be interested in continuing to learn about faculty and instructional designer relationships. If you have feedback, comments, or experiences you would like to share, we invite you to leave a comment on this blog post or reach out to us via email.
When selecting a textbook, there are a number of factors to evaluate. In addition to assessing the textbook for appropriate content, one category that I recommend looking at is how inclusive the textbook is. Here are a few guiding questions to ask when evaluating textbooks for inclusion:
What is the cost of this textbook? Have you looked for open (free) textbooks, perhaps from the Open Textbook Library or considered writing or adapting your own? Affordability is inclusive.
Do the textbook images of people represent diverse cultural heritage and lived experiences?
Are the contributions to the field that are highlighted in the textbook from a diverse range of scholars in the field? If not, is there discussion about why certain voices were historically excluded from the field?
Is the textbook accessible? If there is an e-book, do the images have alt text, for example? Can students with disabilities access all materials in the book?
If the textbook is an e-book, are the concepts presented in multiple ways, such as text, infographic, slide decks, or multimedia elements? Giving students choices in how they explore the course concepts empowers them to use their existing preferences, and helps them develop new strengths and avenues for learning.
What to Do When the Textbook Is Not Ideal
It’s tough to find a textbook that is inclusive and has all of the concepts you are hoping to teach. What can you do when you find a textbook that has the concepts you need but is lacking in inclusive excellence? Here are some simple ideas for addressing this:
Consider giving publisher feedback. Write a brief email to the publisher about your concerns around a lack of representation in the book or whatever it is that you see as missing.
For any text you choose, consider inviting students to write to the publisher if they see areas for improvement, whether that is with cost, bias, or other issues. You could include the contact information for the publisher in your course materials page, inviting students to write in feedback directly to the publisher.
Acknowledge to your students that the textbook isn’t as inclusive as you would like it to be. Share the ways that you are advocating for better quality. You could also invite students to have a bias hunt discussion about the textbook or course materials. Then you could collect that feedback and send it to the publisher.
If the textbook lacks contributions from a diverse range of scholars, consider adding scholarly articles, images, or interviews from diverse professionals in your field to your course learning materials pages, in your LMS course site.
Consider highlighting professional organizations in your field that promote and mentor the professional development of scholars from specific historically underrepresented communities.
Have you had success in this area of evaluating textbooks? Have you found a publisher or textbook that has made gains in this area? If so, please share your resources in the comments.
In recent years, there has been an increasing focus on how cultural background has a deep impact on overall classroom experience. This is true in the physical classroom, and it is no less true online. Cultural experiences not only affect how students navigate meaning in learning contexts, but these experiences also play a role in motivation and satisfaction in a course.
Various comparative studies in the larger educational context have demonstrated the need for consideration of cultural differences in the delivery and design of instruction in the physical classroom. Likewise, in order to improve learner experience for diverse student groups in the online environment, it is important to examine the unique landscape of the virtual classroom.
Communication Frameworks
Frameworks for understanding various cultures, such as Hofstede’s Cultural Dimensions, are frequently relied upon for increasing cultural competence in the workplace or classroom. One particularly relevant type of cultural difference framework is referred to as “High-Context” “Low-Context” (Hall, 1976). In high-context cultures, people rely on nonverbal cues as well relationship building as the backdrop for information exchange (Bai, 2016). In low-context cultures, however, information exchange matters more than relationship building, and nonverbal cues as well as social hierarchy can largely be ignored. Low-context communication is expected to be explicit and highly verbal. Context such as body language and facial expressions are nice in a low-context environment, but they are not a required element of communication and not necessary for receiving the full intended message, as they might be in a high-context culture. Furthermore, expressing personal views in a low-context culture is expected and does not automatically threaten one’s relationships with friends or colleagues who might hold differing opinions.
“In high-context culture, people tend to personalize their disagreement with others. To show one’s disagreement and anger in public is tantamount to admitting loss of control and face, because what is being said is taken personally which may have an influence on interpersonal relationships. Therefore, they will keep their emotions inside or just remain silent to avoid trouble. In this way, they can maintain social harmony and intimate bonds with each other. In the eyes of people from low-context culture, this kind of repression is totally unreasonable. Everyone has their own rights to express opinions, and this explicit criticism has nothing to do with their interpersonal relationships” (p. 22, Bai, 2016).
Because communication norms can be so distinct between cultures, consider their impact on the learning environment. For example, in the physical classroom, communication and meaning rely on multiple forms of interaction. While some communication relies on written language in a face-to-face class, the majority of exchanges are driven by speech. Participants have multiple strategies at their disposal: verbal feedback, social cues, confirmation checks, body language, and even the physical arrangement of the classroom itself. Instructors can have students work in pairs or groups, sit in a circle or rows, and so on. All of these forms of input combined with students’ background knowledge about how various classroom configurations function help students negotiate meaning.
However, in the asynchronous online classroom, the majority of communication relies on written language. Instructors typically use tools within the learning management system (LMS) to provide written feedback and grading. Student interactions take place via discussion forums, blogs, structured peer-review assignments, and other text-driven means of communication. While text-based communication has some advantages for English language learners (ELL) in particular (e.g. opportunities for revision and low-stakes rehearsal), written language is open to interpretation and lacks the visual and physical cues of verbal language. This can be tricky for learners from high-context cultures where communication relies heavily on implicit and contextual cues, such as body language and tone of voice. In fact, one study found that learners from these language and cultural backgrounds tend to misinterpret intent or look for hidden meaning in instructor feedback and discussion posts (Hyland, 2013).
All this is to say, which student would be most comfortable in a setting where nearly all non-verbal cues are absent, direct text-based information is exchanged, and disagreement among peers is encouraged? Students from low-context cultures may feel at home in this environment, but students from the high-context cultures may feel they – as well as the instructor – are regularly violating norms and expectations. This is important for faculty to understand because it affects whether students see themselves and others as active participants who contribute in a meaningful way, which has a direct impact on the learning community in the course.
Tips for Cultural Inclusivity
Survey Students. Use a word cloud, survey, or similar to ask students how comfortable they are expressing disagreement with others. Create guidance for participation based on the responses received.
Provide clear guidance and models. Include guidelines for appropriate language for discussions or debates. Make explicit that respectful disagreement or questioning of ideas is encouraged. Consider includingsentence stemsthat model how to express disagreement or varied perspectives using academic language. Use rubrics that clearly identify expectations for both initial posts and peer response.
Be creative with discussion prompts. Assign roles on occasion to build students’ participation confidence by lowering the personal stakes. Other prompt types include problem statements, cases studies, video analysis, or student facilitation of discussions.
Use multiple means of communication. Include weekly overview videos or podcasts to deliver important details about the course verbally in addition to providing the information in written form so that students may benefit from both high-context and low-context forms of communication. Sharing assignment feedback through video or audio comments (e.g., in Canvas Speedgrader) can provide additional physical cues and context for the communication that written feedback alone cannot convey.
Provide flexible options around communication. Invite students to introduce themselves or participate in discussion posts through video, if they would prefer, while also giving them the option not to appear on camera. Allow them to decide how they would like to exchange information in the course.
Be flexible about tools. Provide information about how to record video or other media used in the course, but keep in mind that preferred or available tools will vary in different regions of the world. Allow students the flexibility to determine which tools are best for communication tasks.
Include anonymous peer review tasks. This can relieve students who are concerned about damaging relationships with their peers through negative feedback. Gather anonymous feedback about the course as well. Consider culturally based perspectives on power dynamics between students and teachers.
Provide feedback in multiple formats. Give specific examples of what students did well while also being clear about areas for improvement. Avoid making assumptions about what students will take from the comments. Feedback delivered through multiple means (video, audio, text) can help to avoid miscommunications.
Extend existing inclusive language practice to include cultural considerations. Be mindful about clarity and use of idiomatic speech that may be less transparent for students from a different language background.
Assignments are an integral component of the educational experience to guide the teaching and learning processes. In fact, Dougherty (2012) contends that assignments are instructional events that aim to teach for learning, that is “recipes for instructional events— lessons in the best sense— and their main function is to create a context for teaching new content and skills and practicing learned ones.” (p. 23). Assignments as instructional plans provide students with the opportunities to apply concepts they studied in the class. Further, through assignments, students can demonstrate the skills developed in a unit of content in more concrete ways and aligned to the goals of the course.
In my consultations with instructors I often hear them raise concerns about course assignments. These concerns range from making assignments more practical and relevant, clarifying the purpose and instructions, integrating problem-solving and critical thinking, to including authentic and experiential tasks. In addition, I hear instructors mention that some assignments that students submit are incomplete, offer superficial and unsubstantiated arguments (i.e., written reports), focus on tangential ideas, have been googled, reflect bias, and are simple opinions using non-credible sources. These concerns are very valid and it is important to examine the assignments deeper. What I have noticed is that some assignment descriptions lack a purpose and clarity. In a word, assignments need to be transparent.
Determining the structure of an assignment bears the questions of how can instructors make the assignments learning events that are clear and relevant enough for students? how can students not only demonstrate what they learn, but also use the assignments as catalysts for further intellectual and academic challenges? Let’s take a closer look at transparency.
Transparency
The first time I heard about transparency in assignment design was at the Wakonse Teaching and Learning conference a few years go. Several sessions and small group activities at the conference showed us that the assignments need to have a clear structure, detailed instructions, and a grading criteria. Obviously! I said to myself at the time. However, the reality is that assignments tend to be reduced to a list of instructions, tasks that students need to complete and submit for a grade. In some cases these instructions vaguely indicate the grading criteria in terms of the format and style (i.e., number of words, font size, spacing).
The underlying framework for transparent assignments is a structure that clearly describes the purpose of the assignment, the instructions or tasks, and the grading criteria (Dougherty, 2012; Winkelmes, 2013; Winkelmes, Bernacki, Butler, Zochowski, Golanics, & Weavil, 2016). Winkelmess and colleagues (2016) draw from three theoretical bases to support the three-stage framework: metacognition, agency, and performance monitoring. Contrastively, Dougherty (2012) draws from instructional strategies informed by backward design and alignment to outcomes to set the assignment structure. In this framework, instructors deliberately design the assignment for high quality learning experience and relevance to students. In their research study, Winkelmess and colleagues (2016) found that students who received transparent assignments showed evidence of greater learning in three areas related to student success: academic confidence, sense of belonging, and mastery of skills.
Designing transparent assignments involve creating a clear and coherent architecture. Through this structure students can think deeper about the concepts studied, focus their attention on particular topics, make connections to real-world contexts, and see the relevance for their future lives and goals (Dougherty, 2012). In doing so, instructors need to create a harmonious structure that clearly explains why students need to do an assignment, what is the assignment about, how to do the assignment, and how they will be graded on it.
When I presented this architecture to one instructor, he replied “you are asking me to tell students the answer! Why would I need to hand-hold students in this way when I want them to be problem-solvers and critical thinkers?” While this comment is valid, and also paralyzed me for a few seconds, I engaged the instructor in discussing what the assignments need to be clear. For instance, we talked about how students will know what to do, why students should care about completing the assignment (besides the grade), and how students will meet the expectations if they don’t know the purpose and the way to complete it. In addition, I said “you want students to be problem-solvers of the content and topics, not problem-solvers of the assignment design.”
A transparent assignment should have the following three basic components: purpose, task, and grading criteria.
Purpose
The starting point in an assignment is to be able to answer the question of why? Why will students learn from this assignment? Why will students need to complete this assignment? Why is this assignment important in students’ learning? Stating the purpose of the assignment serves a two-fold objective. First, it gives the instructor a frame of reference for creating an activity that is relevant and meaningful to students, and that connects to the learning outcomes. Second, the purpose of the assignment gives students a focus and a sense of direction.
Winkelmes (2013) suggests establishing the purpose in terms of the skills students will practice and the knowledge they will gain. In addition, the purpose can also be determined by contextualizing the learning outcomes in practical ways within the activity.
Task
You can call it tasks, details, instructions, steps, or other. In this structure, the instructor describes what students need to do, what resources they can use, and the expectations of the assignments. Having a clear set of instructions makes the assignment more rigorous and helps students produce more high-quality work.
Grading Criteria
Providing the criteria of how the assignment will be graded will also give students a sense of clarity and direction. Clear expectations through a rubric or grading guidelines helps students adhere to the outcomes of the assignment. Winkelmes (2013) suggests including several examples of real-world problems so students can see how the application of knowledge and skills will look like.
Remarks
A transparent assignment should have a well-structured framework or an architecture of steps. Transparency in assignments is a mindset, a way of thinking, the vision that students are given clear and relevant learning events that allow them to demonstrate their learning, and foster their engagement. Transparent assignments can be designed as stand-alone pieces or as a multi-stage assignment. Multi-stage assignments can build on cognitive complexity, include multiple skills, and extend learning to outside the class. In our next blog, I will look at how to design multi-stage assignments.
Sources
Dougherty, E. (2012). Assignments matter: Making the connections that help students meet standards. Alexandria, VA: ASCD.
Winkelmes, M. 2013. “Transparency in Learning and Teaching: Faculty and Students Benefit Directly from a Shared Focus on Learning and Teaching Processes.” NEA Higher Education Advocate, 30(1), 6-9.
Winkelmes, M. A., Bernacki, M., Butler, J., Zochowski, M., Golanics, J., & Weavil, K. H. (2016). A teaching intervention that increases underserved college students’ success. Peer Review, 18(1/2), 31-36.
In 1989, Kimberlé Williams Crenshaw, a lawyer and scholar of Critical Race Theory (CRT), coined the term intersectionality to describe the multiple and layered oppressions experienced by African American women. Over time, this term has been used to describe many aspects of social identity, particularly focusing on race, gender, and class oppression. Intersectionality allows us to consider the impact of multiple oppressions on individuals and groups. For example, asking what it means to be poor in the United States is different from asking what it means to bea poor, Black, woman in the United States, which is different from asking what it means to be a poor, Black, disabled woman in the Southern United States.
Intersectionality matters because, if we don’t recognize and support our most marginalized citizens, they will continue to fall through the cracks. In colleges and universities, this means that our most marginalized students may need additional support to perform to their full potential. Addressing one source of oppression may not provide enough support to students who are working to overcome multiple sources of oppression.
Disability in Higher Ed
Disability is an obstacle for many college students. Consider these statistics:
19% of undergraduate students report having a disability.
28% of American Indian/Alaska Native students reported a disability.
21% of White students reported having a disability (rounded to nearest percent).
17% of the students with disabilities are Black. (National Center for Educational Statistics [NCES], 2019).
When considering disability–or any other identity–we need to consider how other characteristics might compound the marginalization of students with disabilities. Let’s consider how race intersects with disability.
While the percentage of Black people with disabilities in higher education is lower than the percentage of White people with disabilities in higher education, in the general population, the reverse is true. According to Courtney-Long, E.A., Romano, S.D., Carroll, D.D. et al. (2017), 1 in 4 Black people have a disability, while 1 in 5 White people have a disability. This means that more white people with disabilities are accessing and progressing through higher education.
It is also important to recognize that the actual percentages of students with disabilities is higher as many students choose not to disclose their disabilities to their institutions. According to one study, “9% of students who identified as disabled did not disclose this information to their college or university” (Taylor & Shallish, 2019, p. 10).
There are clearly opportunity and equity issues that disproportionately impact students of color with disabilities in higher education.
Yet, when we work to create learning environments that are inclusive of students with disabilities, we often neglect to address intersecting sources of oppression. For example, accessibility requirements do not consider how disability intersects with other oppressions, such as class or race.
Universal Design for Learning
Universal Design for Learning (UDL) is an approach that is commonly cited as a way to meet the needs of all learners. UDL includes a framework with three general principles (multiple means of engagement, multiple means of representation, and multiple means of action and expression) each of which includes multiple guidelines and checkpoints for actual practice (CAST.org, n.d.). The goal of UDL is to increase access and usability for the greatest number of people possible. A UDL approach is structured and practical and, despite the critiques included here, is lauded for its utility by course designers and teachers alike.
UDL, however, does not meet the needs of all learners, particularly our most marginalized learners. Let me repeat: UDL does not meet the needs of our most marginalized learners, as much as we would like to believe it does. Let me highlight a few of the reasons for this.
As Dolmage (2017) explains in the book Academic Ableism: Disability and Higher Education, UDL’s emphasis on universality is problematic because universality is connected to normativity. (p. 134). Dolmage (2017) states that UDL has gained recognition by appealing to the majority, but in doing so “the needs of the majority once again trump the needs of those who have been traditionally excluded—people with disabilities” (p. 135). UDL is viewed as a framework for addressing the needs of disabled students, but its actual emphasis is on meeting the needs of the majority.
With its emphasis on “multiple means” UDL aims to include multiple learner identities and preferences; however, it “overlooks the importance of feedback from its own users” (Dolmage, 2017, p. 126). In this way, UDL ignores the individual circumstances of actual students.
By focusing on the “means,” over the students themselves, UDL is not an intersectional approach to design and teaching. Defining what a Universal Design looks like without considering the particularized realities of actual students results in the continued marginalization and erasure of students who are not in the majority.
UDL has popularized educational practices that serve many students, but in doing so, it has effectively erased the needs of some of the most marginalized students–those with disabilities. Those students with disabilities who are also part of other oppressed groups are increasingly at a disadvantage.
There’s no doubt that UDL is an incredibly useful tool and makes our course designs better, but we must not fail to recognize that UDL is not a panacea. UDL should be one of many tools we use to meet the needs of students, but let’s not forget that we need a truly intersectional approach to design and teaching. Without this, we, unwittingly or not, are contributing to the marginalization and erasure of our most disadvantaged students.
Courtney-Long, E.A., Romano, S.D., Carroll, D.D. et al. (2017). Socioeconomic Factors at the Intersection of Race and Ethnicity Influencing Health Risks for People with Disabilities. J. Racial and Ethnic Health Disparities, 4, 213–222. https://doi.org/10.1007/s40615-016-0220-5
Dolmage, J. (2017). Universal Design. In Academic Ableism: Disability and Higher Education (pp. 115-152). Ann Arbor: University of Michigan Press. Retrieved June 8, 2020, from www.jstor.org/stable/j.ctvr33d50.7
Taylor, A. & Shallish, L. (2019). The logic of bio-meritocracy in the promotion of higher education equity, Disability & Society, DOI: 10.1080/09687599.2019.1613962
U.S. Department of Education, National Center for Education Statistics. (2019). Digest of Education Statistics, 2017 (2018-070), Chapter 3. Retrieved June 10, 2020 from https://nces.ed.gov/fastfacts/display.asp?id=60
Audio is a term we all understand on some level. Commonly we think of audio as the transmission, reception, or reproduction of sound. Audible sound is what we hear. So, when we consider audio as an element of course design we are thinking about how sound can be used to communicate or support learning in an online course environment. Integrating audio in course delivery can be quite powerful if done well.
Perhaps the most common method of using sound in an asynchronous online course is via narrated lecture. The narrated lecture is typically a voice over a slide presentation or screencast. It can be an essential tool of instruction if designed effectively.
Another way the voice of an instructor can be incorporated into a course is via audio feedback on student assignments. These are short audio recordings that have been found to help build a sense of instructor presence in an online course.
Highly focused use of audio is also utilized in subjects where audio is, in essence, the topic at hand. Here we are considering language, music, and media arts courses as examples.
Other valued voices are often brought into the online learning experience via guest interviews in either audio or video format. Not to be left out, the voices of students are increasingly present in online course via tools such as VoiceThread and university provided video portals such as Kaltura Media Space. And with tools such as Zoom recording audio interviews and voice overs is easier than ever. With the availability of media platforms such as YouTube, Vimeo, Amazon Prime, Audible, and Apple Music and Podcasts the ability to bring external audio resources into the online experience seems almost limitless.
Audio is also used as a supporting resource for text-based content in online courses. In this case screen readers may typically provide the audio support. When using audio as a primary learning resource it is necessary to provide text-based transcripts as accessibility options. A fuller description of making audio accessible can be found at the W3C website.
The examples listed above are common ways we integrate audio elements into online learning. Are there other perhaps different ways we might consider? In the next few paragraphs we will explore a few ideas of how we might use audio in specialized ways in course design.
Specialized Audio Use
Orientation & Review Audio
Audio is a great tool to use when smaller segments of media should be used to orient students to a part of an online course. Think about short, more ephemeral, voice messages that can be easily produced and updated from term to term as the course changes. These audio segments need not be highly produced but should be of good quality. This type of audio segment reconnects students with the instructor via voice.
Listen to the orientation audio sample below that was used in an online course: RE 270 – Outdoor Recreation Resources, Behavior and Values | Module III Orientation by Dr. Craig Rademacher; Northern Michigan University c. 2012. (00:02:56). [download orientation audio transcript from Temi.com]
Similar short segments of audio may be used to review sections of content. The review audio may be produced by instructors or students. These audio reviews may be used in preparation for an exam, major project, or collaborations within the course. The goal of such reviews may vary but certainly one goal would be to re-focus the listener to the task at hand while providing timely tips or learning objectives.
One of the things you may have noticed about the audio clip above is the integration of music to the orientation message. Purposeful music selections can support the emotional feel of a course or module being introduced. Music can also serve as an audio cue, or audio branding, for a course. So, selecting audio stingers, or music introductions, can highlight that a particular message or topic is coming or reinforce an emotional tone if carefully planned.
The primary benefit of using audio for orientation and review is that audio is less production intensive making it a quick way to provide feedback. Audio is also fairly easy to edit with a free cross-platform audio tool such as Audacity.
Narrative
The oral traditions of learning go back centuries. Prior to print, learning was interwoven in spoken traditions, legends, and cultural stories. Today story remains a potent vehicle for learning. As you might imagine audio is a great vehicle for story.
An example of this is an Oregon State University political science course titled Governing after the Zombie Apocalypse which was designed and taught by Dr. Rorie Solberg. The story that underpins this course is that a natural disaster has caused the breakdown of the U.S. government. In response citizens must create a new government including a bill of rights and constitution. Students become the citizens creating that new government taking into account marginal populations such as the surviving zombies which are called “blues”due to their virus-caused color.
Audio is used creatively in the course by periodically inserting radio broadcasts about odd happenings around the country. Although not the heart of the effort of the course, these audio presentations, really imagined radio newscasts, provide situational tenor and decision points as students go about creating a new government. Listen to a segment of a mock radio broadcast below.
Mock radio broadcast excerpt: Story by Dr. Rorie Solberg. Produced by Oregon State University Ecampus. Voice acting credit: Warren Blyth (00:02:04)
This segment highlights how audio can be used to shape and carry a narrative through an online course. You might imagine how different narrative audio presentations may support history, literature, or science courses.
Soundscapes & Nature Sounds
Experiencing authentic places or environments is believed to be a valuable form of learning. This idea is a driving force behind field-based learning and experiential learning. Audio soundscapes provide access to authentic acoustic environments that can support online learning about the context of an environment. The environment may be urban, rural, or perhaps in a wilderness-like setting. Soundscapes may also be used to create sense of cognitive and emotional world building that can be used in instruction. Soundscapes typically feature a molar perspective of the acoustic environment.
Other, more specific sounds of nature are potentially positive resources for online instruction. Below is a sound sample of the call of a common raven. Listen to the raven’s call.
Audio recorded in the Beaver Basin Wilderness at Pictured Rocks National Lakeshore in Michigan in 2010 | Craig Rademacher. (00:00:26)
Now imagine how this simple recording may be utilized in a course. It might be used to help identify ravens from crows in an ornithology course. Perhaps this audio clip could be used to help communicate a sense of isolation for listeners studying wilderness values. Or, perhaps it could be used to introduce the poem by Edgar Allen Poe titled The Raven. In each case the audio would be used intentionally to deepen the context or experience of the learner.
Fidelity Matters
Sound quality is an important factor when selecting and using audio in an online course. Audio files need to be of sufficient quality to clearly indicate what you are expecting students to hear. Poor audio, or a confusing sound recording, is experienced as distracting and student will likely tune out. Recording and editing audio does require some knowledge and practice. And there are many places where you can learn to produce audio. LinkedIn Learning offers courses in audio production, podcasting, and even how to select a microphone. So, if you are interested give it a try.
If you are not inclined to produce your own audio content there are resources available where you can find high quality audio for use in courses. Some of these resources are royalty free. Others may require licensing of audio for use.
Final Thoughts
We have reviewed how audio is commonly used in online courses and how we might think about new ways of integrating audio. As you explore the links to resources below start to think about your next course design. How can you augment the text and video you normally use with audio? How might you leverage voice, narrative, or soundscapes to connect online students to the context, authenticity, and humanity of learning? You might want to experiment with audio at first. Start small. If it works, then you will have truly found a sound idea for online course design.
Podcast Resources
There are several ways to find audio podcasts to review for inclusion in a course. Apple Podcasts is a dominant resource in this area. Apple streams over 750,000 podcast shows with over 20 million episodes. Google Play Music is another good resource. Podcast feeds can also be found simply by browsing for podcasts online.
Audiobooks
Audiobooks are found in many online book seller sites such as Amazon (Audible.com). Additionally some more specific sites such as audiobooks.com also provide resources.
