{"id":1443,"date":"2018-11-28T09:51:03","date_gmt":"2018-11-28T17:51:03","guid":{"rendered":"http:\/\/ecampus.oregonstate.edu\/research\/?page_id=1443"},"modified":"2021-01-07T13:40:06","modified_gmt":"2021-01-07T21:40:06","slug":"methodology","status":"publish","type":"page","link":"https:\/\/ecampus.oregonstate.edu\/research\/projects\/report-reader\/checklist\/methodology\/","title":{"rendered":"Report Reader Checklist: Methodology"},"content":{"rendered":"

This section should describe the details and process of how a study was conducted from start to finish. This information should be descriptive enough that you could conduct the study again (or replicate the study) if you wanted to. When information about the methodology of a study is missing, you may have difficulty understanding specifically what the study investigated and how it was carried out. This lack of understanding might prevent you from being able to evaluate how and where the study\u2019s results can be applied. The following are general components to look for relating to methodology. The Sample<\/a> area of the checklist also contains components to look for about the participants in the methodology section.<\/p>

 <\/div>\n
a. The report has a methodology section.<\/p>\n

It is important for you to understand how the study was conducted, including who were the participants (see Sample<\/a> area of checklist), how were they recruited (see Sample<\/a> area), and how were the data collected and analyzed (see b and c below). This level of detail gives you a better understanding of the results of the study. Often, this kind of information will be included in one section of the report, or in a report appendix, and labeled as the methodology section.

[see examples<\/a>]<\/div><\/div>\n
b. It is clear how data were collected.<\/p>\n

Understanding how data were collected or how researchers went about gathering information from people for the study can provide you with a better understanding of the overall results of a study. For example, were the participants interviewed? Were the data collected through an anonymous online survey? Was the study based on archival (already-existing) data? Knowing more about the data collection helps you to better assess the validity<\/span><\/span> of the study results.

[see examples<\/a>]<\/div><\/div>\n
c. It is clear how data were analyzed.<\/p>\n

In addition to understanding how the data were collected, it is important to know how the data were analyzed, that is, how the researchers combined data from multiple participants and presented them as results. If the results are quantitative<\/span><\/span>: Did the researchers provide frequency data? Did they calculate averages? Did they calculate new variables from the data? If the results were qualitative<\/span><\/span>: Were the data coded? If so, what methods were used to code the data? Sometimes studies will use both or a mixed-methods approach. This kind of methodological information can be described prior to the presentation of the results in a section of the report detailing the methodology used to collect and analyze the data. Alternatively, if this detail is not in the main report, readers may be directed to an appendix that has this information. To fully understand the results that are being presented, it is important to know, in detail, how the data were analyzed.

[see examples<\/a>]<\/div><\/div>\n
d. If statistical analyses were used, specific tests are named.<\/p>\n

If the data analysis included statistical testing, the authors should describe the statistical tests used to formulate the study results. It is important for you to understand what statistical test was used on which variables. This allows you to determine if the statistical analyses were appropriate for the data type and research questions. Good reports will describe why they chose a particular statistical test. This description might occur as the data are being presented or in a section of the report detailing the methodology used to collect and analyze the data.

[see examples<\/a>]<\/div><\/div>\n
e. If coding was performed, the coding procedure is described.<\/p>\n

If the methodology for the study includes qualitative data (such as responses from interviews or focus groups), the report should address how that data were handled. Qualitative data is frequently analyzed through a process called \u201ccoding\u201d where researchers look for patterns in the data to draw conclusions. To determine whether a report includes this kind of analysis, you will want to look for the following: Did the study author(s) begin with predetermined themes before they analyzed the study data or did themes emerge as they reviewed the data? Did more than one person code the data to ensure agreement on data themes? It is important for you to understand how the authors created categories or synthesized qualitative data to draw conclusions, and a description of the coding procedure is an important element of the study methodology.

[see examples<\/a>]<\/div><\/div>\n

<\/p>\n

Examples<\/h2>\n

a. The report has a methodology section.<\/h3>\n
\nFox, K. Bryant, G., Lin, N., & Srinivasan, N. (2020). Time for class \u2013 COVID-19 edition part 1: A national survey of faculty during COVID-19. <\/em>Tyton Partners and Every Learner Everywhere. [link<\/a>]\n