Background: Developing online, widely accessible educational courses, such as Massive Open Online Courses (MOOCs), offer novel opportunities to advancing academic research and the educational system in resource-constrained countries. Despite much literature on the use of design-related features and principles of different pedagogical approaches when developing MOOCs, there are reports of inconsistency between the pedagogical approach and the learning activities, content, or resources in MOOCs.
Objective: We present a protocol for a scoping review aiming to systematically identify and synthesize literature on the pedagogical approaches used, and the learning activities, content, and resources used to facilitate social interaction and collaboration among postgraduate learners in MOOCs across the health sciences.
Methods: We will follow a 6-step procedure for scoping reviews to conduct a search of published and gray literature in the following databases: Medline via Ovid, ERIC, SCOPUS, Web of Science, and PsychINFO. Two reviewers will screen titles, abstracts, and relevant full texts independently to determine eligibility for inclusion. The team will extract data using a predefined charting form and synthesize results in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews checklist.
Results: The scoping review is currently ongoing. As of March 2022, we have performed initial data searches and screened titles and abstracts of the studies we found but revised the search string owing to inaccurate results. We aim to start analyzing the data in June 2022 and expect to complete the scoping review by February 2023.
Conclusions: With the results of this review, we hope to report on the use of pedagogical approaches and what learning activities, content, and resources foster social and collaborative learning processes, and to further elucidate how practitioners and academics can harvest our findings to bridge the gap between pedagogics and learning activities in the instructional design of MOOCs for postgraduate students in the health sciences.
International Registered Report Identifier (IRRID): DERR1-10.2196/35878
Developing online, widely accessible educational courses, such as Massive Open Online Courses (MOOCs), offer novel opportunities to advance academic research and the educational system in resource-constrained countries [, ]. A PhD proposal or research protocol is one of the first important academic texts aspiring researchers write worldwide [ ], however; there is little emphasis on developing academic writing at the postgraduate level [ , ]. To facilitate social equity in the health sciences, an accessible course focusing on PhD proposal writing could offer opportunities for students from low- and middle-income countries (LMICs) to increase their writing capacity and successfully compete for PhD scholarships and scholarly positions at research-intensive higher education institutions (HEIs) [ , ].
MOOCs are “online courses designed for large numbers of participants that can be accessed by anyone anywhere as long as they have an internet connection (massive), are open to everyone without entry qualifications, and offer a full/complete course experience online for free” . As such, a MOOC brings together participants from diverse backgrounds and disciplines worldwide with a common interest to learn and coconstruct knowledge in a socially networked, nonformal, computer-mediated learning environment [ , ].
Traditionally, there are 2 main types of MOOCs depending on the pedagogical approach underpinning the design, affordances of the course platform, and the degree of openness (nonprofit or for profit) [, ]. A cMOOC has a social learning, or connectivist approach [ ]. In cMOOCs, active social interaction and collaboration is viewed as required to coconstruct knowledge as a member of an active open, online community [ , ]. Through the learning activities, “learners decide their own objectives, share their knowledge and collaboratively build their ideas and artifacts” [ ]. An xMOOC uses an individual learning or cognitive behavioral approach with didactic or transmission models of teaching, with somewhat limited openness (often for profit) and less emphasis on learners’ coconstruction of knowledge [ , ]. Here, the teaching methods often include information transmission, computer-marked assignments, and peer assessment [ ]. Other more recent MOOCs use a blended-learning approach, also referred to as hybrid MOOCs (eg, bMOOCs and smMOOCs) combining elements from the different pedagogical approaches [ , ].
Although literature reviews have summarized the use of design-related features and principles of different pedagogical approaches [, , - ], the quality of the learning experiences in MOOCs have been questioned [ , ]. Common learning activities of most types of MOOCs include video or face-to-face lectures, blogs, discussion forums, social networks, lecture notes, PowerPoint slides, and PDFs [ ]. However, in cMOOCs or blended-learning MOOCs, where social interaction and collaboration is anticipated to foster key learning throughout or in parts of the course, studies have demonstrated that there are limited learning activities in which social learning could take place [ , ]. In addition, a review of cMOOCs and xMOOCs found that only one-third of them had material or resources, which learners viewed relevant for the learning outcome [ ]. Considering the dynamic and complex individual learning process in a computer-mediated setting [ ], it is important that learning activities, content, and resources reflect an a priori pedagogical approach underpinning the MOOC design, and that the materials offered are viewed by learners as helpful and relevant to achieve the stated learning objectives [ ].
In this paper, we present a protocol for a scoping review aiming to systematically identify and synthesize literature on the pedagogical approaches used, and the learning activities, content, and resources used to facilitate social interaction and collaboration among postgraduate learners in MOOCs across the health sciences.
We will follow the 6-step systematic procedure for scoping reviews as outlined by Arksey and O'Malley . This procedure includes (a) identifying the research question; (b) identifying relevant studies; (c) selecting studies; (d) charting the data; (e) collating, summarizing, and reporting the data; and (f) an optional consultation exercise. We will also use the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist to further guide what we report in this protocol and the final review [ ] while adhering to recommendations made in recent methodological scoping review papers that advance the procedure proposed by Arksey and O’Malley [ - ].
As the findings of the proposed scoping review can inform a potential revision of the MOOC How to Write a PhD Proposal, we account for the theoretical positioning underpinning the work to develop this MOOC and the pedagogical approach used in the design phase. The work aligns with the knowledge creation metaphor of learning, where learning is viewed “as a process of knowledge creation which concentrates on mediated processes where common objects of activity are developed collaboratively” . In a computer-mediated online learning setting, such as an MOOC, an underlying principle of the knowledge creation approach to learning is that each learner’s effort to advance knowledge and skills occurs in, and benefits from, participation in the social online community [ , ]. This understanding is in line with the pedagogical approach of cMOOCs and the blended learning approaches used in MOOCs. However, when embarking on developing the MOOC in PhD proposal writing, we had to adjust the thinking underlying the course design to accommodate the blended learning approach integrated in the online learning platform FutureLearn. Although we designed the key activities in accordance with the social knowledge creation approach to learning, the blended learning principles integrated in the FutureLearn platform also accommodated other ways of learning (eg, acquisition learning [ ]).
Each learner who registers for the MOOC on PhD proposal writing is assumed to have a research idea, but to develop the research idea into a plan or proposal (the common object of activity), the learners are encouraged to actively participate in the key learning activities or tasks that require social interactions and extensive collaboration with other learners (eg, reviewing the work of others together with other participants). Through these forms of feedback and collaborative work, participants could learn from others by reading the work of others and integrating key elements of PhD proposal writing into their own work.
To ensure consistency throughout this scoping review, the lead researcher will spearhead the work during each stage of the review process . We will set up an iterative process to ensure a well-crafted scoping review that demonstrates procedural transparency and methodological rigor [ , ]. We are a research team with experience in conducting exploratory research and various types of literature reviews in the fields of medicine, nursing, education, and sports science. Throughout the process, we will also track refinements of the proposed protocol [ ].
Identifying the Research Question
The first step of the systematic scoping review process involves identifying and defining the parameters of the research question . This is usually done a priori to guide the literature searches, followed by an iterative process of refinement based on familiarization with the identified research evidence [ ]. Our preliminary research question is “What are the pedagogical approaches, and the learning activities, content, and resources used to foster social interaction and collaborative learning processes when designing MOOCs for postgraduate students in the Health Sciences?” We designed our research question to be broad, to generate a large but also manageable pool of data to establish an overview of the topic investigated [ , ]. We have formulated the following subquestions to further guide our scoping inquiry:
- What are the pedagogical approaches used to guide the design of MOOCs in the health sciences?
- What are the learning activities, content and resources used in MOOCs to facilitate social interaction and collaboration?
- What are the social, collaborative learning activities, content, and resources do participants report as helpful and relevant in MOOCs?
Defining and operationalizing the concepts of a review question is useful to clarify the scope . It guides the subsequent phases of exploration and clarifies the final review report [ ]. For this study protocol, the term “design” refers to the development phase of MOOCs preceding the online launch. We define postgraduate studies as academic studies above the master’s level (eg, MA, MSc, and MPhil) or equivalent (eg, MD). As such, we refer to postgraduate students as a group of people who have completed their master’s thesis or similar academic degrees. When using the phrase “PhD proposal,” we refer to a research protocol’s specific format, genre, and style of writing needed when developing a PhD project proposal or presenting a stringent research plan, and, as such, preparing for work that meets the requirements for writing research articles or manuscripts of project results.
Identifying Relevant Studies
The second step of the scoping review is to identify relevant literature . Clear definitions and operationalization of the research question, and a specification of the inclusion criteria that underpins the search strategy are necessary [ ]. However, developing the search strategy is an iterative and reflexive process that requires assessment and refinement based on both the pilot searches and the materials yielded from the searches [ ]. We will start with a comprehensive and broad search including published and gray literature. In line with the search guidelines developed by Aromataris and Riitano [ ], we will (i) explore published literature in electronic databases, (ii) hand-search reference lists in the identified literature, and (iii) review the table of contents in a few key journals. We will search gray literature (i) by hand-searching selected organization, government, and conference websites, (ii) using selected gray literature databases, and (iii) using online search engines.
To ensure breadth and optimize consistency of the literature searches, an experienced research librarian will build, structure, and conduct all literature searches across databases . As the search may yield a wide variety of studies across both published and gray literature, we will not exclude any published literature on the basis of the study design.
We will adapt a Population (or participants)/Concept/Context (PCC) mnemonic and use the Boolean logic of “AND” and “OR” to build the various searches across databases. Findings in each database and the full combination of key terms used will be registered for each individual grid. We will pilot the search strategy using a 3-step approach, which includes (a) an initial search of a key database and analyze alternative terms and index or MeSH terms used to describe the concepts and (b) search the chosen database for the keywords and index terms identified . Based on the piloting of our search strategy and familiarization with the research field, we may refine or add keywords and index or MeSH terms before searching the remaining databases. We will report the full details of our searches in the electronic databases as an appendix in the main scoping review paper [ ].
Our search will cover the period from 2004 to March 2022. Searching for literature before 2004 is not likely to yield relevant results since the first MOOC was launched in 2008. Written material to be included in the scoping review is, for practical and financial reasons, limited to English and the Nordic languages. Using the reference management software EndNote, we will collate and organize all studies extracted per database  and use the integrated algorithm in EndNote to screen for duplicate references [ ]. One reviewer will then carry out an initial screening of all references identified in the literature search to exclude clearly irrelevant studies; for example studies or reports emphasizing online courses targeting children, primary school students, or high school students. If the reviewer has any doubt about whether to exclude a study or report, it will be included for further scrutiny by other reviewers at a later stage.
We will search the multidisciplinary electronic databases (eg, SCOPUS) illustrated in. Some electronic databases have a somewhat similar interface but index subject headings differently. The research librarian will tailor our search of each database to include the specific subject headings listed.
|Education Resources Information Centre||International literature database within pedagogy|
|Medline via Ovid||Medicine, nursing, dentistry, biomedical research|
|Web of Science||Cross-disciplinary, international database with mainly articles|
|SCOPUS||Large bibliographic database containing abstracts and citations for academic journal articles (peer-reviewed journals in medical, technical, and social sciences, including the arts and humanities)|
|PsycINFO via OVID||Centered on psychology and the behavioral and social sciences|
Hand-Searching Reference Lists and Key Journals
We will investigate the bibliographies of literature findings and the table of contents in a few key journals that publish studies on pedagogical practices and instructional designs in MOOCs for additional relevant papers. A special emphasis is on ensuring that we also select journals that are not well indexed in available commercial or public databases to avoid missing out on potentially eligible studies .
Searching for gray literature in repositories that specifically target this form of literature is useful to ensure breadth and limit the publication bias in literature reviews . We define gray literature as conference proceedings and literature produced at all levels of government, academic, business, and industry, available in electronic and printed formats not controlled by commercial publishing houses. This includes any paper or study not formally published or peer reviewed; that is, reports, working papers, theses and dissertations, conference posters and presentations, unpublished protocols and guidelines, market reports, government documents, and white papers [ ].
We will search a selection of the gray literature databases illustrated in. As these repositories may not have advanced search modes, we will use a few carefully selected critical key terms from our search grid [ ]. To ensure comprehensiveness, we will also search the unpublished literature on the internet using online search engines, such as Google, in their incognito, advanced search mode with a stringent search query based on the terms used in the search string. Using relevance filtering, we will further limit the assessment to the first 50 hits per combination of key terms [ ]. We will list the potential findings using EndNote [ ].
Synthesizing gray literature and gray data in a systematic way is a difficult task . To ensure replicability, we will list the findings from each stringent combination of key terms used in each database and add as an appendix to the main review paper.
|Open-Grey||An open access gray literature database that includes technical or research reports, doctoral dissertations, some conference papers, some official publications, and other types of gray literature. OpenGrey covers Science, Technology, Biomedical Science, Economics, Social Science, and Humanities |
|ProQuest Dissertations and Theses||ProQuest has one of the largest collections of electronic theses and dissertations available worldwide|
|Google Scholar||International database that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines, including educational science|
The third step of the scoping review is an iterative assessment of the identified literature using the defined eligibility criteria [, ]. A flowchart will illustrate the selection process of the literature [ ]. It will specify the number of duplicates that are removed and the references excluded on the basis of either the title or abstract or the full text [ ].
We will include all studies that report on conceptual or theoretical frameworks or pedagogical approaches used to guide design considerations; selection or structuring of content, learning activities, and resources that are assumed to stimulate social interaction and collaboration; or participant views of learning material anticipated to foster social learning in cMOOCs and hybrid MOOCs. We will exclude studies or reports that describe learning material, content, or resources in xMOOCs or comparisons of online learning platforms, grading, or assessment of learner achievements, or outcome or operational or technical issues (eg, weekly supervision of courses and considerations during the launch of MOOCs), as well as studies that target participants at the undergraduate level at HEIs.
We will review studies using an iterative 2-step approach . The first step involves importing all references into the app-based literature review program Covidence to screen the title and abstract in accordance with inclusion or exclusion criteria. To do so, we will hold team meetings at the beginning, midpoint, and end to discuss progress as well as any challenges and uncertainties (the number of references identified dictates the number of meetings required) related to study inclusion. If necessary, the search strategy will be refined. The second step involves the full reading of the retained studies and reports to decide on final inclusion. We will first extract the references from Covidence back into EndNote to acquire the full-text version of the studies or reports before the references are imported back to Covidence for the full read.
During the study selection process, two reviewers will screen titles, abstracts, and relevant texts independently to determine eligibility for inclusion in this scoping review [, ]. If the first reviewer chooses to include a reference, the second reviewer will verify his or her selection of the specific study. If there are divergences between reviewers, a third reviewer will be consulted.
We will use the Mixed Methods Appraisal Tool (MMAT) to appraise the quality of the published literature identified . The MMAT tool includes an extensive assessment procedure of qualitative, quantitative, and mixed methods studies. To appraise the quality of gray literature, we will use the AACODS (Authority, Accuracy, Coverage, Objectivity, Date and Significance) checklist [ ]. We will not perform a full assessment of methodological robustness as we argue that it is more important and appropriate for a scoping review to capture breadth in the literature rather than a limited sample of methodological quality-assessed studies used for systematic reviews.
The fourth step of the scoping review process entails organizing and synthesizing the data in accordance with key themes . Based on initial discussions, the research team plans to extract and chart a mixture of general information about the studies or reports in this scoping review. More specifically, we will extract the names of the authors, year of publication, study location, type of document source, aims, and any assessments of target groups or participants. Underpinned by the knowledge creation approach to learning, we will extract specific information from each study and report related to conceptual or theoretical frameworks and pedagogical approaches; selection or structuring of social, collaborative learning activities, content, and resources; and participant evaluations of any material viewed as helpful in cMOOCs or hybrid MOOCs to stimulate interaction and collaboration.
We will develop a predefined Microsoft Excel data charting form. Refinement of the charting form may be necessary, as data extraction is an iterative process involving continual assessment, evaluation, and updating . To ensure consistent data extraction, the lead researcher will undertake a pilot of the form on a random sample of 5-10 studies. In consultation with the full research team, we will modify, if necessary, to mitigate inconsistencies and uncertainties of use to ensure coherence with the research question [ , ].
Collating, Summarizing, and Reporting the Results
The fifth phase of the scoping review process entails summarizing and reporting findings in a clear and consistent way . Throughout the reporting phase, we will follow the recommendations of the PRISMA-ScR guidelines when writing up the final review [ ]. We will summarize and describe each identified study’s characteristics (eg, author name, study location, and year of publication), as well as the pedagogical and theoretical or conceptual approach used in the design of the MOOC, the learning activities, content, and resources adopted and any participant evaluation of materials used in MOOCs. We will then report the output with an emphasis on describing how the findings relate to the subquestions guiding the scoping review before discussing the implications for pedagogical practice and instructional design of MOOCs targeting prospective postgraduate students in the health sciences.
Consulting Relevant Stakeholders
The sixth stage of the scoping review process is an optional consultation stage . We agree with other researchers who argue that consultation could be useful for knowledge translation [ , ]. We will consider inviting relevant stakeholders, such as international PhD students from LMICs and experts in online, computer-mediated pedagogies, to advice on how to provide appropriate, useable learning resources, and how learning activities or material identified can be used to underpin inclusive, collaborative learning processes among students with diverse backgrounds in a potential revision of the MOOC How To Write a PhD Proposal.
To disseminate findings, we seek to publish the scoping review in an international peer-reviewed journal with a scope that includes the use of collaborative pedagogies in computer-mediated and online learning environments. We also aim to disseminate the findings to professionals and other colleagues in postgraduate studies to potentially inspire improved instructional designs of MOOCs.
The scoping review is currently ongoing. As of March 2022, we have completed an initial data search and screened the title and abstract of all references. However, as the initial search captured a large bout of irrelevant published studies beyond the field of study, we decided to revise the search string across all databases in a second literature search. We aim to start analyzing the data in June 2022 and expect to complete the scoping review by February 2023. The proposed scoping review will inform a potential revision of the MOOC How To Write A PhD Proposal. The MOOC is currently delivered on the FutureLearn platform by colleagues from the University of Oslo. The MOOC is funded by the University in Oslo and the NORPART: EXCEL SMART program (2016/10213).
We have developed a scoping review protocol aimed at investigating the pedagogical approaches used, and the type of learning activities, content, and resources used to foster social interaction and collaboration when designing MOOCs at postgraduate level in the health sciences.
The findings of the proposed scoping review may have direct implications for academic staff and professionals working with the instructional design or pedagogical practices of MOOCs or in other online, collaborative courses. First, it could directly influence the way key learning activities and tasks are structured or designed in cMOOCs or blended-learning MOOCs. As the proposed scoping review can inform a potential revision of the MOOC in PhD proposal writing, where the first runs had a wide range of participants from both high-income countries and LMICs worldwide, identifying how social or collaborative learning activities and resources can be adjusted to accommodate the needs of students from LMICs or resource-constrained educational system (eg, internet connectivity issues) can result in a higher completion rate of students enlisting for the course . To this end, it could influence the extent of which participants improve the quality of their PhD proposal writing and thus have social implications for the recruitment of candidates to PhD programs in increasingly internationalized research education programs across the globe. Second, understanding how current learning activities, content, or resources can be modified to underpin inclusive, collaborative learning processes among participants with diverse backgrounds, we could potentially improve course designs and the quality of the individual learning experience in MOOCs using a social learning or blended learning approach [ ].
We followed the PRISMA-ScR to inform what we reported throughout this protocol . Very few scoping reviews are reported, and even fewer published [ ]. We contend that publishing this protocol can contribute to increased transparency and methodological rigor of the final review and future review studies, as many scoping reviews often lack sufficiently detailed descriptions of search strategies and procedures [ , ]. In accordance with the PRISMA-ScR [ ], we have therefore detailed our research strategy, the theoretical position, and how this could influence the specific phases of the review procedure. We have also described our search strategy for both published and gray literature databases in a detailed way. As there is usually no methodological assessment of included studies in scoping reviews, using quality appraisal tools such as the MMAT [ ] and the AACODS checklist [ ] can further increase the rigor, accuracy of interpretation of data, and synthesis of findings [ , , ]. Through these measures, the potential for reproducibility increases and allows readers of the proposed scoping review to fully engage with and assess the assumptions underpinning the work, as well as the specific steps of the scoping review procedure [ ].
However, there are limitations related to the scoping review procedure as described in this protocol. As the research team comprises relatively few members, there is a risk that the literature searches can produce an excessive amount of data, which is unfeasible to process and analyze. To mitigate this, we have recruited a research librarian to compile accurate but comprehensive search strategies. If the body of literature proves unmanageable or the database searches are inaccurate, we will further adjust the search string and continuously assess the scope of our search.
We would like to thank Nicoline Lokdam at University of Oslo (UiO) for her contributions during the initial planning of the scoping review process leading up to this protocol, and the research librarian Marte Ødegaard at the Faculty of Medicine, UiO. We acknowledge the financial support from the NORPART-2016/10213 - EXCEL SMART-Exceeding Learning through Strategic Mobility and Academic Partnership Project grant from The Directorate for Higher Education and Competence. The funding agency had not impact on the planning or development of the protocol.
Conflicts of Interest
- Jansen D, Schuwer R. Institutional MOOC strategies in Europe: Status report based on a mapping survey conducted in October - December 2014. EADTU. 2015. URL: https://www.robertschuwer.nl/download/Institutional_MOOC_strategies_in_Europe.pdf [accessed 2022-05-16]
- Lambert SR. Do MOOCs contribute to student equity and social inclusion? A systematic review 2014–18. Computers & Education 2020 Feb;145:103693. [CrossRef]
- Elliot DL, Reid K, Baumfield V. Beyond the amusement, puzzlement and challenges: an enquiry into international students’ academic acculturation. Stud High Educ 2015 Apr 07;41(12):2198-2217. [CrossRef]
- Langum V, Sullivan KP. Writing academic english as a doctoral student in sweden: narrative perspectives. Journal of Second Language Writing 2017 Mar;35:20-25. [CrossRef]
- Aitchison C, Lee A. Research writing: problems and pedagogies. Teach High Educ 2006 Jul;11(3):265-278. [CrossRef]
- Bennett A, Southgate E, Shah M. Chapter 15 - Global Perspectives on Widening Participation: Approaches and Concepts. In: Widening Higher Education Participation: A Global Perspective. Cambridge: Chandos Publishing; 2016:241-253.
- Toth CT. Massive Open Online Courses (MOOCs). In: Massive Open Online Courses im Kontext von Persönlichkeit und Prokrastination. Wiesbaden: Springer; 2020.
- Liyanagunawardena TR, Adams AA, Williams SA. MOOCs: A systematic study of the published literature 2008-2012. IRRODL 2013 Jul 05;14(3):202. [CrossRef]
- Joksimović S, Poquet O, Kovanović V, Dowell N, Mills C, Gašević D, et al. How Do We Model Learning at Scale? A Systematic Review of Research on MOOCs. Rev Educ Res 2017 Nov 14;88(1):43-86. [CrossRef]
- Ichimura Y, Suzuki K. Dimensions of MOOCs for quality design: Analysis and synthesis of the literature. Int J Educ Media Technol 2017;11(1):42-49 [FREE Full text]
- Ebben M, Murphy JS. Unpacking MOOC scholarly discourse: a review of nascent MOOC scholarship. Learn Media Technol 2014 Jan 22;39(3):328-345. [CrossRef]
- Downes S. Connectivism. AsianJDE 2022 [FREE Full text]
- Toven-Lindsey B, Rhoads RA, Lozano JB. Virtually unlimited classrooms: Pedagogical practices in massive open online courses. Internet High Educ 2015 Jan;24:1-12. [CrossRef]
- Yousef A, Chatti M, Schroeder U, Wosnitza M, Jakobs H. A Review of the State-of-the-Art. 2014 Presented at: 6th International Conference on Computer Supported Education; April 1-3, 2014; Barcelona. [CrossRef]
- Zawacki-Richter O, Bozkurt A, Alturki U, Aldraiweesh A. What Research Says About MOOCs – An Explorative Content Analysis. IRRODL 2018 Feb 23;19(1). [CrossRef]
- Oh EG, Chang Y, Park SW. Design review of MOOCs: application of e-learning design principles. J Comput High Educ 2019 Nov 21;32(3):455-475. [CrossRef]
- Zhu M, Sari AR, Lee MM. A comprehensive systematic review of MOOC research: Research techniques, topics, and trends from 2009 to 2019. Education Tech Research Dev 2020 Jun 22;68(4):1685-1710. [CrossRef]
- Zhu M, Sari A, Lee MM. A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education 2018 Apr;37:31-39. [CrossRef]
- Margaryan A, Bianco M, Littlejohn A. Instructional quality of Massive Open Online Courses (MOOCs). Comput Educ 2015 Jan;80:77-83. [CrossRef]
- Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005 Feb;8(1):19-32. [CrossRef]
- Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018 Oct 02;169(7):467-473 [FREE Full text] [CrossRef] [Medline]
- Peters M, Marnie C, Tricco A, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth 2020 Oct;18(10):2119-2126. [CrossRef] [Medline]
- Colquhoun HL, Levac D, O'Brien KK, Straus S, Tricco AC, Perrier L, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014 Dec;67(12):1291-1294. [CrossRef] [Medline]
- Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010 Sep 20;5:69 [FREE Full text] [CrossRef] [Medline]
- Paavola S, Hakkarainen K. The Knowledge Creation Metaphor – An Emergent Epistemological Approach to Learning. Sci Educ 2005 Aug;14(6):535-557. [CrossRef]
- Paavola S, Lipponen L, Hakkarainen K. Models of Innovative Knowledge Communities and Three Metaphors of Learning. Review of Educational Research 2016 Jun 30;74(4):557-576. [CrossRef]
- Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med Res Methodol 2013 Mar 23;13:48 [FREE Full text] [CrossRef] [Medline]
- Armstrong R, Hall BJ, Doyle J, Waters E. Cochrane Update. 'Scoping the scope' of a cochrane review. J Public Health (Oxf) 2011 Mar 23;33(1):147-150. [CrossRef] [Medline]
- Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs 2014 Mar;114(3):53-58. [CrossRef] [Medline]
- Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods 2014 Dec;5(4):371-385 [FREE Full text] [CrossRef] [Medline]
- Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc 2016 Jul;104(3):240-243 [FREE Full text] [CrossRef] [Medline]
- Natal G. Assembling the Pieces of a Systematic Review: A Guide for Librarians. J Med Libr Assoc 2018 Jul 02;106(3):393. [CrossRef]
- Tillett S, Newbold E. Grey literature at The British Library: revealing a hidden resource. Interlend Doc Supply 2006;34(2):70-73. [CrossRef]
- Peters M, Godfrey C, Khalil H, McInerney P, Parker D, Soares C. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 2015 Sep;13(3):141-146. [CrossRef] [Medline]
- Adams J, Hillier-Brown FC, Moore HJ, Lake AA, Araujo-Soares V, White M, et al. Searching and synthesising 'grey literature' and 'grey information' in public health: critical reflections on three case studies. Syst Rev 2016 Sep 29;5(1):164 [FREE Full text] [CrossRef] [Medline]
- Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, et al. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud 2012 Jan;49(1):47-53. [CrossRef] [Medline]
- The AACODS Checklist. 2010. URL: https://dspace.flinders.edu.au/xmlui/bitstream/handle/2328/3326/AACODS_Checklist.pdf;jsessionid=CE04F9BFF9825B732BC5EA6D04B15261?sequence=4; [accessed 2022-05-16]
- Khalil H, Ebner M. MOOCs Completion RatesPossible Methods to Improve Retention - A Literature Review. 2014 Presented at: World Conference on Educational Media and Technology; 2014; Tampere.
- Tricco AC, Lillie E, Zarin W, O'Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol 2016 Feb 09;16:15 [FREE Full text] [CrossRef] [Medline]
|HEI: higher Education Institutes|
|LMIC: low- and middle-income country|
|MMAT: Mixed Methods Appraisal Tool|
|MOOC: Massive Open Online Course|
|PRISMA-ScR: Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews|
Edited by T Leung; submitted 21.12.21; peer-reviewed by S Yamane, J Kimmerle, E Toki; comments to author 22.02.22; revised version received 29.03.22; accepted 30.03.22; published 30.05.22Copyright
©Øystein Røynesdal, Jeanette H Magnus, Anne Moen. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 30.05.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.