EXTENDING THE SCHOPING REVIEW FRAMEWORK 1

Extending the scoping review framework: A guide for interdisciplinary researchers

Georgia Teare* and Marijke Taks

School of Human Kinetics, University of Ottawa, Ottawa, Ontario, Canada

*Corresponding author: [email protected]

Biographical notes

Georgia Teare is a PhD student at the University of Ottawa, School of Human Kinetics, under

the supervision of Dr. Marijke Taks. Georgia researches the relationship between passive

consumption of sport and active sport participation, primarily among youth populations. She also

researches the impacts of sport events on local communities.

Marijke Taks is a professor in the School of Human Kinetics at the University of Ottawa. Dr.

Taks’ area of expertise is in socio-economic aspects of sport and leisure. Her current research

focusses particularly on impacts, outcomes and leveraging of small and medium sized sport

events, and their meaning for host communities. She also studies sport consumer behaviour of

various groups in society. Mass participation and the “Sport for All” philosophy guide her

research.
EXTENDING THE SCHOPING REVIEW FRAMEWORK 2

Extending the Scoping Review Framework: A Guide for Interdisciplinary Researchers

Abstract

Reviews of literature are conducted within academia to summarize and highlight gaps in findings

and research approaches associated with a particular line of inquiry. Identifying sources to be

included in reviews in rigorous ways can be challenging for interdisciplinary research topics, as

relevant articles may be found in a wide variety of sources. Guidelines on conducting reviews

rarely include step-by-step methods of identifying articles. Thus, this research note extends

Arksey and O’Malley's (2005) scoping review framework by adding a more systematic method

of manual searching to the common data-base search. As demonstrated through the application,

one single rigours method of source identification is not sufficient for identifying articles in an

interdisciplinary line of inquiry. The described systematic manual search presents a

complementary method to the database search, rather than as a supplementary method to assists

researchers in their searches of interdisciplinary topics.

Keywords: Systematic Search; Database Search; Hand Search; Journal Search; Manual Search
EXTENDING THE SCHOPING REVIEW FRAMEWORK 3

Extending the Scoping Review Framework: A Guide for Interdisciplinary Researchers

This research note considers the scoping review process. Scoping reviews conduct a

preliminary assessment of potential size and scope of available research literature (Grant &

Booth, 2009). Arksey and O’Malley (2005) present a comprehensive framework for scoping

reviews that involves five stages, and an optional sixth stage. The first stage involves identifying

the research question. This will help the research team decide what type of review is most

appropriate, as well as help shape the parameters of the review (Arksey & O’Malley, 2005). The

second stage of identifying relevant studies involves the research team identifying articles to read

that are related to the purpose of the review. This preliminary reading helps to determine

appropriate inclusion criteria, search terms, time span, language, and importantly, what sources

to search and search methods to use (Arksey & O’Malley, 2005). Potential sources include

electronic databases, reference lists, hand searching of key journals, conference proceedings,

organizations’ manuals, and so on (Arksey & O’Malley, 2005). Next, in the third step of study

selection, inclusion and exclusion criteria are used to identify the specific studies to be included

in the review (Arksey & O’Malley, 2005). Once the articles are identified, step four calls for the

data to be charted (Arksey & O’Malley, 2005). For a scoping review, the data that is extracted is

not the data collected by the authors in the identified studies, but rather the characteristics of the

article that will help the researchers address their research question, such as year of publication,

theoretical framework, and methodology (Arksey & O’Malley, 2005). Stage five, collating,

summarizing, and reporting results, involves presenting an overview of the breadth of the

literature (Arksey & O’Malley, 2005). Within a scoping review, a synthesis of the literature is

not the focus, as the purpose is not to synthesize, but rather to present an overview of the scope

of the current literature (Arksey & O’Malley, 2005). The final, and optional stage, is consultation
EXTENDING THE SCHOPING REVIEW FRAMEWORK 4

with external stakeholders for additional sources, if it would support the purpose of the review

(Arksey & O’Malley, 2005).

There are many ways to go about identifying sources for articles, including a database

search (Grant & Booth, 2009), as well as a manual search (Vassar, Atakpo, & Kash, 2016).

Guidelines on conducting reviews rarely include step-by-step methods of identifying articles,

with an emphasis being placed on reporting of results (e.g., Grant & Booth, 2009; Vassar et al.,

2016). Often, manual searching is mentioned, however, as a supplemental search method to the

database search. Moreover, a guide on how to systematically conduct a manual search has yet to

be identified; researchers who employ the manual searching method do so at their own

discretion, and their methods of going about manual searching are often not reported (Vassar et

al., 2016). Though manual searching is mentioned (e.g., reference lists, hand searching of key

journals), Arksey and O’Malley's (2005) framework still considers manual searching to be

supplemental to database searching, and the recommended steps remain vague and not

systematic. Thus, there is an opportunity to advance the method of conducting manual searches

to become more systematic and rigorous. As such, the purpose of this research note is to extend

Arksey and O’Malley's (2005) framework to be more comprehensive and systematic, and

advocate for this suggested method of manual searching to be used as a complementary method

to the database search, rather than as a supplementary method. This extension will be of

particular use to interdisciplinary researchers, as we will discover below, the current practices of

database searches alone for article identification are insufficient. The following sections will

detail specifically how to conduct a systematic database search, and a systematic manual search,

both as stand-alone searches that complement each other, rather than supplement.

Method
EXTENDING THE SCHOPING REVIEW FRAMEWORK 5

Systematic database search

The most common way of selecting studies is through database searches. Although there

are databases that cater to specific social science fields, the interdisciplinary nature of many

social science domains means that potentially relevant articles could span several different

databases and use different terms to describe the phenomenon. We argue that for

interdisciplinary topics, more than one systematic search strategy is required. Databases are

chosen based on the research question and which databases are likely to contain relevant articles

or other sources. Key search terms are entered into the database search bar, and the produced

articles go through two rounds of screening to determine if they should be included or excluded

in the final pool of sources. First, titles and abstracts are screened against the inclusion and

exclusion criteria. If the source either fits within the search criteria, or if more information is

needed, the source moves on to the second round of screening. Articles that do not meet the

inclusion criteria, or ones that meet the exclusion criteria, are excluded. The second round of

screening involves reading the full text and comparing against the inclusion/exclusion criteria for

fit with the purpose. Relevant articles are included in the final pool of articles to be included in

the review, while non-relevant articles are excluded.

Systematic manual search

To date, manual searching has not been conducted in as rigorous of fashion as the

database search counterpart. Manual searching involves the researcher or research team manually

looking at selected journals for additional articles (hand search), or select reference lists

(reference list search) and is typically used to supplement a database search (Vassar et al., 2016).

Reference lists of key articles (as determined by the researcher) or journals (selected by the

researcher) are examined for additional sources with very little best practices associated with the
EXTENDING THE SCHOPING REVIEW FRAMEWORK 6

process (e.g., Arksey & O’Malley, 2005; Vassar et al., 2016). The proposed outline for

systematic manual searching integrates hand searches of journals and reference list searches in a

clearly defined procedure that involves specific, replicable steps that researchers can follow to

conduct a more rigorous search. The steps involved in the proposed manual search guideline are

as follows:

1. The top field-specific journals should be chosen to begin the search, as opposed to

the researcher choosing one. Top journals are determined by impact factors, and

the primary field is identified through the preliminary reading by the researcher

that is used to form the research question.

2. All of the issues within the top field-specific journals, within specified timeframe,

should be screened for relevant articles. Similar to the database search, two

rounds of screening should take place: titles, abstracts, and keywords should

comprise first round of screening; followed by full text screening in the second

round. The articles that fit within the inclusion criteria are included in the final

pool of articles, while the articles that are not relevant or meet the exclusion

criteria are excluded from the final pool of articles.

3. Once the articles from the top field-specific journals are identified, the reference

lists of the identified articles should be examined for additional relevant journals.

Step two and three should be completed for the identified journals, and repeated

until no new journals arise.

Context for the Application

To illustrate the need for, and effectiveness of this extension to Arksey and O’Malley's

(2005) scoping review framework, the following section applies the systematic manual search to
EXTENDING THE SCHOPING REVIEW FRAMEWORK 7

an interdisciplinary topic. This systematic method for article identification extends stage two of

Arksey and O’Malley’s (2005) framework. The application is to a scoping review of literature

that addresses sport participation impacts, outcomes, and legacies from sport events, and

leveraging events for such outcomes. The most relevant field that this topic falls under is sport

management. Sport management incorporates several social science disciplines including

sociology, psychology, leisure studies, economics, management, marketing, anthropology, and

the list goes on depending on the particular area within sport management the scholar wishes to

focus.

Based on preliminary readings of the topic, the following research question was

developed: what is the current state of the academic literature that addresses sport participation

impacts, outcomes, and legacies from sport events, and leveraging events for such outcomes?

The research team set the parameters of the search to include peer reviewed journal articles and

book chapters published in any year up to and including 2018, that were available on-line, and

written in English. Additional inclusion criteria called for the article or book chapter to be related

to sport participation or physical activity as an outcome of a sport event. The application scoping

review undertook two systemic search methods: the systematic database search and the

systematic manual search. The database search took place in January in 2019, and the manual

search included all issues up to and including 2018.

Systematic database search. Based on preliminary reading, five databases were used to

search for articles: ABI Inform; Business Source Complete; Physical Education Index;

SPORTDiscus; and, Web of Science. The following search words were used in the database

searches: “sport* N2 participat*” OR “physical* N2 activ*” AND “sport* event*” OR Olympic*

OR championship* OR “FIFA” OR “World Cup”. The search outputs were imported into
EXTENDING THE SCHOPING REVIEW FRAMEWORK 8

Covidence, a software that assists with organizing a review, where the research team reviewed

the titles, abstracts, and key words independently. The articles that were deemed to be relevant

by both researchers were moved to the second round of screening. The research team met to

discuss the articles that were disagreed upon. After the discussion, the articles that were agreed

to be relevant were moved to the second round of screening, where the research team then

independently read the whole text of the articles. Again, the articles that were agreed upon were

included in the final pool of articles, and discrepancies were discussed. The systematic databases

search yielded 111 articles.

Systematic manual search. The following top three journals in sport management were

selected based on their impact factors: Sport Management Review, Journal of Sport

Management, and European Sport Management Quarterly. The titles, abstracts, and key words of

all articles in all issues of the three journals were examined using the inclusion criteria and the

guiding terms of ‘sport participation’ or ‘physical activity’ related to any sport event. The full

text of the articles that were identified to be potentially relevant were then read. The reference

lists of the final identified articles were then examined for additional relevant journals. The

proposed search strategy underwent four rounds of journal searches: the first round (of the three

top journals in the field of sport management) yielded 23 new journals; the second round of 23

journals yielded an additional 23 new journals; the third round of 23 new journals yielded 3 new

journals; no new journals arose after the fourth round. The proposed systematic manual search

yielded 103 articles.

Results and Discussion

Collectively, the database search and the manual search yielded a total of 146 unique

articles. Of these, 68 articles were uncovered by both search techniques, while 43 were unique
EXTENDING THE SCHOPING REVIEW FRAMEWORK 9

through the database search, and 35 were unique through the manual search. The difference in

outcome between the database search and the manual search should alert researchers that one or

the other method alone is insufficient to generate a comprehensive overview of the work in a

specific area, and provides support for the need for more than one rigorous search strategy.

Interestingly, when looking at the articles that were produced by the systematic manual search,

there were several articles included from the same journals, meaning that is it possible that the

particular journals were not included in the databases used, or perhaps they were not properly

catalogued. When looking further into the distinctive outcome, a possible explanation may have

to do with the search terms. The systematic manual search may allow for more leniency with

search terms as humans can use their discretion, as opposed to database searches which employs

strict search algorithms.

[INSERT TABLE 1 ABOUT HERE]

A prevalent issue for interdisciplinary research is different disciplines may describe the

same phenomena with slightly different terminology. This may become an issue when searching

databases, as the research team may only be familiar with the key words used in a few of the

possible fields in which the phenomenon is researched. Using the systematic manual search

method in conjunction with the database search allows for additional fields to be revealed that

may use different terminology. For instance, and as illustrated in Table 1, many of the journals

that contain articles that were produced by only the systematic manual search had to do with

policy, education, and events, while the journals that published many of the articles that only

appeared in the database search tended to be from a health or medical field. The journals that

published articles that appeared in both searches were predominantly sport management and

leisure focused. These results offer further support for the need for interdisciplinary studies to
EXTENDING THE SCHOPING REVIEW FRAMEWORK 10

include more than one systematic search method when conducting reviews. Though the two

different search techniques revealed articles from the main field (sport management), they

revealed very different additional fields. When looking at the key words used in the articles that

were revealed in only the systematic manual search versus only the database search, there were

no obvious differing themes. Indeed, it is evident that these two search methods are

complementary as they provide the research team with a set of articles that would not have been

included in the review had only one systematic search strategy been used.

Conclusion

A typical way of synthesizing knowledge is to conduct a review, and there are several

types of reviews for researchers to choose from, depending on the purpose of research question.

Rigorous reviews take on a systematic approach of determining which sources should be

included, as well as a less-systematic process of searching select journals for additional articles.

As demonstrated through the application presented here, these less-systematic methods should

start to follow more comprehensive guidelines, as one single rigours method of source

identification shows deficiencies.

References

Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework.

International Journal of Social Research Methodology, 8(1), 19–32.

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and

associated methodologies. Health Information & Libraries Journal, 26(2), 91–108.

Vassar, M., Atakpo, P., & Kash, M. J. (2016). Manual search approaches used by systematic

reviewers in dermatology. J Med Libr Assoc, 4.

Whittemore, R., Chao, A., Jang, M., Minges, K. E., & Park, C. (2014). Methods for knowledge

synthesis: An overview. Heart & Lung, 43(5), 453–461.

https://doi.org/10.1016/j.hrtlng.2014.05.014
EXTENDING THE SCHOPING REVIEW FRAMEWORK 12

Table 1. Results
Search type Number of Dominant type of journals Examples of journals
unique articles
Database search 43 Health; medical BMJ Open; International
Journal of Health Promotion
and Education; Journal of
Public Health
Manual search 35 Policy; education; events European Physical
Education Review; Event
Management; International
Journal of Sport Policy and
Politics
Both search 68 Sport management; leisure European Sport
techniques Management Quarterly;
Journal of Sport
Management; Leisure
Studies
Total 146