Article title: The Questionnaire Surveying Research Method: Pros, Cons and Best Practices

Authors: Liam Murphy[1]

Affiliations: [1]
Orcid ids: 0009-0001-2348-4445[1]
Contact e-mail: [email protected]
License information: This work has been published open access under Creative Commons Attribution License
http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, distribution, and reproduction in any
medium, provided the original work is properly cited. Conditions, terms of use and publishing policy can be found at
https://www.scienceopen.com/.
Preprint statement: This article is a preprint and has not been peer-reviewed, under consideration and submitted to
ScienceOpen Preprints for open peer review.
DOI: 10.14293/S2199-1006.1.SOR-.PP3WYS8.v1
Preprint first posted online: 12 March 2023
Keywords: Research methods, survey, questionnaire
The Questionnaire Surveying Research Method:
Pros, Cons and Best Practices

Abstract
Liam Murphy is a
Doctoral student studying
This paper serves as a brief introductory summary to the Questionnaire surveying
at the School of Science,
Business & Enterprise research method, aiding new scholars to avoid common pitfalls and consider how to
utilise the method most successfully in their studies.
University of Chester, UK

ScienceOpen, March 2023

1. Introduction to Surveys

Surveys are a popular strategy used to answer exploratory and descriptive research
(Saunders et al. 2016) and enable the efficient collection of standardised data from a
large population of participants. This enables researchers to conduct comparisons
and conclude correlations within response data. There are multiple methods involved
in surveying, such as questionnaires, structured observations and structured
interviews. Cycyota and Harrison (2006) concluded that surveys in the form of mailed
questionnaires (postage or email) are one of the best-researched data collection
methods within the organisational sciences. In fact, survey questionnaires are widely
used across first world countries by organisations and governments (Alrdige and
Levine, 2001; Saunders et al. 2016). They are often accessed by members of the
public via websites, newspapers and online bulletins etc. Aldridge and Levine (2001)
state their widespread use is evidence of their success and research participants’
ability to understand them.

One of the first tasks when conducting a survey is to identify a sample of the
population which data is to be collected from (Heeks et al. 2017). The sample is a
sub-section of people who should ideally have the same characteristics of the wider
population. There are numerous different sampling techniques that can be used,
dependent upon the relevance to a particular study. Floyd and Fowler (2014) state the
importance of carefully considering the sample size and design of the selection
procedures to minimise potential sampling bias (discussed further in following
section).
The Questionnaire Surveying Research Method: Pros, Cons and Best Practices

Both Baruch and Holtom (2008) and Floyd and

2. Pros and Cons to Surveys Fowler (2014) concluded that if a survey did not

Researchers can receive numerous benefits by receive 100% of responses, the probability of

employing surveys as their data collection method. As statistical bias is heightened.

previously discussed, surveys are a popular and

Finally, Heeks et al. (2017) claim that when
historically successful method used widely across the
conducting surveys, researchers may
globe (Aldridge and Levine, 2001). Additionally, Williams
unconsciously focus on their own geographical
(2007) found that such quantitative research methods
area and participants who share similar
have been in use, in some form or other, since 1250 A.D.
characteristics such as age, sex, gender,
and have since dominated western culture.
education level etc. Thus, biasedness is likely to
Consequently, one can assume that research participants
occur in any scenario.
find surveys familiar and easier to understand than other
research methods (Aldridge and Levine, 2001).

Furthermore, Baruch and Holtom (2008) conclude that 3. Best Practices for Applying Surveys
surveys (specifically questionnaires) have great strength
Following best practices can mitigate some of the
in assessing organisational concerns and observing
aforementioned criticisms of a survey. Firstly,
associated trends. Their strength lies in the fact they can
Saunders et al. (2016) discuss the importance of
be distributed across large numbers of subjects (Heeks et
triangulating data to test validity. To cater for this,
al. 2017) and be subsequently used to suggest reasons
it is recommended to design the survey to be able
for relationships between variables and produce models
to gather a mix of both qualitative and quantitative
of such relationships (Saunders et al. 2016).
data (Allen et al. 2008; Woodside, 2010).
However, while surveys have their merits, they also have
many critiques. Numerous scholars have challenged the
In response to the issue of survey response rates,
scientific and humanistic validity of surveys due to the
Baruch and Holtom (2008) made numerous
wide range of variables they cover, and the question of
suggestions. They found that survey response
whether a small sample of participants can truly
rates increased when participants were
determine the views and beliefs of a wider population
guaranteed anonymity and felt invested in the
(Aldridge and Levine, 2001).
topic of study. The latter of which can be achieved
Further challenges arise with the response rates of by distributing the survey with a covering letter
surveys. Saunders et al. (2016) note that a researcher’s detailing the purpose and objectives of the study
progress can be delayed, or even halted, while awaiting (Saunders et al. 2016; Cycyota and Harrison,
responses. Baruch and Holtom (2008) raised the risk of 2006). Scholars also appear to unilaterally agree
not receiving some responses at all, concluding that a that forging a relationship between the researcher
100% response rate is rarely achieved. They suggest this and participant and gaining pre-consent before
usually occurs for two reasons: survey delivery failure sending the survey will both increase response
(i.e., incorrect participant details) and respondents’ rates (Saunders et al. 2016; Cycyota and
unwillingness to partake. They found poor job satisfaction Harrison, 2006; Baruch and Holtom, 2008).
or organisational commitment impacted participants’
willingness to respond.

Science Open – March 2023 Liam Murphy 2

The Questionnaire Surveying Research Method: Pros, Cons and Best Practices

A further method of best practice is to test the survey for One method, which compliments the survey
reliability and validity (Saunders et al. 2016). It is questionnaire, is the matrix of comparison. The
imperative to ensure that the survey is comprehended, survey questionnaire can produce open or closed
and all questions are correctly interpreted. To conduct answers, producing both qualitative and
this, Saunders et al. recommend distributing a pilot quantitative data. The matrix of comparison can
questionnaire to ensure participants respond in the combine and correlate both sets of data by using
anticipated manner. Any bias should be eliminated by different axis. For example, the horizontal axis
distributing the pilot to a separate sample audience. may hold the type of responder (i.e., project
manager, business analyst etc.) while the vertical
axis may hold the themes of the data collected.
The matrix can contain qualitative quotes of data
4. Data Collection/ Analysis
or counts of the number of times a pattern or
Numerous survey methods can be used to collect data theme has occurred in the participants’ answers.
from participants. Surveys are usually administered via This method therefore allows a researcher to
face-to-face, post, telephone, email or the Internet combine both sets of qualitative and quantitative
(Heeks et al. 2017; Saunders et al. 2016). The method data instead of focusing on just one type.
most examined in this paper is the questionnaire, due to
being one of the most researched (Cycyota and Harrison
2006). Saunders et al. (2016) state that delivering a
References
questionnaire via a structured interview yields a higher
response rate as opposed to self-completed Aldridge, A. & Levine, K. (2001) Surveying the
questionnaires. This being said, it also takes a lot longer social world: Principles and practice in survey
to conduct and can present logistical issues. The self- research, 1st edn, Open University Press,
completed questionnaire, which can also be sent via an Buckingham, UK.
Internet web portal, can account for a larger sample size
Allen, T.D., Eby, L.T., O’Brien, K.E. & Lentz, E.
and be easily analysed via computer. If a researcher
(2008) ‘The state of mentoring research: A
considers the best practices discussed in the previous
qualitative review of current research methods
section, the self-completed questionnaire should
and future research implications’, Journal of
hopefully receive a high response rate.
Vocational Behaviour, (73)3: 343–357.
With the Internet based questionnaire chosen as a data
Baruch, Y. & Holtom, B.C. (2008) ‘Survey
collection method, Cresswell (2009) states that the next
response rate levels and trends in organizational
step is to identify the type of data to be collected, and the
research’, Human Relations, (61)8: 139-1160.
method for collecting that data (i.e., via open or closed
answers). Saunders et al. (2016) re-iterate this and warn
Cresswell, J.W. (2009) Research Design:
that questionnaire questions must be defined precisely,
Qualitative, Quantitative and Mixed Methods
as they do not allow for follow-up questions in the same
Approaches, 3rd edn, Sage Publications Inc,
way an interview does.
California, USA.

Once the data collection method has been decided,

Cresswell discusses several data analysis techniques.

Science Open – March 2023 Liam Murphy 3

The Questionnaire Surveying Research Method: Pros, Cons and Best Practices

Cycyota, C.S. & Harrison, D.A. (2006) ‘What (Not) to Expect When Surveying Executives: A Meta-Analysis of Top
Manager Response Rates and Techniques Over Time’, Organizational Research Methods, (9)2: 133-160.

Floyd, J. & Fowler, Jr. (2014) Survey Research Methods, 5th edn, Sage Publications Inc, California, USA.

Heeks, R., Johnston, M. & McCourt, W. (2017) Designing Research, Research Methods course unit 3, GDI,
University of Manchester.

Heeks, R., Johnston, M. & McCourt, W. (2017) Designing Research, Research Methods course unit 5, GDI,
University of Manchester.

Saunders, M., Lewis, P. & Thornhill, A. (2016) Research methods for business students, 7th edn, Pearson, Harlow,
UK.

Williams, C. (2007) ‘Research Methods’, Journal of Business & Economics Research, (5)3: 65-72.

Woodside, A.G. (2010) ‘Bridging the chasm between survey and case study research: Research methods for
achieving generalization, accuracy, and complexity’, Industrial Marketing Management, (39)1: 64–75.

Corresponding author:
Liam Murphy can be contacted at: [email protected]

Science Open – March 2023 Liam Murphy 4