RES6101 Research Preparation: Methods of Research Xuan Huyen Le

1
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

Introduction
The writer of this paper was given a detail description of a quantitative study done by a student
named Sarah. Sarah was doing her Master thesis on “how rural communities deal with the
decline of public services”. From this research description, a quantitative analysis is done and
elaborated in detail below.

1. Research question and Hypothesis

From the letter sent to her supervisor, Sarah stated that she wants to “see whether the Kingston
community becomes progressively more alienated as the dispute over the school closure
unfolds.” By using the word ‘alienation’, which, according to Cambridge dictionary, means “the
feeling that you have no connection with people around you” (Dictionary, 2018). It can be
understood that Sarah would want to find out if the Kingston residents stay indifferent or stand
together and fight in order to stop the school from being closed. Moreover, Sarah might also
want to figure out whether the controversy over the school closure would make the townspeople
become isolated and no longer stay connected or the opposite.
According to the afore mentioned elements, the research question and hypothesis of Sarah’s
study can, therefore, be inferred as follow

 Research question: “To what extent does the debate on closing the school affect the level of
coherent among Kingston community?”
 Hypothesis: “the school closure dispute will increase the sense of alienation among Kingston
community”
2. How Sarah design her questionnaire

As stated in Kingston file, Sarah decided to use quantitative research method using
questionnaires and interviews as research tools. Although Sarah claimed to use quantitative
research, since she combined two tools (survey and interview) which helped her collect both
quantitative and qualitative data, Sarah was using a method that looks like ethnography research
method. (Whitehead, 2002)
The design of Sarah’s research was done quite carefully with multiple refinement of the attitude
scale and questionnaire. Besides that, there was a pre-test in which she collected the data using

2
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

her alienation scale before any official closure announcement. The participants of this pre-test
were random 185 ratepayers. The pre-test was then followed by a post-test whose response came
from random choice of citizens, collected after the school announced its partial closure.

From the outline of her thesis, it can be seen that Sarah meets almost all necessary steps of a
research paper with introduction, literature reviews of relevant issues, description of her research
design and methods used, and analysis as well as discussion on the subject.

It would be better if Sarah chose other people with different backgrounds to help trial the scale.
Using her peers from university is easy and fast but it limits the variety of answers collected
since the students from the University are people who share similar background with her, hence,
they tend to give somewhat the same response to what Sarah expected. Moreover, according to
the study description, 30% of the participants were ratepayers. There was not enough elaboration
on why Sarah targeted at this type of people. Ratepayers are people who pay taxes or bills, so it
is possible that they are just temporary tenants who live in the area for a short time before
moving to another place. Therefore, those ratepayers may not share the same level of concern on
issues related to the area. Moreover, posting notices in hotels would hardly help Sarah collect
information from the desirable target as hotel guests are definitely not the local residents.

Another point that can be done to improve the study is the number of choice in the Likert scale
Sarah used in the questionnaire. It is recommended that she omit the ‘Uncertain’ option in order
to avoid neutral bias because by doing so, the participants “are forced to use cognitive effort to
think about their true feelings on the subject” (Edwards & Smith, n.d.).
3. The questionnaire reliability and validity

The questions in Sarah’s survey were chosen based on the framework from previous literatures.
When putting the pre-test data into calculation using SPSS, we have the result as shown in figure
3.1.

In order to see the level of reliability of the questionnaire, Cronbach’s Alpha is computed. A
closer look at the Crombach’s Alpha relveals that the pre-test questionnaire has very good
internal consistency reliability with 0.901. However, the reliability can increase even more if
item 6 is deleted. According to the figure 3.1, Cronbach’s alpha of the questionnaire can be up to

3
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

0.922 if item 6 is removed.

figure 3.1. Reliability of the pre-test

From the factors above, it can be concluded that the pre-test questionnaire has high level of
reliability, yet the questionnaire would have higher level of validity if item 6 is modified or
removed.

Regarding the post-test questionnaire, figure 3.2 is presented below. The Cronbach’s Alpha in
the post-test questionnaire is 0.924. This shows that the scales have very good internal
consistency reliability, even more reliable than the pre-test scales. Unlike the pre-test
questionnaire, the post-test one would have higher Cronbach’s alpha, with 0.928 if item7r was
deleted.

4
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

figure 3.2. Reliability of the post-test

All in all, the data shows that both pre-test and post-test questionnaires that Sarah designed had
high level of reliability and would ensure her to receive consistent responds regardless of times
used.

Validity

In order to discuss the validity of the questionnaires that Sarah used in her study, it is necessary
to look at the following: wording and level of lexical in the questions as well as the correlation
among all items in the questionnaire.

The questions are formed with simple structures and vocabulary which is easy to understand and
readable for people regardless of their educational background, as long as they are not illiterate.

5
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

The questionnaire has clear layout with labels for all Likert scales. The questions are relevant to
the research objective as they mention the school closure and how people feel about its impact on
the community. Moreover, the key words such as ‘closure’, ‘town’, and ‘community’ are
repeated multiple times in the questionnaire.

In terms of correlation, which represents the validity of the questionnaire, it is shown in the
Corrected Item-total Correlation column that almost all items in the questionnaire are highly to
moderately correlated except for item 6, with -0.18. This number is too low, meaning that this
item is not a good component for this scale.

Similar to the pre-test, half of the items in the post-test questionnaire are well correlated with the
index of Corrected Item-total Correlation are above 0.7 (item2r, item3r, item4r, item5r, and
item9r) while the other items are moderately correlated with Corrected Item-total Correlation of
around 0.4 to 0.6. Therefore, the questions in the post-test questionnaire are good components.

All in all, the questionnaire from Sarah study is considered to be valid for her research, which
means it can be used to achieve accurate results and measure what it is supposed to do. Other
types of validity such as construct validity and concurrent validity is unable to be assessed due to
the lack of data.

4. Paired sample t-test and null hypothesis

From the paired sample t-test of the pretest and posttest mean scores, it can be seen that the mean
post test alienation score (23.68) was lower than the pre test one (34.27). (t (103) = 12.11,
P=.000). From this statistical number, P is less than 0.05, so it can be concluded that there is a
significant difference between the pre test and post test.

The hypothesis of Sarah study, as mentioned earlier, is “the school closure dispute will increase
the sense of alienation among Kingston community”. Hence, null hypothesis would be “the
school closure dispute will NOT increase the sense of alienation among Kingston community”.
Moreover, according to the way Sarah code the key for alienation scale, higher scores is
associated with positive attitude; thus, the higher the score, the higher sense of cohesion and
lower sense of alienation. The paired sample t-test shows that the post test mean score was lower

6
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

than that of the pre test; therefore, the level of alienation among the residents is higher. From the
aforementioned factors, null hypothesis can be rejected.
5. Descriptive statistic
From the descriptive statistic of Sarah’s research, it can be seen that most parents in the Kingston
community have children who is going to high school next year (57.69%), and half of them live
quite close to Kingston school, about 4km away from school at most. Besides that, the majority
of Kingston residents live quite far from the new high school at Beganup. About 70% of them
will have to drive about 41 to 60 kilometers from their houses to Beganup high school.
Therefore, it is easy to understand why most of the parents (61.54%) are consider boarding
school for their children once they move to the new school. Half of the community have their
youngest members whose ages range from 6 to 18, the age of secondary education. Similarly,
half of the residents are farmers.
Regarding the total scores from both pre test and post test, as Sarah mentioned in her study, the
higher the scores, the more positive the answer is, hence, the lower the level of alienation. From
the descriptive statistic of the pre test total score, the mean score is 34.27, meanwhile, that of the
post test total score is 23.68 which is lower, showing that the positivity feeling among the
community is lower. Therefore, it can be concluded that the level of alienation among Kingston
community after the announcement of the school closure is higher.

7
RES6101 Research Preparation: Methods of Research Xuan Huyen Le

Reference
Dictionary, a. (2018). alienation Meaning in the Cambridge English Dictionary.
Dictionary.cambridge.org. Retrieved 20 April 2018, from
https://dictionary.cambridge.org/dictionary/english/alienation
Edwards, M., & Smith, B. THE EFFECTS OF THE NEUTRAL RESPONSE OPTION ON THE
EXTREMENESS OF PARTICIPANT RESPONSES, 6. Retrieved from
http://blogs.longwood.edu/incite/2014/05/07/the-effects-of-the-neutral-response-option-
on-the-extremeness-of-participant-responses/
Whitehead, T. (2002). What is ethnography? Methodological, Ontological, and Epistemological
Attributes. Cultural Ecology Of Health And Change, 6-7.