Module 6:

SURVEY RESEARCH RESEARCH METHODOLOGY

CHAPTER LEARNING OUTCOMES

Malaysian Teens Grade Dad a C-

When you have completed this module you will be able to: A sample young people from Hong Kong, Indonesia, Philippines, Define whatof is 3212 a survey Singapore, South Korea, Taiwan, Malaysia aged between 14 and 18 Compare the different types Thailand of surveyand methods took in the survey required to techniques grade their parents part Explain thesurvey. process The of selecting a sample participants using different from A to F for 35 statements given. List the seven major steps in conducting survey research The survey was designed to allow children to reflect on and rank their Discuss the and disadvantages of different data collection parents ability to advantages show respect and love, have integrity and survey patience and be openmethods minded, among a myriad of other skills and characteristics. Elaborate onthe the Malaysian ethics involved conducting surveys Although dad in ranks lower than the Malaysian mum, a general analysis of the survey results reveals fascinating details of Malaysian fathers as seen through the eyes of young adults. For instance: Dad got a B for telling jokes and C for being fashionable Dad does not really allow them much independence (C-) but respects their privacy (B) and trusts them (B). Dad takes the trouble to teach them right from wrong (B+) Dad accepts their own taste in music (B) Dad talking about sex (D).
[Source: Sunday Star, 27 July, 2005, p.22]

LEARNING LEARNING ACTIVITY ACTIVITY Read the newspaper Read the newspaper report report on on Malaysian Malaysian teens teens grade grade dad dad a a C- C- What What are are your your views views on on the the findings findings of of the the survey? survey? What What other other information information would would you you need need about about the the survey survey to to accept accept the the findings? findings?

INTRODUCTION 1

Do you want to know what people are thinking, feeling or doing? If you do, than the survey would be the best research method. You could survey them orally through an interview or you could give them a questionnaire. By asking questions, you are tapping into peoples attitudes, beliefs, perceptions and behaviour. The survey has proven to be a powerful tool in gathering information about the opinions, attitudes and intentions of people concerning different social, cultural, economic and political issues. It is quite common for newspapers, radio and television to report on the opinions and perceptions of people obtained through surveys (at times using survey data to sensationalise issues). Surveys are used by various government agencies, non-governmental agencies, business and scientific organisations probing into peoples reactions to different issues and trends. For example: TV stations rely on surveys to profile the people who watch programmes over their stations. Automobile companies use survey to find out about customer satisfaction with the cars they produce or sell. Statistics Departments conduct surveys to measure the consumer price index with the purpose of advising policy. Engineering companies conduct surveys gathering information about the habits of road users. Magazine companies conduct surveys to find out about the reading habits and interests of their subscribers. Manufacturing companies conduct surveys to determine consumer acceptance of their products.

WHAT IS A SURVEY? What is a survey? According to Kerlinger (1973), survey research involves the studying of large and small populations selecting and studying samples chosen from the populations to discover the relative incidence, distribution and interrelations of sociological and psychological variables. It is a method of obtaining information about a population from a sample of individuals. Surveys can provide a quick, inexpensive and accurate means of obtaining information from a large group of people. If you want to know about the opinions, attitudes and perceptions of respondents, the survey is an appropriate method of collecting data. Besides, describing surveys can also be used to explain the relationship and differences between variables. The term sample survey is often used because a sample which is representative of the target population is used. The survey method is widely used in the social sciences, education, business and medicine. Basically, information is obtained by asking people questions either orally or by responding to a written paper or computer screen concerning: 2

What they know? (Who was the first Prime Minister of Malaysia?) What they believe? (Should students be given freedom to express themselves?) What they expect? (Do you think you expect to be a famous person?) What they feel? (Do you think your school principal is fair?) What they have done? (How often do you use the computer in a week?) What they plan? (Do you intend to continue studying or work?)

TYPES OF SURVEYS

Cross-Sectional Cross-Sectional Survey Survey TYPES OF SURVEY Longitudinal Longitudinal Survey Survey
Trend studies

Cohort studies Panel studies

Figure 6.1 Types of Survey Surveys provide an important source of basic scientific knowledge. Economists, psychologists, health professionals, political scientists, educationists and sociologists conduct surveys to study such matters as income and expenditure patterns among households, the roots of ethnic or racial prejudice, the implications of health problems on people's lives, comparative voting behaviour, factors influencing academic performance, the effects on family life of women working outside the home and so forth. To serve these different needs, there are two main types of survey (see Figure 6.1). The types of survey used will depend on the objectives of the study. If the study aims to get a snapshot of opinions and practices than the cross-sectional survey would be most appropriate. If the objective is to compare differences in opinion and practices over time than the longitudinal survey would be the obvious choice. a) Cross-Sectional Survey Just like all surveys, the cross-sectional collects information from a sample drawn from a population. It involves collecting data at one point of time. However, the time period for collection of data can vary from 1 week to 6 months. If you are using a questionnaire to collect data, you can ask respondents about the past, present or the 3

future. For example, you administered a questionnaire on habits and attitude towards smoking to 500 students in secondary school aged between 14-16 years on 6 th September, 2005. The students included male and females from different socioeconomic backgrounds in the state of Selangor. The data you obtained is a cross-section of the population at one point of time. b) Longitudinal Survey In longitudinal surveys, data are collected at different points in time in order to study changes. There are 2 common types of longitudinal surveys: Cohort studies: You identify a specific population (e.g. teachers in Perak who have a masters degree) and list the names of all members of this population. At each data collection point, you select a sample of respondents from the population of Perak teachers with a masters degree and administer a questionnaire (e.g. about their aspirations). At another point you might select another sample from the same population of teachers and administer the same questionnaire. Thus, although the population remains the same, different individuals are sampled each time. Your aim is to see if there are changes in perceptions or trends that are present. Panel studies: You identify a sample from the beginning and follow the individuals over a period of time with the aim of noting changes in specific respondents and explore reasons why these individuals have changed. For example, you want to find out about changes in racial attitudes among a group of primary school children. You administer an attitude scale at year 5 and than administer the same scale when they in year 5 and so on. You analyse the data to see if there are changes in racial attitudes as children grow older. The only problem is the loss of subjects which you cannot replace. LEARNING ACTIVITY LEARNING ACTIVITY SAMPLING Define survey in Define survey in your your own own words. words. What What is is the the main main difference difference between between cross-sectional cross-sectional surveys surveys and and longitudinal longitudinal surveys? surveys? Unlike a census, where all members of the population are studies, survey gather information from only a portion of the population of interest the size of the sample will depend on the purpose of the study. [We will discuss this issue later]. In a good survey the sample is not selected haphazardly or only from persons who volunteer to participate. It is scientifically chosen so that each person in the population will have a measurable chance of being selected. This procedure is called sampling. This way the results can be reliably projected from the sample to the larger population. Two key words involved in sampling are population and sample. The word population has is defined as all people, objects or events found in a particular group the researcher is planning to generalise to (Borg and Borg, 1983). 4

POPULATION POPULATION :: (Total = 10,000 persons) (Total = 10,000 persons) Characteristics: Characteristics:

Gender: 5100 males & 4900 females Gender: 5100 males & 4900 females Socioeconomic status: (2500 high, 4000 Socioeconomic status: (2500 high, 4000 middle, 3500 low) middle, 3500 low) Ethnicity: 5000 Malays, 3000 Chinese, 1000 Ethnicity: 5000 Malays, 3000 Chinese, 1000 Indians, 1000 Ibans. Indians, 1000 Ibans. GENERALISATION
Sample Selected from

SAMPLE SAMPLE (n=500) (n=500) Characteristics: Characteristics:

SAMPLING

-- Gender: Gender: 255 255 males males & & 245 245 females females -- Socioeconomic status: Socioeconomic status: (125 (125 high, high, 200 200 middle middle & & 175 175 low) low) -- Ethnicity: Ethnicity: 250 250 Malays, Malays, 150 150 Chinese, Chinese, 50 50 Indians Indians & & 50 50 Ibans. Ibans.
population (5%)

Figure 6.2 Selection of a Sample Representative of the Population For example, the population of primary school students, population of graduate teachers, population of medical doctors in Malaysia, population of senior citizens in Selangor, population of secondary school students in Sarawak and so forth. If the population is small (e.g. school principals in Kuala Lumpur), the researcher may decide to study all subjects. When the population is large, the researcher is not able to study all individuals. It would be too expensive and time consuming even for large research organisations. Only in a census is the whole population studied involving enormous expenditure, time and many research assistants. Just imagine having to administer a questionnaire to 100,000 sixteen year olds in Malaysian schools! So the researcher has to select a certain number of subjects or a sample from the population to study. Regardless of the method used in selection of the sample, the most important thing is to ensure that the sample is representative of the population. See Figure 6.2 which shows a population of 10,000 individuals who possess certain characteristics in terms of gender, socioeconomic status and ethnicity. If you intend to select 5% of individuals from the population to form a sample, you should ensue that the sample has similar characteristics to the population. This is called population validity where the researcher is trying to show that the sample is representative of the population 5

according to the variables specified. Using a sample is more economical and if the sample is selected appropriately, the researcher can make conclusions about the population based on the results from the sample. This is called generalisation.

SAMPLING TECHNIQUES LEARNING LEARNING ACTIVITY ACTIVITY Refer Refer to to Figure Figure 6.2 6.2 and and explain explain how how the the sample sample selected selected is is representative representative of the population. of the population.

As mentioned earlier, surveys rely on samples to make projections about the population. How does one select a sample? The sampling techniques used will depend on the objectives of the study and the resources available. Generally, there are two types of sampling techniques: Probability sampling: Probability sampling includes techniques that select samples based on the concept of random selection. Among the techniques that are based on the concept of random sampling are; random sampling, systematic sampling, stratified sampling, and cluster sampling. Non-probability sampling: Non-probability sampling techniques are not based on random selection. Among the common techniques are; quota sampling, purposive sampling and convenience sampling. A) Non-Probability Sampling Techniques i) Random sampling One of the techniques to increase the probability that the sample selected represents the population random sampling is used. Using this technique, each member of a population has an equal chance of being selected (Refer to Chapter 2 on random assignment) For example you are interested in surveying the attitudes of graduate teachers toward the teaching profession. The sample you select will have to represent the attitudes of the target population (see Figure 6.3). Due to financial and time constraints you are unable to survey the attitudes of all graduate teachers across the whole of Malaysia (N=100,000) and so you decide to confine your study to graduate teachers in Perak (n= 15,000) which is called the accessible population. From the accessible population a sample of 100 teachers is drawn. Even though the sample is selected from the accessible population, you may want to know the degree to which the results can be generalised to the target population which requires two steps. In the first step, you generalise from the results of the sample to the accessible population (teachers in Perak). Second, you generalise from the accessible population to the target population (graduate teachers in Malaysia). The leap from sample 6

to the accessible population presents no problem if a random sample of the accessible population is obtained. However, in order to make the leap from the accessible population to the target population, you must gather data to determine the degree of similarity between graduate teachers in Perak and graduate teachers in the whole of Malaysia. If you can demonstrate, based on a number of variables (such as gender, age, experience, ethnicity) that the accessible population is closely comparable to the target population, you have established population validity, i.e. the accessible population is representative of the target population.
TARGET POPULATION Graduate Teachers in Malaysia (N= 100,000)

ACCESSIBLE POPULATION

All Graduate Teachers in Perak (N=10,000)

SAMPLE

Sample Randomly Selected (n= 100)

Figure 6.3 Random Selection of a Sample How do you select a sample randomly? The usual definition of a random sample is that it is a procedure in which all individuals in the defined population have an equal and independent chance of being selected as a member of the sample. Independent means the selection of an individual is not affected by the selection of another individual. In other words, each individual, event or object has the same probability of being selected. For example, the number of graduate teachers in Perak is 10,000 and you intend to draw a sample of 100 teachers (see Figure 6.3). When you select the first teacher, he or she has a 1:10,000 chance of being selected. Once this teacher is selected, however there are only 9999 cases remaining so that each teacher has 1:9999 of being selected as the second case. Thus, as each case is selected, the probability of being selected next changes slightly because the population from which you are selecting has become one case smaller. The table of random numbers is often used in the selection of a random sample (see Table 6.1). You need to obtain a list of the 10,000 graduate teachers in Perak and assign number 1 to 10,000. Using the table of random numbers, randomly select a row or column as the starting point. Say, that you select Column 5. Select all the numbers that follow in that column. So, you will select teacher assigned number 7332, followed by teacher assigned number 6516, then teacher assigned number 4553 and so forth. If you need more numbers, you can proceed to the next column until you have enough numbers to make up the desired sample (i.e. n=100).

1 2 3 4 5 6 7 8 9 10

1 2345 7612 6823 4598 8956 9059 2312 8787 9063 1298

2 3445 7989 5732 2564 357 2691 6120 8236 7190 3820

3 2678 5456 3702 9860 3934 8395 6425 5153 2590 6737

4 9156 4523 3808 6360 3495 2634 6556 0202 8696 0932

5 7332 6516 4553 5245 9112 2189 3720 9530 6267 6719

6 9332 1345 4589 2347 9472 3465 0639 6490 3923 3154

7 8345 1123 0467 0391 1254 8223 5490 6220 5360 4532

8 5950 5636 8506 3623 3998 0745 2614 3523 4937 4046

9 5023 9189 6612 0620 1390 3487 2950 9691 4854 2860

10 5189 27452 12136 0850 3194 6709 7556 4917 5223 3191

Table 6.1 Table of Random Numbers ii) Systematic sampling Similar to random sampling, systematic sampling is used to draw a sample from a population. It is often used instead of random sampling. It is also called an N th name selection technique. You begin by having a list of the names of members in the population in random order. You want to select a sample of 100 students from a population of 1000 students. You divide the population by the sample needed (1000 / 10) = 10 You select at random a number smaller than 10 You start with that number (e.g. 7) and select every seventh name from the list of the population.

After the required sample size has been calculated, every N th record is selected from a list of population members. As long as the list does not contain any hidden order, this sampling method is as good as the random sampling method. Its only advantage over the random sampling technique is simplicity. Systematic sampling is frequently used to select a specified number of records from a computer file. iii) Stratified sampling In some surveys, you want to ensure that individuals with certain characteristics are included in the sample to be studied. For this purpose, the stratified sampling technique is used. For example, if you are interested in studying inductive reasoning among 12 year olds according to ability and gender in Petaling District. You want to ensure that you have a proportionate number of high and low academic achievers as well as a proportionate number of males and female. In order to avoid a sample that does not include a sufficient number of students of each sex at each ability level, a stratified sample may be selected. All 12 year old students in the district are divided into one of the following four groups: male high achievers, 8

female high achievers, male low achievers and female low achievers. Subsamples are then selected at random from the population to fill each of the four groups (see Table 6.2). Males n=? n=? Low achievers Table 6.2 Sample Stratified According to Gender and Academic Performance However, the proportion of students randomly selected to fill each of the groups is the same as the population. For example, if male high achievers made up 25% of all 12 year olds in Petaling District, proportion of male high achievers in the sample should also be 25%. If you do not follow this procedure, results obtained from the sample will produce an inaccurate view of the population. Stratified sample are most appropriate when you want to make comparisons between various subgroups and to ensure the sample is representative of the population in terms of critical factors you want to study. Females n=? n=? High achievers

ACTIVITY ACTIVITY 5.3 5.3 You You have have been been appointed appointed to to lead lead a a research research team team assigned assigned with with the the task task of finding the reasons teenagers smoke. The team has decided to conduct of finding the reasons teenagers smoke. The team has decided to conduct a a nation-wide nation-wide survey survey involving involving students students between between 14-16 14-16 years years of of age age in in secondary schools. secondary schools. Suggest Suggest THREE THREE research research questions questions that that will will guide guide data data collection. collection. iv) Cluster sampling Explain Explain how how you you plan plan to to draw draw the the sample sample of of students students using using stratified stratified In the techniques of sampling discussed thus farm the unit of sampling is the sampling. What subgroups would you include? sampling. What subgroups would you include? individual What student, teacher or principal. In need cluster the unit of sampling information do draw sample? What further further information do you you need to tosampling, draw a a representative representative sample? is not the individual but rather a group of individuals. For example, in a district there 25 villages and in each village there are an estimated 20 pre-school children. You need a sample of about 100 pre-schoolers but this would mean going to many villagers if random sampling is used. In cluster sampling, you select 5 villages randomly from the total 25 villages. You study all the pre-school children in the 5 villages you have selected. The advantage of using cluster sampling is that it saves time and money, especially if the population is dispersed. The weakness is that it is less accurate compared to the other techniques of sampling discussed because the subgroups may be more heterogeneous rather than heterogeneous.

LEARNING LEARNING ACTIVITY ACTIVITY What is meant What is meant by by random random sampling? sampling? What What is is the the difference difference between between stratified stratified sampling sampling and and systematic systematic sampling? sampling? How How is is cluster cluster sampling sampling different different from from all all the the other other types types of of sampling sampling techniques techniques discussed? discussed?

LEARNING LEARNING ACTIVITY ACTIVITY

N=134 N=105 N=118 N=113 N=154 N=111 N=129 N=98 N=109

The The above above is is a a diagram diagram showing showing the the location location of of 9 9 housing housing areas areas dispersed dispersed all all over over a a city city with the number of senior citizens indicated in each area. with the number of senior citizens indicated in each area.

1. 1. Explain Explain how how you you would would select select a a sample sample of of about about 200 200 senior senior citizens citizens you intend to interview regarding how they spend their time using you intend to interview regarding how they spend their time using cluster cluster sampling. sampling. 2. What factors 2. What factors should should you you consider consider when when using using cluster cluster sampling? sampling? B) Non-Probability Sampling In many situations it is not possible to ensure that the sample will be selected based on random selection. So the sample has to chosen by some other way. Non-probability sampling is based on the researchers judgement and hence biasness will enter in sample selection and distort the findings of the study. Nonetheless, non-probability sampling techniques are used because of practical reasons. For example, non-probability techniques are used to save costs and time; when its use can satisfactorily meet the objectives of the study and it may be the only feasible method given the spread and features of the population. Among the more common non-probability techniques are: quota sampling, purposive sampling and convenience sampling. a) Quota sampling In situations where you are unable to ensure that subjects are randomly assigned to the subgroups you can use quota sampling. Like stratified sampling, you first identify the subgroups and their proportions as they are represented in the population. Following which you select subjects based on convenience and your judgment to fill each subgroup. When using this method, you must be confident that the chosen sample is truly

10

representative of the population. Obviously, you should state clearly the criteria for your selection of the sample; especially when you make generalisations of the results to the population. b) Purposive sampling The sample is selected on the basis that members conform to certain criteria. The researcher uses his or her judgement to select cases to answer certain research questions. The form of sampling is usually used when the population is small (such as in case study research) and when the main purpose is to select cases that are particularly informative. It is very useful in the early stages of an exploratory study. The main weakness of this technique is that the sample may have characteristics which differ from population characteristics. c) Convenience sampling In this sampling technique, researchers have the freedom to choose whomever they find. You simply sample people who are easy to survey. The sample is chosen rather haphazardly until the required sample size is met. It is less expensive and easy to conduct; and is considered the most useful procedure to test ideas in exploratory research. This sampling technique is considered the least reliable because of its high incidence of biasness. Newspapers ask people to mail in their responses to a survey question; radio stations ask people to call in their opinions to a question. As you can see, the convenience sample can provide you with a lot of data but you do not know whether the sample represents your population.

LEARNING ACTIVITY LEARNING ACTIVITY 5.5 SAMPLE SIZE What is the major What is the major difference difference between between probability probability and and non-probability non-probability sampling sampling techniques? techniques? Why Why are are non-probability non-probability sampling sampling techniques techniques used used despite despite their their many many weaknesses? weaknesses?

How large must the sample size be? This is a question that is uppermost in the minds of many beginning researchers. The general rule is to use the largest sample possible. The rule is good because we are interested in learning about the population based on the results of the sample from which it is drawn. The larger the sample the more likely the sample mean and standard deviation will be representative of the population mean and standard deviation. The sample size required for a survey partly depends on the statistical quality needed for survey findings; this, in turn, relates to how the results will be used. Even so, there is no simple rule for sample size that can be used for all surveys. Much depends on the professional and financial resources available. Often a moderate sample size that is sufficient statistically and operationally is preferred. For example, the well-known Gallup Poll in the United States frequently use samples of 11

about 1,000 persons to get reasonable information about national attitudes and opinions. The following are some guidelines you can use in deciding how large should you sample be: When the sample selected has to be broken down into smaller groups involving comparisons of groups, a larger sample is advisable. For example, you want to compare the self-esteem of males from low socioeconomic backgrounds from single-parent families against males from high socioeconomic backgrounds from single-parent families. When attrition is expected, especially in longitudinal studies. The longer the duration of a study, the higher will be the number of subjects who drop-out. To reduce attrition you should keep demands on subjects to the minimum, fully inform subjects about the study, obtain a strong commitment from subjects before the study and make frequent contacts with subjects to maintain interest and rapport. When the population is highly heterogeneous on the variables being studied, you need to have a larger sample so that the different characteristics of individuals are satisfactorily represented. If members in the population are more or less similar, then you will need a small sample as most of the characteristics can be captured.

LEARNING LEARNING ACTIVITY ACTIVITY What are the What are the factors factors you you will will consider consider when when making making decisions decisions about about sample sample size size in in a a survey? survey?

LEARNING LEARNING ACTIVITY ACTIVITY Sample Size Sample Size Calculator Calculator (www.surveysystem.com/sscalc.htm) (www.surveysystem.com/sscalc.htm) Click Click the the above above website website to to access access a a Sample Sample Size Size Calculator Calculator.. What What is is meant meant by by confidence confidence level level and and confidence confidence interval? interval? Calculate Calculate the the sample sample size size required required from from a a population population of of 9000 9000 students students with with a a confidence confidence level level of of 95% 95% and and confidence confidence interval interval (or (or margin margin of of error error)) of of 5. 5.

THE PROCESS OF CONDUCTING A SURVEY

12

Step 1: Defining the Objectives of the Study A successful survey requires careful preparation. The first and often the most difficult question is Why am I doing this survey? Many studies start with a general hope that something interesting will emerge, and they often end in frustration. Putting together a group of questions you feel are important to students does not constitute a valid and reliable survey. This initial planning requires some idea of the final analysis; and it may be useful at the outset to outline the key tables for the final report (better still if you are able to create dummy tables), and to consider the numbers of cases expected in each major cells or subgroups. Conduct a review of literature to define terms and justify theory underlying research questions. Step 2: Writing the Items and Construction of the Questionnaire Survey data is mostly obtained by means of questionnaires. Generally, questionnaires are an inexpensive way to gather data from a large number of respondents. They may be either self administered (that is, completed by the subject), administered at an interview or administered to a group in face-to-face session. The crux of a survey is the questionnaire (i.e. what we ask our respondents). A well-designed questionnaire can provide valuable information about the opinions, beliefs and practices of groups of individuals. Questionnaire design is a long process that demands careful attention and should not be taken lightly. [We will discuss in detail how to design questionnaires in Module 7 Instrumentation]. In this section the general qualities of a good questionnaire are discussed briefly.

1) 1) Defining Defining the the objectives objectives of of the the survey survey

2) 2) Writing Writing the the items items and and constructing constructing the the questionnaire questionnaire

5) 5) Administering Administering the the questionnaire questionnaire

4) 4) Pilot-testing Pilot-testing the the questionnaire questionnaire

3) 3) Select Select method method of of conducting conducting the the survey survey

6) 6) Data Data entry entry and and analysis analysis

7) 7) Writing Writing the the report report

13

Figure 6.3 The Seven Steps of the Survey Process Clarity: Questions must be clear, precise and unambiguous. This is to eliminate the chance that the question will mean different things to different people. Avoid the use of colloquial or ethnically sensitive terms. Technical terms that assume a certain background should also be avoided. Leading Question: A leading question forces or implies a certain type of answer. The researcher feels strongly about an issue and assumes that everyone will be of the same opinion. e.g. Does your principal treat men and women teachers differently? Hypothetical Question: A hypothetical question is one in which you are asking respondents to indicate what they think they would do under particular imaginary circumstances. While they are used in some attitudinal research, they are difficult to interpret and often give rise to unreliable answers. e.g. If you were buying a house, what features would you most want it to have? Question Requiring Prior Knowledge: A question that taxes a respondents memory leads to non-response or inaccurate replies. e.g. What did you do in the last school holidays? Order of Questions: o Take into account the sensitivities of the person to whom they are addressed it is better to start with "What is your date of birth?" rather than jump straight into "Has there been a death in the family?" o Begin with interesting items. If the first items are boring, there is little chance that the person will complete the questionnaire.

Step 3: Method of Conducting the Survey You have to decide the procedures for conducting the survey; how many people you will survey (the sample size and how they will be obtained) Decide how you will survey your subjects (by phone, in class, mail format, interview). Decide how you intend to distribute and collect the questionnaires; make followup contacts. Decide on the level of response that is acceptable (refer to your research questions and the population the question is being asked). What percentage of questionnaires returned or answered will you accept? What will you do if you do not have enough respondents? Or a certain sector of respondents is underrepresented? e.g. insufficient number of females.

14

Step 4: Pilot-Testing of the Questionnaire Pilot-testing a questionnaire is necessary to avoid problems that may arise when the questionnaire is administered to the whole sample. Unfortunately, some people consider pilot-testing nothing more than a ritual while others do not do it. Before pilot-testing, get some of your colleagues to review the questionnaire, particularly those who are interested or familiar with what you are doing. The questionnaire should be pilot-tested with a small group of individuals who are in many respects similar to the sample in the actual study. When they have finished, ask for their opinions and suggestions (e.g. Was it too long? Which items were difficult?). The pilot-test helps you find out if the content and form of the questions are satisfactory. You can also get information about: o Length of time to complete the questionnaire o The appropriate order of the questions o Sufficient space is provided for responses o The instructions or directions are understood o Extent to which the questionnaire is reliable (if you give the questionnaire again four weeks later would you get the same responses) o Extent to which the questionnaire is valid (does it measure what it is supposed to measure) [We will discuss further these issues in Module 7]

Step 5: Administering the Questionnaire You should gain approval from the respondents you intend to survey. For example, permission from the Ministry of Education (Educational, Planning and Research Division), State Education Departments and school principals should be sought. In a small study you can administer the questionnaire yourself. But in a large survey, you will need helpers. Your field assistants have to be trained, especially if they are required to observe or conduct interviews. Such training should be completed before the start of the main study. Teachers are gatekeepers to the students. Gaining cooperation of teachers is very important when conducting a survey involving students. Teachers will be more receptive in conducting the survey during class time if you explain to them clearly the objectives and contents of the survey. Even better if you can demonstrate that the survey can be incorporated into the curriculum. Schedule the survey far enough in advance to allow teachers to make class time available. Consent may be necessary for students to participate in the survey. If so, parents may play a key role in students participation. Ensure that parents understand why you are asking their childrens opinion. Do not administer the questionnaire on a day when there are school activities such as open day, celebration of festivals, sports events, examinations. Students will be distracted with these events and may not give the survey their full attention.

15

If you are surveying parents, you could send the questionnaire home with students. How can you ensure of receiving enough returned surveys? One suggestion is to reward students for returning questionnaires, e.g. a coupon for a free soft drink. Thank the groups that assisted in the survey process. An appreciation goes a long way toward support and participation.

Step 6: Data Entry and Analysis Coding and entry of data. Check to ensure accuracy of data entry and ensure that all codes are valid (for example, 1 is for male, 2 for female and there should not be a 3!) and look for any inconsistencies. Selection of software package to analyse data. Statistical analysis should only begin when the data set is as clean as possible. If you are unclear of certain statistical procedures, obtain advice from a person who is well-versed in statistics.

Step 7: Writing the Report The final phase of any research process is documentation or writing the research report. The usual parts of a research report are: [ We will discuss in detail writing the research report in Chapter 10] o introduction (background of the study, rationale for the study, the aims and research questions, limitations and significance of the survey); o the review of literature (previous work done in the field, underlying theory or core of theories) o methodology (description of the sample, how the sample was drawn, description of the questionnaire used, administration of the questionnaire). o results (data presented in the form of tables and graphs; statistical analysis, description of the findings) o conclusion (explanation for the findings, relate to previous literature and theory, making generalisations and recommendation for future research)

LEARNING LEARNING ACTIVITY ACTIVITY 1. List some 1. List some of of the the qualities qualities of of a a good good questionnaire. questionnaire. 2. 2. Why Why is is pilot-testing pilot-testing necessary? necessary?

DATA COLLECTION METHODS USING SURVEYS 16

Surveys can be classified by their method of data collection. Generally, there are two main types of data collection methods: self-administered and investigator administered (see Figure 6.3). The most common self-administered method of data collection is the mail survey and more recently the web survey where the respondent expected to respond to the questionnaire without the presence of the investigator. The investigator administered method of data collection requires the presence of the investigator such as the telephone interview, face-to-face interview and group administered questionnaire. Besides the above, extracting data from samples of medical and other records are also frequently done. In newer methods of data collection, information is entered directly into computers using devices attached to TV sets that automatically record the channels being watched. a) Mail Surveys Surely you would at one time or another received a questionnaire in the mail (e.g. credit card companies, automobile companies). There are many advantages to mail surveys. This method of data collection can be relatively low in cost. You can send the exact same questionnaire to many people and they allow respondents to fill it out at their own convenience. Mail surveys can be most effective when directed at particular groups, such as subscribers to a specialized magazine or members of a professional association. The disadvantage of the mail survey is the low response rate. Also, since the researcher is not present, there is no way for the respondent to seek clarification if questions are unclear. b) Web Survey A more recent method of data collection is using the web. The questionnaire is uploaded to a website and respondents are invited to respond to the questionnaire. While it is less expensive and you can reach out to large audience there are many weaknesses with this method of data collection. The authenticity of the person responding can be difficult to prove, response rate may be low and persons responding to the questionnaire would be confined to those who have internet access which may not be representative of the population. c) Telephone Interviews Telephone interview are an efficient method of collecting some types of data and are being increasingly used. They lend themselves particularly well to situations where timeliness is a factor and the length of the survey is limited. The telephone interview gives respondents the feeling of anonymity since the interviewer cannot see them. d) Face-to-face Interview Face-to-face or in-person interviews in a respondent's home or office are much more expensive than mail or telephone surveys. They may be necessary, however, especially when complex information is to be collected.

Self-Administered

Mail Surveys

17

Questionnaire Survey Investigator Administered

Face-to- Face Interview Telephone Interview Group Administered Questionnaire

Figure 6.3 Data Collection Methods Using a Questionnaire

e) Group-Administered Questionnaire A sample of respondents are brought together and invited to respond to a structured sequence of questions. This is convenient method because you are able to capture a relatively large sample of respondents in one sitting (e.g. classroom). Also, the response rate is relatively high. If the respondents are unclear about the meaning of questions they could ask for clarification. However, the presence of the researcher may make respondents feel that their answers are less anonymous and as such they may be less candid. ETHICS IN SURVEYS What about confidentiality and integrity in surveys? The confidentiality of the data supplied by respondents is of prime concern to all who conduct surveys. For example, in many countries the data collected by the census department is protected by law. There are acts that guarantee the confidentiality of data collected by the relevant agencies Several professional organizations dealing with survey methods have codes of ethics that prescribe rules for keeping survey responses confidential. The recommended policy for organizations or individuals to safeguard such confidentiality includes

As far as possible use only number codes to link the respondent to a questionnaire and store the name-to-code linkage information separately from the questionnaires. For example 001 for respondent Azlina Darus, 002 for respondent Ong Mei Ling and so forth. The names and addresses of survey respondents should not be made available to anyone outside those involved in the survey after the responses have been entered into the computer (Individuals and organisations have been known to sell such databases to companies for marketing purposes without the consent of individuals involved!).

18

Omitting the names and addresses of survey respondents from computer files used for analysis Presenting statistical tabulations using broad enough categories so that individual respondents cannot be singled out.

Respondents should be informed about the purpose of the survey and have the option not to participate or not to divulge information that he or she feels not comfortable with. For example, respondents may be reluctant to disclose income. To overcome this, you may want to use categories (e.g. RM1500 RM2000 per month) which may be less intrusive. You should determine in the pilot-test which items respondents are uncomfortable with, so that you do not have too many unanswered questions to the point that some research questions cannot be answered. The questions asked should not in any way attempt to deceive respondents. The integrity of a survey is enhanced if respondents are clear about the purpose of the study.

DISCUSSION DISCUSSION QUESTIONS: QUESTIONS:

You You wish wish to to study study moral moral reasoning reasoning among among fifteen fifteen year year olds olds in in a a large large district. district. A A total of 4100 fifteen year olds are enrolled in 105 classrooms in the district. You total of 4100 fifteen year olds are enrolled in 105 classrooms in the district. You plan plan to to obtain obtain a a total total of of 250 250 students students using using cluster cluster sampling sampling technique. technique. Describe Describe the steps you would take in selecting the sample? the steps you would take in selecting the sample? Discuss Discuss some some of of the the problems problems with with telephone telephone interviews interviews that that may may affect affect the the results results of a survey. of a survey. One One of of the the drawbacks drawbacks of of mail mail surveys surveys is is the the low low response response rate. rate. Suggest Suggest how how you you would would increase increase response response rates. rates. When When would would you you use use a a longitudinal longitudinal survey survey rather rather than than a a cross-sectional cross-sectional survey? survey? What What are are some some problems problems with with surveys surveys conducted conducted over over the the internet? internet? How How can can you you overcome these problems? overcome these problems?

Books a) Mitchell, M. and Jolley, J. Research Design Explained (1988). New York: Holt, Rinehart and Winston. Chapter 12: An introduction to survey 19

research. 284-311. b) W. Borg & M. Borg, (1988), Educational Research: An Introduction, Chapter 6: Population and sample.

20