WEB-BASED SURVEY MANAGEMENT SYSTEM (W-SMS): USEFULNESS AND EASE OF USE Suleiman Yafao Elhejaj, Ariffin Abdul Mutalib,

*Mohd Helmy Abd Wahab. Universiti Utara Malaysia 06010 Sintok, Kedah [email protected] * Universiti Tun Hussein Onn Malaysia (UTHM) 86400 Batu Pahat, Johor Abstract This paper discusses an ongoing project. The aim of the project is to develop an in-house web-based survey management system. From a preliminary study, lecturers reported that inhouse survey-management system may help them in administering their research work more efficiently. The idea of this proposal is based on a number of related studies. In the review of literature part, it is also discussed about the process of survey management and tips to motivate response. At the end, this paper describes about the methodologies to be used in achieving the objectives. Key words: Web-based, survey management system, usefulness, eases of use 1.0 Introduction Surveys provide a means of measuring a populations characteristics, self-reported and observed behaviors, awareness of programs, attitudes or opinions, and needs (Sekaran, 1992). (Hair et al., 2006) adds that it is an ideal mechanism to gather and analyze large amounts of direct feedback about someones members, prospects, and employees. In supports of gathering big amount of data, computer technology may be a good option. In fact, it is commonly experienced that surveys are distributed through emailing services. Also, there are Web-based systems developed for administering survey practices. A web-based survey is the collection of data through a self-administered electronic set of questions on the Web (Thomas, 2003). Web-based surveys are able to conduct large-scale data collection. Web-based survey management system encompasses how the organizations organize, run and manage various types of surveys through the internet networks. It lets the user not only to build questionnaires but also to publish questionnaires to the respondents. This technology provides an inexpensive mechanism for conducting surveys online instead of through traditional survey methods. Also, it speeds up the distribution and response cycles. Web-based surveys are expected to be popularly used. However, it is observable that most people or organizations manage their survey using traditional method by distributing their survey through the mail or by telephoning, and some may afford to self-distribute by hand (Ariffin & Norshuhada, 2008; Zulikha & Ariffin, 2005; Tronstad, Phillips, Garcia, & Harlow, 2009). In current age, where digital is the theme, this is not a timely solution for gathering information because it does not have fast circular returning and responding from the respondents. Other issues such as cost, time and effectiveness are also within considerations.
Proceedings of Regional Conference on Knowledge Integration in ICT 2010 445

Therefore, a conceptual model of web based survey management system is required to be the solution for these issues. This initiative is proposed to solve problems as described in the next section. 1.1 Problem Statement

There are many types of surveys carried out by researchers which are by distributing the questionnaires to potential respondents manually through mail. This method requires cost, time, and efforts (Sekaran, 1992; Hair et al., 2006). A preliminary study involving 12 lecturers found that they agree to implement any tool that can assist them in administering survey practice. There are many commercial web-based survey management systems, but users have to pay for use besides having to register such as SurveyGizmo 1, SurveyMonkey2, and QuestionPro3. When these were asked to the lecturers, they prefer not to use the commercial web-based survey management system because they do not have trust on the system. This shows that the lecturers want to use any tool that help them in administering their survey practice, but the system is more preferably be in-house. Based on the described problem, a web-based system will be developed. In developing and make sure the developed web-based survey management system, two research questions were formulated: (1) How to design the system so that it is perceived useful? (2) How to design the interaction style so that the system is perceived easy to use? The system will be designed to cater the needs of traditional survey practice; including functions to disseminate questionnaire, gather feedback and store data, specify the period, and analyze the data (Sekaran, 1992; Barnum, 2002; Hair et al., 2006). Accordingly, this study is proposed to achieve objectives as outlined in the next section. 1.2 Project Objective

The main aim of the study is to develop a web-based system that administers survey practice which is called Web-based Survey Management System (W-SMS). To achieve the main aim three objectives were formulated: To determine functional components of W-SMS To develop a prototype of W-SMS. To evaluate the prototype of W-SMS in terms of usefulness and ease-of-use. 1.3 Scope of study

This research will be carried out for use by researchers and lecturers in UUM involving researchers of three main colleges i.e. CAS, COB, and COLGIS. 1.4 Significance of study

1 2

http://www.surveygizmo.com/ http://www.surveymonkey.com/ 3 http://www.questionpro.com/ Proceedings of Regional Conference on Knowledge Integration in ICT 2010 446

This study is expected to contribute significantly to different parties, including to the body of knowledge and to the researchers. To the body of knowledge: Results gathered from the evaluations will be used to propose whether W-SMS is useful and easy to use by researchers. This is important because researchers that are going to be asked include all levels of computer background. To the researchers: By implementing W-SMS, researchers can reduce the cost, time, and efforts in administering their survey. Previous studies have shown that web-based survey management systems have been used for many reasons. There are benefits out of it, and there are also disadvantages. Reviews on previous works similar to this study are discussed in the next section. 2.0 REVIEWS ON USABILITY AND WEB-BASED SURVEY MANAGEMENT SYSTEM This section elaborates about related topics to this study including web-based survey; types of web-based system, advantages and disadvantages of web Surveys, web-based survey administration process, and mechanisms to improve the Response of web-based Survey. 2.1 Definition of Terms Today the word survey is used most often to describe a method of gathering information from a sample of individuals. Surveys can be classified by their method of data collection (Leedy & Ormrod, 2001). Mail, telephone interview, and in-person interview surveys are the most common types (Scheuren, 2004). Besides, surveys are also distributed and collected through the means of electronic application over the Internet (Leedy & Ormrod, 2001; Catherine, Dimitrion, & Mike., 2001) There are two methods of using the Internet as survey mechanisms: (1) electronic mail (email) and (2) the World Wide Web (the Web). With e-mail, researchers can send surveys to e-mail addresses as text messages, in which the recipient can then read, save, respond to, or throw away, much like a paper survey (Carey, Mao, Smith, & Vredenburg, 2002). Surveys can also be posted on the Web and may include text, pictures, and forms to be filled in by the respondent (Catherine, Dimitrion, & Mike., 2001). According to (Galin, 1998), the primary difference between these two response modes is that e-mail is a push technology while the Web is a pull technology. That is, with e-mail, the sent messages are automatically received in the potential respondents mailbox, whereas respondents must be attracted in some way to a Web page. Because of this difference, one might expect a higher response rate to an e-mail survey than to a web-based survey (Catherine, Dimitrion, & Mike., 2001). In short, a web-based survey is a survey conducted by a researcher or the public through the internet. They can create their online questionnaire and distribute it through the Internet. On the other hand, a web-based survey management system is a system which can manage the survey through internet. In this study, a database will be integrated with the W-SMS to store data. When respondents answer the survey at anytime, data will be stored in the database. At the end of data collection period, researchers can retrieve the responses to analyze.

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

447

2.2 Types of Web-based Surveys Nowadays, Web-based surveys are everywhere on the Internet. There are various categories of Web-based surveys. According to (Couper, 2000), there are two major categories of Webbased surveys: probability-based or non probability-based surveys. (Couper, 2000) included the following in the non-probability-based category: Entertainment surveys Self-selected Web-surveys Surveys made up of volunteer panel of Internet users Entertainment surveys consist of questionnaires that request a vote on particular questions and other instant polls. Usually, people need to spend a lot of time to complete an entertainment survey. These surveys do not lead to generalizations of viewpoints across populations, and are not intended for that reason. Other non probability-based surveys include dedicated survey sites maintained by owners of Web sites. Such surveys could allow multiple submissions, and make no attempt to be representative of the whole Internet population. On the other hand, there are several types of probability-based Web-based surveys as listed below (Couper, 2000): Intercept surveys which are frequently used to survey on customer satisfaction (Couper, 2000): Surveys that obtain respondents from an e-mail request. Mixed-mode surveys where one of the options is a Web survey. Pre-recruited panel of a particular population as a probability sample In addition, the Intercept survey polls every n th visitor to a Website and prevents multiple submissions from the same computer by using cookies. This is important to make sure that every response is sent by different respondent. In the survey that obtains respondents from an e-mail request, the survey will be sent to respondents who agree to complete a survey in response to an e-mail invitation to participate. Non-response is a big concern with this type of Web survey. The mixed-mode survey let the participants complete the survey on the Web or by paper. With a pre-recruited panel as a probability sample, respondents are provided with passwords or personal identification numbers. In some cases, the participation of probabilitybased samples of the full population is obtained by providing equipment in exchange for participation in the survey. From these, it could be concluded that the type of the web-based survey is determined by the objectives of the survey and also the type of questions that are asked by the researchers. 2.3 Advantages and Disadvantages of Web-based Surveys Referring to (Rena et al., 2004) Web-based surveys are important today, in the digital age, and most organizations are moving towards digitizing products and process. Also (Rena et al., 2004) mention that technologies are always available to serve the needs. In fact, these technologies are anticipated as one of the factors to gain advantage in the competitive business square (Dillman and Bowker, 2001). When deploying a web-based survey management system, the organization is able to gain advantages, as well as facing some disadvantages as discussed by (Rena et al., 2004).

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

448

2.3.1 Advantages The advantages can be gained obviously in terms of fast processing job, expenditure, and survey management (Dillman and Bowker, 2001; Rena et al., 2004; Couper, 2000; Leedy & Ormrod, 2001). Savings in printing, postage, data entry. No data entry errors from hand-entry. Shortened timeframe to administer surveys (3 weeks with web-based survey Vs. 6 weeks or more with paper surveys). Easier and cleaner in providing skip patterns or survey sections customized to different respondent populations. Almost immediate access to data for analysis. Can easily link to background data, if appropriate (e.g., gender, grades, rank). 2.3.2 Disadvantages Surveys are meant for human to answer. So, the surveys should be designed in a way it helps respondents understand well. Platform compatibility is another aspect to consider. In this case, the system should be able to run on multiple platforms such as Windows and Mac. Below is a list of possible disadvantages of deploying web-based survey management system (Wyatt, 2000; Fricker & Schonlau, 2002; Information Technology Services, 2008). Generality and validity of the results. Needs programming and IT expertise. Certain populations are not comfortable using computers. Must have accurate email lists. Web-based surveys are not recommended for email software that does not support web access. Must be able to click on a URL provided in an email and to have it bring respondents to the system. There may be problems finding software that is appropriate for both PCs and Macs, or developing surveys that run on both platforms. Data provided via a web-based survey are not anonymous, although the survey administrators may choose to keep the results confidential. It should now be quite clear about the benefits and limitations of using web-based survey. So, choosing a web-based survey or other survey method depends on the purpose of the organization. 2.4 Web-based survey administration process According to (Rena et al., 2004), there are four stages when doing web-based survey projects. Stage One: Survey Background Building a survey team. Defining survey content. Choosing web surveys. Selection of the survey population. Stage Two: Designing the Web-based survey. Questionnaire design. Choosing HTML software and format. Turning a survey questionnaire into the web-based survey: technical issues. Stage Three: Administering a Web-based survey Administering a web survey: technical issues.
Proceedings of Regional Conference on Knowledge Integration in ICT 2010 449

 Pre-testing. Privacy issues. Survey incentives. Technical support for duration of survey. Output of survey data. Stage Four: Reporting In this study, a clear picture about the web-based survey and the administration process are needed in order to come out with the W-SMS. All stages must be done one by one. 2.5 Mechanisms to Improve the Response of Web-based Survey Low response rate of online survey is one of the major concerns in many organizations which are involved in online survey management. Since online survey is one of the ways which can help organization gauge expectations, understand participants reaction and meet the organizations target markets, it is very important to increase the response rate of online survey. In response to that, (Jennifer, 2006) suggests several ways to increase response rates of online survey: Target the audience. Personalize email invitations. Keep the invitation email short and simple. Make simple instruction at the first page of the survey. Be clear about privacy protections. Send reminder emails. Consider offering incentives - gifts, prizes, etc. Use graphics and Internet features strategically. Publish your results online for survey participants. 2.6 The Perceived Usefulness and Ease of Use Davis (1989) has shown that Technology Acceptance Model (TAM) can explain the usage of information technology. He applied (Fishbein &Ajzens) theory of reasoned action to show that beliefs influence attitudes which lead to intentions, and therefore generate behaviors. Davis thus conceived that TAMs belief- attitude-intention-behavior relationship predicts user acceptance of IT. Davis asserted that perceived usefulness and ease of use represent the beliefs that lead to such acceptance. Perceived usefulness is the degree to which a person believes that a particular system would enhance his or her job performance (i.e., by reducing the time to accomplish a task or providing timely information). Perceived ease of use is the degree to which a person believes that using a particular system would be free of effort. Two other constructs in TAM are attitude towards use and behavioral intention to use. Attitude towards use is the users evaluation of the desirability of employing a particular information systems application. Behavior intention to use is a measure of the likelihood a person will employ the application. TAMs dependent variable is actual usage. It has typically been a self-reported measure of time or frequency of employing the application (see Figure 1). The arrows represent the relationships of interest in the current study. Some authors have considered additional relationships. Some have ignored intention to use and instead studied the effect of attitude on usage. The current research took that approach. Such theories and models as self-efficacy theory, cost- benefit research, expectancy theory, innovation research, and channel disposition have supported TAM.

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

450

Figure 1: The Technology Acceptance Model 3.0 Research Methodology Research methodology is more than just collections of methods to perform a research; it is a systematic way to solve the research problem. This study will adapt a methodology from (Hofer et al, 2004) and is shown in Figure 2. 3.1 Selection, Planning and Analysis Planning is the most important step. The failure of doing a good plan is a hazard to the whole process of implementing the project. For the beginning, a comprehensive study has to be in order to get the clear idea of what will be achieved at the end of the project. Document review including articles in proceedings and journals, magazines, newspapers, and books will be done to get the idea of problem solving techniques, and issues. In the analysis, a few activities will be carried out such as document review, comparative study of current web-based survey management system to identify functional components, interviews with users regarding common process in survey management, and understanding users background. The aim of this phase is to gain an in-depth understanding of W-SMS. By the end of this phase, the requirement of W-SMS could be figured out such as best practices, common procedures and features of web-based survey management. 3.2 Development Methodology After obtaining user requirement in the analysis phase through comparative study and user interviews, it is appropriate to justify on how it meets the identified requirements. In this stage, the interfaces for the prototype will be designed. Besides, the database will also be designed and developed. The prototyping process comprises of three steps which are adapted from Prototyping Process (Laudon, 1995), as shown in Figure 3. Prototyping provides end users with artifacts that allow them to gain insight into the behavior of the system before the final delivery.

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

451

PHASES

ACTIVITY

Selection, Planning and Analysis

Understand the background of the problem. Analysis back ground of the problem. Determine project objective and scope. Need analysis for the proposed system.

Development

Identify application requirements. Design of prototype. Development of prototype.

Evaluation

Usability study - Usefulness and ease of use.

Figure 2: Methodology Design

Using Prototype Revising & Enhancing Prototype Figure 3: Prototyping Process

3.3 Evaluation Finally, the W-SMS will be evaluated. 60 lecturers of UUM will involve in the evaluation; 20 of each college, CAS, COB, and COLGIS. First, the lecturers will be demonstrated with the W-SMS, and then they will try the system on their own. A week will be allocated for the lecturers to try the W-SMS. Then, their responses on the usefulness and ease of use of the W-SMS will be gathered through an instrument. This study has decided to utilize an existing instrument. The instrument by (Davis, 1989) that measures Perceived Usefulness and Ease of Use (PUEU) has been widely used in various fields of study such as banking , education, and marketing. REFERENCES Ariffin A.M. & Norshuhada, S. (2008). Usable but not entertaining eLearning materials. In Proceedings of World Conference on e-Learning in Corporate, Government, Healthcare, and Higher Education (e-Learn), USA. AACE Barnum, C.M. (2002). Usability testing and research. USA: Pearson Education, Inc.

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

452

Carey, T., Mao, J., Smith, P., & Vredenburg, K. (2002). A survey of user-centered design practices. In Proceedings of the 2002 SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press. 471 478. Catherine C., Dimitrios, B., & Mike, P. (2001). Enhancing SMTEs business performance through the Internet and e-learning platforms. The Centre for eTourism Research (CeTR), School of Management, University of Surrey, Guildford, UK. Couper, M.P. (2000). Web-based surveys: A Review of Issues and Approaches. Public Opinion Quarterly. 64, 464-494. Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13(3). 3 19-339. Dillman, D.A. and Bowker, D.K. (2001). The Web Questionnaire Challenge to Survey Methodologists, Retrieved on 3rd March 2010 from http://survey.sesrc.wsu.edu/dillman/zuma_paper_dillman_bowker.pdf,. Fricker, R.D. & Schonlau, M. (2002) Advantages and Disadvantages of Internet Research Surveys: Evidence from the Literature. Field Methods, 14(4). 347-367 Galin, M. (1998). Collecting data through electronic means: A new chapter in the evolution of survey methodology? In Proceedings of the American Evaluation Association Annual Conference. Chicago Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E., & Tatham, R.L. (2006). Multivariate Data Analysis 6t Edition. Pearson Education International: USA Information Technology Services. (2008). Online Surveys. Retrieved on 3 March 2010 from http://www.utexas.edu/learn/surveys/disadvantages.html. Leedy, P. & Ormrod, J. (2001). Practical research: Planning and design. Upper Saddle River, NJ: Prentice-Hall. Sekaran, U. (1992). Research methods for business: a skill-building approach 2nd ed. USA: John Wiley & Sons, Inc. Thomas, M. A. (2003). Web-Based Surveys. Leader, Program Development and Evaluation. Ohio State University Extension Columbus, Ohio Tronstad, B., Phillips. L., Garcia, J. & Harlow, M.A. (2009). Assessing the TIP online information literacy tutorial. Reference Services Review. Wyatt, J.C. (2000). When to use Web-based surveys. Journal of The American Medical Informatics Association. 7(4). 426-429. Zulikha, J. & Ariffin, A.M. (2005). IT-graduate abilities: performance gap as an input for curriculum improvement. In Proceedings of 3rd International Conference on Information Technology: Research and Education (ITRE 2005). Taiwan. IEEE.

Proceedings of Regional Conference on Knowledge Integration in ICT 2010

453