Measuring The Performance of Information PDF
Measuring The Performance of Information PDF
Usman
Department of Accounting, STIE Bank BPD Jateng, Jalan Pemuda 4A,
Semarang 50136, Indonesia
Abstract. Information systems (IS) function is defined as all individuals, group, or department in an
organisation daily responsible for information systems related activities in the organisation. This paper
presents the development of a model and scorecard to measure the IS functions performance based on a
number of earlier models such as Pitt and Watson (1995), Seddon (1997), and Heo and Han (2003).
Founded on literature study in the areas of IS success, IS organisations effectiveness, IS resources and
capability, and IS function performance, 66 performance items were identified. 132 data set from 29
national and multinational organisations were used to validate the measurement instrument. The resulted
performance scorecard covers four main dimensions, namely:1) Product Quality, 2) Service Quality, 3)
Planning, Adaptation and Operational Support Capability, and 4) Net Benefit. This study highlights the
importance of IT/IS-Business partnership to increase the performance of the IS function, especially related
to the capability of IS function to perform a good quality planning, effective system adaptation and good
operational support.
Several earlier works on IS functions performance were 1999, Jones 2004). Finally, Several works in the area of IS
studied (e.g. Saunders and Jones 1992). Results of studies in resource and capability (e.g. Zhu 2004; Ravichandran and
the area of IS success were explored (e.g. DeLone& McLean Lertwongsatien 2005) were also used in this study
1992, Myers et al. 1997, Seddon 1997, Heo and Han 2003, considering that users view on the capability of IS function
DeLone & McLean 2003). Looking at the IS function as a in performing their tasks and the quality of IS resources
special unit within an organisation, studies that analysed the available can influence the willingness of users to propose
effectiveness of the IS organisation are studied (e.g. Carnall IT-based innovations in the organisations.
The identification of dimensions of the performance Further, not all dimensions within both Output and Outcome
measurement model is done through an analysis using the categories are further included, since some of the dimensions
input-output logic model as a basic framework. This basic are more the concern of management people. Those
framework had been used in a number of studies, e.g. dimensions include: Productivity, Profitability, and
Heavisides and Price (2001) and Brophy and Wynne (1997), Operational efficiency from Output category, and the
to develop a performance measurement system. The effectiveness of organisational business processes,
framework suggested that performance measurement organisational performance and growth for Outcome
dimensions can be derived by analyzing performance category. It is necessary to include other organisations
measures in the following four different categories/groups: stakeholders such as senior management to evaluate/measure
Input, Process, Output and Outcome. In this study, the those dimensions. At this stage, there are 17 measures
identification of the dimensions based on the Input-Output included in the measure list. In designing and developing a
logic model is done based on results of previous studies in performance measurement instrument, it is necessary to
four literature domains namely IS Success, IS Organisations remove the duplications in the measures (Folan and Browne
Effectiveness, IS Resource and Capability, and IS Function 2005). Further, combining a few closely related measures into
Performance. The result of dimensions identification is one measure is suggested to get a simple list of measures.
presented in Tabel 1. The identification was done for each After removing the duplications and performing some
theoretical domain. combinations in the measure list, the following four main
dimensions were resulted, namely: Product Quality (1),
Considering that the IS functions performance
Service Quality (2), Planning, Adaptation and Operational
measurement model developed in this study is focused on the
Support Capability (3), and Net Benefit (4).The process to
user perspective, only dimensions in Output and Outcome
come into the four main dimensions is globally presented in
categories are included in further model development stage.
Table 2.
The next step in the development of the measurement identification is done based on literature as shown in Table 1
scorecard is to identify measures of IS functions (Dimension Identification). As can be seen in Table 2, the
performance within the four main dimensions. The measures first dimension (Product Quality) addresses system quality
and information quality (included information output quality) 17. Ease to understand the information provided
of the products provided by IS function. For Product Quality, 18. Information precision
20 measures were identified and they are presented in Table 19. Information concision
3. 20. Organisation of information
The second dimension (Service Quality) addresses many
aspects related to the services provided by IS function. Service Quality
Besides relevant studies which presented in Table 1, a 1. Usefulness of training delivered
number of other literature related to SERVQUAL were also 2. Variety of training delivered
used in identifying the relevant service quality measures (e.g. 3. Socialization of proper technology development
Jiang et al. 2002, Kettinger 1997, Pitt et al. 1997, Pitt et al. insights to other stakeholders
1995, Van Dyke et al. 1997, Watson et al. 1998, and Zeithaml 4. Suggesting ideas for improvements
et al. 1990). For Service Quality, 16 measures were identified 5. Willingness to help
(See Table 3). 6. Willingness in give help more than expected
The third dimension (Planning, Adaptation and 7. Responsiveness in giving service
Operational Support Capability) addresses many aspects 8. On time delivery of service needed by users
related to the capability of IS function in planning and 9. Ability to give personalized service
developing systems, giving operational support, identifying 10. Willingness to give emergency service
and assimilating new technology that is suitable to the need 11. Willingness to give service to users (business)
of business, providing flexibility in managing the relation
organisations IS and finally the capability of IS function in 12. Efficiency and effectiveness in giving service
adapting the organisations IS in line with the changes in 13. Hospitality of IS people
business. For Planning, Adaptation and Operational Support 14. Sincerity of IS people
Capability, 9 measures were identified and they are presented 15. Ability of IS people to cooperate with business
in Table 3. people
16. IS people attitude towards giving good response to
The forth dimension (Net Benefit) addresses measures
the business people
related to the impact of using the products provided and
services given by IS function. Relevant factors found in
Planning, Adaptation and Operational Support Capability
literature such as user satisfaction, individual benefit, work
1. Capability to identify and overcome problems
group benefit, organisational benefit, and image were used.
2. Capability to comprehend specific need of users
For Net Benefit, 21 measures were identified and they are
3. Capability to overcome emergency conditions
presented as follow.
4. Planning sophistication
Product Quality 5. Capability to identify technology change
1. Ease of use 6. Capability to convey ideas
2. Ease to learn 7. Capability to adapt to new technologies
3. Ease to access features available 8. Capability to adapt to the changes in the environment
4. Response time during usage 9. Transparency in managing activities
5. System downtime
6. Presentational quality of information Net Benefit
7. Tidiness of the systems arrangement 1. Benefit on ease to complete the tasks
8. Integration capability 2. Benefit on work efficiency
9. System reliability 3. Benefit on user and department productivity
10. System flexibility 4. Benefit on product quality
11. Ability of systems to meet user expectation 5. Benefit on decision making capability
12. Ease to handle error 6. Benefit on communication effectiveness among
13. Ease to up-date information organisations members
14. Ease to access information 7. Benefit on client satisfaction toward product and
15. Ease to verify information service provided by individuals and organisation
16. Information security
8. Benefit on information exchange between users and Y 11k
their client
Y 21k
9. Benefit on delivery time of products or services
M
10. Benefit on work monitoring Factor 1k M
Y l1k
11. Benefit on business process management
Y 12k
12. Benefit in cost efficiency
13. Benefit on collaboration effectiveness kth Prime Factor 2k Y 22k
Dimension M
14. Benefit on collective decision making process M M
15. Benefit in learning new things M Y l2k
Easy of 0.84
0.21 Product Use
0. Interpersonal
Intrinsic Qual. 89 Character 0.49
0.37 of System 2
0 .7 9 0 .7
PRODUCT SERVICE Service
G eneral 0.45 0.80
Characteristic 0. 86 QUALITY Flexibility
0.25 QUALITY 0 .8
of System 0 6
0 .7 0. Technology
Intrinsic Qual. 77 Assim ilation 0.26
0.51 of Inform ation
073 050 Responsive-
ness 0.41
061
069
Change Capab.
0.
Benefit on
0.85 Business Proc. 054
Benefit on
0.60 Know ledge
Transfer