Evaluating: To of
Evaluating: To of
Sutanto SOEHODHO
Center for Transport Studies
Departrnent of Civil Engineering
University of Indonesia
Depok 16424, Indonesia
Fax.62-21-127N28
E-mail :tanto @eng.ui. ac.id
Abstract: Evaluating urban public transport system, especially bus system, comes up as a
crucial activity in Jakarta since the local government currently imposes various performance
measurements to urban transport systems for efficiency and effectiveness. On one hand the
service capacity of city bus service has been declining due to the continuing economic crisis
for highly tremendous maintenanie cost. On the other hand the policy of local government
has been shifted to encourage more use of public transport as to reduce urban traffic
congestion, and inviting more roles of private sector for the undertakings. To such extent,
scruiinized evaluation on bus service is required to identify critical seryice components for
further investment. In this work a multidimensional approach to evaluate such service is
developed wherein the canonical correlation analysis (CCA) is proposed as a tool. Data from
main city bus operators in Jakarta is extracted and analyzed to indicate the main service
indicators. This approach is further expected to provide sound tool for private sectors in
evaluating prominent service indicators to justify their participation and investments in the
urban public transport service.
l.INTRODUCTION
Jakarta as capitol city has a long history of urbanization, and this situation keeps on while the
country has been experiur:cing an economic crisis that makes most socio-economic activities
rtugnunt if not declining. Other problem of urban transport in Jakarta city is the limited space
urui6bitity through which the transport infrastructures to be continuously developed. Air and
noise pollutions, although are not yet considerQd as critical counter-measure has made the
urban environment worse off. As an illustration, in 1998 it was identified that no less than 16
million person trips per day were recorded. Their mobility would then split as 50.87o using
their private vehicles including motorcycles and 49.27o using public transport with majority
of urban bus mode.
These worsening situations in the city has made Jakarta local government impose different
paradigm to solve urban transport problem from which the transport demand management is
aptly idopted. Basically within such approach physical development on infrastructure would
Ue sirictty limited, and instead the more utilization of existing facilities would be encouraged.
ln consequence role of bus system as major urban transport service is to be enhanced to cope
with existing demand as well as potentially shift passenger car users.
In spite ol worsening condition of declining urban public transport system evaluation criteria
are hard to define. Literature review has shown that evaluation studies which focus on transit
system vary in scope, emphasis and methodology. Some studies emphasized financial and
economic indicators (Heathington and Graeub, 1975; Ire, 1976), efficiency and effectiveness
Proceeclings of the Eastem Asia Society for Transportation Studies, Vol.3, No.1, October, 2001
140
SUIANIO SOEHODHO
(Fielding, et.al., 1978), and equity and impacts (Fielding and Lyons, iggl; Savas, l97g).
Other investigated the effects of environmental factors (Guiliano, 1980) and govemment
policies (Pucher, et.al., 1983). Recently, a group of studies has tried ro determine rhe impa4s
of service area characteristics on transit performance (cervero, 1990; yu, 1990).
Route level performance evaluation studies have focused on two broad approaches: a) service
standards related to design and operations, and b) economic and financial measures. Despite
the advancement of research in the development of performance indicators for evaluating
transit services in 1970's and 1980's, problems in their applications still prevail. Further
indication can be made in the literature that three broad limitation of current research in
public transport or transit sen,ice evaluation remain.
The first limitation pertains to the conceptualization of service performance. Research studies
have tended to see service performance in terms of dichotomous classifications such as best-
performing routes versus worst-performing routes or good routes versus bad routes. While
these labels may be appropriate in one set of circumstances, such as assisting transit
managers to implement simple improvement packages, forecasting service impacts and
satisfying equity and disabilities' requirements. Thus, an expanded conceptualization of the
notion of service performance is needed.
The second limitation pertains to public transport systems' almost exclusive use of
performance standards. While service standards seem to be the only logical measure of
performance levels, there are a number of problems associated with their development and
application. These include lack of clear achievable service objectives from which
performance standards ca be derived, inconsistent measurement and application of standards
by different public transport system, lack of timely and proper review of standards and the
almost exclusion of exogenous measures, such as network structure, area density patterns and
service area characteristics. Other problems include cost and quality of data collection and
processing, budgetary and financial constraints and political pressure exerted on the 'rational'
application of standards.
The third limitation is of a methodological nature and relates to the approach uscd to specify
and test relationships between service evaluation characteristics and service performance.
Previous research has treiated service evaluation and its performance components as if they
were unidimensional. Analysis of their interrelationships has been done using longitudinal
methods such as means, time series, and regression analysis, peer comparisons, performance
rating or ranking based on established standards, and portfolio technique.
To overcome the foregoing limitations, this study aims to redirect service evaluation research
by addressing evaluation process and performance outcome as two different dimensions. The
primary purpose of this study was to examine conceptual and methodological bases of service
evaluation by developing and testing, as case 'of Jakarta Urban Bus System, a multi
dimensional framework that looks comprehensively at somewhat wider span of evaluation
including the contcxt, process and outcome of the evaluation. After discussing the concept
and necessary criteria for public transport service performance, it describes the choice of
multi dimensional analysis, surveyed data, analyses and results.
Proceedings of the Eastern Asia Societv for Transportation Studies, Vol.3, No.1, Ocrobcr. 2001
l4t
A Multidimensional Approach for Evaluating Bus Service
To measure the service performance of urban bus service is a big task since it involves
various perspective dimensions of criteria. This research would deliberately look at two
perspective dimensions of such criteria, namely evaluation process and service performance.
The first dimension would certainly focus on use of data evaluation to increase effectiveness
of service performance. The second dimension would focus on specific performance level
related to efficiency, effectiveness, and approach to performance prediction. Furthermore,
first dimension may have to comprise various criteria such as planning and design,
operational, and service area. While second dimension may comprise object performance
fulfillment, use of evaluation data, and improvement of agency capacity and public image. In
brief Figure 1 illustrates the relation of the two dimensions'
For each the dimension within the scope of evaluation would further consist of criteria used
as indicators. In this multidimensional analysis the service performance evaluation dimension
is supposed to have: a) Planning & Design Standard Criteria (e.g., route length, passenger
trarei ii*. savings, spacing between stops, area coverage, route duplication), b) Operational
Standard Criteria (e.g., available fleet during peak hours, vehicle ratio at peak hours, vehicle
revenue per trip kilometer, vehicle revenue per operating hour, driver's skill improvement)
and, c) area couerage criteria (e.g., population density, minority population, medium to low
income households, car ownership).
This hump of criteria is quite general and has practical implications. It is, however, worth
noting that data on such information for the Jakarta City Bus System may hardly be found in
practice. So in this research, as would be explored in the ensuing section, only some of them
are affordable and explored further within the multidimensional analysis.
Given the task of examining the interrelationships between a set of service evaluation
dimensions and a set of servicc performance dimensions a proper method of multi
Proceedings of the Eastem Asia Society for Transportation Studies, Vol.3, No.1, October, 2001'
r42
suhnto soEHoDHo
dimensional analysis is required. Canonical correlation analysis (CCA) is chosen as the most
appropriate, because CCA lends itself well to a multi dimensional approach to observing a
complex phenomenon (expressed as a set of variables) via its relation to other phenomena
(expressed as individual variables). Thus the CCA would allow the complex phenomenon
involving the technical and social processes of service evaluation to be addressed.
To summarize, the CCA was chosen over other approaches for three basic reasons: a) it can
specify, test and explain relationships in one data set as well as between two data sets, b) it
of the dimensions through canonical structure (loading and
captures the interactive effects
crossJoading) and redundancy analysis and c) it is most suited and empirically accepted
method for analyzing multi dimensional evaluation problems (Ramanujam and Venkatraman,
1987; Yu, 1990).
To comply with the analysis the followings are steps to obtain canonical correlation
coefficients;
R=[R-
*'l . (1)
IR Rtyl
L yr
I
, and in accordance with the variables analyzed each component would have the following
dimension;
Ro =8x8
R) = 6x8
R,, = 8x6
R, = 8x8
)zxv ->X:y
Rr= ...(2)
Jtsrr' -(:x)'Jn ' -(>y)'
. Step 2. Search the Eugen Value (),).
Eugen values are to be determined since they become the canonical coefficients (canonical
weights). Such eugen values can easily be derived from the followings;
Proceedings of the Eastem Asia Society for Transportation Studies, Vol.3, No.1, October, 2001
t43
A Multidimensional Approach for Evaluating Bus Service
. Step 3. Perfonn Canonical Variables. These variables are composed by specific vectors
amount to the min(p,S)pairs with the followings;
Vr=arX Wr=brY
Vt = W"=,brY
.arX
.....'....(6)
Vo=arX Wn=boY
rvhere;
xl Yl
x2 Y2
X_ f=
xp Y
,-@
r_ a'Rob
(8)
Step 5. Detennine the dominant variables. Dominant variables can be calculated with
following formulation;
Evaluating the service performance of city bus in Jakarta is big task since it covers large scale
of bus network involving heterogeneous capacities (e.g., small, medium and large size).
Apart from such coverage data availability is hard to search, and only large size buses
normally operated by big companies could be located with fairly reliable information. In this
research five main operators of large bus service were identified and explored for the related
data, they are namely PPD, Mayasari Bhakti (MB), Steady Safe (SS), Bianglala (B), and
Kopaja (K) (see Appendix). To anticipate any bias in data capture some primary data were
collected nearby the terminals (e.g., Blok M, Kalideres, Tanah Abang, Senen, and Irbak
Proccedings of the Eastern Asia Society for Transportation Studies, Vo1.3, No.1, October, 2001
144
sutanto soEHoDHo
Bulus). The flat tariff for each service is different (i.e., Regular = Rp. 500,-, PATAS = Rp.
800,-, PATAS AC = Rp. 2,500,-), and different operating hours (i.e., Regular = 2 x 8 hours,
PATAS = 2 x 8 hours, and PATAS AC = 12 hours). As for number of routes for each
company are PPD = 18 routes, NIB = 22routes, SS = 14 routes, B = 5 routes, and K = 2
routes. ln consequence with number of routes and fleetsize the number of operators/drivers
are PPD = 2,812 persons, lvfB = 2,872 persons, SS = 1,162 persons, B = 438 persons, and K =
176 persons). The operating cost/vehicle for each company is different in the following PPD
Rp. 110,000,-, MB = Rp. 100,000,-, SS = Rp. 100,000,-, B = Rp. 105,000,-, K = Rp.
105,000,-, in which Rp = Indonesia rupiah.
In order to achieve high accuracy in calculation some variables are rationalized in such a way
that only 14 variables would be anaiyzed further, These 14 variables are then divided into two
sets of dimensions, namely service evaluation dimension (.IO and service performance
dimension (I'). Evaluation dimension comprises the following operational variables (refer to
Table A. in appendix);
Service performance dimension comprises the following operational variables (refer to Table
A. in appendix);
To have reliability values of service measures some statistic is done and values of mean and
standard deviation for each are summarized in Table 1.
Testing the strength of correlation among variables (X,D is necessary to adopt the most
appropriate or influencing variables. Results of correlation test are summarized in Table 2. It
can be learnt from the table that most of correlation appears weak although most of them
show positive correlation. Correlation among the service performance variables are
apparently stronger than the service evaluation ones, and correlation between the vehicle
revenue hour per operating cost and vehicle revenue per operating hour (e.g., 0.9891). This
implication has apparently addressed further investigation on multidimensional analysis (i.e.,
CCA).
Proceedings of the Eastern Asia Society for Transportation Studies, Vol.3, No.l, October, 2001
145
A Multidimensional Approach for Evaluating Bus Service
The CCA would suggest to develop canonical variables amount to Min [X,Y] = Min [8,6] =
6 pairs of canonical variables. So implicitly there will be 6 independent canonical variables
(If and 6 dependent canonical variables (lI.) {i.e, Vi: Wi), wherein Vl is a linear combination
of {X}, and Wi is a linear combination of { Y}. Having these canonical variables coefficient of
each linear combination known as canonical weight is to be determined. Due to different
units used for each of operational variables the coefficient to be determined would have to
comply with standardized canonical coefficients as summarized in Table 3 and 4 for {V}--+A;1
Vi j and { I7}-+81 respectively.
Proccedings of the Eastern Asia Society for Transportation Studies, Vol.3, No.1, October, 2001
146
Sutanto SOEHODHO
Based on Table 3 and 4 any linear combination equation of the canonical variables can be
composed as [Vi = Ei A,iXil and []Vi = 4
BijYi) such like;
Vl = -0.0096X1 - 0.07 33X2 - 0.0337X3 - 0.0 17 1X4 + 0.3 l05XS - 0,07 57X6 + 0.0452X7 +
0.9597 X8.
Since there are 6 pairs of canonical variables there would consequently 6 canonical
correlation coefficients as analyzed and summarized in Table 5. It can be learnt from the table
that there are strong correlations in the first, second and third canonical correlation
coefficients. These coefficients have indicated strong relations between set of {X} and set of
{Y} variables simultaneously. On the basis of such indication of strong correlation, it is
necessary to determine which combinations would be sigrificant and conclusive. To
investigate further two examinations namely cumulative varied proportion and canonical
correlation significance test are conducted. Results of such examinations are given in Table 6
and 7 respectively.
Proceedings of the Eastcm Asia Society for Transportation Studies, Vol.3, No.1, October, 2001
r17
A Multidimensional Approach for Evaluating Bus Service
The two examinations have suggested similar confirmation on which linear combinations
significantly determine prominent variables in the multidimensional analysis for bus service.
This determination would address further investigation to first and second combinations. To
comprehend such correlations more specific and explicit relation is to be analyzed through
evaluation of canonical loading that denotes correlation between original and canonical
variables. Canonical loading of X would denote correlation of each X variable to its
counterpart V variable, and likewise for canonical loading of I that denotes correlation of
each y variable to its counterpart W variable. Summary of canonical loadings for {V} and
{ }7} are given in Table 8 and Table 9 respectively,
Table 8. Correlatioq between the 'VAR' Variables and Their Canonical Variables
V1 v2 V3 V4 V5 v6
xl 0.3714 -0.6708 0.489',1 0.2278 -0.3197 0.1181
X2 0.41'75 0.2543 -0.2280 0745'1 0.1930 -0.1380
X3 0.4507 0.4747 -0.1r50 0.6662 0.1083 0,0054
X4 0.0292 -0.1355 0.o796 0.344',1 -0.2379 -0.1537
X5 0.3022 0.8934 0.1168 0.2225 -0.142'7 0.1541
x6 0.1 166 0.4081 -0.2825 -0.2572 -0.6405 0.4468
X7 0.8202 0.2942 -0.2217 -0.3333 0.1252 -0.2000
x8 o.9667 -o.2438 -0.0647 0.0121 0.0008 -0.0278
Table 9. Correlation between the 'WITH' Variables and Their Canonical Variables
W1 w2 w3 W4 W5 W6
Y1 0.9680 -0.225',t -0.0808 0.0248 -0.0696 0.0066
Y2 -0.2574 o.7945 0.3692 -0.3625 -0.1648 0.0873
Y3 0.9036 -0.2637 -0.0620 -0.0997 0. r 585 0.2't40
Y4 -0.1317 0.8525 0.36'71 0.2324 -0.0851 0.2449
Proceedings of the Eastern Asia Society for Transportation Studies, Vol.3, No.1, October, 2001
148
Sutanto SOEHODHO
a) The first strong conelation has a value of 0.993381 that composed with 5770 varied
data. The Y variables that dominate are Y1 (vehicle hour revenue per operating cost)
and Y3 (operating revenue per operating cost) with canonical loading values of
0.9680 and 0.9036 respectively. The X variablcs that dominate are XJ (vehicle
revenue per kilometer trip) and X8 (vehicle revenue per operating hour) with
canonical loading values of 0.8202 and0.9667 respectively.
b) The second strong correlation has a value of 0.989484 that composed with 35.67E0
varied data. The X vanables that dominate are Xl (route length) and X5 (number of
vehicles at peak hour) rvith canonical loading values of -0.6708 and 0.8934
respcctively. The Y variables that dominate are Y2 (total number of passengers per
vehicle revenue hour), Y4 (total vehicle hours per total operators), Y5 (total vehicle
kilometers per vehicle at peak hour), and Y6 (total number of data collecred) wirh
canonical loading values of0.7945, 0.8525, -0.5969 and 0.8582 respectively.
5. CONCLUSIONS
In this research a case of evaluating city bus service in Jakarta is made based on
multidimensional approach. Although, it is learnt that such multidimensional approach should
consider both quantitative and qualitative components, only the quantitative one is discussed.
Two dimensions of criteria have been explored and determined as process evaluation
dimension and service performance dimension with several planning, design and operational
variables in each.
The canonical correlation analysis (CCA) is adopted for analysis to determine the complex
correlation of the sets of various variables mentioned above. Through CCA a linear
combination of one dimension of independent variables correlates maximally with a linear
combination of other diniension of dependent can be determined.
Sets of data from several major city bus companies were collected to comply with the above
dimensions. CCA onto the data has resulted in two prominent conelations. The first strong
correlation has a value of 0.993381 that composedwith 5'71o varied data. The Y variables that
dominate nre Y1 (vehicle hour revenue per operating cost) and Y3 (operating revenue per
operating cost) with canonical loading values of 0.9680 and 0.9036 respectively. The X
variables that dominate are X7 (vehicle revenue per kilometer trip) and X8 (vehicle revenue
per operating hour) with canonical loading values of 0.8202 and 0.9667 respectively. The
second strong correlation has a value of 0.989484 that composed with 35.67Vo vaied data.
The X variables that dominate are Xl (route length) and X5 (number of vehicles at peak
hour) rvith canonical loading values of-O.6708 and 0.8934 respectively. The Y variables that
dominate are Y2 (total number of passengers per vehicle revenue hour), Y4 (total vehicle
hours per total operators), Y5 (total vehicle kilometers per vehicle at peak hour), and Y6
(total nunrber of data collected) with canonical loading values of 0.1945,0.8525, -0.5969 and
0.8582 respectively.
Proceedings of the Eastern Asia Socicty for Transportation Studies, Vol.3, No.l. October, 2001
149
A Multidimensional Approach for Evaluating Bus Service
As can be expected two prime correlations have practical implications for both individual and
collective operators. Such findings are necessary for investor candidates that would take roles
in providing service in city bus system. It is expected further that the proposed CCA can be
utilized to perform financial calculation as part of investment policy.
REFERENCES
Proceedings of the Eastern Asia Society for Transportation Studies, Vol.3, No.1, October, 2001