Social Science & Medicine 56 (2003) 803–814

Eliciting preferences of the community for out of hours care

provided by general practitioners: a stated preference discrete
choice experiment
Anthony Scotta,*, M. Stuart Watsonb, Sue Rossc
a
Health Economics Research Unit, University of Aberdeen, Foresterhill, Aberdeen AB25 9ZD, UK
b
Department of Public Health, University of Aberdeen, UK
c
Health Services Research Unit, University of Aberdeen, UK

Abstract

Access to primary care services is a major issue as new models of delivering primary care continue develop in many
countries. Major changes to out of hours care provided by general practitioners (GPs) were made in the UK in 1995.
These were designed in response to low morale and job dissatisfaction of GPs, rather than in response to patients’
preferences. The aim of this study is to elicit the preferences of patients and the community for different models of GP
out of hours care. A questionnaire was sent to parents of children in Aberdeen and Glasgow in Scotland who had
received a home visit or attended a primary care emergency centre, or were registered with a GP. The questionnaire used
a discrete choice experiment that asked parents to imagine their child had respiratory symptoms. Parents were then
asked to choose between a series of pairs of scenarios, with each scenario describing a different model of out of hours
care. Each model varied by waiting time, who was seen, location, and whether the doctor listened. The response rate
was 68% (3893/5718). The most important attribute was whether the doctor seemed to listen, suggesting that policies
aimed at improving doctor–patient communication will lead to the largest improvements in utility. The most preferred
location of care was a hospital accident and emergency department. This suggests that new models of primary care
emergency centres may not reduce the demand for accident and emergency visits from this group of patients in urban
areas. Preferences also differed across sub-groups of patients. Those who had never used out of hours care before had
stronger preferences for waiting time and the doctor listening, suggesting higher expectations of non-users. Further
research is required into the demand for out of hours care as new models of care become established. r 2002 Elsevier
Science Ltd. All rights reserved.

Keywords: Out of hours; General practice; Discrete choice experiments; Patients’ preferences; Scotland; UK

Introduction primary care, increasing public expectations, and the

effects on stress and family life for primary care
There have been many changes to the delivery of physicians, have led to reform of out of hours primary
primary care in many countries, as health care reforms care in several countries. A common theme in these
introduce and experiment with new models of care. The reforms has been a degree of centralisation and
provision of out of hours (or after hours) care by integration of services through system-wide telephone
primary care physicians is a crucial element of primary triage (Vedsted & Olsen, 1999; Hansen & Munck, 1998)
health care in many countries. Concerns about access to and growth in co-operative models of out of hours care,
including the establishment of primary care out of hours
*Corresponding author. Tel.: +44-1224-553866; fax: +44- emergency centres (Department of Health, 2000; Scot-
1224-662994. tish Out of Hours Study Group, 2001). These changes
E-mail address: [email protected] (A. Scott). have reduced the on-call commitment of primary care

0277-9536/03/$ - see front matter r 2002 Elsevier Science Ltd. All rights reserved.
PII: S 0 2 7 7 - 9 5 3 6 ( 0 2 ) 0 0 0 7 9 - 5
804 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

physicians, increased the proportion of telephone con- characteristic means less of another. It is impossible for
sultations, and changed the location and provider of patients to receive a home visit within 5 min of their call,
health care. The effects of these major changes in service from their own GP. It is therefore important to find out,
provision have therefore been supply-led, rather than from the patients’ perspective, which attribute they
demand-led. There is little evidence of the effects on would most like to be improved, given that they cannot
patients. have the best level of every attribute. In short, priorities
In the UK, the provision of out of hours care by need to be set.
general practitioners (GPs) has changed radically since The aim of this study was to elicit the preferences of
1995. As a result of deepening concern felt by GPs about users and non-users (i.e. the community) for different
increasing workload out of hours and the ability to models of out of hours care, and to examine the relative
maintain their 24 hour contractual commitment, the importance of attributes of out of hours care. The study
arrangements for providing care out of hours were uses a discrete choice experiment which has its origins in
altered in 1995. Funding was made available to set up mathematical psychology, market research, and eco-
GP co-operatives, where many practices formally nomics and has been developed as a method of
collaborate to provide out of hours care. This funding examining preferences for attributes or characteristics
paved the way for the set up of new primary care of goods and services (Brunel University, 1993). Its
emergency centres open during evenings, nights and application in health care is growing (Ryan, 1999).
weekends, and staffed by GPs from member practices.
In many areas, this has led to reductions in the numbers
of home visits as more patients are asked to travel to the Method
emergency centre and more advice is given over the
telephone. The structure and organisation of co- A postal questionnaire was developed which pre-
operatives vary, with some providing nurse triage, sented respondents with a number of pairs of scenarios,
patient and doctor transport, and being supported by where each scenario described a particular model of out
a formal management board and administrative struc- of hours care. For each pair, respondents were asked to
ture. Others have a less formal structure, and are more choose which scenario they preferred (Fig. 1). The
similar to rotas. These changes have influenced mainly attributes for the scenarios were chosen from the
urban areas of the UK, with care in rural areas provided existing literature and from face-to-face and postal
along more traditional lines with GPs providing their piloting of the questionnaire (Table 1). Several studies
own cover for their own patients (Hallam and have suggested that who the patient consults is
Henthorne, 1999; Department of Health, 1998). important, as is the time between initial contact and
These arrangements were devised and implemented to actual consultation with a doctor (McKinley et al.,
reduce stress and improve the morale of GPs, with little 1997a; Sawyer & Arber, 1982; Prudhoe, 1984; Bollam,
recourse to the preferences of patients and the commu- McCarthy, & Modell, 1988; Dixon & Williams, 1988;
nity. The changes have had important effects on the Cragg, Campbell, & Roland, 1994).
process of care, including location of care, waiting times, Furthermore, the current policy context suggests that
and who is seen as well as potential effects on health the location of the out of hours consultation is
status. important. The doctor–patient relationship is also a
Several patient satisfaction studies have been con- relevant factor and has been shown by many previous
ducted which have used a variety of instruments, and patient satisfaction studies to be more important than
asked about a variety of different models of care other aspects of care in general practice (Smith &
(McKinley et al., 1997a, b; Salisbury, 1997; Bain, Armstrong, 1989; Calnan et al., 1994). The attribute
Gerrard, Russell, Locke, & Baird, 1997). However, representing the quality of the doctor–patient relation-
satisfaction studies suffer from several known short- ship was ‘whether the doctor seems to listen to what you
comings. Although they ask about patients’ experience have to say’. This was chosen on the basis of the results
of the care they have had, which is relevant, they do not of a previous choice experiment examining the impor-
directly ask about preferences for alternative models of tance of different attributes of the GP consultation (Vick
care. It is difficult to determine the relative importance & Scott, 1998). ‘Whether the doctor listens’ was found to
of attributes from satisfaction studies, since dissatisfac- be the most important to patients. The scenarios
tion (or satisfaction) with an attribute does not therefore described out of hours care in terms of the
necessarily indicate that it is the most important to location of the consultation, time between initial contact
patients (Scott & Smith, 1994; Carr-Hill, 1992). and consultation, who was seen, and whether the doctor
Satisfaction studies also ignore the notions of sacrifice seemed to listen (Table 1).
and opportunity cost: resources for out of hours care are A vignette was used to place the hypothetical
finite and choices need to be made about where they are scenarios in a recognisable and realistic setting. The
likely to have the best effect. To have more of one scenarios were therefore presented in the context of
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 805

• Imagine that during the night, your child is short of breath, wheezing and coughing
and that you decide to call a doctor. You have several options about the care you
receive. These differ according to who your child sees, where they are seen, the
time it takes between making the telephone call and receiving treatment, and
whether the doctor seems to listen to what you have to say.

• For each question below, you are asked to choose which type of consultation you
would prefer for your child during the night (Consultation A or Consultation B).

1. Which consultation would you prefer? (please tick box below)

Consultation A Consultation B
Where your child is seen: Emergency centre run Your home
by GPs
Who your child sees: A GP who doesn’t work A GP who doesn’t
at your practice/health work at your
centre practice/health centre
Time taken between the 60 minutes 20 minutes
telephone call and treatment
being received:

Whether the doctor seems The doctor seems to The doctor seems to
to listen to what you have to listen listen
say:

Prefer consultation A Prefer consultation B

(please tick one box)

Fig. 1. Example of a choice presented to patients.

Table 1 Gilmour, & Sullivan, 1991). Parents of young children

Attributes and levels used in the questionnaire have also been shown to be the most dissatisfied users of
Attribute Levels of each attribute the service in one of our study sites (Glasgow) (Wilson
et al., 2001).
Where your child is seen Emergency centre run by The attributes and levels (Table 1) were organised into
GPs scenarios using a factorial experimental design. From a
Your home possible 48 scenarios, 16 were generated using a
A hospital accident and
fractional factorial design. This provides the minimum
emergency department
Who your child sees A GP from your number needed to maintain an orthogonal design. One
practice/health centre scenario was chosen to be constant across all pairs
A GP who doesn’t work (Scenario A), and the others compared with it (Fig. 1).
at your practice/health The constant scenario was chosen to represent the new
centre model of an emergency centre. The remaining 15
Time taken between the 20 min scenarios were then divided randomly across two
telephone call and 40 min questionnaires, one with eight pairs and one with seven.
treatment being received 60 min Each type of questionnaire was allocated randomly to
80 min respondents.
Whether the doctor The doctor seems to
Once scenarios are paired (whether using a constant
seems to listen to what listen
you have to say The doctor does not scenario or a random pairing), the orthogonality of the
seem to listen original main effects design is not guaranteed. Ortho-
gonality was therefore checked by examining the
associations between attribute differences (i.e. the
children with respiratory illness (Fig. 1). Children difference between the levels of each pairwise choice)
represent a large proportion of consultations out of in the 15 scenarios. Appendix A shows that associations
hours, with respiratory disease contributing significantly between attribute differences were low and not statisti-
to these consultations (Cragg et al., 1994; Morrison, cally significant.
806 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Face validity was assessed during piloting in a records of the emergency centres. Records of GDOCS
respiratory clinic at Aberdeen Children’s Hospital, and users were available for a 12-month period and those for
at the Glasgow and Aberdeen primary care emergency GEMS were available for a 6-month period. On the
centres. Internal consistency was examined by analysing basis of previous experience of using choice experiments
those pairs of scenarios where one scenario was in the community, expected response rates and expected
obviously dominant. The experimental design produced sub-group analysis, we estimated sample sizes of 1800
three scenarios that were dominant, assuming a priori non-users and 715 in each user group from each area
that individuals prefer shorter waiting times, prefer a were required to obtain sufficient responses for analysis
home visit to a GP emergency centre visit, and prefer to (Pearmain, Swanson, Kroes, & Bradley, 1991). Remin-
see a GP they know. One questionnaire had two ders were sent at 3 and 6 weeks if necessary.
dominant scenarios and the other version had one. If a
respondent chose inconsistently (i.e. choose the non- Econometric model
dominant scenario), then all of the respondent’s choices
were removed from the data set. Theoretical validity Each model of out of hours care is represented by an
was assessed using the signs of the coefficients to indirect utility function. With two models of out of
test prior hypotheses about the nature of preferences hours care (i and j), y
n is a latent variable representing
(e.g. shorter waiting time is preferred to longer waiting the difference in utility between them, with n individuals
time). making a choice. Since it is the choice that is observed
To examine the effect of respondent characteristics rather than the difference in utility, y
n is binary.
and past experiences on the relative importance of Therefore:
attributes, additional questions were asked for demo-
graphic details, past experiences of out of hours care and yn ¼ 1 if y
n > 0 and 0 else
wheezing illness, and about a range of access factors and
such as the use of a car at night, and availability of
someone else to look after children. For example, it y
n ¼ ða þ bxi þ qsn þ ein Þ  ða þ bxj þ qsn þ ejn Þ; ð1Þ
might be expected that parents with younger or fewer where a; b and @ are coefficients, x are the attributes of
children are more likely to prefer shorter waiting times each model of care, s are socio-economic characteristics
or the doctor listening. Parents’ own characteristics may reflecting influences on tastes and e is the random
also influence their preferences, such as gender and level component of utility accounting for the analyst’s
of education. It may also be expected that past inability to accurately observe individual’s behaviour
experiences of out of hours care might influence (Manski, 1977). Further, assume that there are
preferences, indirectly testing hypotheses related to taste variations such that the marginal utility of x
stability of preferences and path-dependence. depends on s:
The questionnaire was sent to parents of children in
b ¼ p þ lsn : ð2Þ
Aberdeen and Glasgow, allowing comparison of two
communities with contrasting social structures (affluent This gives
and one of the most deprived in the UK, respectively). y
n ¼ ða þ pxi þ lsn xi þ qsn þ ein Þ
As a guide to relative deprivation using the Carstairs
 ða þ pxj þ lsn xj þ qsn þ ejn Þ: ð3Þ
deprivation index, around half of Greater Glasgow
Health Board’s population were classified in categories Because the discrete choice experiment presents each
six and seven (where seven is the highest category of respondent with several pairs of scenarios, multiple
deprivation). This is compared with none of the observations from each respondent mean that errors are
population in Grampian (including Aberdeen) being not independent and so an error term mn capturing
classified in these categories (McLaren & Bain, 2000). random variation across respondents is included:
Each area had a new co-operative, Grampian doctors on
y
n ¼ ða þ pxi þ lsn xi þ qsn þ ein þ mn Þ
call service (GDOCS) based in Aberdeen, and Glasgow
emergency medical service (GEMS) in Glasgow.  ða þ pxj þ lsn xj þ qsn þ ejn þ mn Þ: ð4Þ
From each area (and with ethical approval), three Taking differences for each pairwise choice (k), the
random samples of children aged under 13 years old equation to be estimated becomes
were identified: those who had visited a primary care
y
kn ¼ pxk þ lsn xk þ ekn : ð5Þ
emergency centre; those who had received a home visit;
and those from the lists of GPs (i.e. who may or may not Terms common to both indirect utility functions drop
have used out of hours care before). Duplicates from the out of the model (i.e. a; qsn ; and mn ).
samples of users and temporary residents were excluded However, the inclusion of a constant term (a) and
from this latter sample. Information about those who error term across respondents (mn ) can be used to test for
had a centre visit or home visit was gathered from the mis-specification due to unobservable attributes and
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 807

unobservable interaction terms between GPs’ socio- of observations where data on the dependent and
economic characteristics and attributes. The constant independent variables were missing (2391; 8%), and
term can be interpreted as the difference in the average inconsistent responses (2023; 7.5%). The main effects
utility of scenario i and j; caused by the use of a constant model is shown in Table 4. r; measuring the correlation
scenario, left/right bias or an omitted dummy variable between observations of the same respondent, was
that is a function of other included attributes (Scott, statistically significant, suggesting that a random effects
2000). The model to be estimated then becomes specification was appropriate and that there may be
unobservable interaction terms between attributes and
y
kn ¼ a þ pxk þ lsn xk þ ekn þ mn : ð6Þ
characteristics of respondents.
This model was estimated using random effects probit The constant term is statistically significant and
regression. A model with main effects was estimated negative, suggesting a general preference for scenario
first, and used to calculate marginal rates of substitution A. Scenario A represented the new co-operative model
between each attribute and the time attribute. A full with a primary care emergency centre, and Scenario B
model was estimated including main effects and inter- represented other models of care. Although this suggests
action terms for which hypotheses existed. This was then there was a general preference for new models of care,
reduced to a more parsimonious model by excluding other explanations for the significant constant term
variables, one at a time, with p-values >0.10. should not be ruled out (i.e. ‘right’ bias and omitted
attributes).
Face and content validity were established through
Results piloting, there was a low level of inconsistent responses
(8%), and the results were consistent with prior
Excluding those questionnaires that were not deliv- hypotheses (e.g. shorter waiting times are preferred to
ered (586), the final response rate was 3893/5718 longer), confirming the techniques’ theoretical validity.
(68.1%). Two questionnaires were completed by chil- Using data from this study that has been reported
dren and were excluded from analysis. Time to complete elsewhere, the instrument used was also found to be
the questionnaire ranged from 2 to 60 minutes (mean reliable (San Miguel et al., forthcoming).
12 minutes), and 1760/3757 (46.8%) of respondents For the main attributes, the signs of coefficients
considered that the questionnaire was easy to answer. suggest that respondents preferred to see a GP from
Characteristics of the respondents and their last out of their own practice or health centre, preferred shorter to
hours consultation are presented in Tables 2 and 3. longer waiting times, and preferred the doctor to listen.
For the regression analysis, the final sample size was These are as expected and confirm the theoretical
3326 individuals (24,789 observations), after the deletion validity of the technique. In terms of the location of

Table 2
Descriptive characteristics of respondents

Characteristic Response

From Aberdeen 44% (1701/3890)

Age of parent/guardian (mean years, range) 34 (16–75)
Gender of parent/guardian (% female) 87.4% (3394/3883)
Age of children (mean years, range) 6.9 (0–25)
Number of children (mean, range) 2.2 (0–12)
Highest level of education of parent/guardian University 17% (659/3858)
College 32% (1215/3858)
Secondary 50% (1937/3858)
None 1.1% (43/3898)
Health status of child Excellent 47% (1812/3878)
Good 41% (1586/3878)
Fair 10% (374/3878)
Poor 2% (106/3878)
Child has ever suffered from asthma or other ‘wheezy’ illness Yes 38% (1463/3881)
Child has other medical problems for which they see a doctor regularly Yes 16% (630/3869)
Access to a car during the night Yes 73% (2805/3830)
Availability of others to help look after children during the night Yes 74% (2756/3715)
Used out of hours care before Yes 42% (1634/3890)
808 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Table 3 marginal rates of substitution (Table 4). Waiting time

Characteristics of last out of hours visit for those who had used becomes more important than ‘whether the doctor
an out of hours service listens’ if the difference in waiting time was longer than
Characteristics Response 66 minutes. This indicates that respondents would be
of last out of willing to wait up to 66 minutes to see a doctor who
hours visit: seems to listen. Waiting time became more important
than ‘who was seen’, if the difference in waiting time was
When did your Less than 1 3% (106/3879)
longer than 14 minutes, suggesting that respondents
child last see a month ago
doctor during 1–6 months ago 15% (591/3879)
would be willing to wait 14 minutes to see a doctor from
the night? 6–12 months ago 17% (660/3879) their own practice. Similarly, waiting time was more
More than 1 24% (938/3879) important than being seen in an A&E department if the
year ago difference waiting time was longer than 33 minutes. This
Never 33% (1266/3879) also represents the extra time respondents are willing to
Can’t remember 8% (318/3879) wait to be seen in an A&E department rather than a
Doctor seemed 84% (2197/2623) primary care emergency centre. Waiting time became
to listen more important than being seen at home rather than in a
Doctor did not 10% (258/2623) primary care emergency centre, if the difference in
seem to listen
waiting time was longer than 11 minutes.
Can’t remember 6% (168/2623)
Place of visit A&E 13% (353/2631) The reduced model is shown in Table 5. The main
Emergency 33% (865/2631) effects are of a similar sign, except for the constant term
centre and whether there is a home visit or a primary care
Home visit 48% (1263/2631) emergency centre visit. The constant term is now
Can’t remember 6% (150/2631) positive, suggesting a general preference for Scenario
Time between Around 20 min 29% (742/2592) B. This will be a function of the addition of interaction
phoning and Around 40 min 33% (845/2592) terms that involve dummy variables, and show that once
seeing a doctor Around 60 min 15% (394/2592) respondents’ characteristics are included in the model,
Around 80 min 10% (256/2592) the general preference is for more traditional models of
>80 min 0.03% (14/2592)
care, rather than the new co-operative model with a
Can’t remember 13% (341/2592)
Who was seen GP from 25% (641/2616) primary care emergency centre. However, other expla-
practice nations for the significant constant term should not be
GP not from 68% (1781/2616) ruled out (i.e. ‘right’ bias and omitted attributes). The
practice sign of the variable examining preferences for either a
Can’t remember 7% (194/2616) home visits or a visit to a primary care emergency centre
has also changed, suggesting that once respondents’
characteristics are included, a primary care emergency
care, respondents preferred a home visit compared to a centre visit is preferred to a home visit.
primary care emergency centre run by GPs, and a The coefficients of the interaction terms show how
hospital A&E department to a primary care emergency preferences differ depending on respondents’ character-
centre. istics and past experiences. For location of care, home
The relative importance of each attribute can be visits and A&E visits were more likely to be preferred to
examined by using the main regression coefficients, primary care emergency centre visits for those who had a
which are measured on the same scale, as well as college and university education, compared to those who
marginal rates of substitution. The most important had a secondary school education. Those with children
attribute was ‘whether the doctor seemed to listen’, in fair/poor health compared to excellent health also
followed by being seen at an A&E department compared preferred to receive a home visit.
to a primary care emergency centre, seeing a GP from There was also evidence that the location of their last
your own practice, and being seen at a home visit out of hours visit influenced preferences. A home visit
compared to a primary care emergency centre. Waiting was preferred to a primary care emergency centre visit
time was also a significant predictor, with the impor- by those who had previously had an A&E visit and a
tance of waiting time compared to other attributes home visit compared to those who never visited before.
depending on how long they had to wait. The coefficient Emergency centre visits were more likely to be preferred
gives the relative importance for a difference of 1 minute to home visits for those who previously had an
in waiting time between each model in the scenarios. emergency centre visit, and those who received a visit
It is possible to examine the approximate ‘thresholds’ more than a year ago. An A&E department was more
at which waiting time becomes more important than the likely to be preferred to a primary care emergency centre
other attributes, by examining the implied trade-offs or visit for those who previously received a home visit.
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 809

Table 4
Regression results (main effects only)

Variable b (SE) MRSa (SE) 95% CI

Constant 0.2015 (0.0399)* — —

Who is seen 0.4686 (0.0254)* 14 (0.83) 15.73 to 12.48
Waiting time 0.0332 (0.0006)* — —
Whether the doctor listens 2.1925 (0.0275)* 66 (1.11) 68.17 to 63.82
Home visitb 0.3638 (0.0422)* 11 (1.34) 13.58 to 8.32
Hospital A&E department visitb 1.0892 (0.0363)* 33 (1.25) 35.23 to 30.35
r 0.3227 (0.0121)* — —

2 log likelihood 9832 % 0’s predicted correctly 96%

Model chi-squared (d.f.) 6959 (5)* % 1’s predicted correctly 66%
Psuedo R2 29% Number of observations 24789
Number of individuals 3326

Notes: *=po0:0001:
a
MRS ¼ bx =bwaiting time : Standard errors calculated from a Taylor series approximation to the variance of a function of random
variables, where varðMRSÞ ¼ 1=b2waiting time ½varðbx ÞF2MRS covðbx ; bwaiting time Þ þ MRS2 varðbwaiting time Þ (Propper, 1988).
b
Relative to a primary care emergency centre.

Preferences for seeing a GP from their own practice sampled from centre visits, and who had a college or
(or not) also differed across sub-groups. Those sampled university education compared to those who had a
from Glasgow were more likely to prefer to see a GP secondary education. Respondents’ childrens’ character-
from their own practice, compared to those from istics also had an effect, with the doctor listening being
Aberdeen, as were those sampled from the general preferred by those whose children had no other medical
population compared to those who were sampled from problems, and who were in excellent rather than poor or
home visits. Seeing a GP from their own practice had a fair health. Past experiences also had an impact. The
higher marginal utility for respondents with older doctor listening was preferred by those who had never
children, with a secondary rather than university received an out of hours visit compared to those who
education, and for those who saw a GP from their visited up to 1 year ago, and by respondents whose
own practice at their last visit. doctor had seemed to listen at their last visit.
A shorter waiting time was preferred by respondents The version of the questionnaire respondents were
whose child had not previously suffered from asthma, by sent also influenced preferences. Although the scenarios
those with fewer and younger children, and by those were allocated to each version of the questionnaire
with children in excellent compared to good, fair or poor randomly, this does not guarantee that there will be no
health. Preferences for waiting time were also influenced systematic differences in preferences. The inclusion of
by parent’s characteristics, independent of their chil- these variables in the model controls for these biases.
dren’s characteristics. A shorter waiting time was
preferred by those sampled from Glasgow compared
to Aberdeen, by those with a college and university Discussion
education compared to secondary education. Past
experiences also influenced the marginal utility of The most important attribute was whether the doctor
waiting times, as shorter waiting times were preferred seemed to listen. This was independent of whether the
by those who had never had an out of hours visit patient knew the doctor, perhaps reflecting the real
compared to those who visited up to 1 year ago, and by reason why other studies have reported low satisfaction
those who waited 20 minutes at their previous visit. with deputising services. This finding is consistent with
Those who waited more than 1 hour for their last visit other studies examining patient satisfaction with general
were more likely to prefer longer waiting times, practice and the doctor–patient relationship (Williams &
compared to those who had never had a visit. Calnan, 1991; Savage & Armstrong, 1990; Wissow,
Preferences for the doctor–patient relationship were Roter, & Wilson, 1994), with out of hours services
also influenced by respondents’ characteristics. Those (Scottish Out of Hours Study Group, 2001), as well as
who were more likely to prefer the doctor who seem to previous discrete choice experiments examining the
listen included respondents who were younger, who were doctor–patient relationship (Vick & Scott, 1998) and
sampled from the general population compared to those out of hours care (Morgan, Shackley, Pickin, & Brazier,
810 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Table 5
Regression results (model with interaction terms)

Variable B SE

Constant 0.3586 0.0593***

Who is seen 0.3975 0.0624***
Waiting time 0.0618 0.0024***
Whether the doctor listens 3.3710 0.1189***
Home visita 0.7080 0.1011***
Hospital A&E department visita 1.7586 0.0809***
Centre visit sample * home visitc 0.2574 0.0531***
Home visit sample * home visitc 0.2061 0.0606**
College education * home visitd 0.1926 0.0676**
University education * home visitd 0.1566 0.0786**
Fair/poor health * home visite 0.2012 0.0729**
Last visit more than 1 year ago * home visitf 0.1538 0.0627**
Last visit at A&E * home visitf 0.2747 0.0897**
Last visit at home * home visitf 0.1555 0.0609**
College education * A&E visitd 0.1238 0.0643*
Last visit at home * A&E visitf 0.1732 0.0476**
Grampian Health Board * who is seenb 0.0987 0.0411**
Age of children * who is seen 0.0139 0.0054**
Centre visit sample * who is seenc 0.0988 0.0594*
University education * who is seend 0.1640 0.0710**
Saw a GP from own practice at last visit * who is seenf 0.1377 0.0534**
Grampian Health Board * waiting timeb 0.0027 0.0008**
Whether child has asthma * waiting time 0.0028 0.0009**
Number of children * waiting time 0.0013 0.0004**
Age of children * waiting time 0.0003 0.0001**
College education * waiting timed 0.0029 0.0013**
University education * waiting timed 0.0049 0.0016**
Good health * waiting timee 0.0024 0.0008**
Fair/poor health * waiting timee 0.0056 0.0017**
Last visit up to 6 months ago * waiting timef 0.0029 0.0013**
Last visit between 6 months and 1 year * waiting timef 0.0048 0.0012**
Waited 20 min at last visit * waiting timef 0.0023 0.0011**
Waited 60 min at last visit * waiting timef 0.0036 0.0013**
Waited over 80 min at last visit * waiting timef 0.0052 0.0016**
Other health problems of child * doctor listens 0.1170 0.0374**
Centre visit sample * doctor listensc 0.0861 0.0414**
Age * whether the doctor listens 0.0167 0.0024***
College education * doctor listensd 0.3137 0.0497***
University education * doctor listensd 0.1551 0.0539**
Fair/poor health * doctor listense 0.1727 0.0462**
Last visit up to 6 months ago * doctor listensf 0.2258 0.0461***
Last visit between 6 months and 1 year ago * doctor listensf 0.1429 0.0433**
Doctor listened at last visit * doctor listens 0.1058 0.0331**
Questionnaire * home visit 0.7898 0.1245***
Questionnaire * A&E visit 1.4554 0.0647***
Questionnaire * waiting time 0.0128 0.0014***
r 0.3097 0.0126***

2 log likelihood 9199 % 0’s predicted correctly 90%

Model chi-squared (d.f) 5916 (45)*** % 1’s predicted correctly 72%
Psuedo-R2 39% Number of observations 24789
Number of individuals 3326

Notes: ***=po0:0001; **=0.0001Xpo0:05; *=0.10Xpp0.05.

a
Relative to a primary care emergency centre.
b
Relative to Greater Glasgow Health Board.
c
Relative to general population sample.
d
Relative to secondary education.
e
Relative to excellent health.
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 811

2000). The finding suggests that if GP out of hours preferences may be modified by past experiences and
organisations want to increase patients’ utility the most, may also exhibit a degree of path-dependence. This
then they should focus on improving doctors’ commu- highlights the importance of gathering information
nication skills. about these past experiences in preference elicitation
Preferences differed by respondents’ characteristics. studies.
Socio-economic differences are reflected by respondents’ The most preferred model of GP out of hours care for
education. Highly educated respondents had a stronger parents of children who have respiratory symptoms in
preference for A&E department visits to primary care urban areas, is a hospital accident and emergency
emergency centre visits and home visits to primary care department where the doctor seems to listen, where
emergency centre visits. They were also less concerned waiting time between the initial phone call and treatment
about seeing a GP from their own practice, more being received is low, and where they see a doctor from
concerned about a lower waiting time, and had a their own practice. This describes a model of care that
stronger preference for the doctor listening. This may does not exist in most places of the UK, although there
have implications for the provision of services and the are examples of GPs working in A&E departments, and
pattern of demand in affluent compared to deprived of primary care emergency centres located adjacent to
areas. A previous satisfaction study of out of services in A&E departments. For example, GEMS in Glasgow has
Glasgow showed that those from affluent areas were six primary care emergency centres, several of which are
more likely to be dissatisfied (Wilson et al., 2001). located alongside A&E departments.
The characteristics of parents’ children also influenced These preferences were elicited without any informa-
preferences. Respondents with children in fair or poor tion on the costs of providing this model of care and so
health were more likely to prefer a home visit than a it may not be possible, for example, for patients to see a
primary care emergency centre visit. Those with children GP from their own practice in an A&E department. The
in excellent health, and who had not previously had results of this study would need to be combined with
asthma had a stronger preference for a shorter waiting costs of different combinations of attributes to establish
time. This may be related to limited experience of health the most cost-effective model of care.
care services or illness where expectations with respect to Nevertheless, the results have important implications
waiting times may be high. Parents of children with for the demand for new models of GP out of hours care
other health problems were less likely to prefer the which are based in primary care emergency centres and
doctor to listen. Parents with younger and fewer where the patient may not see a GP from their own
children were more likely to prefer a shorter waiting practice. For parents of children with respiratory
time, whilst those with older children had a stronger symptoms, these new models may not be the most
preference to see a GP from their own practice. preferred. The consequence of this may be that new
Previous utilisation of GP out of hours services models of GP out of hours care may not reduce the
influenced preferences. Those who had never visited an demand from parents of young children with respiratory
out of hours service before were more likely to prefer a problems, who may continue to use A&E services. New
visit at a primary care emergency centre compared to a primary care emergency centres may not therefore
home visit, and more likely to prefer a shorter waiting reduce the demand for A&E services. It also suggests
time and for the doctor to listen. Those sampled from that primary care emergency centres that are located
the general population had a stronger preference for the within or adjacent to A&E departments may be more
doctor listening and seeing a GP from their own effective at reducing demand at A&E departments, than
practice. This has implications for the education of centres located away from acute hospitals.
potential users about appropriate use of out of hours These results should, however, be interpreted with
services, and also shows that the potential users may several issues in mind. First, the results presented here
have higher expectations of out of hours care compared apply to the parents of children under 13, when their
to those who have used the service before. child has respiratory symptoms. Although this is a
What happened at the respondent’s last out of hours relatively common condition seen by GPs out of hours,
visit also influenced preferences, such that respondent’s preferences may differ for other groups of individuals,
preferred what they had previously experienced. This such as the elderly, and for other types of symptoms of
confirms previous studies, and has implications for when differing severities. In addition, the study applied to
preferences should be elicited during a health care urban areas only and extrapolation to rural areas would
episode (Salkeld et al., 2000). Along with the influence of be difficult.
past utilisation, it suggests that to a certain extent, Second, some individuals responding may not be
preferences are endogenous. It is not necessarily the case using compensatory decision rules. This can be proble-
that individuals hold a set of stable and complete matic when a lexicographic ordering exists since utility
preferences as implied by standard neo-classical micro- functions cannot be defined (Scott, 1998). These
economic theory. Once a dynamic element is introduced, individuals have very strong preferences for a given
812 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

attribute since they are not prepared to trade off with Finally, there are several issues about the experi-
other attributes. Generally, individuals may have used a mental design. When placing scenarios into pairs, the
variety of non-compensatory decision making rules property of orthogonality that was in the original linear
and heuristics that lie between the extremes of fully design is no longer guaranteed. What is required is a
compensatory decision making and lexicographic design that minimises colinearity between attribute
orderings (Payne, Bettman, & Johnson, 1992). The differences. However, it is not necessarily the case that
implications of non-compensatory decision making correlations have to equal zero as there are other aspects
are that policy recommendations based on the marginal to the design that need to be considered, such as balance
rate of substitution should be treated with caution. (that the levels of attributes appear with equal
Our results found that individuals were prepared to wait frequency) and minimal overlap of attribute levels. It
an additional 66 minutes to see a doctor who seems has been recognised that, ‘‘y for most combinations of
to listen. This implies that a policy that reduces waiting attributes, levels, alternatives and parameter vectors, it is
times by 66 minutes would generate the same improve- impossible to create a design that satisfies (all of) these
ment in utility as a policy that improved doctors’ principles’’ (Zwerina, Huber, & Kuhfeld, 1996).
communication skills. However, individuals with a A common assumption in choice experiments is that
lexicogrpahic ordering would not be willing to interactions between the main effects are negligible.
wait at all. In this case, any policies to reduce Because there were no specific hypotheses that the main
waiting times would not influence their utility, or effects in this study were interdependent, we used a
demand. For those sub-groups of the sample with a fractional factorial design and so assumed interaction
lexicographic ordering, the implication is that reducing effects were negligible. If there are interaction effects,
waiting times would be of no value to them. A further then reliable estimation of these is only possible if they
implication is that inclusion of those with lexicographic are built into the experimental design (or if a full factorial
orderings will overestimate the size of regression design is used). However, this does not rule out the
coefficients for those attributes where individuals have possibility that interaction effects exist. This may reduce
dominant preferences. the explanatory power of the model and could lead to
It is important to find ways of identifying these sub- incorrect estimates of regression coefficients for the main
groups of individuals, although this may only be effects, and must be borne in mind when interpreting
possible through face-to-face interviews that explore results. Although there have been few studies in health
individuals’ preferences in more depth. However, there that have examined interactions, previous empirical
may be different reasons for the existence of these studies in transport suggest that interactions between
heuristics, that include the complexity of the question- main effects are negligible (Louviere, 1988).
naire (bounded rationality) or because they genuinely Further research is required into the demand for out
have a dominant preference. Alternative ways of of hours services, especially as alternative models begin
examining preferences should also be explored, such as to emerge. It is not sufficient to reconfigure services to
those reviewed by Ryan et al. (2001). suit the preferences of providers, as this takes no
There are also other issues when interpreting the account of patterns of demand. Preferences of both
attributes. The study has not investigated why respon- users and non-users need to be accounted for.
dents preferred the A&E department. This may because
of a higher perceived ‘quality’ of service and faster
access to other hospital facilities. This implies other Acknowledgements
attributes of potential importance that require further
research. Furthermore, different wording of the attri- Thanks go to Nicola Torrance for collecting data, and
butes may have influenced responses, especially for the to the staff of G-DOCs and GEMS. Thanks also go to
doctor listening. Further research should be conducted anonymous referees and Cristina Ubach for helpful
on framing effects in choice experiments. comments. This project was funded by the NHS R&D
A further issue is the treatment of inconsistent Primary–Secondary Care Interface Programme. The
responses. Eight percent of responses were inconsistent, Health Economics and Health Services Research Units
suggesting those individuals did not understand the task are funded by the Chief Scientist Office of the Scottish
being set. This compares favourably with other methods Executive Health Department (SEHD). The views in this
of eliciting preferences in health care (with levels of paper are those of the authors and not SEHD.
inconsistency of between 7% and 77%) (Vick & Scott,
1998). Although the inclusion of dominant scenarios to
check for inconsistency may reduce the number of Appendix A. Testing whether the design is orthogonal
choices that involve trading-off attributes, this is not a
major concern and has not manifested itself in a poor The differences between the levels of the paired
quality econometric model. attributes were tested for orthogonality. The sample
 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814 813

Table 6
Measures of association between attribute differences

D WHERE D WHO D LISTEN D TIME

a b c b
D WHERE — 0.07 /0.38 0.006 /0.094 0.22d
D WHO 0.07a/0.38b — 0a/0.07b 0.03d
D LISTEN 0.006c/0.094b 0a/0.07b — 0.03d
D TIME 0.22d 0.03d 0.03d —
a
l: A measure of association which reflects the proportional reduction in error when values of the independent variable are used to
predict values of the dependent variable. A value of 1 means that the independent variable perfectly predicts the dependent variable. A
value of 0 means that the independent variable is no help in predicting the dependent variable.
b
Measures of association based on chi-square including: contingency coefficient (measure of association based on chi-square—the
value ranges between zero and 1, with zero indicating no association between the row and column variables and values close to 1
indicating a high degree of association between the variables. The maximum value possible depends on the number of rows and
columns in a table.); f (a chi-square based measure of association that involves dividing the chi-square statistic by the sample size and
taking the square root of the result); cramer’s V (a measure of association based on chi-square).
c
Goodman and Kruskal t: Similar to l; except the marginal proportions are used for predictions, rather than modal category.
d
Z is a measure of association that ranges from 0 to 1, with 0 indicating no association between the row and column variables and
values close to 1 indicating a high degree of association.

size for these tests was 15 (the number of pairwise review of GO out of hours services in England. Department
choices in the questionnaires), and so the power to of Health, London.
detect statistically significant correlations was low, and Dixon, R. A., & Williams, B. T. (1988). Patient satisfaction with
so none of the values in the table below are statistically general practitioner deputising services. British Medical
significant. However, all are close to zero suggesting that Journal, 297, 1519–1522.
Hansen, B. L., & Munck, A. (1998). Out of hours services in
the attribute differences exhibit low associations. Since
Denmark. The effect of a structural change. British Journal
there are a mixture of nominal and interval data, the of General Practice, 48, 1497–1499.
measures of association used varied and so are not Hallam, L., & Henthorne, K. (1999). Cooperatives and their
directly comparable with each other (see Table 6). primary care emergency centres: organisation and impact.
Combined report on seven case studies. Health Technology
Assessment 3.
References Louviere, J. (1988). Analysing decision making: Metric conjoint
analysis. California: Sage Publications Inc.
Bain, J., Gerrard, L., Russell, A., Locke, R., & Baird, V. (1997). Manski, C. F. (1977). The structure of random utility models.
The Dundee out of hours co-operative: Preliminary out- Theory and Decision, 8, 229–254.
comes for the first year of operation. British Journal of McKinley, R. K., Cragg, D. K., Hastings, A. M., French, D. P.,
General Practice, 47, 573–574. Mankau-Scott, T. K., Campbell, S. M., Van, F., Roland,
Bollam, M. J., McCarthy, M., & Modell, M. (1988). Patients’ M. O., & Roberts, C. (1997a). Comparison of out of hours
assessment of out of hours care in general practice. British care provided by patients’ own general practitioners and
Medical Journal, 296, 829–832. commercial deputising services: A randomised controlled
Brunel University. (1993). The valuation of changes in quality in trial. II: The outcome of care. British medical Journal, 314,
the Public Services. Report prepared for HM Treasury, 190–193.
HMSO, London. McKinley, R. K., Mankau-Scott, T. K., Cragg, D. K.,
Calnan, M., Katsoyiannopolous, V., Ouchavev, V. K., Pro- Hastings, A. M., French, D. P., & Baker, R. (1997b).
khorskas, R., Ramic, H., & Williams, S. (1994a). Major Reliability and validity of a new measure of patient
determinants of consumer satisfaction with primary care in satisfaction with out of hours primary medical care in the
different countries. Family Practice, 11, 468–478. United Kingdom: Development of a patient questionnaire.
Carr-Hill, R. (1992). The measurement of patient satisfaction. British Medical Journal, 314, 193–198.
Journal of Public Health Medicine, 14, 236. McLaren, G., & Bain, M. (2000). Deprivation and health in
Cragg, D., Campbell, S. M., & Roland, M. O. (1994). Out of Scotland. Information and Statistics Division, Common
hours primary care centres: Characteristics of those attend- Services Agency, Edinburgh.
ing and declining to attend. British Medical Journal, 309, Morgan, A., Shackley, P., Pickin, M., & Brazier, J. (2000).
1627–1629. Quantifying patient preferences for out of hours primary
Department of Health. (1998). GP out of hours services. care. Journal of Health Services Research and Policy, 5,
Working Group Report, Department of Health, Scottish 214–218.
Office, Edinburgh. Morrison, J. M., Gilmour, H., & Sullivan, F. (1991). Children
Department of Health. (2000). Raising standards for patients. seen frequently out of hours in one general practice. British
New partnerships in out of hours care. An independent Medical Journal, 303, 1111–1114.
814 A. Scott et al. / Social Science & Medicine 56 (2003) 803–814

Payne, J. W., Bettman, J. R., & Johnson, E. J. (1992). ences for health and health care. Discussion paper 01/98,
Behavioural decision research: A constructive processing Health Economics Research Unit, University of Aberdeen,
perspective. Annual Review of Psychology, 43, 87–131. 1998.
Pearmain, D., Swanson, J., Kroes, E., & Bradley, M. (1991). Scott, A. (2000). Agency and incentives in general practice. Ph.D.
Stated preference techniques: A guide to practice. Steer Thesis, University of Aberdeen.
Davies Gleave, Hague Consulting Group. Scott, A., & Smith, R. (1994). Keeping the customer satisfied:
Propper, C. (1988). Estimation of the value of time spent on NHS Issues in the interpretation and use of patient satisfaction
waiting lists using stated preference methodology. Discussion surveys. International Journal for Quality in Health Care, 6,
Paper 49, Centre for Health Economics, University of York. 353–359.
Prudhoe, R. H. (1984). Deputising services. British Medical Scottish Out of Hours Study Group. (2001). A comparison of
Journal, 288, 718. models of delivery of out of hours general medical services in
Ryan, M. (1999). Using conjoint analysis in health economics. Scotland. Final Report to Scottish Executive, Department
Paper presented to the International Health Economics of General Practice, University of Edinburgh.
Association, Rotterdam. Smith, C. H., & Armstrong, D. (1989). Comparison of criteria
Ryan, M., Scott, D. A., Bate, A., van Teijlingen, E., Russell, E. derived by government and patients for evaluating
M., Napper, M., Reeves, C., & Robb, C. (2001). Eliciting general practitioner services. British Medical Journal, 299,
public preferences for health care: a systematic review and 494–496.
evaluation of methods. Health Technology Assessment, 5(5). Vedsted, P., & Olesen, F. (1999). Effect of re-organised after-
Salisbury, C. (1997). Postal survey of patients’ satisfaction with hours family practice service on frequent attenders. Family
a general practice out of hours co-operative. British Medical Medicine, 31, 270–275.
Journal, 314, 1594–1598. Vick, S., & Scott, A. (1998). Agency in health care. Examining
Salkeld, G., Ryan, M., & Short, L. (2000). The veil of patients’ preferences for attributes of the doctor–patient
experience: do consumers prefer what they know best? relationship. Journal of Health Economics, 17, 511–644.
Health Economics, 9(3), 267–270. Williams, S. J., & Calnan, M. (1991). Key determinants of
San Miguel, F., Ryan, M., & Scott, A. (forthcoming). Testing consumer satisfaction with general practice. Family Prac-
the assumptions of completeness and stability of preferences tice, 8, 237–242.
using discrete choice experiments. Journal of Economic Wissow, L. S., Roter, D. L., & Wilson, M. E. H. (1994).
Behaviour and Organisation, forthcoming. Pediatrician interview style and mothers’ disclosure of
Savage, R., & Armstrong, D. (1990). Effect of a general psychosocial issues. Pediatrics, 93, 289–295.
practitioner’s consulting style on patient’s satisfaction: A Wilson, P., McConnacchie, A., O’Donnell, C. A., Ross, S.,
controlled study. British Medical Journal, 301, 968–970. Moffat, K. J., & Drummond, N. (2001). Assessing
Sawyer, L., & Arber, S. (1982). Changes in home visiting and dissatisfaction with an out of hours service: Reasons and
night and weekend cover: The patient’s view. British remedies. Health Bulletin, 59, 37–44.
Medical Journal, 284, 1531–1534. Zwerina, K., Huber, J., & Kuhfeld, W. (1996). A general method
Scott, A. (1998). Giving things up to have more of others. The for constructing efficient choice designs. Working Paper,
implications of limited substitutability in eliciting prefer- Fuqua School of Business, Duke University.