Scandinavian Journal of Primary Health Care, 2008; 26: 174
180

ORIGINAL ARTICLE

Data quality and confirmatory factor analysis of the Danish EUROPEP

questionnaire on patient evaluation of general practice

PETER VEDSTED, INETA SOKOLOWSKI & HANNE N. HEJE

Research Unit for General Practice, Institute of Public Health, University of Aarhus, Denmark

Abstract
Objective. The Danish version of the 23-item EUROPEP questionnaire measuring patient evaluation of general practice has
not been evaluated with regard to psychometric properties. This study aimed to assess data quality and internal consistency
and to validate the proposed factorial structure. Setting. General practice in Denmark. Subjects. A total of 703 general
practitioners (GPs). Some 83 480 questionnaires were distributed to consecutive patients aged 18 or more attending
practice during the daytime. A total of 56 594 eligible patients responded (67.8%). Main outcome measures. Data quality
(mean, median, item response, missing, floor and ceiling effects), internal consistency (Cronbach’s alpha and average inter-
item correlation), item-rest correlations. Model fit from confirmatory factor analysis (CFA). Results. The distribution was
skewed to the left for almost all items with a small floor effect (0.1
9.3%) and a ceiling effect larger than 15% (18.6
56.3%).
Item response was high. For seven items ‘‘not applicable/relevant’’ represented more than 10% of the answers. Internal
consistency was good. Item-rest correlations were below 0.60 for three items, and four items had lower correlations with
their own domain than with other domains. CFA showed that four domains were highly correlated and that model fit was
good for two indices (TLI and SRMR), acceptable for one index (CFI), and poor for three indices (chi-squared, RMSEA
and WRMR). Conclusions. This study revealed high ceiling effects, a few items with low item-rest correlation and low item
discriminant validity, and an uncertain model fit. There seems to be a need for developing response categories to bring down
the ceiling effect and it is also unclear how to use the proposed domains. Future research should focus on evaluating the
factorial structure when ceiling effect has been lowered, on whether items should be deleted, and on assessing the
unidimensionality of each domain.

Key Words: Denmark, factor analysis, family practice, patient evaluation, questionnaires

Patient evaluation of healthcare services contributes content and construct validity and some on criter-
to the basis of quality development in the healthcare ion validity [5
10]. The questionnaire was devel-
sector [1,2], which, in turn, presupposes the avail- oped so that each item provided information and
ability of scientifically sound instruments facilitating not as sum-scales. However, the items were cate-
valid and relevant data collection [3,4]. gorized into five qualitatively developed domains
In 1993, the EU BIOMED study EUROPEP was (doctor
patient relationship, medical care, informa-
launched to develop an international patient eva- tion and support, organization of services, and
luation questionnaire for general practice [5,6]. As accessibility).
in other European countries, a national project Thus, the question remains whether these do-
based on this questionnaire, entitled ‘‘Danish Pa- mains can be used to categorize the items and as
tients Evaluate General Practice’’ (DanPEP), was sum-scales. Further, data quality and internal con-
launched in 2002. This Danish version of the sistency must also be assessed [11
13].
questionnaire was introduced as a tool for assessing The aim of this study was to assess data quality
patients’ evaluation of general practice and conse- and internal consistency and to validate the factorial
quently psychometric issues arose. So far, the structure with five domains of the EUROPEP
development had focused on aspects of especially questionnaire.

Correspondence: Peter Vedsted, Research Unit for General Practice, University of Aarhus, Vennelyst Boulevard 6, DK-8000 Aarhus C, Denmark. E-mail:
[email protected]

(Received 17 April 2006; accepted 18 June 2008)

ISSN 0281-3432 print/ISSN 1502-7724 online # 2008 Taylor & Francis
DOI: 10.1080/02813430802294803
 Data quality and factor analysis of the Danish EUROPEP questionnaire 175

The EUROPEP evaluation questionnaire was consultation [9]. Patients completed the question-
developed using a comprehensive approach naire at home and returned it to the DanPEP
comprising literature studies, patient inter- secretariat. A total of 56 594 eligible patients
views, and priority and validation studies. responded (67.8%). Questionnaires were produced
However, we lack knowledge about its psycho- and optically scanned using Cardiff TELEform.
metric performance.

. Data from more than 50 000 patient evalua- Analyses

tions indicated good quality data, good The analyses consisted of two parts: First, we
internal consistency, and low floor effect. assessed the data quality, internal consistency and
. The ceiling effect was consistently very high. correlations between items and domains and be-
. Two of six goodness-of-fit indices showed tween domains. Second, we explored the five-do-
poor, two showed good, and one index main structure using confirmatory factor analysis.
showed acceptable fit of the questionnaire Analyses were performed with Stata 9 and factor
domains. analysis with Mplus 4.1 [15]. The analyses included
. There is a need to develop the response questionnaires in which at least 50% (12) of the
categories and the factorial structure of the items had been answered.
questionnaire. Data quality was assessed in terms of mean with
standard deviation, median, percentage of missing
Material and methods data, number of ‘‘not applicable/relevant’’ answers
and extent of ceiling and floor effects. Floor
The questionnaire and ceiling effects between 1% and 15% were
The EUROPEP questionnaire was developed on the defined as optimal [16]. Internal consistency was
basis of a systematic literature review, patient inter- assessed using Cronbach’s alpha and average inter-
views, and an empirical study designed to determine item correlation. Only answers on the 1
5-point
patient priorities [6
8]. A cross-national validation scale were included. We defined an alpha of 0.80 as
study of the 23-item questionnaire was performed the lowest acceptable value [3,4]. In contrast to
before its introduction [9,10]. alpha, the average inter-item correlation is indepen-
The patients were asked to evaluate their GP on a dent of the number of items and sample size when
five-point scale (1
5) ranging from ‘‘poor’’ (1), measuring internal consistency. We regarded an
through ‘‘acceptable’’ (3) to ‘‘excellent’’ (5) (2 and average inter-item correlation of at least 0.50 as
4 had no text). Alternatively, patients could use the good [12].
response option ‘‘Not applicable/relevant’’ [5]. All The correlation analyses assessed whether each
items were scored in the same direction. The item had a high correlation with the sum score of the
EUROPEP questionnaire was not developed as a rest of the scale (internal item convergence) and a
rating scale, but as a collection of several equal items higher correlation with the items in its own scale
(‘‘indicator’’ variables [11]) measuring a construct. than with those of other scales (item discriminant
validity) [3,17,18]. Correlations were fixed at a
minimum of 0.60 to reflect a high level of internal
GP and patient populations
convergence [11]. We defined a sufficient item
From 2002 to 2005, 703 GPs participated in the discriminant validity as a correlation with the
DanPEP survey. Each GP personally distributed items in its own scale two standard deviations above
questionnaires to a number of consecutive patients that obtained with other scales (calculated as the
aged 18 or more attending the surgery and able to 95% confidence intervals for the coefficients based
read and understand Danish. The GPs were divided on Fisher’s transformation [Stata, ci2-option]) [12].
into three groups because other aims of the project The factorial structure was evaluated by confir-
were to study the effect of reminders and of using matory factor analysis (CFA) where items were
postal questionnaires to patients. Hence, 121 GPs analysed as categorical measures with a variance-
each distributed 130 questionnaires to consecutive adjusted weighted least-squares method (WLSMV)
patients [14], 391 GPs each distributed 100 ques- estimator. The objective of the CFA was to explore
tionnaires to consecutive patients, whereas the to what degree the correlations between the variables
questionnaires from 191 GPs were sent directly to could be explained by the five domains (factors).
150 patients by the secretary. A total of 83 480 Thus, we defined a basic model where an item was
questionnaires were distributed. Non-responders linked to its own domain (see Table I), with
from the two latter groups received a reminder unspecified correlation between domains. A number
from the DanPEP secretariat 3
5 weeks after the of indices are available to assess the fit of a model
 176
P. Vedsted et al.
Table I. Data quality (mean value (mean), standard deviation (SD), median, item response, answers in the ‘‘Not applicable/relevant’’ category, missing answers, and answers in lowest (floor) and
highest (ceiling)) and internal consistency (Cronbach’s alpha and average inter-item correlation for the domain).

n Mean1 SD Median1 Item response ‘‘Not relevant’’ Missing Floor Ceiling

Doctor
patient relationship (average inter-item correlation 0.64, alpha0.92)

1 Making you feel you had time during consultations? 55877 4.16 0.91 4 99.7 0.3 0.4 1.3 43.5
2 Interest in your personal situation? 55630 4.23 0.88 4 99.6 0.7 0.4 1.0 47.0
3 Making it easy for you to tell him or her about your problems? 55390 4.22 0.88 4 99.6 1.1 0.4 1.0 45.9
4 Involving you in decisions about your medical care? 52936 4.15 0.89 4 99.2 5.1 0.8 0.9 39.8
5 Listening to you? 55841 4.31 0.85 5 99.6 0.3 0.4 0.8 51.6
6 Keeping your records and data confidential? 43838 4.68 0.61 5 99.0 21.0 1.0 0.1 57.9
Medical care (average inter-item correlation 0.68, alpha0.92)
7 Quick relief of your symptoms? 49908 4.03 0.94 4 99.2 10.5 0.8 1.5 33.0
8 Helping you feel well so that you can perform your normal daily activities? 47444 4.04 0.93 4 99.0 1.0 1.0 1.3 31.0
9 Thoroughness? 55260 4.27 0.89 5 99.4 1.1 0.6 1.0 49.9
10 Physical examination of you? 54567 4.23 0.88 4 99.4 2.3 0.6 0.9 46.0
11 Offering you services for preventing diseases? 45339 4.05 1.09 4 99.1 18.5 0.9 2.8 36.1
Information and support (average inter-item correlation 0.72, alpha0.91)
12 Explaining the purpose of tests and treatments? 52692 4.20 0.86 4 99.6 5.9 0.4 0.7 41.4
13 Telling you what you wanted to know about your symptoms and/or illness? 53574 4.10 0.94 4 99.7 4.4 0.3 1.2 39.7
14 Help in dealing with emotional problems related to your health status? 43825 4.04 1.01 4 99.3 21.4 0.7 1.8 32.0
15 Helping you understand the importance of following his or her advice? 51870 4.16 0.84 4 99.4 7.2 0.6 0.6 37.5
Organization of services (average inter-item correlation 0.67, alpha0.80)
16 Knowing what she/he had done or told you during previous contacts? 52404 4.05 0.96 4 99.6 6.4 0.4 1.5 37.0
17 Preparing you for what to expect from specialist or hospital care? 38895 3.97 0.98 4 99.2 30.0 0.8 1.3 25.1
Accessibility (average inter-item correlation 0.52, alpha0.86)
18 The helpfulness of the staff (other than the doctor)? 53071 4.06 0.93 4 98.4 4.0 1.6 1.3 36.9
19 Getting an appointment to suit you? 55449 3.92 1.10 4 99.1 0.5 0.9 3.7 38.2
20 Getting through to the practice on the phone? 54522 3.40 1.23 3 98.9 1.9 1.1 9.3 22.1
21 Being able to speak to the GP on the telephone? 49547 3.56 1.20 4 98.9 10.8 1.1 6.3 24.1
22 Waiting time in the waiting room? 55407 3.49 1.02 3 99.1 0.6 0.9 4.4 17.7
23 Providing quick services for urgent health problems? 38168 4.22 0.94 4 98.6 30.7 1.4 1.1 33.8

Notes: Only questionnaires with at least 12 items answered were included. Each item began as follows: What is your opinion of the GP and/or general practice over the last 12 months with respect to . . .
1
Based on answers on answers to the categories 1
5 and excluding ‘‘Not applicable/not relevant’’.
 Data quality and factor analysis of the Danish EUROPEP questionnaire 177

based on categorical data and we present the six 4. Root mean square error of approximation
indices which have been shown to be useful in (RMSEA) expresses the lack of fit per degree
assessing model fit [19
22]. of freedom of the model. Values are interpreted
as follows: 50.05 indicates very good, 0.05

1. Chi-squared goodness-of-fit statistic assesses 0.08 good and ]0.10 poor fit.
the discrepancy between the sample and fitted 5. Standardized root mean square residual
covariance matrix (the null hypothesis is that (SRMR) is the average of the differences
the model fits the data. An insignificant test between the observed and predicted correla-
indicates good fit (p-value above 0.1). The chi- tions and has a range from 0 to 1. Values of less
squared is extremely sensitive to sample size than 0.08 indicate good fit.
(about 200 cases) and in large samples it tends 6. Weighted root mean square residual (WRMR)
to result in a rejection of the model. For this is also a residual based measure where values
reason, additional fit indices that are less above 0.90 indicate poor fit.
sensitive are recommended (using the non-
centrality parameter and taking into account
sample size and degrees of freedom). Results
2. Comparative fit index (CFI) assesses fit relative
Data quality analysis
to a null model and ranges from 0 to 1 with
values of 0.90
0.95 indicating acceptable and For all items except 20, 21, and 22 the distribution
over 0.95 good fit. was skewed to the left (Table I). The item response
3. Tucker Lewis index (TLI) adjusts for the was high with a small number of missing answers
number of parameters of the model and is (0.3
1.7%). The response category ‘‘not applicable/
interpreted as CFI. relevant’’ was used by 0.3
30.7% of responders. For

Table II. Correlations between domains and correlations between items and (1) the rest of the items in its own scale (item-rest correlation)
and (2) the other domains (Dim 1 to Dim 5).

Item-rest1 Dim 1 Dim 2 Dim 3 Dim 4 Dim 5

Dim 1 Doctor
patient relationship
0.89 0.92 0.86 0.64

1 Making you feel you had time during consultations? 0.73
0.67 0.69 0.62 0.51
2 Interest in your personal situation? 0.82
0.72 0.75 0.67 0.47
3 Making it easy for you to tell him or her about your problems? 0.82
0.71 0.75 0.65 0.46
4 Involving you in decisions about your medical care? 0.77
0.72 0.75 0.66 0.48
5 Listening to you? 0.84
0.73 0.75 0.66 0.47
6 Keeping your records and data confidential? 0.55
0.53 0.55 0.48 0.37
Dim 2 Medical care

0.92 0.87 0.65
7 Quick relief of your symptoms? 0.78 0.66
0.70 0.63 0.50
8 Helping you feel well so that you can perform your normal daily activities? 0.79 0.66
0.70 0.62 0.49
9 Thoroughness? 0.81 0.77
0.77 0.68 0.49
10 Physical examination of you? 0.80 0.76
0.77 0.68 0.48
11 Offering you services for preventing diseases? 0.69 0.65
0.70 0.63 0.46
Dim 3 Information and support


0.92 0.65
12 Explaining the purpose of tests and treatments? 0.79 0.74 0.74
0.71 0.50
13 Telling you what you wanted to know about your symptoms and/or illness? 0.83 0.76 0.76
0.72 0.51
14 Help in dealing with emotional problems related to your health status? 0.80 0.76 0.75
0.71 0.49
15 Helping you understand the importance of following his or her advice? 0.76 0.74 0.75
0.70 0.52
Dim 4 Organization of services



0.66
16 Knowing what s/he had done or told you during previous contacts? 0.67 0.67 0.66 0.70
0.49
17 Preparing you for what to expect from specialist or hospital care? 0.67 0.68 0.71 0.76
0.52
Dim 5 Accessibility





18 The helpfulness of the staff (other than the doctor)? 0.58 0.44 0.44 0.46 0.44

19 Getting an appointment to suit you? 0.71 0.42 0.43 0.43 0.42

20 Getting through to the practice on the phone? 0.70 0.36 0.37 0.38 0.37

21 Being able to speak to the GP on the telephone? 0.71 0.41 0.42 0.43 0.42

22 Waiting time in the waiting room? 0.58 0.39 0.40 0.40 0.42

23 Providing quick services for urgent health problems? 0.61 0.56 0.58 0.58 0.54

Only questionnaires with at least 12 items answered were included.

1
Item-restitem-rest correlation correlation between the item and the sum of the other items in its own domain.
178 P. Vedsted et al.

7 items (6, 7, 11, 14, 17, 21, and 23) this category Discussion
represented more than 10% of the answers. Floor
In agreement with other authors we found a skewed
effect was small (range 0.1
9.3%) and all items had a
distribution for almost all items [23]. Consequently,
ceiling effect larger than 15% (range 18.6
56.3%).
the ceiling effect was high indicating that the full
Cronbach’s alpha ranged from 0.80 to 0.92, and
evaluation range was not captured [16], which may
average inter-item correlation ranged from 0.64 to
0.72. lower the responsiveness of the questionnaire [24].
Item-rest correlation (Table II) (internal item This obviously calls for a change in the response
convergence) (range 0.55
0.84) was below 0.60 for categories as the ceiling effect was seen for all items.
three items (6, 18, and 22). For four items (6, 11, We saw a small number of missing responses, but for
16, and 17) the correlation with own domain (item seven items more than 10% of respondents found
discriminant validity) was lower than with other the questions irrelevant or not applicable. These
domains. Four domains (dim 1 to dim 4) were items represented situations where respondents were
highly correlated (ranged 0.86
0.92). These do- supposed to have experienced health problems and
mains had lower correlations with the ‘‘Accessibility’’ thus they were not relevant to all respondents in
domain. general practice. Answers to these questions may
consequently be divided depending on which pa-
tients respond.
Confirmatory factor analysis We found a few items with low item-rest correla-
The CFA showed high factor loading for all items tion and higher correlations to other domains than
(Table III). The indices for model fit (Table IV) their own. Thus, it is possible that some items should
show that the model fits the data well for two indices be assigned to another domain. We also saw high
(TLI and SRMR), acceptably fit for one index correlations between four of the five domains, which
(CFI), and poorer for three indices (chi-squared, may be a result of cross-correlation. Still, we found a
RMSEA, and WRMR). high internal consistency of the domains which
Table III. Results of the confirmatory factor analysis showing the standardized factor loadings and standardized residuals for each item
when modelled with its own domain.

Standardized factor Standardized residual

loading variance

Doctor
patient relationship
1 Making you feel you had time during consultations? 0.867 0.249
2 Interest in your personal situation? 0.930 0.135
3 Making it easy for you to tell him or her about your problems? 0.923 0.148
4 Involving you in decisions about your medical care? 0.904 0.182
5 Listening to you? 0.943 0.110
6 Keeping your records and data confidential? 0.782 0.389
Medical care
7 Quick relief of your symptoms? 0.883 0.221
8 Helping you feel well so that you can perform your normal daily activities? 0.879 0.227
9 Thoroughness? 0.946 0.105
10 Physical examination of you? 0.935 0.126
11 Offering you services for preventing diseases? 0.844 0.288
Information and support
12 Explaining the purpose of tests and treatments? 0.909 0.173
13 Telling you what you wanted to know about your symptoms and/or illness? 0.930 0.135
14 Help in dealing with emotional problems related to your health status? 0.919 0.155
15 Helping you understand the importance of following his or her advice? 0.908 0.175
Organization of services
16 Knowing what s/he had done or told you during previous contacts? 0.871 0.241
17 Preparing you for what to expect from specialist or hospital care? 0.914 0.164
Accessibility
18 The helpfulness of the staff (other than the doctor)? 0.788 0.379
19 Getting an appointment to suit you? 0.816 0.334
20 Getting through to the practice on the phone? 0.836 0.301
21 Being able to speak to the GP on the telephone? 0.868 0.246
22 Waiting time in the waiting room? 0.734 0.461
23 Providing quick services for urgent health problems? 0.934 0.128
 Data quality and factor analysis of the Danish EUROPEP questionnaire 179
Table IV. Model statistics of the CFA of the Danish version of the EUROPEP questionnaire (n20 072).

Statistics Chi-squared df p CFI1 TLI1 RMSEA SRMR WRMR

Model fit for basic model with five domains 19748.3 79 B0.0001 0.928 0.991 0.111 0.041 8.871
1
Number of free parameters 33. Chisquaredchi-squared goodness-of-fit with dfdegrees of freedom and pp-value (reference:
p]0.1). CFI Comparative fit index (reference: 0.90
0.95acceptable,0.95good). TLITucker Lewis index (reference: 0.90

0.95acceptable,0.95good). RMSEAroot mean square error of approximation (reference: 50.05good, ]0.10poor fit).
SRMRstandardized root mean square residual (reference: B0.08good fit). WRMRweighted root mean square residual (reference:
0.90 poor fit).

indicates that the items behave reasonably similarly sample-size sensitive (there is no simple rule here,
within the domains. However, this would also be but the results should be treated with caution if
seen if there was a high cross-correlation between alpha is below 0.85 in a large sample [11]).
items from different domains.
The high correlations between domains seem to Implications and future research
be confirmed by the CFA. The proposed model of
the EUROPEP questionnaire did not fit the data for This study revealed problems with the Danish
each goodness-of-fit measure. The chi-squared was version of the EUROPEP questionnaire: high ceiling
expected to reject the model because of the high effects, some items with low item-rest correlation
sample size. Even if we ignored this test, two indices and low item discriminant validity, and an uncertain
showed good, two poor, and one acceptable fit. model fit of the proposed factorial structure. Con-
Thus, we are not able to determine the fit of the sequently, there seems to be a need to improve the
proposed factorial structure. We made additional Danish version of the questionnaire, and the study
analyses (not shown) with different values for especially emphasizes the need to develop new
correlations between the domains, but these models response categories to lower the ceiling effect.
did not improve the results compared with the basic Subsequently, there also is a need to assess the
model. We also considered removing one or more factorial structure of the questionnaire again due to
items, but as seen in Table III, no item had a changed variances and to explore possible composite
particularly low loading. The goodness-of-fit indices subscale scores and how the categorical response
may, however, have been strongly affected by the categories behave based on item response theory.
highly skewed response distributions. Further, future research should focus on trying to
provide an external anchor which would make it
possible to analyse whether the indices (multi-item
Strengths and weaknesses model) can be used without loss of information
We obtained a high response rate, minimizing the compared with the use of a single-item model,
risk of selection bias due to dropout. The ques- evaluate whether items should be deleted and finally
tionnaires were handed out personally by the GPs, assess the unidimensionality of each domain.
which may have given the GPs the possibility of
excluding some patients (e.g. those with the most Acknowledgements
negative attitudes) from the study. The patients were
included when attending the surgery, and frequent The authors would like to thank Ms Gitte Hove,
attenders were thus more likely to be included. MLISc, for data management. This part of the
However, this selection method ensured that the DanPEP project was funded by the Central Com-
patients were able to evaluate the GP, which in turn mittee on Quality and Informatics in General
enabled them to better answer the questions. These Practice, the Ministry of the Interior and Health,
issues of selection bias may have affected the and the regional committees on quality development
actual scores, but are unlikely to have changed the in the counties of Aarhus, Frederiksborg, Funen,
factorial structure of the questionnaire. We have Southern Jutland, and Vejle and the municipalities of
previously shown that although non-responders may Copenhagen and Frederiksberg. They are grateful to
evaluate their GPs more negatively, this would only the many patients and their general practitioners for
result in a small change in the complete evaluation their contributions to this study.
[14].
However, in addition to providing very high
statistical precision the large sample size also led to References
narrow confidence intervals for correlation coeffi- [1] Wensing M, Baker R. Patient involvement in general practice
cients and affected Cronbach’s alpha, which is care: a pragmatic framework. Eur J Gen Pract 2003;9:62
5.
/ /
180 P. Vedsted et al.
[2] Anden A, Andersson SO, Rudebeck CE. Concepts under- [13] Cook DA, Beckman TJ. Current concepts in validity and
lying outcome measures in studies of consultations in general reliability for psychometric instruments: Theory and appli-
practice. Scand J Prim Health Care 2006;24:218
23.
/ / cation. Am J Med 2006;119:166. / /

[3] Sitzia J. How valid and reliable are patient satisfaction data? [14] Heje HN, Vedsted P, Olesen F. A cluster-randomized trial of
An analysis of 195 studies. Int J Qual Health Care 1999;11: / / the significance of a reminder procedure in a patient
319
28. evaluation survey in general practice. Int J Qual Health
[4] McDowell I, Newell C. Measuring health: A guide to rating Care 2006;18:232
7.
/ /

scales and questionnaires. Oxford: Oxford University Press; [15] Muthén LK, Muthén BO. Mplus user’s guide. Los Angeles:
1996. Muthén & Muthén; 2007.
[5] Wensing M, Mainz J, Grol R. A standardised instrument for [16] McHorney CA, Tarlov AR. Individual-patient monitoring in
patient evaluations of general practice care in Europe. Eur J clinical practice: Are available health status surveys ade-
Gen Pract 2000;6:82
7./ / quate? Qual Life Res 1995;4:293
307. / /

[6] Grol R, Wensing M. Patients evaluate general/family prac- [17] Foster SL, Cone JD. Validity issues in clinical assessment.
tice. The EUROPEP instrument. EQuiP, WONCA Region Psychol Assess 1995;7:248
60. / /

Europe, 2000. [18] Ware-JE J, Gandek B. Methods for testing data quality,
[7] Wensing M, Mainz J, Ferreira PL, Hearnshaw H, Hjortdahl scaling assumptions, and reliability: The IQOLA Project
P, Olesen F, et al. General practice care and patients’ approach. International quality of life assessment. J Clin
priorities in Europe: An international comparison. Health Epidemiol 1998;51:945
52.
/ /

Policy 1998;45:175
86.
/ / [19] McDonald RP, Ho MHR. Principles and practice in report-
[8] Grol R, Wensing M, Mainz J, Ferreira P, Hearnshaw H, ing structural equation analyses. Psychological Methods
Hjortdahl P, et al. Patients’ priorities with respect to general 2002;7:64
82.
/ /

practice care: An international comparison. European Task [20] Muthen B, Kaplan D. A comparison of some methodologies
Force on Patient Evaluations of General Practice (EUR- for the factor-analysis of non-normal Likert variables. Br J
OPEP). Fam Pract 1999;16:4
11./ / Math Stat Psychol 1985;38:171
89. / /

[9] Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, [21] Bentler PM. Comparative fit indexes in structural models.
Hearnshaw H, et al. Patients in Europe evaluate general Psychol Bull 1990;107:238
46.
/ /

practice care: An international comparison. Br J Gen Pract [22] Hu L-T, Bentler PM. Cutoff criteria for fit indices in
2000;50:882
7.
/ / covariance structure analysis: Conventional criteria versus
[10] Mainz J, Vedsted P, Olesen F. How do the patients evaluate new alternatives. Structural Equation Modeling: A Multi-
general practitioners? (in Danish). Ugeskr Laeger 2000;162:/ / disciplinary Journal 1999;6:1
55. / /

654
8. [23] Milano M, Mola E, Collecchia G, Carlo AD, Giancane R,

[11] Fayers PM, Machin D. Quality of life: Assessment, analysis Visentin G, et al. Validation of the Italian version of the
and interpretation. Chichester: Wiley; 2000. EUROPEP instrument for patient evaluation of general
[12] Streiner DL, Norman GR. Health measurement scales. A practice care. Eur J Gen Pract 2007;13:92
4. / /

practical guide to their development and use. New York: [24] De Vet HC, Terwee CB, Bouter LM. Current challenges in
Oxford University Press; 2003. clinimetrics. J Clin Epidemiol 2003;56:1137
41. / /