International Journal of Quality & Reliability Management

A comprehensive internal quality assurance system at University of Minho

Isabel M. Santos, Graciete Dias,
Article information:
To cite this document:
Isabel M. Santos, Graciete Dias, (2017) "A comprehensive internal quality assurance system at
University of Minho", International Journal of Quality & Reliability Management, Vol. 34 Issue: 2,
pp.278-294, https://doi.org/10.1108/IJQRM-04-2015-0063
Permanent link to this document:
https://doi.org/10.1108/IJQRM-04-2015-0063
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

Downloaded on: 14 March 2018, At: 21:50 (PT)

References: this document contains references to 27 other documents.
To copy this document: [email protected]
The fulltext of this document has been downloaded 436 times since 2017*
Users who downloaded this article also downloaded:
(2017),"The European standards and guidelines for internal quality assurance: An integrative
approach to quality management in higher education?", The TQM Journal, Vol. 29 Iss 2 pp. 342-356
<a href="https://doi.org/10.1108/TQM-01-2016-0009">https://doi.org/10.1108/TQM-01-2016-0009</a>
(2017),"Evolution of internal quality assurance at one university – a case study", Quality Assurance
in Education, Vol. 25 Iss 2 pp. 189-205 <a href="https://doi.org/10.1108/QAE-03-2016-0011">https://
doi.org/10.1108/QAE-03-2016-0011</a>

Access to this document was granted through an Emerald subscription provided by emerald-
srm:603747 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.

*Related content and download information correct at time of download.

 The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0265-671X.htm

IJQRM
34,2 A comprehensive internal
quality assurance system at
University of Minho
278 Isabel M. Santos
Quality Assurance Services, University of Minho, Braga, Portugal, and
Received 30 April 2015
Revised 13 September 2015 Graciete Dias
Accepted 3 November 2015
University of Minho, Braga, Portugal

Abstract
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

Purpose – The European Standards and Guidelines (ESG) for quality assurance (QA) adopted at the Bergen
Ministerial Meeting in 2005 in the scope of the Bologna process call upon higher education institutions to take
up a systematic approach to internal QA. Standard 1.1 of the ESG establishes that institutions should have a
policy and associated procedures for the assurance of the quality and standards of their programmes and
awards. University of Minho (UMinho) has a longstanding experience on innovative methods for the
coordination and management of the teaching and learning processes, including, since 1991, systematic
mechanisms for the evaluation of teaching. However, to fulfil the new demands raised by the ESG UMinho felt
the need to define a formal institutional quality policy, building upon the existing procedures and
mechanisms in order to set up a comprehensive internal QA system (SIGAQ-UM) fully compatible with the
ESG. The purpose of this paper is to present the distinctive features of SIGAQ-UM, the procedures involved in
its certification by Agency for Assessment and Accreditation of Higher Education (A3ES), as well as on the
perceived impact on the university’s activities, including some comments on the difficulties to develop and
consolidate a quality culture embedded in all the academic community.
Design/methodology/approach – Case study approach.
Findings – SIGAQ-UM is a fully operational comprehensive internal QA system certified by the Portuguese
Agency A3ES in January 2013, with considerable impact on the university’s operation.
Originality/value – Disclosure of best practices on QA in higher education.
Keywords Quality culture, Internal quality assurance systems, Quality assurance in higher education
Paper type Case study

1. Introduction
Quality assurance (QA) in higher education stays high in the agenda of the European Higher
Education Area, as it plays a key role in the Bologna process. A fundamental principle of
QA is that institutions themselves hold the main responsibility for ensuring the quality of
their activities. This is also an underlying principle of the European Standards and
Guidelines (ESG) adopted in Bergen, which call upon higher education institutions to take
up a systematic approach to internal QA. However, the follow-up reports on the Bologna
process show that, in many cases, there is still no holistic approach to quality enhancement
within institutions.
Portuguese higher education institutions are since 2007 required by law to develop their
own internal QA systems and data from the Portuguese Agency for Assessment and
Accreditation of Higher Education (A3ES) shows that significant progress was already
made by many institutions (Fonseca, 2011). Indeed, as a result of the former national
evaluation system (1995-2005) several institutions had already started to implement some
QA mechanisms as a way to embed a quality culture in their activities.
University of Minho (UMinho), which was a pioneer in this field in Portugal, has a
International Journal of Quality &
Reliability Management longstanding experience on innovative methods for the coordination and management of
Vol. 34 No. 2, 2017
pp. 278-294
the teaching and learning processes, including, since 1991, systematic mechanisms
© Emerald Publishing Limited
0265-671X
for assessing the quality of teaching in all its graduation programmes on a yearly basis.
DOI 10.1108/IJQRM-04-2015-0063 The internal QA system of UMinho (SIGAQ-UM) can thus be seen as a case study of
 continuous development from the initial mechanisms of tracking and feedback from Internal
students to a comprehensive system covering all activities and services, certified by A3ES quality
under its institutional audit process. assurance
system
2. The European and Portuguese contexts of internal QA
The systems for the QA in higher education play a crucial role in the prosecution of the
goals stated in the Bologna Declaration, for their potential contribution towards the mutual 279
recognition of degree awards and periods of study and the subsequent portability of
academic and professional qualifications which is so essential to a true and meaningful
mobility of the European citizens. According to Harvey and Williams (2010), citing
Westerheijden (2001), the aim of the Bologna process was indeed to increase transparency of
European higher education, thus encouraging clearer QA processes.
QA has, therefore, acquired a growing importance as the Bologna process moved along.
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

It started with a short statement in the Bologna Declaration, advocating “promotion of

European co-operation in quality assurance with a view to developing comparable criteria
and methodologies” (Bologna Declaration, 1999), followed in the Prague Communiqué (2001)
by a call upon relevant partners to the establishment of a common framework of reference
and the dissemination of best practices aiming at the mutual acceptance of evaluation
and accreditation/certification mechanisms. At the Berlin meeting ministers took a more
pro-active approach, establishing some mutually agreed elements to be included into the
national systems for QA and calling upon the E4 Group[1] “to develop an agreed set of
standards, procedures and guidelines on quality assurance” (Berlin Communiqué, 2003).
This decision led to the adoption, in the Bergen Communiqué (2005), of the Standards and
Guidelines for QA in the European Higher Education Area (ESG), which constitute the main
framework for QA systems all over Europe.
An essential underlying assumption of the ESG is the centrality of internal QA.
As stressed in the Berlin Communiqué, “consistent with the principle of institutional
autonomy, the primary responsibility for quality assurance in higher education lies with
each institution itself and this provides the basis for real accountability of the academic
system within the national quality framework”, reflecting a common position from the
European Universities, stated in the Graz Declaration issued under the auspices of the
European University Association (2003).
The ESG explicitly assume, amongst its basic principles, that “providers of higher
education have the primary responsibility for the quality of their provision and its
assurance” and “there should be encouragement of a culture of quality within higher
education institutions”. Having this in mind, the ESG dedicate a full set of standards to the
internal QA of institutions. There is, however, a limitation inherent to the ESG, since they
relate only to the three cycles of higher education and “are not intended to cover the area of
research or general institutional management” (ENQA, 2009).
The European Association for Quality Assurance (ENQA) document on the ESG published
by ENQA puts a great emphasis on the importance that should be given to internal QA
systems, recommending as a principle of good practice that “use should be made, wherever
possible, of the results of institutions’ own internal quality assurance activities” (ENQA, 2009),
in accordance with the first standard of part 2 of the ESG which establishes that external QA
procedures should take into account the effectiveness of the internal QA processes. External
and internal QA procedures are, therefore, complementary and mutually supporting processes,
requiring a proper balance of intensiveness and effort. In particular, as suggested in the Trends
IV report, “external quality assurance should be reduced in direct correlation to the evidence of
robust internal quality processes” (Reichert and Tauch, 2005).
The integration of the ESG into the national legal frameworks for higher education that
has been occurring in most European countries created a great dynamics in QA, but the 2009
 IJQRM stocktaking report on the Bologna process stated that, “although the implementation of
34,2 external quality assurance is proceeding at a rapid pace, development of internal quality
assurance systems at HEIs is progressing more slowly, especially because in some countries
the internal QA systems are still thought to amount only to writing a self-assessment report
for external review” (Rauhvargers et al., 2009). The Trends V (Crosier et al., 2007) and Trends
2010 (Sursock and Smidt, 2010) reports also refer a growing awareness of higher education
280 institutions in relation to the potential benefits and challenges of effective QA and
enhancement activities, visible in the work that has been done to develop internal quality
processes in institutions, but few institutions are taking a holistic approach to quality
improvement, pointing out that, for example, only a few countries have developed assessment
of support services for students in the majority of their institutions.
More recently, the Trends 2015 report states that “an increasing number of institutions
have institutional QA policies and processes that are used for institutional planning and
improvement” (Sursock, 2015), but still mentions some shortcomings, for example, in
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

relation to peer feedback, the use of results from student questionnaires, academic staff
development and research on learning and teaching. The Yerevan Communiqué (2015) also
refers that QA tools are sometimes used incorrectly or in bureaucratic and superficial ways.
In this respect Reichert (2008) argues that one of the achievements of the ESG is on the
emphasis that “internal quality assurance should not be reduced to formalised processes
but should be likened more to a set of institutional and individual attitudes, a ‘quality
culture’, aiming at ‘continuous enhancement of quality’ ”. However, at the concluding
session of the 2nd EQAF, Harvey (2008) expressed the concern that “an overemphasis on
compliance with the standards, rather than treating the ESG as advisory, may lead to a
tick-box mentality, with institutions becoming ‘slaves to the ESG’ instead of being creative
in their developments on quality assurance”. The same apprehension had been conveyed
in the final report of the European University Association (EUA) Quality Culture Project
on the development of an internal culture of quality within universities, which concluded
on the risk that internal QA processes might become mere bureaucratic procedures, even
when they are undertaken properly (European University Association (EUA), 2006).
These problems on the implementation of the ESG are “largely related with academic
conservatism and institutional inertia, which have made it difficult to involve not only
teachers but also managers, students and (other) staff in enhancement processes which
require significant change in practices and mentalities” (Santos, 2011).
European higher education institutions have, therefore, been facing big challenges to
design and implement a systematic approach to internal QA as a permanent process aiming
at embedding a quality culture to foster enhanced quality. In the aforementioned EQAF,
Harvey (2008) suggests the following key features for such a quality culture:
[…] academic ownership and engagement; recognition of the need for a quality system (but not one
driven by bureaucracy); a focus on changing people’s behaviour rather than the mechanics of a
system of reporting and review; clarity of purpose; centrality of students; encouragement of
partnership and co-operation; focusing (on) community engagement and team working; a
leadership style that inspires rather than dictates; welcoming of external critical evaluation; an
integrated and continuous process of self-reflection; providing the context to take the initiative to
improve, even where it is risky.
In Portugal, the external QA of higher education institutions started in 1993 in the scope of
a pilot project on QA launched by the Portuguese Rectors Conference with the main
objective of quality enhancement of teaching. In 1994 a law was passed by Parliament,
establishing an innovative national QA system, which allowed the normative
development of the law to be made by protocols agreed with entities representative of
the institutions. The pilot project was integrated into the new framework, under the
coordination of an Evaluation Council established as an independent body within the
 Foundation of the Portuguese Universities, and considered as the first round of Internal
evaluations to cover all public university programmes, to be concluded in 1999. In the year quality
2000 the evaluation system was extended to all sectors of higher education (universities assurance
and polytechnics, public and private, and military higher education) and a new evaluation
round was set up for the period 2000-2005, keeping the programme-oriented approach system
(Amaral et al., 2013).
The evaluation rounds had a significant impact within higher education institutions, as 281
stated in the self-evaluation report of the national evaluation system prepared as part of an
ENQA-coordinated external evaluation, which identifies as a system’s strength that many
institutions assumed the participation in the evaluation process as an opportunity to reflect
on their organisation and activities and to take up quality culture and QA as strategic
elements for their institutional development, contributing to a positive movement towards
the institutionalisation of internal QA procedures (Santos et al., 2006).
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

This system was discontinued, due to a political decision to introduce programme

accreditation in line with the prevailing trends in Europe. A new law, passed in Parliament
in 2007, created an Agency for the evaluation and accreditation of higher education (A3ES)
whose main task is to perform programme accreditation in relation to the three cycles of
studies. Under the new legal framework, each institution is required to develop its own
internal QA system, certifiable along the lines of the ESG.
Although overwhelmed by the legal requirement to promote the accreditation of all study
programmes, but aware that a number of mature QA systems in Europe were moving away
from programme accreditation towards less burdensome institutional audit approaches
(Crosier et al., 2007), the Agency included in its initial objectives “the establishment of
institutional audit mechanisms leading to the possibility of certification of internal quality
assurance systems […] as an essential tool for a subsequent simplification of procedures in
the external assessment and accreditation processes” (Santos, 2011).
Aiming to assist institutions in designing and implementing their QA systems the
Agency, after an extensive consultation process, set up the criteria for certification, in middle
2011, by adopting a set of reference points (10 “open” standards presented as
“non prescriptive proposals, which describe the main characteristics of a well designed
and properly implemented quality assurance system”) which follow closely the seven
standards in part 1 of the ESG but also include the self-assessment and improvement of
research, interaction with society and internationalisation, thus covering the whole
institutional mission (A3ES, 2013a). An institutional audit process was then started, as
referred ahead in Section 6.

3. The early stages of QA at UMinho

Since its beginning, in 1975, the UMinho has in place an innovative process for the
coordination and management of teaching that considers each degree programme
as an “educational project” with its own coordination structures, which include the
degree committee and the degree director. The degree committee includes a student
(delegate) for each curricular year and an equal number of professors. A Degrees Council
( presently, a Pedagogical Council) was also created for each scientific area, to coordinate
common aspects of different programmes (e.g. the engineering degrees council,
for the whole set of Engineering programmes). This was quite a novelty, since the
coordination of teaching in Portugal, at the time, was mainly performed in a fragmented
way, at subject level (the Chairs or Cathedras), by the Chair holders, and students had no
participation in it. This structure proved to be useful to keep track on the quality of
teaching and promote improvement.
In June 1991 a formal system for the evaluation of teaching was established by a Rector’s
Resolution, after extensive consultation with the University Academic Council, containing
 IJQRM three main elements (a more detailed description of this early system is available at
34,2 Montenegro et al. (2010):
(1) A procedure for the application of a survey to students, covering all course units of all
undergraduate programmes, on a semi-annual basis, whose outcomes included
individual and aggregate results, as well as percentiles. Individual results concerning
the quality of teaching were only sent to the corresponding teacher. Overall results,
282 aggregated by degree programme (global and for each curricular year), department,
school and the whole university, were widely publicised and published as a book.
(2) The systematic monitoring of the students’ success in every course unit including
data on the number of students registered in the unit, students that attended classes
regularly, students assessed, pass and failure rates and part time students.
(3) An internal reflection system, in two complementary dimensions:
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

• degree directors, departments and schools were called upon to analyse the results
from the surveys and take remedial/improvement actions as deemed necessary; and
• in the case of pass rates below predefined standards, the course unit coordinator
was asked to analyse, as deeply as possible, the reasons for the abnormal non-
success and propose measures to improve the efficacy of teaching; a second lower
critical value was defined for pass rates, below which the Degree Committee had
necessarily to analyse the suggestions from the course unit coordinator and
prepare formal recommendations aiming at overcoming the detected problems.
This system went on quite smoothly for over one decade, with small adjustments in the
questionnaires. It proved to be very useful for the self-evaluation of degree programmes in
relation to the external evaluations rounds 1995-2000 and 2000-2005, because a lot of data
collection and analysis on students and teachers’ performance and students’ satisfaction
was already available.
In 2004 the system was reinforced with the establishment of a QA administrative structure,
under the coordination of the Pro-Rector for the assessment and quality of teaching, with the
objective to plan and coordinate the evaluation procedures and interact with the national
evaluation agency. With the support of this new structure, existing questionnaires were
updated and new ones were devised, aiming at more comprehensive procedures to collect and
interpret data on students and teachers’ performance and satisfaction.
This system ran up to 2009/2010, making use of an extensive set of mechanisms and
procedures for the QA of teaching, including:
• Internal feedback mechanisms for systematic collection of data from students, teachers
and services (particularly, students supporting services), through the application of a
wide set of surveys covering, inter alias, social-demographic context and expectations
of freshman students, assessment of learning/teaching processes at course unit level by
students and teachers, assessment of organisational and curricular aspects at degree
programme level by students, teachers’ satisfaction with the working conditions and
learning environments, enquires to incoming and outgoing ERASMUS students on the
special support facilities concerning students in mobility schemes, and the satisfaction
of the main users of the different university services. The results from the surveys are
made available to key decision makers in the different levels within the university.
• External feedback, mainly collected by means of the AlumniUM site, set up with the
objective to follow-up the professional careers of the alumni and to provide them with
information about on-site and distance educational offers, access to an employment
pool and other services favouring a permanent relation with the university.
 Systematic feedback is collected annually from alumni and employers. Additional Internal
ways include formal consultation of professional associations, enterprises and public quality
services, as deemed relevant, when a new degree programme is designed and assurance
developed, and reports from external advisory committees set up in connection with
some Schools or degree programmes. system
• Internal reflection on the data collected from monitoring the operation of the degree
programme and from internal and external feedback, leading to the preparation of a 283
report on the progress of learning/teaching processes, to be discussed and analysed in
the Degree Committee and Pedagogical Council. Issues needing correction are identified
and some measures for improvement are taken as a result of the assessment.

This system has been helpful in the promotion of quality of teaching, but two handicaps
could be identified. On one hand, although the degree council issued recommendations on
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

some corrective or improvement measures, a proper action plan was not consistently drawn
up and subjected to the approval of the relevant bodies at School and/or university level,
which led to somewhat heterogeneous results. Second, no follow-up procedure concerning
the implementation of the actions for improvement was in place in a comprehensive way:
• External reviews at programme and institutional level:
– Every degree programme went through two external evaluation processes,
conducted by the former National Evaluation Council in the context of the
evaluation rounds 1995-2000 and 2000-2005. These exercises, especially in
the preparation of the self-evaluation reports, were very useful to promote
participation and internal reflection.
– At institutional level, UMinho has been reviewed for three times by the EUA, on a
voluntary basis, under the (EUA/Institutional Evaluation Programme). The first
review took place in 1997 and the second one ten years later, with a follow-up in 2009.

It is interesting to compare the comments of the two external review teams, since they help
to understand the sometimes difficult and casuistic way that has been followed in the
implementation of internal QA mechanisms, as well as the progresses achieved. In the 1997
report it can be read (CRE, 1997):

The review team has the impression that there is a real interest for the exercise of self-evaluation and
external quality assurance, and that “quality” is a permanent concern for all UMinho actors. During
the visit, the team was told that five years ago the scientific staff was very sceptical about evaluation
exercises and even refused them. Now, on the basis of previous, discipline based self-evaluations the
process is well embedded even in the teaching exercise. At the level of Departments or Schools, we
feel that there is no clear policy yet for quality assurance, the Schools are not yet ready for the idea of
continuous self-evaluation. […] However, we must mention that people are better prepared for and
conscious about it and to carry out such an exercise is much easier now than it was 5 years ago.
In the 2007 review, the external report stresses the different processes for QA in place, namely
the large scale monitoring of results of academic activities and the annual surveys of student
and teacher satisfaction about teaching and learning processes, as well as the degree council
system, through which the quality of teaching can be assessed, and concludes (European
University Association, 2007):
The Team found that there is a good quality culture prevailing at UM. The Team especially
appreciates the way educational programmes are managed through the matrix system by degree-
based committees and degree leaders; this structure makes it possible to develop, manage and assess
these programmes in co-operation with teachers and students. Such a system may be better than
traditional student satisfaction surveys, since it provides an opportunity for immediate reactions.
 IJQRM 4. Moving towards a comprehensive internal QA system
34,2 As confirmed by the external reviews, in 2009 UMinho had already in place most of the
usual instruments to monitor the quality of teaching and take appropriate action towards its
improvement. The same was true for research activities, as the university applied the
criteria used by the Foundation for Science and Technology to monitor the performance of
its research units.
284 As mentioned above, the ESG appeal to higher education institutions to adopt a
systematic approach to internal QA. The first standard of the ESG establishes that
institutions should have a policy and associated procedures for the assurance of the quality
and standards of their programmes and awards, and consequently should develop and
implement a strategy for the continuous enhancement of quality, adding that “the strategy,
policy and procedures should have a formal status and be publicly available” (ENQA, 2009).
UMinho felt that, to comply with this standard, it needed to develop further three main
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

elements of its QA system:

(1) the definition of explicit quality standards for its activities, a matter that at the time
had no tradition in the Portuguese Universities;
(2) the organisation of its QA instruments as a well documented system, applied in a
consistent and holistic way; and
(3) the definition of procedures for the monitoring and continuous improvement of the
QA system itself.
Aiming at full compliance with the ESG, UMinho has therefore adopted a formal
institutional quality policy, building upon former QA procedures and mechanisms
while addressing the questions raised above, in order to set up an internal QA system
(SIGAQ-UM) certifiable by the national agency A3ES. This system, presented in the next
sections, was tested, on a trial basis, in the academic year 2010/2011 and has been running
regularly since then.

4.1 The institutional quality policy

UMinho assumes an unambiguous commitment towards quality and QA, explicitly
mentioned in its statutes as a fundamental vector for its activities and accomplishment.
The institutional policy for quality assumes, as a nuclear underlying principle, the
shared vision of the university’s mission and objectives by all the academic community,
translated into a strategic institutional development plan and its furtherance throughout the
whole university, and developed and implemented through the following key elements:
• the strategic programme of UMinho adopted by the Board of Trustees (Conselho
Geral) under proposal from the Rector;
• an operational plan (quality plan) that translates the strategic programme into
concrete action plans, both at institutional level and at the level of organisational
units and services; and
• the definition of formal methodologies and procedures for monitoring, assessment
and feedback action for quality enhancement (Plan-Do-Check-Act approach),
implemented at the various levels of management, all suitably documented in a
quality manual.
In practice, based on the detailed description of the objectives and actions defined in the
quality plan, as well as on the elements collected by means of monitoring, each unit, service
or pedagogical entity, through appropriate bodies and in accordance with properly
institutionalised methodologies, promotes an analysis of the results achieved and the level
 of conformity with the objectives and targets fixed. The purpose of this analysis is to Internal
produce annual self-assessment reports which reflect on the processes and results and quality
propose amendments to be introduced, expressed in improvement plans and the possible assurance
reformulation of objectives or goals.
The quality policy encompasses, furthermore, the consolidation of a quality culture system
embedded in all the academic community and university’s activities, where continuous
assessment is to be seen as a natural element of the institutional life, in a double perspective 285
of quality enhancement and of providing relevant and updated information on the degree of
fulfilment of the university’s mission.

4.2 The internal QA system (SIGAQ-UM)

The institutional quality policy is developed and embodied in a formal QA system – the
SIGAQ-UM – whose object is the different dimensions of the institutional mission, covering
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

systematically the QA of all activities within the university.

The objectives of SIGAQ-UM are to promote the definition and documentation of the
structuring elements for the implementation of the quality policy, namely:
• the institutional strategy and the standards for quality;
• the responsibilities of the different bodies and management levels in the QA processes;
• the processes for monitoring, control, reflection and subsequent intervention for
quality improvement;
• the ways for the participation of students, teachers, researchers and other staff, as
well as external stakeholders;
• the production and dissemination of information, including the publication of
information that is relevant to external stakeholders; and
• the organisation and continuous improvement of the system itself and the methods
for the monitoring and revision of the quality policy.
The two main documents for the definition of SIGAQ-UM are:
(1) the quality plan, grounded on the strategic programme; and
(2) the quality manual, as a normative document that describes the system’s
organisation and implementation.
4.2.1 The quality plan. The quality plan addresses, in a comprehensive and dynamic way,
the question of the quality standards for the university. In reality, by fixing strategic
objectives, hierarchic operational objectives, the actions to be developed and the
corresponding schedule and goals, the quality plan establishes the quality standards to
be pursued.
As mentioned earlier, the strategic document that underpins the quality plan is the
strategic programme for the four-year Rector’s mandate. It develops around strategic
development vectors relating to the key areas of the university’s mission (mission vectors)
and to activities which support a harmonious and sustainable development of the university
(support vectors).
Vectors are developed into action plans, which include the definition of the strategies,
methodologies and performance indicators involved in the prosecution and monitoring of
the foreseen actions, the goals to be met and the schedule and the levels of responsibility for
their implementation.
Additionally, the quality plan includes a set of action plans concerning transversal
measures associated to the learning, working and living environments in campus.
 IJQRM Monitoring the development of actions plans, as well as the level of achievement of the
34,2 expected outcomes, is the responsibility of bodies or entities identified in the plans, and
requires the systematic collection of appropriate performance indicators, which can be of
two different types:
(1) quantitative indicators, associated to concrete goals that can be quantitatively
assessed; and
286 (2) products, to be developed within a predefined timetable – in this case, a four-level
scale is used to assess the stage of development of a product, in order to monitor the
progress of its implementation in an objective way.
4.2.2 The quality manual. The quality manual, formally approved by a Rector’s Resolution,
defines the overall conception of the institutional quality policy and its translation into an
internal QA system. The following aspects, central to the ESG, are addressed and developed
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

in the manual:
• the statutory mission, vision and objectives of UMinho, as well as its strategy for quality;
• a brief presentation of the university’s organic model;
• the macro-organisation of SIGAQ-UM, in terms of its ambit, objectives, coordination
structures and levels of responsibility in the scope of quality and QA;
• the methodologies for the monitoring, assessment and action for quality
enhancement, discriminated for each of the different dimensions of the
institutional mission (research, teaching and learning, and interaction with society)
and also the transversal dimensions of human and material resources and services;
• the interface of SIGAQ-UM with the strategic management of the university;
• the participation of internal and external stakeholders in the QA system;
• the production and dissemination of information (mechanisms to collect, analyse and
use information, and to publicise information relevant to external stakeholders); and
• the ways in which the QA system is monitored and revised.
The manual also establishes a set of performance indicators covering the main dimensions of the
university activities, which, in its evolution over time (as time series), constitute an Institutional
Progress Chart that will allow for a dynamic view of the development of the university. The
chart provides data for the three most recent years in the fields (and subfields) of teaching
(educational offer, qualification of demand, student body, teaching organisation, educational
efficiency, professional insertion of graduates and social support), research (organisation of
research and level of activity, scientific output and valorisation of knowledge), interaction with
society (institutional action in the external environment), internationalisation (in teaching and
research, and student and staff mobility), human resources (faculty and non-teaching staff),
physical resources (facilities, document collection, ICT) and financial resources.
Having in mind the complexity of the teaching and learning processes, the manual pays
special attention to the methodologies for the QA of teaching. It includes, therefore, a
particularly detailed specification concerning the strategy for the assessment of teaching, the
hierarchic structure used for monitoring, assessment and preparation of improvement action
plans, the identification of best practices, the flagging and treatment of (quantitative or
qualitative) results bellow predefined goals, and the follow-up of adopted action plans for
correction and improvement. It must be stressed, however, that the strategy for the
monitoring, assessment and consequent action for quality enhancement regarding the areas of
research, interaction with society and support services is basically the same, although with a
 simpler hierarchic structure. For example, in relation to R&D each research unit draws up its Internal
annual self-assessment report along the lines outlined in Section 4.3 and the Scientific Council quality
at each school analyses and discusses the reports of all research units operating within its assurance
remit, producing a summary report on the assessment of quality and adequacy of the research
activity carried out, which will then become an integral part of the school’s annual report. system
4.2.3 QA supporting structures. The main structures for the coordination and support of
SIGAQ-UM are: 287
• The QA follow-up Committee (Comissão de Acompanhamento), acting as a
coordination and advisory body chaired by the Vice-Rector with the QA portfolio and
integrating the QA Manager, representatives from schools, cultural units and
services and from students. It may also include an external consultant. The
Committee is responsible for the meta-evaluation of SIGAQ-UM, including the
approval of an annual report on the system’s operation and enhancement.
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

An important feature of the Committee is that the schools’ representatives are required to
act as “quality promoters” in their organic units, liaising with the QA Office and assisting
the Dean in regard to the local coordination of teaching and learning QA processes, thus
contributing to a proper vertical articulation of SIGAQ-UM:
• The QA Office (Serviços para a Garantia da Qualidade (SGAQ)), responsible for the
functional coordination of the QA system, acting as a logistics support centre for the
Committee and the SIGAQ-UM. The quality manager acts as head officer of SGAQ.
• A specific information system, properly integrated into the university information
system, which provides support for data gathering and analysis (including on-line
surveys), monitoring of action plans in the quality plan and on-line preparation of
reports, allowing for a paperless environment in the whole system.
4.2.4 Integration in the institutional governance and management system. The QA
structures mentioned in the previous section are strictly supporting the QA system and do
not duplicate or overlap with governance or academic bodies. In fact, all of the self-reflection
work leading to the elaboration of the self-assessment reports concerning the different levels
and activities is conducted by the actual coordination and management bodies: the course
units coordinators, degree programme directors, degree programme committees and
pedagogical councils, regarding the teaching dimension; the directors and directive
committees of the research units, and the Scientific Councils, for the research dimension; the
cultural units coordinators and the cultural council, as regards the scope of these units; the
Heads of Services, with respect to the dimension of support services.
With respect to the use of the information produced by SIGAQ-UM as a tool for
strategic management, the Unit Councils in the Organic Units and the Rector, based on the
self-assessment reports and summary reports, analyse on an annual basis the level of
achievement of the goals and targets proposed in the quality plan, starting from the panel of
indicators and the analyses conducted, and promote an extended discussion about these
issues. Based on such discussions and analyses, they validate the quality plan on their level
of competences, suggesting the adjustment, whenever necessary, of strategic objectives,
operational objectives or goals to be achieved.
As a result, a close interconnection has been established between SIGAQ-UM and the
governance and management bodies.

4.3 Strategy for QA of teaching and learning

As mentioned above, the methodologies for the QA of teaching are particularly detailed in
the quality manual. Indeed, the complexity of teaching and learning processes, the lower
 IJQRM experience with regard to the assessment of teaching and the level of detail
34,2 which characterises the external QA standards in this particular dimension of the
institutional mission justify a substantially increased degree of explicitness to ensure
the quality of teaching.
The strategy for monitoring, assessing and improving teaching is essentially developed
at successive levels of intervention which are gradually aggregated and consists mainly in
288 the preparation of annual self-assessment reports, in accordance with previously defined
specifications and format. The starting point for this analysis is the subject level (Course
Unit), which constitutes the foundation for the organisation, planning and operation of
teaching, and is hence located at the core of the processes of teaching and learning as well as
of the interaction among its main actors (students and teachers). The mechanisms of QA for
teaching are thus developed in progressive “quality cycles”, from the level closest to the
learning environments upwards to the following levels of analysis – the degree programme,
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

the Organic Unit (School) and UMinho as a whole.

Detailed specifications have been devised for the structure of the self-assessment reports,
aiming at the development of forms that facilitate their on-line preparation. A model
structure has been adopted for the different reports (initially for teaching, but extended later
to self-evaluation of research units, cultural units and services), which establishes a
distinction between the background information that underlies the self-assessment process
and the self-assessment exercise per se:
• The background information is imported from sources and databases in UMinho’s
information system, including the aggregated appraisals submitted at the previous
levels of analysis. It becomes automatically available upon opening of the report. An
agile access to information is thus provided through selective viewing interfaces,
which allow for an easy and interactive navigation among a highly significant
volume of information.
• The electronic form for the elaboration of the report is pre-formatted, including a
self-reflection on nuclear aspects of the organisation, functioning and results of
teaching at the level of analysis in question, the identification of strengths and
weaknesses (or SWOT analysis), and specific fields to suggest improvement
measures and identify best practices liable to be included in a best practices portfolio.
Since the background information is treated separately and in advance, the preparation of
the self-assessment reports (either for teaching research or services) corresponds to a
predominantly analytic process, devoid of bureaucratic load and relatively light on the
academic community. However, although light, the process ensures the systematic
identification of potential situations of poor quality thanks to a process of automatic
identification of results that require further examination, in accordance with flagging
criteria defined in the quality manual. In case a course unit is flagged with results in need of
further examination (i.e. results that stray significantly, by default, from the goals and
targets set out in the quality plan and remaining criteria stipulated by the Pedagogical
Councils), the self-assessment report must include a reflection from the course unit
coordinator on the reasons that may have been at the root of those results, as well as a
suggestion for an action plan to overcome or minimise the problems detected. Such
reflections and suggestions are resumed at the next level of analysis – the degree
programme self-assessment report – which shall explicitly include a proposal for an action
plan aimed at the resolution of the problems encountered, or a proper rationalisation for the
non-necessity of a formal action plan to tackle the situation in question. It is for the
pedagogical council to analyse and validate the action plans for improvement proposed by
the degree programme directors. Consequently, there are appropriate safety measures to
ensure that the situations under reference are necessarily analysed and treated.
 The recommendations for improvement are not limited to those situations in which Internal
there is evidence of results in need of further examination. The structure of the reports quality
ensures that, even in course units and degree programmes without instances of assurance
flagging, there is the possibility to suggest recommendations for improvement, particularly
in terms of pedagogical innovation, as part of an unremitting contribution towards system
continuous improvement.
Follow-up on the implementation of the action plans for improvement is guaranteed as 289
well. The software application for the management of the action plans allows for the
recording of the improvement actions comprised in the plan, the respective coordinators and
of those responsible for their implementation and schedule, as well as the monitoring of plan
implementation. It should be further noted that the self-assessment reports always require
an appreciation of the answer given to the recommendations and improvement plans
included in the previous assessment.
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

4.4 Constraints on the implementation of SIGAQ-UM

In the setting up of the system the main constraints were related not so much with the
design of the QA system itself, since a clear quality policy and strategy for QA had been
previously defined, but rather with some problems of pedagogical organisation and timely
availability of reliable information on the teaching/learning and research processes.
Indeed, the comprehensiveness of the QA system and the on-line application of its
main instruments, such as surveys and the preparation of self-evaluation reports, implied
the need for a better integration of sometimes autonomous information systems into a
single institutional information system, which, in turn, required well-defined procedures for
the pedagogical management of programmes and course units. The tackling of these
“upstream” constraints had, in practice, a very positive impact on the organisation of the
teaching and learning processes.
A key element for the success in the design and implementation of SIGAQ-UM was the
participative way in which it was developed, in permanent contact with the academic
community through many direct contacts and meetings whenever possible, the
dissemination of information and personal support from the SGAQ to answer questions
and difficulties in the application of the QA instruments. However, it is not easy to reach all
the intended recipients, particularly students and some academics. So, especial attention has
been paid to internal communication mechanisms, namely in the upgrading and proper use
of institutional mail boxes. A comprehensive institutional communication policy under
preparation in the Rectors Office may contribute to the adoption of better procedures for a
dynamic dissemination of information.
Clarity of language in the definition of concepts, drafting of procedures and preparation
of forms for self-evaluation reports also raised some problems and required permanent
attention. Along the development of SIGAQ-UM several clarifications and simplifications
were introduced to improve the understanding of the QA instruments by all those involved
in their use. A paradigmatic example was the revision of the section of the quality manual
dealing with the flagging of course units. In reality, in many sectors of the academic
community the flagging of a course unit was initially seen as a penalisation of the teaching
team and not as a mechanism for systematic signalling of potential problems (whatever
their possible causes) for the purpose of a thorough analysis of the situation and, if
necessary, the proposal of improvement action plans, thus as an opportunity for quality
enhancement. The straightforward change of the expression “unsatisfactory results” into
“results to reflect upon” introduced in the revision of the quality manual helped to somehow
mitigate the problem and provide a better understanding of the flagging mechanism.
As said earlier, the composition of the QA follow-up committee favours the vertical
articulation of SIGAQ-UM both upwards and downwards throughout the university.
 IJQRM However, the role of the “quality promoters” in the local coordination and support of QA
34,2 mechanisms is still heterogeneous in the different organic units, contributing to a larger
than desirable centralisation in the operation of the system. The added value of the
Committee’s composition is therefore still liable to a better exploiting.
UMinho is aware of the danger of QA systems slipping into routine forms of bureaucratic
compliance with established procedures instead of contributing towards the attainment of a
290 culture of quality, conscious that the focus on continuous improvement ultimately
depends on the attitude and true involvement of each actor. So, particular care is taken to
further the awareness of all stakeholders for the importance of the quality system and to
foster their committed participation in the QA mechanisms. The system streamlining, the
emphasis on self-reflection and the monitoring mechanisms in place are also important
safeguards for the system’s efficacy.
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

5. Perceptions on impact
The impact of SIGAQ-UM on the pedagogical component of the institutional mission is
particularly visible in aspects related to the organisation and coordination of teaching, as
mentioned before. In effect, the integrated approach of all the dimensions and aspects
connected with teaching, including the integration of the different areas of UMinho’s
information system, has brought to light the existing problems of articulation between
sectors, as well as issues of information organisation, allowing for the respective
characterisation and treatment, with several visible improvements namely in the quality
and timely availability of the required information.
Likewise, a significant number of action plans for improvement of teaching and learning
are laid out every year as a result of the reflections contained in the self-evaluation reports.
For example, in the academic year 2013/2014 a total of 620 course units (in a total of
3.607 pertaining to 178 different degree programmes) were flagged as requiring further
examination and as a consequence 11 formal action plans for improvement have been
proposed at programme level, from which six were adopted by the corresponding schools
and five were considered as recommendations for improvement not requiring a formal plan.
Comparing with the academic year 2010/2011, 115 action plans were then proposed by the
degree programme coordinators and 72 were adopted by the school bodies, meaning that
most action plans have been really effective and the most problematic situations have been
dealt with, particularly those concerning pedagogical organisation and restructuring of the
educational offer, although self-assessment reports at course unit, programme and school
levels continue to include recommendations for improvement, most relating to flagged
situations, having in mind the continuous enhancement of teaching and learning, from the
schools’ annual reports it is apparent that Pedagogical Councils are becoming more
interventional on QA issues as a result of the operation of SIGAQ-UM.
As regards research, an important added value of the strategy defined for its QA lies in
the fact that the schools and the university now rely on a systematic collection of
performance indicators on research and development activities. The Scientific Councils have
systematic access to the self-assessment reports of the research units, which is bound to
facilitate the definition and development of research policies at school level. This is an
important asset for UMinho, since it has positioned itself as a research university and the
vast majority of its academic staff integrates research centres rated by the Foundation for
Science and Technology as very good or excellent.
Similarly, the hierarchical structure adopted for the preparation and appraisal of
self-assessment reports of services may bring gains in coordination and inter-service
articulation, thus providing improved support for students and for the academic activities.
SIGAQ-UM has gained an important external visibility which impacts positively on
the university’s image. All the structural documents of the system are publically available in
 the website, as part of the open-access policy adopted by UMinho. It is interesting to notice Internal
that the structure of the quality manual has been used as a reference model by several other quality
higher education institutions. assurance
The annual meta-evaluation report on SIGAQ-UM always includes a section on the
system impact, including, for example, concrete information on the main problems system
encountered, the improvement actions planed and undertaken and, wherever possible, the
perceptions of academic staff and students on impact. It also includes an action plan for 291
the system continuous improvement. A recent example is the design of a specific extensive
action plan to improve the participation of students in the QA system, particularly in
responding to questionnaires, which involves a large number of academic actors in its
implementation (SGAQ, 2015).

6. The certification of SIGAQ-UM

Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

In late 2011 the Portuguese Agency A3ES set up an institutional audit process with the
objective to “assess and certify the internal quality assurance systems developed by the
institutions” (A3ES, 2013a) and invited higher education institutions to express their
interest in participating in the first experimental exercise of applying the audit model.
Fourteen institutions responded to the request, but only five met the requirements, namely
the existence of a quality manual formally approved and with a minimum of one year’s
effective application.
UMinho had the privilege of being one of the five selected institutions to integrate the
experimental audit process, which proved to be very useful for the enhancement and
consolidation of SIGAQ-UM, due in particular to the following aspects:
(1) the exercise comprised an intensive and interactive preparation phase, including a
national workshop with the participating institutions and a local seminar which
counted with the participation of a large number of members of the university,
allowing for the clarification of external and internal QA processes;
(2) the preparation of the self-evaluation report on SIGAQ-UM, in particular, showed to
be a good opportunity to involve the academic community and increase its
awareness on QA matters;
(3) the developmental approach of the audit process, which assesses the degree of
development of the different QA processes (including the system as a whole) rather
than their full conformity with rigid externally defined standards, stimulates the
adoption of flexible QA procedures more suitable to the complexities of the teaching/
learning and research processes and their acceptance and appropriation by the
academic community;
(4) the discussions in the meetings with the external assessment team (EAT) during the
on-site visit contributed also to the involvement and awareness of many academic
and non-academic staff members and students, particularly those engaged in
teaching and research coordination bodies; and
(5) the very positive assessment from the EAT, as well as its recommendations
supporting the improvement measures identified by UMinho in the self-evaluation
report, strengthened the motivation for the continuous enhancement of the QA
system and its permanent focusing on quality improvement while keeping the
bureaucratic burden on UMinho as low as possible.
The audit report (A3ES, 2013b) highlights “the existence of a well-defined and documented
institutional policy for quality, which covers the different aspects of the institutional
mission” and considers that the system “has all the conditions to contribute to the
 IJQRM continuous quality improvement of UMinho activities”. In January 2013, following this
34,2 positive assessment from the EAT, the agency issued a full certification of SIGAQ-UM for a
period of six years.

7. Conclusion
The ultimate goal of an internal QA system should be the embedment of a quality culture in
292 all the academic community, “in the understanding that quality assurance activities are
not activities which run parallel to the life of the institution, as the responsibility of only
one or several specific people, but rather the responsibility of each and every member of the
community, in a concern with quality which should be present in all the activities of
the institution” (Santos, 2011. p. 39). Key factors for the development and consolidation of
such a quality culture, as identified in the concluding report of the EUA Quality Culture
Project 2002-2006, are adequate strategic planning, proper organisational structures,
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

adequate internal evaluation processes and feedback loops, committed institutional

leadership, active participation of academic and administrative staff, involvement of
students and of external stakeholders, and an integrated information system for data
collection and analysis (EUA, 2006).
SIGAQ-UM encompasses all these elements for fostering an internal quality culture.
Its certification by A3ES validates UMinho’s institutional quality policy, demonstrates the
system’s substantial stage of development and endorses it as an example of best practice.
Indeed, the EAT audit report not only endorses, as strong points, the system’s
comprehensiveness, coherence, harmony and enhancement-led approach, as well as its close
coordination with the academic management bodies, but also explicitly identifies the
following five areas of best practice recommended for diffusion to other institutions:
deep involvement of university leadership in the SIGAQ-UM; periodical tracking of
SIGAQ-UM to ensure its consistency; the concern in progressive streamlining of the system,
emphasising the practices of reflection among the different actors; the process of quality
improvement in education/learning in the 1st and 2nd cycles, with all procedures associated
with the Course Unit portfolio; and the good integration of the computer system supporting
SIGAQ-UM (A3ES, 2013b).

Note
1. The E4 Group includes the European Association for Quality Assurance (ENQA), the European
Students Union (ESU), the European University Association (EUA) and the European Association
of Institutions in Higher Education (EURASHE).

References
A3ES (2013a), Auditing Internal Quality Assurance Systems in Higher Education Institutions – Manual
for the Audit Process, Agency for the Assessment and Accreditation of Higher Education,
Lisbon, available at: www.a3es.pt/sites/default/files/Manual_for_Audit_Process_0.pdf
(accessed 15 October 2014).
A3ES (2013b), External Assessment Report of SIGAQ-UM, Agency for the Assessment and
Accreditation of Higher Education, Lisbon, available at: www.uminho.pt/docs/sigaq-um/2013/
01/16/relatorio-cae_asigq-12-00011_u-minho_2012.pdf (accessed 15 October 2014).
Amaral, A., Rosa, M.J. and Fonseca, M. (2013), “The Portuguese case: can institutions move to quality
enhancement?”, in Land, R. and Gordon, G. (Eds), Enhancing Quality in Higher Education:
International Perspectives, Routledge, London, pp. 141-152.
Bergen Communiqué (2005), “The European higher education area – achieving the goals”, Communiqué
of the Conference of European Ministers Responsible for Higher Education, Bergen, 19-20 May.
 Berlin Communiqué (2003), “Realising the European higher education area”, Communiqué of the Internal
Conference of Ministers Responsible for Higher Education, Berlin, 19 September . quality
Bologna Declaration (1999), “The European higher education area”, Joint Declaration of the European assurance
Ministers of Education, Bologna Declaration, Bologna, 19 June.
system
CRE (1997), “Institutional review of the University of Minho: CRE reviewers’ report”, Association of
European Universities, Genève.
Crosier, D., Purser, L. and Smidt, H. (2007), “Trends V: universities shaping the European higher 293
education area – an EUA report”, European University Association, Brussels.
ENQA (2009), Standards and Guidelines for Quality Assurance in the European Higher Education Area,
3rd ed., European Association for Quality Assurance in Higher Education, Helsinki.
European University Association (2003), “Graz Declaration 2003 – forward from Berlin: the role of the
Universities”, European University Association, Brussels, September.
European University Association (EUA) (2006), “Quality culture in European Universities: a bottom-up
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

approach, report on the three rounds of the quality culture project 2002-2006”, EUA
Publications, European University Association, Brussels.
European University Association (2007), “Universidade do Minho – EUA evaluation report”, European
University Association, Brussels, available at: www.uminho.pt/docs/avaliacao-institucional-pela-
eua-(2007)-e-follw-up-(2009)/2012/10/15/minho-final-report-eua.pdf (accessed 15 October 2014).
Fonseca, M. (2011), “2010: Acreditação Ano Zero. Os Sistemas Internos de Garantia de Qualidade das
Instituições de Ensino Superior em Portugal”, Agency for the Assessment and Accreditation of
Higher Education, Lisbon, available at: www.a3es.pt/sites/default/files/SIGQ_IES_PT_0.pdf
(accessed 15 October 2014).
Harvey, L. (2008), “Using the European standards and guidelines: some concluding remarks”, in EUA (Ed.),
Implementing and Using Quality Assurance: Strategy and Practice – A Selection of Papers from the
2nd European Quality Assurance Forum, EUA Case Studies, European University Association,
Brussels, pp. 80-85.
Harvey, L. and Williams, J. (2010), “Fifteen years of quality in higher education”, Quality in Higher
Education, Vol. 16 No. 1, pp. 3-36.
Montenegro, I., Morais, N., Santos, I.M., Fernandes, J. and Dias, G. (2010), “Quality culture and quality
assessment in higher education – the experience of University of Minho”, Proceedings of the 13th
Toulon-Verona Conference, Faculdade de Economia da Universidade de Coimbra, pp. 145-158.
Prague Communiqué (2001), “Towards the European higher education area”, Communiqué of the
Meeting of European Ministers in Charge of Higher Education, Prague, 19 May.
Rauhvargers, A., Deane, C. and Pawels, W. (2009), “Bologna process stocktaking report 2009”, Report
from working groups appointed by the Bologna Follow-up Group to the Ministerial Conference,
BFUG, Leuven/Louvain-la-Neuve, 28-29 April.
Reichert, S. (2008), “Looking back – looking forward: quality assurance and the bologna process”,
in EUA (Ed.), Implementing and Using Quality Assurance: Strategy and Practice – A Selection of
Papers from the 2nd European Quality Assurance Forum, EUA Case Studies, European
University Association, Brussels, pp. 5-10.
Reichert, S. and Tauch, C. (2005), Trends IV: European Universities Implementing Bologna, European
University Association, Brussels.
Santos, S.M. (2011), Comparative Analysis of European Processes for Assessment and Certification of
Internal Quality Assurance Systems, A3ES Readings 1, Agency for the Assessment and
Accreditation of Higher Education, Lisbon, available at: www.a3es.pt/sites/default/files/
ESTUDO_SIGQ_EN.pdf (accessed 15 October 2014).
Santos, S.M. (Rapporteur), Gonçalves, L.C., Silva, J.D., Fonseca, L.A., Filipe, A.F., Vieira, C., Lima, M.J.
and Oliveira, M.F. (2006), “Review of the quality assurance and accreditation policies and
practices in the Portuguese higher education – self-evaluation report”, CNAVES, FUP,
ADISPOR, APESP, Lisbon.
 IJQRM SGAQ (2015), SIGAQ-UM – Relatório de Acompanhamento do Sistema (2013/2014), Serviços para a
34,2 Garantia da Qualidade, Universidade do Minho, Braga, maio de 2015.
Sursock, A. (2015), Trends 2015: Learning and Teaching in European Universities, European
University Association, Brussels.
Sursock, A. and Smidt, H. (2010), Trends 2010: A Decade of Change in European Higher Education,
European University Association, Brussels.
294 Westerheijden, D.F. (2001), “Ex oriente lux? National and multiple accreditation in Europe after the fall
of the wall and after Bologna”, Quality in Higher Education, Vol. 7 No. 1, pp. 65-75.
Yerevan Communiqué (2015), “Communiqué of the meeting of European Ministers in charge of Higher
Education”, Yerevan, 15 May .

Corresponding author
Downloaded by Universitas Negeri Semarang At 21:50 14 March 2018 (PT)

Isabel M. Santos can be contacted at: [email protected]

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: [email protected]