Chapter 15
Chapter 15
Chapter 15
Chapter 15
This chapter describes a course on the modelling and validation of concurrent systems based on this textbook which we have been giving at the Department of Computer Science, at Aarhus University. The course uses CP-nets as a formal modelling language for concurrency and exposes students to the benets and applications of modelling for designing and reasoning about the behaviour of concurrent systems. The course introduces participants to the CPN modelling language, its analysis methods, and its supporting computer tools. It also includes a presentation of industrial projects where CP-nets have been used for the modelling and validation of systems. After the course, the participants will have a detailed knowledge of CP-nets and practical experience in the modelling and validation of concurrent systems. The course emphasises the practical use of modelling and validation and has less focus on the formal foundation of CP-nets. The slide sets, CPN models, and suggestions for exercises and projects from the course are available via the Web pages for this textbook. Section 15.1 describes the overall organisation of the course and explains its context in the curriculum. Section 15.2 discusses the intended learning outcomes of the course, and Sect. 15.3 presents the teaching and assessment methods used. Section 15.4 gives an example of a representative student project conducted during the course. Section 15.5 discusses our experiences obtained when developing and giving the course.
K. Jensen, L.M. Kristensen, Coloured Petri Nets, DOI 10.1007/b95112 15, c Springer-Verlag Berlin Heidelberg 2009
363
364
concurrent systems. The aim of the second part is that the participants will gain practical experience in the application of CP-nets and CPN Tools for the modelling and validation of larger concurrent systems. The working methods of the second part are also intended to train the participants to plan and complete projects and to communicate professional issues. The only prerequisite for the course is that the participants must have completed the rst two short introductory programming courses of their bachelors degree studies. These two programming courses correspond to 10 ECTS. This means that we assume that the participants are familiar with conventional programming-language concepts such as variables, types, procedures, and modules. The overall approach taken in the course is to introduce the CPN modelling language in a way similar to that in which programming languages are introduced, i.e., through concrete examples that illustrate the constructs of the modelling language and also the more general concepts of concurrency, synchronisation, and communication. The course is an optional advanced course, and the majority of the participants are in their third to fth year of studies when taking the course. The course usually has 1520 participants. It is important to emphasise that the course is a specialised course on the CPN modelling language and its supporting computer tools. There are several other courses in the curriculum at our computer science department aimed at giving a more general introduction to the theoretical and practical aspects of concurrency. The theoretically oriented courses include courses on automata, concurrency, and model checking introducing the students to labelled transition systems, communicating sequential processes (CSP), the calculus of communicating systems (CCS), and temporal logic. The practically oriented courses include courses on network protocols and internetworking, operating systems, and distributed systems.
15.2 Intended Learning Outcomes Table 15.1 The ve levels of the SOLO taxonomy (see [6], pp. 3940)
365
Level 5: Extended abstract Characterised by verbs such as theorise, hypothesise, generalise, reect, and generate. These verbs represent competences at a level extending beyond what has been dealt with in the actual teaching. Level 4: Relational Characterised by verbs such as apply, integrate, analyse, and explain. These verbs represent competences in orchestrating facts, theory, actions, and purposes. Level 3: Multistructural Characterised by verbs such as classify, describe, and list. These verbs represents solid competences within each topic and a basic understanding of the boundaries of each topic. Level 2: Unistructural Characterised by verbs such as memorise, identify, and recognise. These verbs represent a minimalistic, but sufcient understanding of each topic viewed in isolation. Level 1: Prestructural This is the very bottom level, where no competences have been obtained.
pose of the new grading scale is to measure more explicitly than earlier the extent to which course participants have achieved the intended learning outcomes (ILOs). In Tables 15.2 and 15.3, the verbs that map into the ve levels of the SOLO taxonomy are highlighted using bold italic type. The SOLO level to which a given verb belongs is written in superscript following the verb. For the rst part of the course, seven ILOs, given in Table 15.2, have been dened. These intended learning outcomes express what the participants are expected to be able to do at the end of the course. In the following, we discuss each of these learning outcomes in more detail. ILO1 (constructs and concepts) is concerned with learning the constructs of the CPN modelling language, which include the net structure, the CPN ML inscription language, and the concepts related to hierarchical and timed CPN models. ILO1 also includes concepts such as binding elements, steps, concurrency, and conict. In ILO2 (syntax and semantics), we require the participants to be able to formally dene and explain the syntax and semantics of CP-nets. The purpose of ILO2 is to make the participants understand that CP-nets rely on a formal foundation. When they are introduced to the formal denitions, the participants explore CP-nets from a different angle than the example-driven introduction to the language. In this sense, the formal denitions represent a complementary view of the modelling constructs that can help the participants to consolidate their understanding. ILO2 does not require the participants to be able to formally dene hierarchical CPN models and timed CPN models. The formal denitions for this limited subset of the CPN modelling language can be introduced using simple mathematical concepts. In ILO3 (behaviour of concurrent systems), we require the participants to be able to dene and explain the standard behavioural properties of CP-nets (such as boundedness properties, dead markings, and live transitions) and quantitative performance properties (such as delays, throughput, and utilisation). These concepts are
366
Table 15.2 Intended learning outcomes of the rst part of the course ILO1 Explain4 the constructs and concepts of the CPN modelling language. ILO2 Dene2 and explain4 the syntax and semantics of non-hierarchical untimed CP-nets. ILO3 Dene2 and explain4 properties used for characterising the behaviour of concurrent systems. ILO4 Explain4 the basic concepts and techniques underlying state space analysis methods. ILO5 Explain4 the basic concepts and techniques underlying simulation-based performance analysis. ILO6 Apply4 CP-nets and CPN Tools to the modelling and validation of small concurrent systems. ILO7 Judge4 the practical application of CP-nets to the modelling and validation of concurrent systems.
used when the students work with the analysis methods of CP-nets, which include state space analysis and simulation-based performance analysis. ILO4 (state space analysis) is concerned with the state space analysis methods of CP-nets. Here we require the participants to be able to explain the concepts of state spaces and strongly-connected-component graphs. Furthermore, we require the participants to be able to explain the techniques used to check the standard behavioural properties of CPN models from the state space and the stronglyconnected-component graph. Finally, we require that the participants are able to explain the basic ideas underlying the advanced state space methods. ILO5 (performance analysis) is concerned with simulation-based performance analysis of CPN models. Here we require the participants to be able to explain the techniques underlying simulation-based performance analysis such as workload generation, data collection monitors, and simulation replications. Furthermore, we require the participants to be able to explain the statistical concepts related to discrete- and continuous-time statistics. ILO6 (modelling and validation of small systems) species that the participants must have operational knowledge of the topics taught in the course, i.e., be able to apply the modelling language and the analysis methods in practice. ILO7 (judging the application of CP-nets) requires the participants to be able to determine whether CP-nets constitute an appropriate choice for modelling and validating systems within a given domain, i.e., to determine whether CP-nets are suitable for the modelling of a system and the validation of the properties of interest. For the second part of the course, three intended learning outcomes given in Table 15.3, have been dened. ILO8 (modelling of larger systems) and ILO9 (validation of larger systems) require the participants to be able to model and validate concurrent systems of a size and complexity that appear in real system development projects. ILO10 (discussing application of CP-nets) requires the participants to be able to convey the results of modelling and validation, and issues arising from these results, to colleagues.
15.3 Teaching and Assessment Methods Table 15.3 Intended learning outcomes of the second part of the course
367
ILO8 Construct3 and structure3 CPN models of larger concurrent systems. ILO9 Apply4 analysis methods for CP-nets to the validation of larger concurrent systems. ILO10 Discuss5 the application of CP-nets to the modelling and validation of larger concurrent systems.
We discuss the learning outcomes further in the next section, where we explain how the teaching methods were chosen to support the participants in achieving the intended learning outcomes, and how assessment methods were chosen to measure whether the participants had achieved these outcomes.
368 Table 15.4 Sessions in the rst part of the course Session 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Topic Method
Material
Projects
Why modelling and validation? Lecture Chap. 1 Non-hierarchical CP-nets Lecture Chap. 2 CPN ML programming Lecture Chap. 3 Practical modelling Workshop Exercises Formal denition of CP-nets Practical modelling Hierarchical CP-nets State space analysis (1) State space analysis (2) Practical state space analysis Timed CP-nets Performance analysis Practical performance analysis Industrial applications Lecture Workshop Lecture Lecture Lecture Workshop Chap. 4 Project 1 Chap. 5
Start P1
the participants and are effective in highlighting issues that need to be discussed in more detail and which can then be discussed on demand at the workshops. In this respect the workshops facilitate an interactive teachinglearning environment. The workshops support the intended learning outcomes of the course, in particular ILO6 (modelling and validation of small concurrent systems), but the workshops also facilitate learning outcomes ILO1, ILO3, ILO4, and ILO5 as they stimulate discussions among the participants of the concepts covered. There are three mandatory projects in the rst part of the course: project 1, on modelling; project 2, on state space analysis; and project 3, on performance analysis. The projects start and end as indicated in column Projects of Table 15.4. The projects are conducted in groups of two or three participants and have to be documented in a short 510 page written group report. The rst project is concerned with extending the CPN model of the protocol system shown in Fig. 2.10 to model a sliding-window protocol. The model of the sliding-window protocol must be validated using simulation. The second project is concerned with conducting state space analysis of the model developed in project 1 in order to verify the correctness of the protocol. It is interesting that 5075% of the groups usually discover errors in their design of the sliding window protocol from project 1 errors that were not discovered by means of the simulation conducted as part of project 1. This means that the participants experience at rst hand the power of verication techniques such as the use of state spaces. Finally, project 3 is concerned with analysing the performance of the sliding-window protocol created in project 1 using simulation and comparing it with the performance of the protocol system described in Chap. 12. The three projects must be approved before a participant can enrol for the exam. This ensures that the participants have fullled learning outcome ILO6 (modelling
369
and validation of small concurrent systems) before taking the exam. The exam is a 20-minute oral exam and the participants have approximately one week for preparation for the exam. In the exam, each examinee draws one question, covering ILO15 and ILO7. Table 15.5 lists the topics of the exam questions. Each question corresponds to a chapter in this textbook.
Table 15.5 Exam questions for the rst part of the course Question 1 2 3 4 5 6 7 8 Topic Non-hierarchical Coloured Petri Nets (Chap. 2) Formal denition of non-hierarchical Coloured Petri Nets (Chap. 4) Hierarchical Coloured Petri Nets (Chap. 5) State spaces and behavioural properties (Chap. 7) Advanced state space methods (Chap. 8) Timed Coloured Petri Nets (Chap. 10) Simulation-based performance analysis (Chap. 12) Industrial applications (Chap. 14)
The second part of the course is organised in a different manner, as the main aim is to train participants in the modelling and validation of larger concurrent systems. In this part of the course, the participants conduct a larger modelling and validation project. There is a high degree of freedom in dening the project which is to be done in groups of two to three persons. During the second part of the course there are no conventional lectures, but there are two progress workshops where the groups give a 25-minute oral presentation of the current state of their project. In the rst progress workshop, the focus is on the modelling, and the groups discuss their models with the lecturers and the other participants, who provide feedback. In the second progress workshop, the focus is on the validation part of the project. The project is typically based on a natural-language description of a larger concurrent system. The following is a partial list of the systems that have served as a basis for projects: Distributed le systems. This project was based upon Chapter 8 of the textbook [24]. Dynamic Host Conguration Protocol (DHCP). This project was based upon the IETF Request for Comments document 2131 [31]. Data dissemination protocol. This project was based upon the paper [12]. Dynamic MANET On-demand (DYMO) routing protocol. This project was based upon the IETF Internet-Draft [16]. Internet Key Exchange (IKE) protocol. This project was based upon the IETF Request for Comments document 6306 [66]. Mutual exclusion algorithms. This project was based upon selected algorithms from the textbook [92].
370
PathFinder scheduling mechanism. This project was based upon a description that can be found in the paper [53]. Each year we provide a set of ve to ten project proposals, but participants may also choose other systems as a basis for their projects. Many of the projects have focused on communication protocols and distributed algorithms, but it is possible to choose systems from other domains such as workow systems, manufacturing systems, and embedded systems. In the next section, we give an example of a representative project conducted during the second part of the course. The assessment of the second part of the course is based on an evaluation of a written group report, which is required to have a length of 1520 pages, together with an individual oral exam, where each participant is required to give a presentation of the group project. The nal grade is the average of the grade for the written report and the grade for the oral performance. The act of constructing and validating a larger model supports ILO8 (modelling of larger systems) and ILO9 (validation of larger systems), whereas the progress presentations and the exam support ILO10 (discussing the application of CP-nets).
371
Application Layer
DYMO Layer
Network Layer
Process RREQ
Topology
Process RREP
has to be forwarded to the next destination address, i.e., the next mobile node on the route being established, or the mobile node is the target for the RREP. These two cases are modelled by the accordingly named transitions. If the RREP is to be forwarded, it is put on the place DYMOtoNetwork. If the mobile node is the target for the RREP, the message is put on the place ReceivedRREPs for further processing (not modelled). The CPN model constructed captures a large subset of the DYMO protocol specication. Through the modelling the students demonstrated that they were able to take a complex system (in this case the DYMO protocol) and construct a CPN model at a good level of abstraction (see ILO8, modelling of larger systems). Furthermore, they showed that they were able to divide the CPN model into modules which naturally reected the various operations of the protocol. In the process of constructing the CPN model, the students discovered several ambiguities and missing parts in the DYMO specication, and they used state space analysis to investigate nontrivial issues related to the operation of the DYMO protocol (see ILO9, validation of larger systems). The project was documented in a 20-page written report that introduced the basic operation of the DYMO protocol, presented the CPN model and the assumptions made in the modelling, and discussed the simulation and state space analysis results obtained (see ILO10, discussing the application of CP-nets for larger systems).
372
Application Layer
DYMO Layer
DYMO Layer
Network Layer
Network Layer
Topology
Topology
Fig. 15.2 MANET module: top-level module of the DYMO protocol model
I/O RouteTable
Routing Table rt
SeqNum
I/O SeqNum
(ip, n) p
Forward RREP forwardRREP (p, rt) [#msgtype p = RREP, #targetAddr p <> #destAddr p, #destAddr p = #ip rt]
Out
Network to DYMO
In DYMOPacket
373
organised by the Faculty of Science. Unfortunately, it is typical for all courses at the Faculty of Science that only a few participants ll out the on-line evaluation form. Table 15.5 provides a representative summary of the formal evaluation for one of the years in which the course was given. Altogether there were eight participants who lled out the on-line evaluation form, and each asterisk in a table entry represents the feedback of one participant. This means that a single participant represents 12.5% of the replies, and the evaluation results should therefore be interpreted with some care. Nevertheless, the feedback provides a useful indication of the participants views of the course. The evaluations that we have received are in general very positive. In terms of achieving the course goals, content, and level of interest, the participants are positive. It is also interesting to observe that the participants do not nd the course to be particularly difcult. The participants are expected to spend one-third 1/3 of their study time on the course, which is approximately 15 hours per week, but the feedback shows that they spend less. This is probably related to the participants not nding the course difcult, which in turn may be related to the workshops, where the participants can work on their projects under our supervision. Issues that may arise can thereby be resolved quickly. Participants are also positive with respect to the learning outcomes, the lectures, the workshops, and the textbook. The overall evaluation of the course is also positive. Compared with the old CPN textbooks [60, 61, 63] and the way we taught CPnets earlier, we have added more material on the CPN ML programming language. Mastering the CPN ML programming language is important in order for the participants to be able to apply the CPN modelling language successfully to the modelling and validation of concurrent systems. We have made the deliberate choice of introducing CP-nets directly without rst introducing ordinary Petri nets (e.g., Place/Transitions Nets). The main benet of this is that it enables us to use realistic model examples from the very beginning of the course without having to model data manipulation in an unnatural way using the net structure. Demonstrating that realistic examples can be modelled using relatively simple CPN ML constructs is a factor which contributes to the motivation of the participants. Our teaching activities rely heavily on the integrated use of CPN Tools. This choice is deliberate as it is, in our view, a very motivating factor for the participants and it encourages the participants to work actively with the topics. A key characteristic of CP-nets is that the language has few but powerful modelling constructs. This is an advantage from a teaching perspective since there are relatively few concepts that have to be introduced and mastered. It is also to some extent a disadvantage in practical modelling, since certain parts of systems cannot be modelled in a straightforward, natural way. A further development of the CPN modelling language and CPN Tools to include constructs such as queueing places, capacities, and module parameterisation is therefore of interest also from a didactic perspective and would improve its convenience of use for modelling. The rst part of the course relies heavily on the protocol model that we have used as a running example. In the second part of the course, we have observed that it takes some efforts from the participants to get started on their own modelling project,
374 Table 15.6 Summary of on-line participant evaluation To a very large extent Were course goals achieved? Did content match description? Was the course interesting? Was the course difcult? Hours spend <4 58 *** Very good Learning outcomes Lectures Workshops Textbook Overall evaluation * * * * * Good ****** ***** *** ****** ******* * ** * To a large extent ******* ***** ***** * ** ***
To some extent
To a lesser extent
Not at all
****
1316
1720
Bad *
Very bad
which is concerned with a different system and sometimes lies within a different application domain. In courses with sufcient time it is therefore recommended that additional examples of CPN modelling should be integrated in the course. A number of small and medium examples can be obtained from the CPN Tools Web pages, and a list of larger examples from the literature is available via [40]. Altogether, this can give participants a broader perspective on CPN modelling and validation. This is useful when the participants are working on their nal project, and hence are facing the challenges of modelling a larger system. It also contributes to ILO7 (judging the application of CP-nets). As described above, we have recently adapted the theory of constructive alignment and the SOLO taxonomy [6] for describing course aims and learning outcomes of the course at our department. This has not prompted major changes to the way the course is being taught, but it has been very helpful in making the learning outcomes of the courses much more explicit than earlier. In our opinion, the SOLO taxonomy and constructive alignment provide a very good and practically applicable framework for reecting upon the teaching and assessment methods used in a course.