Adaptive Systems: from intelligent tutoring to autonomous agents1

David Benyon* and Dianne Murray+ ABSTRACT Computer systems which can automatically alter aspects of their functionality or interface to suit the needs of individuals or groups of users have appeared over the years in a variety of guises. Most recently attention has focused on intelligent interface agents which are seen as specialised, knowledge-based systems acting on behalf of the user in some aspect of the interaction. Similar requirements for automatic adaptation have been noted in intelligent tutoring systems, natural language systems and intelligent interfaces. This paper brings together the research which has emanated from a number of backgrounds and provides a unifying perspective on adaptive systems in general. We present an architecture for adaptive systems and a methodology for their development. The paper also describes software support for producing adaptive systems and offers some experimental evidence to justify both the desirability and feasibility of exploiting an adaptive system approach to human-computer interaction. keywords: adaptive systems, intelligent interfaces, user models, domain models, autonomous agents, adaptive system architecture, development methodology 1. INTRODUCTION

The concept that computer systems should be capable of adapting themselves to suit the needs either of individuals or of different classes of users is an apparently attractive, if slightly unusual, one. Adaptive computer interfaces have been discussed over the past ten years or so and have had a number of advocates (Edmonds, 1982; Innocent, 1982; Zissos and Witten, 1985) but it is only now that consideration of how to build such systems is gaining prominence (Kobsa and Wahlster, 1989; Browne, Totterdell and Norman, 1990; Hancock and Chignell, 1989, Sullivan and Tyler, 1988; SchneiderHufschmidt, Kuhme and Malinowski, 1993; Gray, Hefley and Murray, 1993).
1 * +

Some parts of this paper have been published in Benyon and Murray, 1993 Computing Department, Open University, Milton Keynes, MK7 6AA, UK

Social and Computer Sciences research group, Dept. of Sociology, University of Surrey, Guildford, UK page 1

Recent interest in the development of multi-agent systems (Maes, 1991; Laurel, 1990) and intelligent interfaces (Chignell and Hancock, 1988) have also focused attention on this important area. Early applications of the adaptive system concept have been rather disappointing and problems have proved far harder to deal with than was first expected. Will the dream of interface agents go the same way? In this paper we provide a unifying perspective on adaptive systems which facilitates the comparison of different systems and offers guidance on the circumstances under which software designers should consider an adaptive system approach to human-computer interaction design. After briefly considering the arguments for and against adaptive systems in this section, we survey existing applications of the concept (Section 2). This illustrates the range of problems which software developers have sought to solve using an adaptive system approach. Although there are many differences in the design of such systems, we believe that they all share a common architecture. This is presented in Section 3. In the following section we consider a methodology for the development of adaptive systems within the framework provided by the adaptive system architecture. The next section discusses our own experience of developing specific adaptive systems and serves to illustrate in concrete terms the more abstract nature of the discussion of sections 3 and 4. The final section (Section 6) offers our conclusions as to the likely developments of adaptive systems over the next few years. One of the aims of this paper is to provide a reference model for the comparison of adaptive systems. Our intention is to survey the field and to draw together the numerous efforts which have been expended. We believe that much can be learnt from comparing and contrasting the philosophies and experiences emanating from areas as apparently diverse as intelligent tutoring systems and agent-based human-computer interaction. Emerging from this experience is a discipline which promises to impact not just human-computer interaction, but also on the multiple interacting agent systems which we see as being a major development in HCI and computer science over the next few years. 1.1 Why should we have adaptive systems? Computer systems can be difficult to learn and once learnt, may be easily forgotten. As the proliferation of computer applications and different delivery platforms continues apace, the number of individuals exposed to the vagaries of a range of computer environments likewise continues to grow. Computer applications tend to embody specific characteristics which make the chosen design solution better suited to some users than others. However, many systems could potentially have a range of different features to match the diversity of user populations. Even when practised users learn one system well, the software industrys predilection for development of new features, bug-fixes and production of new (i.e. slightly different and improved) versions (Thimbleby, 1990) means
page 2

much interface functionality is continually changing. There is also the problem of users changing their perception of and proficiency with different software systems. Discretionary users are those users who have the option not to use a system. They must be persuaded or otherwise enticed into making use of the facilities. Once discretionary users are convinced that a system is worth using, they may alter their working practices and become reliant on the system. As with novice users, they soon become too constrained by the features of the original system. We need to provide such users with effective methods of graduating to more complex use of systems. Some designers do, of course, implement systems to cater for different users, environments and working practices. Some build different systems for different markets. Others allow individualisation by means of customising facilities or 'macros'. The drawback with such approaches is that the user must learn functions which are tangential to their main task. Although some routines will only have to be set once, others involve learning specialised commands. Tailoring facilities are typically very general, do not take finegrained, individual differences into account and do not cater for a users task needs, either perceived or implicit. Although users may learn and become committed to the use of a piece of software and so are no longer discretionary with respect to the system itself, they are still discretionary with respect to customising the system to better suit their needs. The use of metaphor and analogy as a means of making system functionality accessible to user populations has been claimed to overcome many usage problems. However, this can be a hit-and-miss affair, well-suited for some but not for the population as a whole, dependent as it is upon a closely shared appreciation of the basis of the metaphor. Recent criticism (Kay, 1989) of metaphor in interface design adds weight to this argument. Adaptive systems are systems which can alter aspects of their structure, functionality or interface in order to accommodate the differing needs of individuals or groups of users and the changing needs of users over time (Benyon, Innocent and Murray, 1987). Adaptive systems seek to take over the burden of tailoring systems to individuals and groups. One important question, of course, is whether this goal is feasible technically, operationally or economically. For example, Thimbleby (1990) argues that only complex systems can benefit from an adaptive capability and that, being complex, it is not possible to provide such a capability because the user patterns of usage will be weaker. He asserts that there is simply no enough bandwidth in the user interface to accommodate the required functionality and adaptation. Similar arguments can be made regarding the cost of building an adaptive capability into an application and the technical problems concerned with speed of processing, knowledge representation and so on. Not only has it been asked whether systems can actually incorporate enough suitable knowledge about an individual user in order to make adaptive responses to that person a viable proposition, but the basis upon
page 3

which user characteristics can be inferred has been questioned (Kobsa, 1990; Pullinger, 1989). We do not accept that the feasibility argument can be theoretically proven. A very simple adaptive mechanism may be highly effective. The cost associated with the implementation of adaptive systems can be justified if they significantly improve usability and the quality of interaction. Furthermore, the inclusion of an adaptive capability may not be such a large overhead if it arises as a natural consequence of better attention and metrics being applied to interactive system design. One of the objectives of the work reported here has been to develop cost-effective software support for adaptive systems development. The feasibility of adaptive systems remains a usability issue (Benyon, 1993). Suffice it to say at this point that we strongly believe that computer systems and interfaces to computer applications can be made to adapt to suitable and identifiable user characteristics, to the benefit of the user in terms of effectiveness of performance, speed and accuracy of execution and personal satisfaction, and that appropriate characteristics can be identified, inferred and tested. We propose that some systems must be inherently adaptive if they are to fulfil their purpose and that many systems which have to deal with variety, of users or of environments, may be effective as adaptive systems. 2. APPLICATIONS OF THE ADAPTIVE SYSTEM CONCEPT

Benyon and Murray (1993) provide a detailed review of adaptive user interfaces. Other useful reviews are provided by Kok (19xx) and Norcio and Stanley (1988). One of the problems with the area is that similar ideas have emerged from different disciplines. As is often the case, different disciplines employ their own terminology which make comparisons and generalisation difficult. The classification below has been chosen to reflect the different historical backgrounds which contribute to adaptive systems research. Systems which are described as intelligent may take many forms and are built for many reasons and to many different ends (Elkerton, 1987; Mason and Edwards, 1988). Claims, however, have in the past been somewhat overstretched and few systems have fully lived up to their grandiose expectations. Future systems may well tackle smaller, more manageable areas and deal with issues which are more tractable. 2.1 Intelligent Interfaces Intelligent Interfaces emerged as an important theme in the UK Government funded Alvey initiative in the early to mid-eighties where the term Intelligent Front End (IFE) was preferred. An IFE was then conceived of as an interface to an existing piece of software which would be accessed by a large number of diverse users. The IFE would provide a usable interface to packages which were too complex or too technically daunting to offer easy interaction to such a variety of users (Bundy, 1983; Innocent, 1982; Benyon, 1984). The Monitor system (Benyon, 1984, Benyon and Murray, 1988),
page 4

although implemented in an intelligent tutoring system domain, focused on the structure of adaptive systems in general. The work provided an architecture for adaptive systems which included an explicit representation of individual users and an explicit representation of the domain. This was qualitatively different from other adaptive systems which provided adaptation according to various mechanical or statistical means (Furnas, 1985; Greenberg and Witten, 1985) and which included a limited and implicit user model. There was no attempt to maintain a long-term representation of user characteristics. More recently intelligent interfaces have been applied to information retrieval (Chignell and Hancock, 1988; Branjik, Guida and Tasso,1990). 2.2 Natural Language systems A significant strand in the investigation of adaptive systems has been the extensive attention paid to Natural Language (NL) interfaces, which provide a tough focus for research into aspects of plan, goal and belief recognition. Many systems are simulations or representations of small subsets of an entire dialogue, but they serve to illustrate the difficulties of inferring users intentions and future actions from dialogue only. Related research strands such as speech act and discourse theories, decision theory and rhetorical structure theory have been employed, but the problems remain difficult to overcome, even when only considering written and not spoken interaction. Natural language systems adapt by generating text appropriate to the particular query and characteristics of individual users. To do this they have to infer the users needs and focus of attention from the (ambiguous) use of natural language. Anaphoric references (the use of words such as 'it', 'that', etc.) and ellipsis (where information is missing from a statement) offer difficult syntactic problems, but inferring the semantics of an utterance and the intention which the user had in making that utterance are even more intractable problems which have generated a wealth of research studies in both AI and Computational Linguistics. However, it is now becoming more favourable to consider that NL systems must also include some form of model of the user or dialogue partner in addition to algorithms and heuristics for inferencing and mechanisms of domain knowledge representation. Wahlster and Kobsa (1989) provide several examples of recent systems and current research projects which are more pertinent to HCI concerns. 2.3 Intelligent Tutoring Systems The rationale of computer-based teaching systems is that, for given students and topics, a computer system can alleviate the variance of human-based teaching skills and can determine the best manner in which to present individually targeted instruction in a constrained subject domain. In order to minimise the discrepancy between a users knowledge state and the representation of an identified experts knowledge (a goal state) the ITS must
page 5

be able to distinguish between domain-specific expertise and tutorial strategy. A computer coach assesses the current state of a learners knowledge (Dede, 1986 covers this area comprehensively) and provides instruction tailored to that individuals training needs within an overall context of courses, syllabi and tutorial objectives (Self, 1987). ITS are expert-based systems which must be able to recognise errors and misconceptions, to monitor and intervene when necessary at different levels of explanation, and to generate problems on a given set of instructional guidelines. A student model of the user of an ITS system stores information on how much the student knows about concepts and relationships which are to be learnt and about the students level and achievements. It often contains a history of task performance and some detailed representation of the state of an individuals knowledge in a specified subject area. Some of this may be held in the form of a user profile and can have other uses in management and score-keeping. Student models are explicit representations of a state of knowledge of a specific domain, possibly with individual personal information to maintain records of performance and achievements. Other models - the Tutor model and the Expert model - have very specialised functions. The Expert model is a representation of the knowledge to be imparted, together with the explanatory facility for remedial intervention. It may also contain an error recognition and evaluation feature, and problem-solving model. The Tutorial model arranges teaching strategy, initiates remedial actions and monitors and assesses performance in conjunction with the evaluative function. Problem-generation from the knowledge-base will offer a sequence of problems, adapting the difficulty level on the basis of previous performance, but will also present new problem classes and review or practise already known items. An interface controller arranges the relevant output and accepts student input. The distinction between a student model and other types of model hinges on the use to which the model is put and its location within the interface management system. John Self (Self, 1985) classified the types of knowledge which determine the overall performance of a teaching program as; knowledge of how to teach (which includes knowledge of students in general), knowledge of what is being taught and knowledge of who is being taught (i.e. knowledge of one student in particular). This corresponds to the Tutor model, the Expert model and the Student model. The integration of the Expert and Student models is crucial to the provision of generative, or adaptive tutoring systems (that is, those in which dialogues and teaching frames are not predetermined but, rather, are generated interactively on the basis of an evaluative function operating on the Student model). The field of ITS encompasses many detailed considerations such as the teaching strategy (e.g. intelligent computer coach versus the Socratic tutor, discovery-learning versus guided discovery-learning and task-centred
page 6

learning-by-doing). They are adaptive systems in that they aim to tailor the tutoring to the needs of individual learners. Historically they have attended principally to ensuring that a student attains a certain level of competence in a well-specified domain. This is measured by applying theories of learning instantiated in concrete examples and comparing performance to that of an expert. 2.4 Intelligent Support Systems Another popular application intelligent interface systems is in the provision of context-dependent active help (Fischer, Lemke and Schwab, 1986; Finin, 1983; Hansen, Holgaard and Smith, 1988; Bronisz, Grossi and Jean-Marie, 1989; Stehouwer and van Bruggen, 1989). On-line help systems track the users context and incorporate assistant strategies and a set of action plans in order to intervene when most appropriate or when the user appears to be having difficulty. Intelligent help systems share some characteristics with ITS since a diagnostic strategy is required to provide the most appropriate help for that user in that particular situation. However, they also have to be able to infer the user's high level goal from the low level data available in the form of command usage. Various strategies and approaches have been suggested. (Fischer et al., 1986; Fischer, Morch, and McCall, 1989; Chin, 1986; 1989; Mason, 1986; Jerrams-Smith, 1985). Intelligent help has further developed into critiquing systems (Fischer, 1989), where users must be competent in the subject domain being critiqued, rather than being tutees or learners (Moore and Swartout, 1988; Fischer, 1987; Fischer, 1989). A major problem with these mixed-initiative and co-operative dialogue is concerned with who has the controlling hand. If an assistant or critic offers a piece of advice that an individual overrides, how can that assistant behave sensibly while helping with the remainder of the same task, especially if the task is repeated with slightly different data (as in medical expert systems, for example). If the assistant is an intelligent computer system, how can it avoid giving the same bad advice over and over again and actually learn from the user? The system must have knowledge of how to be a competent assistant by taking into account those things which the user knows best. 2.5 Explanation Systems A similar but slightly different strand has become apparent in recent research - that of being able to provide an explanatory facility for the behaviour of the system (e.g. Paris, 1989); Cohen and Jones, 1989). Expert systems have been criticised for failing to provide adequate and suitably-tailored explanations (Steels, 1987) and to be effective these systems must tailor their explanation to the assumed knowledge of the user (Kass and Finin, 1988; Carroll and McKendree, 1988). An explanation-based system combines the problems of NL generation, help and tutoring and presents extremely stubborn problems to researchers and to system builders. Lehner (Lehner, 1987) suggests that the
page 7

interaction between a user and an expert-system is actually a situation in which two problem-solvers co-operatively attempt to solve common decision problems. Of course, each will have different decision processes, make use of different heuristics and operate on divergent, if fundamentally identical, data. 2.6 Co-operative Intelligent Agents Recent interest in computer supported co-operative work (CSCW), distributed artificial intelligence (DAI) and HCI have taken the adaptive system concept further and led to the development of interface agents. Co-operative systems require models of all the systems and humans participating in the interaction (Seel, 1990). Agents are entities capable of voluntary, rational action carried out in order to achieve goals and holding a representation or belief in the state of the world. They come to hold these beliefs through existing data and by deriving new belief from interaction with external sources and as a result of internal reasoning. Issues which become evident here are those concerned with a number of interacting agents. In more complex systems agents may be interacting in a number of ways, modelling other agents and adapting and responding to a variety of needs. Multiple agent systems incorporating surrogates to filter routine tasks and provide personalised theatrical metaphor interfaces are cited as the wave of the future (Negroponte, 1989) and simple agent-based interaction is a prerequisite of co-operative dialogue, between human or computer as in the interface agents which provide expertise, skill and labour described by Laurel (Laurel, 1990). Such developments can be seen an extension of dialogue assistants (Alty and McKell, 1986; Alty and Mullin, 1987; Coutaz, 1989). Essentially agents are adaptive systems, but systems which are specialised and know about only a very small part of the world. Much of the work presented here is highly relevant to the design of agent-based systems. An important issue for cooperative or support systems is, firstly, how knowledge is actually represented (both task specific details and knowledge about the user or co-operative partner) and, secondly, how the interaction or conversation can be realised. Co-operative support systems can also be thought of as task-orientated dialogue systems, which are actually characterised by the conversational roles which each partner is expected to adopt. Active dialogue partners in mixed-initiative dialogues are those which try to identify a users intentions in order to exhibit co-operative behaviour. To behave co-operatively, the system must discover the presumable plans underlying the users question or statement; represent those plans in its knowledge base; examine them for hidden obstacles and provide information which overcomes those obstacles. In keeping with the needs of ITS, active help systems and more general intelligent support systems, agent-based interaction requires the computer partner to maintain explicit models of a users goals and plans and to have sufficient information about prior knowledge and about known
page 8

misconceptions or errors in the users current state of knowledge. Implementations of intelligent support systems must cope with problems of scale in deciding upon relevant and irrelevant information and with uncertainty in time relationships or object selection. There must also be mechanisms in order to allow the planner to recover when assumptions are unjustified or conditions change. 2.7 Summary Although the system categories described are quite different and work has taken place from different disciplines, with the researchers employing different techniques and specialised language to explain their concepts, they seem to share an underlying architecture. All are adaptive systems in that they automatically alter aspects of the system to suit the requirements of individual or groups of user - or more generally to suit the needs of other agents in the system. All have to infer characteristics of the other agent from the interaction. Work on active help, ITS, NLS, explanation-based and cooperative agent systems contribute to the confusion because they describe systems from distinct perspectives. We need to consider how adaptive systems are designed and how the adaptive functionality can be incorporated into existing architectures. 3. AN ARCHITECTURE FOR ADAPTIVE SYSTEMS

In this section we present a description of the components which all adaptive systems must have. Clearly the complexity of the system and the requirements of the application have an impact on the detail contained in each component. The advantage of developing a generalised architecture for adaptive systems is that it enables researchers to talk in the same language, to compare different systems and to develop appropriate representation techniques. In order to provide a perspective for the following discussion, we may represent the overall structure of an adaptive systems as shown in Figure 1. An adaptive system has a model of the system with which it is interacting. Often this other system will be a human and hence we refer to this representation as the user model. An adaptive system also includes some representation of the application which is to have the adaptive capability. This is the domain model. The interaction of user and system is described in the interaction model. Each of the three components of Figure 1 will be considered in detail, relating their contents to the typology of adaptive systems identified in Section 2. Our aim here is to draw together the strands of research identified in Section 2 and to concentrate on an elaboration of the components of an adaptive system. We do not concern ourselves with the run-time issues which are central to user interface management systems (UIMS), nor to the details of various implementations. We seek to develop a conceptual model of adaptive systems. The design-time issues which are the concern of such
page 9

devices as user interfaces design environments (UIDEs), user modelling shells and adaptive system development environments are discussed in Section 4. Model of the other system. The 'User Model' Model of the system itself. The 'Domain Model'.

Model of the user-system interaction. The 'Interaction Model'

Figure 1 overview of an adaptive system 3.1 The User Model A user model is a representation of the knowledge and preferences which the system believes that a user (which may be an individual, a group of people or a non-human agent) possesses. It is separable by the system from the rest of its knowledge and contains explicit assumptions about the user. The user model is used to provide adaptivity either by intervention or by co-operative agreement with a user. Providing adaptive functionality requires that a user model controls an inference engine and often implies that it samples user behaviour. It may infer perceived goals and courses of actions and act upon such decisions, altering features of the interaction to meet the task and personal needs of individuals. User models may be highly pragmatic in that they represent only what is required in order to facilitate the required adaptation. Other user models may be orientated towards a realistic description of the user. Murray (1987a; 1987b) refers to these as Embedded User Models (EUMs). In contrast to mental models or designers models, which are intangible, user models are representations of user features and decisions which are accessible by the system software. The knowledge represented in the user model may be acquired implicitly from inferences made about the user or it may be explicitly elicited from the user. Explicit acquisition may be achieved through some co-operative behaviour such as asking relevant questions. Alternatively, the user model may be fed with previously stored information, held on whatever medium may be appropriate or transportable. The use of user records, profiles or scores in a personal data store such as a smart card is one mechanism to allow fullscale use of adaptive systems with explicitly acquired data (Murray, 1989). This, however, highlights the need for absolute determination and ratification of some of the moralistic problems we discuss below.
page 10

Knowledge for the user model can be acquired implicitly by making inferences about users from their interaction, by carrying out some from of test, or from assigning users to generic user categories usually called 'stereotypes'. The notion of user stereotypes derives from the work of Elaine Rich (Rich, 1981; 1983; 1989). Stereotypes represent a structured collection of traits or characteristics, stored as facets, to which is attached a value, and optionally a confidence-level and rationale. Some traits are triggers and have an attached probability rating which can mediate or inhibit firing of a whole stereotype. They can be used to 'provide a way of forming plausible inferences about yet unseen things on the basis of things that have been observed' (Rich, 1983). Stereotypes model users on a variety of dimensions and represent characteristics of users in a hierarchy. At the top of the hierarchy is the any person stereotype which defines the characteristics relevant to all users of the system. All stereotypes at lower levels in the hierarchy may inherit the characteristics of this stereotype. Lower level stereotypes will depend on the application, but retain the property of inheriting characteristics from parents. At the bottom of the hierarchy is the individual who may inherit characteristics from a large number of stereotypes. One of the problems with such a representation is to decide just what happens when conflicting characteristics are inherited. Conflict resolution rules must then be included to deal with such situations. Other mechanisms may be employed to infer user characteristics. In terms of our adaptive system architecture, it is the interaction model which is primarily responsible for acquiring implicitly knowledge about users. We deal in detail with these mechanisms in Section 3.3. Rich (1989) describes the space of user models using two dimensions. The first, canonical versus individual, describes whether the model is of one single user or a collection of models of different individuals. A canonical model represents the 'typical' user and is not usually stored explicitly in the system. It is the designer's model identified earlier. Rich also distinguishes long-term models from short-term models. Long-term models are representations of fairly stable user characteristics such as expertise or interest. Short-term models are employed in transitory problem-solving behaviour for the specific task in hand, focusing on specific topics and goals in the immediate interaction. This distinction is also referred to as local (short-term) user models versus global (long-term) user models. Sleeman (1985) provides a more detailed description of short-term individual models with reference to ITS. He describes student modelling techniques as scalar, ad hoc, profile, overlay and process. The profile model encompasses a stereotype model while the others correspond to descriptions of the mechanisms of modelling student knowledge and inferring the state of that students learning. Overlay models are particularly pertinent to ITS. Here the expert's knowledge of the domain is represented and the student's knowledge is captured as a
page 11

subset of that knowledge - overlayed on the expert's knowledge. In this way, aspects of the expert's knowledge which the student does not possess stand out. The problem with this approach is that the student may believe things about the domain which the expert does not. Perturbation models (Kass and Finin, 1989) can be used to represent beliefs which are outside the expert's view of the domain. 'Buggy' models (Burton 1982) and 'mal-rules' (Sleeman, 1981) may be used to represent common misconceptions which users may have about the domain. Short-term individual, or local user models are of particular concern in natural language (NL) systems. The problems of inferring users intentions or their focus of attention from some natural language statement requires that the system models the relevant part of the preceding discourse. NL systems tend to encode a lot of linguistic knowledge concerning how and where anaphoric and elliptic references occur. Inferring the focus of attention of the user is similarly important in such systems. In these cases long-term user models are not required, but short-term use of language is central. Intelligent interfaces on the other hand have to deal with fundamental cognitive characteristics such as users preferences for particular styles of display, basic cognitive capabilities such as spatial ability and preferred learning styles (van der Veer, 1990; Benyon, 1993b). In these systems longterm, cognitively valid models are vital. Similar cognitive capabilities become central to intelligent support systems and explanation systems (Finin, 1989) where the requirements of the system demand that help or explanations take into account users' existing knowledge and the intention which they have in asking for advice. Autonomous agent systems have the added complication of representing 'second level' beliefs. They must not only infer what they believe the other system believes, but must also base their adaptation on what they believe that the other system believes that they believe. User models may contain one or more of three types of knowledge of the user. Firstly, the user model may hold data about what the system believes the user believes about the domain. It is domain dependent data. Because of the similarity of this data to that held by ITS, we refer to this portion of the user model as the student model. Student model data may be kept at one or more of three levels; the intentional, or task, level, the logical level and the physical level. (The reasons for identifying these levels are explicated in Section 3.2.) The task level describes user goals in the domain. For example, an intelligent interface to an information retrieval domain may need to infer that the user is attempting to discover how many items meet some criteria rather than actually trying to obtain a list of those items. Failure to recognize the intentions underlying some user action will result in a less satisfactory interaction. An intelligent support system may need to infer that a user is trying to display the contents of a directory and not list the content of a file in
page 12

response to a query such as 'how do I use ls to display my directory'. A natural language interface should be able to infer that the user has misunderstood a concept and requires further elaboration of that concept rather than simply responding automatically to some syntactic analysis of the dialogue. A second level of description of user knowledge of the domain is the logical level. Here the system records what it believes the user understands about the logical functioning and the logical concepts embodied by the domain. For example, an intelligent interface should be capable of recognising that attempting to execute a database language 'Select' statement in response to a help system prompt is a logical, or semantic error (given that the help system cannot execute database language statements) rather than a syntactic error of attempting to obtain help on the 'Select' statement. Finally the system records the user's (inferred) knowledge at the physical level. For example. an intelligent help system must recognize when a user understands the semantics of a command but has forgotten the syntax if it is to provide appropriate advice. At each of these levels the user model should record the user knowledge and the user's erroneous beliefs. Domain independent data may be considered either as fundamental psychological data or as profile data. Psychological data is concerned with essential cognitive and affective traits of users and is held in the psychological m o d e l component of the user model. There is an ever increasing body of experimental evidence which confirms that users differ in cognitive skills and personality traits which significantly affect the quality of certain interaction styles and user requirements (van der Veer, 1990; Egan, 1988; Jennings and Benyon, 1992). These characteristics of users are particularly resistant to change by the user and hence are particularly important for adaptive systems. If users find it difficult or impossible to change aspects of their make-up, these are exactly the characteristics to which the system should adapt (van der Veer; 1990, Benyon, 1993b). Spatial ability is a characteristic which a[[ears particularly relevant to HCI (Vicente and Williges, 1988; Vicente, Hayes and Williges, 1987; Egan, 1988), particularly where users have to navigate through the conceptual space of file structures or system modes. We know of no system which adapts to users' affective states, but clearly interface agents will have to represent these characteristics if they are to fulfil their promise (Kay, 1990; Laurel, 1990). Data concerning the background, interests and general knowledge of users is held in a user profile component of the user model. This data is not psychological in nature, but may interact with cognitive characteristics in a number of ways. For example, users with poor spatial ability may be able to deal effectively with an interface style if they have a certain level of experience using that style (Jennings and Benyon, 1992). Knowledge of generic applications is stored in the user profile as is much of the stereotype-inherited data such as being a business traveller (Morik, 1989) or feminist (Rich, 1983).

page 13

Student Model

Psychological Model

User Profile

User Model

Figure 2 Main components of a user model The user model thus consists of the three interlinking components shown in Figure 2. The student model component is created directly from the domain model (Section 3.4). Both the psychological and profile components have to be represented explicitly, preferably using some user modelling software (see Section 5). All aspects of the user model will require considerable prototyping, evaluation and refinement before they capture appropriate and salient aspects of users with sufficient accuracy for a given domain. 3.2 The Domain Model The user model is required in an adaptive system so that it can alter aspects of the system in response to certain inferred or given user characteristics. The domain model is required in order to define the aspects of the application which can be adapted or which are otherwise required for the operation of the adaptive system. Other terms which have been used for this concept include application model, system model, device model and task model. The domain model will serve a number of purposes. Firstly, it forms the basis of all the inferences and predictions which can be made from the user-system interaction. It is important therefore that the model is at an appropriate level of abstraction to allow the required inferences to be made. There may be mechanisms which, as a result of some observed behaviour or stated characteristic, predict that some problem will occur or infer a users attempt to achieve some goal. In order to do make these inferences the system must have an appropriate representation of the domain. For example, in UC the fact that a user knows the command r m is used to infer that the user knows ls as well. This is only possible because UC has a domain model which specifies a certain relationship between the r m and ls commands. In TRACK (Carberry, 1989) the domain model includes sequences of actions (plans) which are required to achieve a particular goal. This model is used to infer the user's goal from their observed actions. In addition to inferences, the domain model forms the basis for all the adaptations which the system can make. The system can only change aspects of the application which are described by the domain model. In HAM-ANS (Morik, 1989), for example, the domain model contains a representation of
page 14

hotels which includes quietness as an attribute. The system exploits this representation in making a recommendation of a hotel to a particular user, by emphasizing that a particular hotel is quiet. This adaptation is only possible because the domain model contains this attribute. In TAILOR (Paris, 1989) the domain model represents two levels of description of components of complex devices such as telephones so that it can provide appropriate explanations for different users. Any system which is capable of evaluating its own actions also requires a domain model. The domain model holds the characteristics of the application which are measurable, so that they can be evaluated for effectiveness against the required criteria. The final use of the domain model is to form the basis of the student model component of the user model. The system needs to record what it believes the user believes about certain aspects of the application. The domain model must describe the system so that it can store data about the user's understanding of the various concepts and functions in the application. The Domain Model consists of one or more abstractions of the system. These abstractions allow the adaptive system to reason about the target application, to facilitate adaptations to other agents, to evaluate its effectiveness and to supply the student model part of the user model with its content. If the system is to be capable of adapting the screen displays then these must be described in the domain model. If it is to adapt the functionality of the system, then the domain model must represent alternative functional capabilities and the relationship between functions. Similarly if we want the system to alter the description of concepts then these too must be modelled. If the system is to infer user goals, then goals and plans must be explicitly modelled. In most adaptive systems the domain model is implicit. The representations are embedded in the system code or are only available after a significant amount of processing. For example, in Grundy (Rich, 1983) the classification of books as suitable for particular types of people can only be obtained from the stereotype user models. There is no explicit representation of the domain. Chin (Chin, 1986) recognised the restrictions of this in the UC system and categorised domain knowledge into stereotypical levels of difficulty. Hence, simple concepts in Unix include the commands r m , ls and cat and the concept of a file, mundane commands include v i and diff. Other commands are classified as complex or esoteric. The benefits to be gained from having an explicit and well-defined domain model are considerable and have long been recognized in AI (e.g. Buchanan, 19xx). A separate domain model provides improved domain independence which means that refining the domain model is much easier. This is most important as it is unlikely that any adaptive system design will have a perfect representation of the domain at the first attempt. A separate and explicit domain model is more easily used for multiple purposes such as providing explanations of the systems behaviour.
page 15

We see the domain model as a description of the application which contains facts about the domain, i.e. the objects, their attributes and the relationships between objects. The domain model is the designers definition of the aspects of the application relevant to the needs of the adaptive system. A central question in constructing a domain model is deciding what level of description should be represented. Since the domain model forms the basis of the student model component of the user model, it is important that the domain model is to a large extent cognitively valid. That is, it should capture a view of the domain appropriate to human information processing. One of the areas to consider a realistic cognitive representation of computer systems is the work on Task Analysis techniques (Wilson, Barnard, Green and Maclean,1988; Diaper, 1988; Payne and Green; 1989, Bsser, 1987). These are formalisms which attempt to represent some aspect of the application. In a critical review of cognitive task analysis, Benyon (1992a) identified three levels of description provided by these techniques. The user has something which s/he wishes to achieve outside the computer system (e.g. produce a letter). This goal is then translated into a number of tasks which have to be performed using a specified device such as a particular word processor (e.g. start the word processor, edit the letter, print it). Tasks are decomposed into a hierarchy of simpler subtasks. These can be further decomposed through several levels until some simple task (Green and Payne, 1989) or unit task (Card et al., 1983) is reached. These simple tasks are those which can be characterised by having no problem solving or control structure component. Simple tasks may be different for experts and novices. For example, the edit task is decomposed until we get to the level of a simple task which is to move the cursor one character forward. This general model of task analysis is presented in Figure 4 and the focus of various task analysis techniques is shown in Table 1. Nielsen (Nielsen, 1986) provides a similar argument concerning different representations. He focuses on the difference between the "invisible layers" (conceptual features) of the application concerning the goals, tasks and semantics of the system and the "visible layers" which are concerned with perceptual and physical features.

page 16

External Task or Goal

Mapping

Device Dependent

(Internal) Tasks Device Independent

Mapping

(Physical) Actions

Figure 4 Generalised Task Analysis Model Goal level GOMS TAG ETIT Payne CLG Nielsen External task Problem space Task level Goals Internal Task level Physical Action level Operators Tasks Internal task Device space Semantic level Syntax and lexical levels Visible layers Methods Actions

Invisible layers

Table 1 Emphasis of some task analysis techniques This model reflects the abstraction/presentation dichotomy of the software architectures (Benyon and Murray, 1993, see also IEEE Software 1989), but also includes the goal or external task level of description. Goals are concerned with what the system can be used for. They are external to the system. A central concern of HCI is how they can be mapped onto the functions which are available in the application. Internal tasks on the other hand are high level, logical descriptions of the activities which have to be done in order to achieve that goal. Tasks are dependent on the design of the system and on the functions which are available. For example, the goal of producing a letter is mapped onto the tasks
page 17

of start the word-processor, edit the letter, print it, etc. However, in some systems in may be unnecessary for the user to start the word-processor if the application has been configured in a particular way. Tasks may be allocated to human or machine depending on the facilities available. The physical actions are the actual keystrokes, mouse clicks, cursor movements and so on necessary to enact the tasks. These are dependent on the style of the presentation. These three levels are echoed in Rasmussen's consideration of mental models and HCI (Rasmussen, 1986; 1987). Based on his experience of technicians and operators in complex systems such as power plants, Rasmussen proposes that five levels of abstraction are required to capture the problem solving behaviour. Models at low levels of abstraction are related to a specific physical world. The physical form of an object is the lowest level used. The next level up concerns physical functions. Humans model these in a language related to their physical properties (e.g. electrical, mechanical, etc.). These two levels relate to the physical actions of the task analysis model (Figure 4) and the presentation level of the UIMS model. Rasmussen's generalised function level involves models of the function of system objects without concern for their physical arrangement. An engineer recognises that some object is a multiplexer, say, and therefore it performs certain functions. The multiplexer may be physically realised in a number of configurations of actual physical components. This level is a description which is unconcerned with the physical arrangement (the syntax) of a particular system. It is the application level of the UIMS or the (internal) task level of the task analysis model (Figure 4). Above the level of generalised function is the level of abstract function. Here the model is expressed in a language which depends on universal laws and symbols and not on the local physical and functional properties. The humans mental model at this level can only be formed by considering the purpose or reasons behind a system structure. At the highest level of abstraction, the system may be modelled in relation to its purpose within its overall environment. These two levels thus concern the goals which the system can be used for - the systems purpose. Rasmussen argues that in human systems higher level functions are naturally derived from the purpose of the system whereas physical systems are responding to the laws of nature. Physical systems are causal systems as opposed to intentional systems. Rasmussen's model is similar to the philosophical arguments of Pylyshyn (Pylyshyn, 1984) and Dennett (Dennett, 1989). Pylyshyn argues that what might be called the basic assumption of cognitive science...[is] that there are at least three distinct, independent levels at which we can find explanatory principles....biological, functional and intentional (Pylyshyn, 1984, p.131, Pylyshyns italics). He relates these terms to those used by other writers. For example, Newell (Newell, 1982) calls them the device level, symbol level and

page 18

knowledge level. Pylyshyn himself prefers the terms physical, symbol and representational (or semantic) level. The levels are distinguishable from each other and necessary because they reveal generalisations which would otherwise not be apparent. A functional description is necessary because different functions may be realised through the same physical states and different physical states may realise the same function. Although Pylyshyn is primarily concerned in this discussion with human cognitive systems, it is apparent that both physical and functional descriptions are appropriate for computer systems also. For example, the physical action of pressing ^D will result in the application performing different functions depending on the system. Similarly, the function of moving a cursor can be accomplished in a variety of different ways. The semantic, or representational level of description is required because certain principles or constraints are revealed. We interpret behaviours of systems not only through function, but also through relating function to purpose - by relating the representations of the system to external entities. The purely functional view of someone dialling 911 (or 999 in the UK), does not reveal that that person is seeking help. There is no simple causal connection between dialling a certain number and the seeking of help, it is a semantic, or representational relation. It is this level - of intentions on behalf of the user of a system - that also needs describing. We need a representation of the goals, beliefs and desires which the system has Dennett also recognizes three levels of description. We can understand the behaviour of complex systems by taking a physical view, a design view or an intentional view. The physical view, (also called the physical stance or physical strategy) argues that in order to predict behaviour of a system you simply determine its physical constitution and the physical nature of any inputs and then predict the outcome based on the laws of physics. However , sometimes it is more effective to switch to a design stance. With this strategy, you predict how the system will behave by believing that it will behave as it was designed to behave. However, only designed behaviour is predictable from the design stance. If a different sort of predictive power is required then you may adopt the intentional stance. This is summarized as follows; 1. Treat the system as a rational agent 2. Figure out what beliefs it ought to have given its place in the world and its purpose 3. Figure out what desires it ought to have 4. Predict that this rational agent will act to further its goals in the light of its beliefs and hence 5. Predict what the agent will do on the basis of what it ought to do. It is clear that the intentional stance provides a useful strategy for understanding humans and for predicting what will happen. It is therefore,

page 19

an important part of the domain model which we require since that model will form the basis of our inferences and predictions of future behaviour. The conclusion of this analysis suggests that a cognitively valid domain model should capture descriptions of the application at three levels which we shall refer to as the
Task level Logical level Physical level.

At each of these levels, the structure (the objects and relationships which exist) and the processing of the application need to be described. A comparison of the various domain representations which we have considered is shown in Table 2. Our model Dennett Pylyshyn Rasmussen Task Analysis User Interface Software Task level Representational level Purpose and Abstract function External Task or Goal Logical level Symbol level Generalised function (Internal) Task Abstraction level Table 2 Comparison of domain levels The task level describes the application from the perspective of what it can be used for and the general components or subsystems which it has. This is the level at which a system is related to external entities, to achieve goals in the outside world. The task level is important because the user needs to be aware that (for example) an e-mail system can be used to send messages and that it can be used for elementary word processing, but that it is not ideally suited to that purpose. Unless users understand the purpose of a system they will be left frustrated by many of its shortcomings. The logical level of the domain model equates with the semantic (in task analysis terms) or functional level. It is the level of generalised functions. The term logical is used here because it emphasizes that in order to achieve some purpose, certain functions h a v e (logically) to be performed and certain objects h a v e to exist. This is the design stance where the system is described in logical functions/concepts; a description which concentrates on how something works. In agent-based interaction this level is particularly important. Logically Physical level Physical stance Physical level Physical form and Physical function. Action Presentation level

Intentional stance Design stance

page 20

something has to be done or some data has to be provided in order to fulfil a purpose (Benyon, 1992b). Whether it is done by human or agent is a design decision. The physical level provides a causal description concerned with how simple tasks are sequenced and how objects and displays are laid out. It is concerned with the presentation of the system, dialogue control and physical actions. The domain model must also describe the mappings between the levels. It does not contain everything about the application. The domain model represents the aspects of the application which are to be used in providing the adaptive capabilities. 3.3 The Interaction Model The third component of an adaptive system is its representation of the actual and designed interaction between user and application - the interaction model. This use of the term is thus very different from the interaction model proposed by Norman (1986) which is a theoretical representation of humancomputer interaction in general. It is much closer to the notion of a discourse model, or dialogue model. However, there is still much debate over exactly what is meant by a discourse model and what type of data it contains (see the discussion in Computational Linguistics, vol. 14(3), 1988). We will remain with the term interaction model and consider its relationship with discourse models in the Section 3.4. An interaction is a user making use of the system at a level which can be monitored. Data gathered from monitoring this interaction can be used to make inferences about the users beliefs, plans and/or goals, long-term characteristics, such as cognitive traits, or profile data, such as previous experience. The system may tailor its behaviour to the needs of a particular interaction or, given suitably reflective mechanisms, the system may evaluate its inferences and adaptations and adjust aspects of its own organization or behaviour. In some other representations (Alty and McKell, 1986) the interaction model is seen as a part of the domain model. However, we like to see the components explicit and separated from other parts of the architecture because of the gains which arise from understanding the roles of the different models. For example, Alty and McKell's application model can trap errors. However, an error can only be understood with respect to some notion of what is correct at some level of description. The definition of an error belongs with the representation of domain concepts which is rightly kept in the domain model. It is the role of the interaction model to identify and deal with errors. There are two main aspects to the interaction model; capturing the appropriate raw data

page 21

representing the inferences, adaptations and evaluations which may occur

Raw data is obtained through maintaining a dialogue history or dialogue record (DR) which is a trace of aspects of the users observed behaviour. The dialogue record kept for as long as is required according to the needs of the adaptive system and is then deleted. It may contain details such as the sequence of keystrokes, mouse clicks and mouse movements made, timing information and system messages. It is an abstraction of the interaction since it cannot capture everything which takes place. A portion of the dialogue record from the Basser data project (Thomas, Benyon, Kay and Crawford, 1991) is shown in Figure 5, with an accompanying interpretation. This project is monitoring the use of the editor sam (Pike, 1986) by over 1000 users at the University of Sydney. The purpose of the dialogue record is to enable inferences to be made about how commands are learnt, how they are used, the difficulties which different groups of users have and so on. The dialogue record contains the mouse movements and other commands used by the users. Mouse commands map directly to menu selections and so the data can be sensibly interpreted. However, it was not possible to record various aspects of the interaction such as the size and shape of windows created or the actual position of the mouse on the screen. Other aspects of the interaction left out of the dialogue record were such information as the command arguments entered. Similarly it was decided to record only the total number of keystrokes and backspaces entered when inserting text. The dialogue record is an abstraction of the actual dialogue suitable for the purpose at hand. Although the dialogue record contains only low level data, it is surprising how much can be inferred if the knowledge necessary is available (i.e. provided by a human or stored in a domain model). For example, the command sequence MdMe (position and cut) indicates a possible error since there is no drag (or highlight) operation which indicates the data to be cut. A timestamp (code T) is output every five minutes which shows the total number of characters inserted. This allows us to calculate how many were typed into a command window even though such data is not recorded directly. McMeMdMeMfMdMcMeMdMdMcMe I2 0 3 Md I1 0 10 MaAlA,A$KsAlA,KsEvAlA,KsEvAlA,KsEvM} T667538030 19 292 M{M{M{M}AlA,A$KcKxKwWf

page 22

Explanation Drag out (highlight) some text Cut Position and cut (NB no drag means no change in the file). Paste Position, drag, cut Position, position, drag, cut. Insert 2 characters, no backspaces in 3 seconds Position and Insert one character in 10 seconds. Select command window Substitution command Substitution command Substitution error displayed Two more attempts with errors Scrolling up and down in command window with timestamp output showing time, consistency and 292 characters typed. global substitution Write out file Warning message occurred

Monitor Output Mc Me MdMe Mf MdMcMe MdMdMcMe I2 0 3 Md I1 0 10 Ma AlA,A$Ks AlA,Ks Ev AlA,KsEvAlA,KsEv M} T667538030 19 292 M{M{M{M{M} AlA,A$KcKx Kw Wf

Codes indicate commands e.g. M} is mouse button 3, Me is select cut from pop-up menu, Ks is command search, addresses e.g. A, is to end of file, warnings, errors, etc.,

Figure 5 Portion of the dialogue record from the Basser data project, with associated explanation. The second part of the interaction model is a description of the 'stereotyped' interaction; the Interaction Knowledge Base (IKB). This describes the inferences that can be made from the dialogue, the evaluations of the interaction which are possible and the changes (adaptations) which the system can accomplish. The user model and domain model define what can be inferred. The IKB actually does the inferencing by combining the various domain model concepts to infer user characteristics or by combining user model concepts to adapt the system. The IKB represents the relationship between domain and user characteristics. It provides the interpretation of the dialogue record.

page 23

For example, given the dialogue record in Figure 5, an attribute in the student model part of the user model KSub = knowledge of substitute command values {1...5} and definitions in the domain model for Ks (substitution command), Ev (substitution error), Kc (change command), Kx (global substitution) the IKB might contain a rule IF KsEv OR KsEv occurs in dialogue > 3 times AND KcKx does not occur THEN KSub = 1 OR KSub = KSub - 1 which infers the level of a users knowledge of the substitute command (by noting the errors resulting from Ks) and updates the user model appropriately. Adaptations may be accomplished in a similar fashion. For example IF KSub < 2 AND KsEv occurs in dialogue THEN SubHelp which calls a routine (SubHelp) to offer advice on the substitute command if users have a low value of KSub (knowledge of substitute command) and if they make a substitution error. Notice that SubHelp and any ensuing interaction now becomes part of the dialogue record. A sophisticated adaptive system may subsequently evaluate its own recommendation by analysing the dialogue record further and performing some check and repair activity (a routine called Check) on the SubHelp routine. IF SubHelp has been called AND KsEv occurs in dialogue THEN Check SubHelp The interaction model is a vital part of an adaptive system. The developer of adaptive systems has to decide the level of abstraction which is required for the dialogue record, the individual user data and the interaction knowledgebase. In our approach, the interaction model must be constrained so that all attributes used in the representation are defined either in the domain model or in the user model as appropriate. Automatic mechanisms are required for this through software support (Section 5). 3.4 Relationship between the models We have described a conceptual structure for adaptive systems which consists of three models; the user model, the domain model and the interaction model as illustrated in Figure 6. Although the discussion has generally been as though there is only one of these in each adaptive system this may not be the case.
page 24

Psycholgical model

profile model

task level

logical level

student model user model

physical level domain model

dialogue record evaluation mechanisms inference adaptation mechanisms mechanisms interaction knowledge base interaction model

Figure 6. Overall architecture for an adaptive system For example, in a given system there may be several user roles and hence several user models. The focus of the system may be about a person. In general there may be a large number of user models representing the individual agents in a multi-agent co-operative system. Similarly there may be more than one domain model. Conceptually our approach is that we want to model users, or agents who are characterised by having one sort of data the psychological model, personal profile and the student model - and the domains which are characterised by having another sort of data - the task, logical and physical levels of description. The interaction model as expressed through the adaptive, inference and evaluation mechanisms may be extremely complex, embodying theories of language, pedagogy or explanation. Morik (1988) emphasizes the difference between the interaction-independent theory of language and the interactiondependent nature of the actual dialogue. This reflects the distinction which we have drawn between the dialogue record and the mechanisms in the IKB which use that data. In general, the interaction model will represent the strategies and theory of the particular type of system. 4. METHODOLOGY FOR ADAPTIVE SYSTEM DEVELOPMENT

We take the view that whether or not a system should have an adaptive capability is essentially a human-computer interaction (HCI) activity. Alternatives to adaptive systems will often exist for system developers including training users in the use of a system, the use of other support mechanisms and customisation facilities. The suitability of each of these solutions will depend on the nature of the system, the environments in which it will be used and the characteristics of its users. Adaptive systems are one solution to usability problems (Benyon, 1993a). However, we do expect
page 25

that incorporating more knowledge into the interface and providing systems with an adaptive capability will become an increasingly attractive option for designers. As an HCI activity, developing adaptive systems shares problems with all HCI development. Fischer (Fischer, 1989) summarises these as; there is no real theory of HCI current life cycle models are inadequate because of the ill-structured nature of HCI problems there is a great degree of design instability in HCI and hence the need to evaluate and re-design systems requirements cannot be fixed and used as the basis for deriving a formal design equivalent

Hartson and Hix (Hartson and Hix, 1989) also emphasize that designers take an alternating waves approach to design rather than a strict top-down approach and that evaluation becomes central. HCI needs to deal with cognitive design issues which do not have any generally accepted notations. These problems of designing effective HCI systems have led various researchers to suggest alternatives. These are generally characterised by a changed view of the systems life cycle an approach to systems development based on prototyping, evaluation and iteration the need for effective software support tools

We concur with the spirit of these suggestions and the current discussion should be seen in this context. However, the fact that text is a linear medium can make descriptions of iterative processes difficult. Readers should bear in mind that the textual description inevitably hides the process of continual refinement and iteration which underlies systems development. We deal with the important role of software tools in Section 5. 4.1 A general approach to HCI development The alternative to the traditional life cycle recommended in (Hartson and Hix, 1989) is a star representation which has evaluation as central. We have changed the labels in some of the components in order to correspond to our terminology which is explained below. The star model is shown in Figure 7, but it is important to remember that all the activities are highly interdependent and that two or more activities may occur at the same time.

page 26

Implement Systems Analysis Prototype Evaluate

Design

Specify User Requirements

Figure 7 The star approach to interactive systems development (adapted from Hartson and Hix, 1989) Systems analysis Systems analysis is the process of understanding the problems of any current system and the requirements for any replacement system. Systems analysis inevitably involves some design considerations. It consists of five interrelated and interdependent activities functional analysis which aims to establish the main functions of the system. data analysis which is concerned with understanding and representing the meaning and structure of data in the application. Data analysis and functional analysis go hand in hand to describe the information processing capabilities of the system (Benyon, 1990) task analysis which focuses on the cognitive characteristics required of the users by the system e.g. the search strategy required, cognitive loading, the assumed mental model etc. Task analysis is device dependent (Benyon, 1991) and hence requires some design to have been completed before it can be undertaken. user analysis which determines the scope of the user population which the system is to respond to. It is concerned with obtaining attributes of users which are relevant to the application. This includes aspects of each of the components of a user model identified in Section 3.1. environment analysis which covers the environments within which the system is to operate. This includes physical aspects of the environment and softer features such the amount and type of user support which is required.

page 27

Specify User Requirements During the development process, user requirements will emerge. There are three aspects to the user requirements; functional requirements describing what the system must be able to do data requirements describing the data which the system is to store and manipulate usability requirements which specify the effectiveness, learnability, flexibility and attitude (Shackel, 1990). Design In line with the arguments presented in Section 3.2, the design of a system can be seen as consisting of three levels of description - task, logical and physical - and the mappings between them. Once the requirements have been obtained at a suitable level of detail, the application can be specified at these three levels. The application may be represented in terms of structure and/or functions at each of these levels and any of a number of notations may be used. the task level describes the (external) tasks, or goals which the system is to support the logical level describes the logical functioning and structure of the application. physical design is concerned with the layout of screens and the construction of icons, etc. (the representational aspects) and with the operational aspects (how objects behave) the task/logical mapping describes the cognitive processing required by users as they translate their goals into the systems functionality the logical/physical mapping describes the consistency, learnability and other aspects of the actual system use.

The transition from logical to physical design is particularly important since it is here that functions are allocated either to the human or to the machine and it is here that an adaptive capability is most likely to be considered. A decision to be made by designers is that if a function is allocated to the user, it will require them to face additional cognitive processing, but if it is allocated to the machine, then the system will require an adaptive capability. Prototyping The feature of any prototype is that it does not have a generalised lifetime. It may be built and thrown away or it may evolve into the final system.

page 28

Prototypes must be quick, easy and cheap to construct and hence rely on the provision of good software tools. Implementation This is concerned with completing the system, programming it in an implementation language, thorough testing and other quality assurance processes, acceptance and completion of the documentation. Evaluation Frequent evaluation is central to the methodology. It may take the form of a very quick discussion with users or a formal review procedure. It may require a lengthy experiment or it may demand the skills of an HCI expert. It is also important to recognize that evaluation is applicable at all stages of development - analysis should be evaluated as well as requirements and design. Evaluation may take place in order to help generate ideas or to clarify details later in the development process and thus be primarily formative rather than summative. 4.2 Adaptive system development Section 4.1 describes the development of any interactive system and the issues which must be addressed. However, adaptive systems differ from other interactive systems in that they are characterised by design variety. Rather than the designer trying to obtain a single solution to a problem, the designer specifies a number of solutions and matches those with the variety and the changeability of users, environments and tasks. At some point in the system development process, the designer may decide that the system requires an adaptive capability. This may be to meet a specified user requirement, it may arise from the analysis or it may happen during the transition from a logical to physical design. Once this option has been identified, the adaptive system must be developed in parallel with the application and using the same development approach. The specification of the adaptive system part of the application requires the specification of the domain, user and interaction models. This in turn requires the designer to focus on what features of the system are to be adaptive, what characteristics of the users it will adapt to and how the adaptations will be accomplished. It may be that the whole system is to be adaptive. In this case, the domain model and the system design are the same thing. However, usually only a part of an application will require an adaptive capability and so it is this part which must be isolated and specified in the domain model. We expect there to be considerable experimentation and refinement during the development of the adaptive component of a system and that software support is vital to the success of adaptive systems (see Section 5).
page 29

An important issue in adaptive system development is ensuring that the adaptivity can be controlled and measured and that the reasons for having an adaptive mechanism are known and clear. We believe that the architecture and methodology presented here contribute to this. However others (Browne, Totterdell and Norman, 1987, 1990) suggest that metrics are necessary in order to control the development process. Their experience of developing a number of prototype adaptive systems in the UK led them to the conclusion that the whole issue of how, where and when to adapt needed to be explicitly stated. They recommend that there should be the following measures; an objective metric which describes the purpose of the adaptive facility a trigger metric which identifies what will trigger the adaptive mechanism an assumption metric which describes the rationale behind the objective metric a recommendation metric which describes what recommendations are possible a generality metric which describes the generality of the findings an implementation metric which relates to the effect that the adaptive mechanism will have on the implementation.

The notion of metrics in HCI is currently receiving much of attention, but is still in its formative stages. We generally accept that metrics are an important development in HCI and that they have a role to play in adaptive systems. However, we believe that it is unnecessary to have separate adaptivity metrics. Adaptivity is a usability issue and can be accommodated by usability metrics. In considering adaptive systems as solutions to interface problems, the designer must consider an appropriate level of adaptivity. Totterdell, Browne and Norman (1987; 1991) examine adaptivity in natural and computer systems and identify four levels of adaptive system. (Simple) adaptive systems are stimulus-response systems or simple rule-based systems. They are characterised by their ability to produce a change in output in response to a change in input. Their adaptive mechanism is hard wired. These SelfRegulating systems monitor the effects of the adaptation on the subsequent interaction and evaluate this through trial and error. A mechanism is required which maintains a history or at least provides immediate feedback. This evaluation mechanism then selects from a range of possible outputs for any given input. Self-mediating systems monitor the effect on a model of the interaction. Possible adaptations can be tried out in theory before being put into practice. Self-modifying systems are capable of changing their representations. This allows self-modifying systems to reason about the interaction.

page 30

In terms of the architecture which we have developed in this paper, we can see the levels of adaptivity in respect of the models possessed by the adaptive system. 'Simple' adaptive systems possess a dialogue record and some adaptation mechanism. They do not have to have explicit user and domain models, though refining the mechanisms is made more difficult if they do not. Predictive interfaces (Greenberg et al., 1991) and many natural language systems fall into this category. The move to a learning system - a self-regulating adaptive system - is significant. Such systems now require inference mechanisms. They have to be able to abstract from the dialogue record and capture a logical or task level interpretation of the interaction. Similarly the system must now include a representation of its own purpose in its domain model. Self-mediating systems require a more sophisticated model of the interaction and of the other system and require evaluation mechanisms in order to determine how effective a possible adaptation may be. Self-modifying systems are metaadaptive systems in that they can change the user, interaction and domain models. An important difference between Totterdell et al.s approach and our own is that they see adaptation as being adaptation to an environment whereas we see adaptation as being to another system. Whereas Totterdell et al. argue that: if X occurs in the interaction then the system will do Y our approach is to say: if X occurs then Y is a characteristic of the other system if Y is a characteristic of the other system then the system should do Z In other words Totterdell et al. do not explicitly model the other system. They only react to the behaviour of the other system. We believe that this behaviourist view of interacting systems is restrictive, principally because it is difficult to transfer effects from one situation to another. By building realistic, humanistic representations of people, we are able not just to make better long-term use of the inferred characteristics, but also discover any human cognitive processing requirements demanded by particular system features. 4.3 Software Support The adaptive system developer needs to consider the opportunities for adaptivity provided by the framework of the three models and their relationships identified in Section 3. Developing the user model forces the designer to focus on the psychological, profile and domain knowledge which users will require. The domain has to be described at the task, logical and physical levels. Specifying the interaction model requires the designer to consider what data is available from the interaction (the dialogue record) and the inferences, adaptations and evaluations which can be made from the dialogue record within the context of the domain and user models.
page 31

As with many aspects of HCI design, the designer cannot be expected to code all the contents of the interaction knowledge base at specify the user and domain models using a low-level language. Indeed our experience with the first project (section x) demonstrated this quite convincingly. Hence, software support became an important and integral part of the adaptive system methodology. The Adaptive Systems Development Environment (ASDE, Benyon, Murray and Jennings, 1990; Benyon and Murray, 1993) is a designers tool-kit which tailors the general-purpose nature of an AI Toolkit to the specific needs of an adaptive system developer and which exploits the UIMS facilities of the underlying system at run-time. At present, we do not seek to develop the self-modifying systems described in Section 4.3. Accordingly, the software does not facilitate the development of such systems. Changing the representations is still the job of the designer. The ASDE is similar to the concept of an expert system shell. During the development phase of building an adaptive system, the developer employs the ASDE to specify the characteristics of the target adaptive system and its users, their interaction and the mechanisms which will guide the inferencing, adaptations and evaluations which are to take place. Once the structure of a particular adaptive system has been established, the developer uses the ASDE to specify the values of relevant attributes of individual or classes of user and the inferences, adaptations and evaluations which are to be made from these values. These are instantiated in the target system with which the user interacts. The ASDE reflects the architecture of adaptive systems described earlier. The domain model describes the structure and functioning of the domain of the target (adaptive) system at the three levels of abstraction and provides the basis of an overlay model which is used as a the representation of the users knowledge of the domain. The student model automatically inherits the domain model. Each user (individual or class) is represented in the adaptive system as a system object. When individuals use the target system, they are allocated to one or more stereotypes on the basis of the knowledge which the system has acquired about the user. The individual then becomes a member of that class and inherits the characteristics of the class. Conflicting classifications can be handled through the conflict resolution mechanisms provided by the AI Toolkit. As the system learns more about the user, so the user model becomes increasingly individual. The adaptation and inference rules are specified through the AI tool-kit's own mechanisms. An important aspect of the tool is that the models are explicit and can be displayed and edited if necessary. This facility is vital for the user model, both in respect of the privacy of individuals and their rights under data protection legislation, and in order to maintain the accuracy of the model. Making the interaction and domain models explicit and separable from the other components of the adaptive system facilitates changing the mechanisms as
page 32

details of the interaction are understood. The designer describes these mechanisms through the ASDE. There are several other software systems concerned with developing aspects of adaptive systems. GUMS (Finin, 1989) is strictly a user modelling system and appears strong on the implementation of user stereotypes. These stereotypes include inference rules (which are part of the interaction model in the ASDE). However, GUMS does not place so much emphasis on the domain modelling or adaptation and evaluation rules, preferring to see these as parts of the user model. Kays system (Kay, 1991) is more wide ranging in that it seeks to provide tools for allowing users access to their user models (for viewing, editing etc.). It also provides an architecture to handle various levels of confidence which the system has in its inferred data. As with the system described here, Kay identifies a conceptual and physical level of knowledge which the user may have of the domain. Kobsas system (Kobsa, 19xx) employs a graphical function/concept tool similar in structure to the domain model described here. The UM-tool described by Branjik (Branjik, et al. 1990) takes a similar approach to modelling users and attaches methods of obtaining information to the frame slots. 5. EXPERIMENTAL WORK

During the first project in 1986 - 1988, we were primarily concerned with the mechanisms and feasibility of adaptation. A small ITS was developed and implemented in GC Lisp. This system directed students through the learning material according to their inferred learning style and level of expertise and their performance during the current interaction session. The dialogue was all predetermined and so navigation could be controlled by directing the user to the next node in the system (Benyon and Murray, 1988). Some pilot experiments with this system (Benyon, Innocent and Murray, 1987; Benyon, Milan and Murray, 1987) confirmed that whilst the mechanisms were effective in that they successfully tracked a user through the dialogue, were able to make inferences from this behaviour and hence were able to adapt the dialogue based on these inferences, the inferences which were made did not correlate with either the users own assessment of themselves nor other more objective measures. The mechanisms were laborious to code in Lisp and this represented a serious developmental restriction. As a result of these experiences, the second project was designed which would tackle three main areas. 1. 2. There was the need to find reliable metrics of cognitive factors which genuinely affected the interaction. We wanted to implement the system in a much richer environment in order to exploit the facilities such as automatic inheritance mechanisms, rule specification, frame creation and user interface management offered by an AI toolkit.

page 33

3.

We needed to attend to the development of a shell type system which would facilitate the creation and amendment of adaptive systems by providing a set of tools to enable rapid and incremental changes to be made. The provision of such a toolkit meant that the architecture of an adaptive system had to be carefully understood and specified.

Items 2 and 3 above have been well described in this paper. The experimental work was aimed at finding more out about item 1 and was undertaken with the following aims. To identify one or more reliable and measurable individual differences which affect the interaction To identify aspects of the interaction which could accommodate these differences and improve the performance of the disadvantaged group To identify a way of inferring which group an individual user was in To develop a mechanism which automatically adapted the system appropriately. 5.1 Pilot experiment

In the first experiment, reported fully in (Jennings, et al., 1991) an on-line shopping catalogue database system was developed. The database contained details of all the items which were available from the catalogue. The user could query the catalogue to find out, for example, if there was a vacuum cleaner costing less than 65. All items in the catalogue had three attributes such as colour, size and cost. Five functionally equivalent interfaces - iconic, button, command, menu and question - were developed for the database. These were chosen as being typical of interface styles which are widely available. Twenty-four users performed similar tasks using each of the interfaces. The time taken to complete a number of standard tasks was recorded and taken as the measure of performance. Each user was tested for five individual differences - spatial ability, verbal ability, field dependency, logical/intuitive thought and short term memory ability - using standard psychometric tests. Those with the top 12 scores on each test formed the high group and those with the 12 lowest scores into a low group. The main results of this experiment are shown in Figure 9.

page 34

A. Spatial Ability 400 task completion time (secs) 300 p<0.1 Q M p<0.01 p<0.1 200 low high C I B 200 low 300 400

B. Verbal Ability p<0.05 Q M C

B I high

C. Field Independence 400 task completion time (secs) 300 400 Q M C B I 200 low E. Thinking 400 task completion time (secs) 300 high 200 300

D. Short Term Memory

Q M C B I low high

Q M C B I

B = button interface C = command interface I = iconic interface M = menu interface Q = question interface

200 low high Figure 9 Main results of pilot experiment

page 35

The results from this experiment suggested a number of interesting things. Since there were five different interfaces, those on which users achieved a similar performance indicated something similar about the interface styles. Differences on the same interface indicated something different about the user groups. There are clearly performance differences between users and between interfaces. Some of these are due to the characteristics of the particular interface (for example the iconic interface generally required less mouse clicks than the other interfaces), but the relative differences in the slopes of the lines indicate something more fundamental. Most notable is the large difference in performance on the command interface by the high and low spatial ability groups, but other differences were also important (Jennings, et al., 1991). The feature which we decided to concentrate on was the level of spatial ability and field dependence (which correlated significantly) and the characteristics of the command interface which we took to be the openness and flexibility of the dialogue. Intuitively there appeared to be some conceptual spatial activities concerned with moving around such a dialogue which was reliably measured by the spatial ability test we had used. 5.2 Second Experiment In the light of these results, a second database system was developed which had two interfaces. One (a command interface based on SQL) was designed to be open and flexible and the other (a menu interface) more constrained. The hypothesis we were testing was that users with a good spatial ability would perform better on the open dialogue. By its nature the constrained dialogue was slower and more restrictive and so it would take longer for users to complete a task. However, poor spatial ability users were expected to perform better on the menu interface since they would make more mistakes and spend more time thinking when using the command interface. This experiment is reported fully in (Jennings and Benyon, 1992). Thirty subjects (all graduates) took part. They all performed similar tasks on the two interfaces. Users were tested for spatial ability and were also asked how they perceived their own use of computers - frequent/occasional and their experience with similar interfaces (a scale from 1 = low to 5 = high). Time taken to complete a number of tasks was taken as the measure of performance. Initially only one half of the hypothesis was confirmed, namely that people with a high spatial ability would prefer the command interface. The results did not show a significant performance difference for those with low spatial ability. The results were re-examined and it was discovered that there was an additional factor involved - the level of experience with a command interface. When this was taken into account, the performance difference in the groups was significant. Figure 10 shows that only the low spatial, low experience group (20% of the subjects) performed significantly better using the menu interface than using the command interface. It appeared that
page 36

differences in performance could be explained by spatial ability plus previous command language experience. The number of errors made shows this quite convincingly (Figure 11). The low spatial, low experience group made an average of nearly three errors when using the command interface, whereas the other groups made practically no errors. Since command language experience would change with frequency of computer use, this characteristic of individual users also needed to be considered.

Figure 10 Mean test session times The results of this experiment suggested that an interface style which minimises navigation and constrains the dialogue, such as the menu interface, is suitable for users with both a low spatial ability and low experience of using command language style interfaces. On the other hand, an interface which facilitates an open and flexible dialogue such as the command interface, is suitable for all users with a high spatial ability, whatever their previous experience and for users with a low spatial ability who have a high experience of using command language interfaces. It is not clear to what extent these results would generalise to other users and systems. In the first instance, non-graduate user groups may have much lower levels of spatial ability which cannot be compensated for by high experience levels. Secondly, although the menu interface appears free from the effects of spatial ability, it is considerably slower (and therefore less suitable) for a large proportion of the users. Thirdly, the SQL statements

page 37

which users were required to perform were relatively simple. More complex usage of SQL may demonstrate increased problems for low spatial ability group.

Figure 11 Mean number of errors per person per session 5.3 An Adaptive System2 We believe that the results of this experiment demonstrate exactly the sort of problem faced by many system designers. In terms of the methodology presented earlier, we may consider the experimental work to constitute the systems analysis stage. The designer now has a rationale for adopting an adaptive system solution to the interface design. There is a large group of users (20%) who require one sort of interface and another group who require another. Being infrequent computer users, the users who require the more constrained interface should not be expected to customise the system. Training in the command language is not a viable option as syntax would soon be forgotten. Although the results of the pilot experiment suggest that iconic or button interfaces may be suitable for all users, it is not always feasible to provide these, particularly in large systems. There seems to be good reason to consider the adaptive system approach. The designer should now consider whether data is available from the dialogue record to facilitate the required adaptation and the mechanisms which can be employed to make the relevant inferences and adaptations. A
2A

more detailed version of this example, relating it to the process of development, is provided in Benyon (1993a) page 38

simple error count seems to be suitable for this purpose since errors correlate with spatial ability and command experience. Errors can be easily detected from the dialogue and can be used to infer appropriate characteristics of users. The system is to be adaptive insofar as it will change the interface style in response to one or two characteristics of the users. The design of the adaptive system is shown as Tables 3, 4 and 5 (from Benyon, 1992). The domain model in this case consists of four attributes, one at the task level, two at the logical level and one at the physical level of description. The user model contains data on the users personal profile, cognitive characteristics and the student model. The student model inherits all the attributes from the domain model and is updated from the dialogue record. An important aspect of defining the user model is to establish how the particular data will be obtained. The interaction model consists of the dialogue record which records details of the number of errors made and the number of tasks completed and the Interaction Knowledge-base which describes the inferences, adaptations and evaluations which the system can make. In this case no evaluations are undertaken, but it is a trivial development to incorporate them (Benyon, 1992). Level Task Logical Description A Task is a successful completion of a query An error is defined as; an incorrect formulation of a query (including specifying attributes in the wrong place, etc.) a missing or incorrect operator (such as <, > etc.) an inappropriate command (e.g. typing select.... when in the help system) The average task completion time is calculated as the total time to complete a block of 12 tasks divided by 12. An interface is a coherent style of interaction Attribute Name tasks errors Values {1...N} {1...N}

Logical

ATCT

{1....N} seconds

Physical

interface

{menu, command}

Table 3 Domain model for sample adaptive system Model Attribute Name How Obtained Values

page 39

Cognitive

spatial ability

inferred from the interaction (see inference rule 1) 1. by asking user 2. from inference rule 1 by asking user from dialogue record from dialogue record from dialogue record from dialogue record

{high, low}

Profile

command experience computing tasks ATCT errors interface

{high, low, none}

Profile Student Student Student Student

{frequent, occasional} {1...N} {1...N} seconds {1...N} {menu, command}

Table 4 User model for sample adaptive system

page 40

Dialogue Record It consists of the data items; (tasks, errors) Interaction Knowledge base Inference rule 1 If interface = command and errors> 1 and tasks = 12 then spatial ability = low and command experience = low Adaptation Rule 1 if spatial ability = high then interface = command Adaptation Rule 2 if spatial ability = low and command experience = none and computing = frequent then interface = command Adaptation Rule 3 if spatial ability = low and command experience = none and computing = occasional then interface = menu Table 5 Interaction model for sample adaptive system. The adaptive system described here has been developed as a demonstrator but has not been used in any further experiments. The system as it stands is inevitably very simple, but it does serve to illustrate much of the theory presented in this paper. It shows not just the feasibility of the adaptive mechanisms, but the feasibility of finding and recording cognitive characteristics which effect the interaction. The contention is that the user makes mistakes because they have a poor spatial ability and a low level of command language experience and the command interface requires them to have a high level of spatial ability and/or a high level of command language experience. We are able to infer the domain independent characteristic of spatial ability because of the characteristics of the command interface. Since spatial ability is domain independent we can then use this knowledge when the user interacts with other systems. This data can be unobtrusively and quickly obtained. A word of caution is required here. It may be the effect which we observed was not the spatial ability per se, but something related to spatial ability. Indeed it may be ability to move around a computer system and may not have an analogue elsewhere.

page 41

6.

SUMMARY

In this paper, we have surveyed the field of adaptive systems and argued that all these systems share a common architecture consisting of three models the user model, the domain model and the interaction model. Each of these can be further understood as detailed in Section 3. In this endeavour we have sought to present a reference model, thus enabling existing systems to be compared in terms of the ways in which they implement the architecture. We offer a simple definition of adaptive systems. A system is adaptive if: it can receive and process signals from another system (it can interact) it can automatically change its state and/or its behaviour appropriately

The sophistication of the adaptive mechanism depends on the quality of the models which it possesses and the use to which it can put them. An adaptive system: possesses a model of the application which is to be adaptable (the domain model) so that it knows what characteristics it can alter. This model is maintained at the three levels of tasks, physical and logical possesses a model of the system with which it can interact (the user model). This model is developed and enhanced by monitoring the interaction possesses a model of the interaction which includes inference, evaluation and adaptation mechanisms.

We have also described a methodology for the development of adaptive systems which indicates where and when a system developer might consider implementing an adaptive capability. The software support necessary to underpin the methodology was presented in Section 5. The experimental work illustrates the methodology in action. The work described here is important in a number of respects. Firstly the results of the experimental work add weight to the growing awareness of the importance of individual differences in human-computer interaction (Egan, 1988). Our results suggest that spatial ability, verbal ability and field dependence are important cognitive characteristics of users which have a significant effect on the quality of the interaction. These results confirm those found by Vicente and Williges for spatial and verbal ability (Vicente and Williges, 1988) and by Fowler and Murray (Fowler and Murray, 1987) regarding field dependence. The system designer has to decide what to do in the face of these findings. From our results, we can make the general recommendation that interfaces should not be designed which require users to traverse a number of systems or a multi-level hierarchy and which do not provide adequate reference points because such systems require users to have a high level of spatial ability and many users do not have this characteristic. However, since many
page 42

such systems do exist, and will continue to exist if systems are to provide a high-level of functionality, then the designer must choose between providing increased levels of support for users, providing customising facilities or building intelligence into the interface. Our results also indicate that the user profile is important. We found that frequency of computer use and knowledge of generic systems (command languages in this case) were factors affecting the quality of the interaction. The Basser Data project (Thomas, et al., 1991) has also highlighted the impact which previous experience of computer systems and more basic skills such as typing and familiarity with a mouse has on learning command sets. The future for adaptive systems seems promising. There are three arguments in favour of adaptive systems; the a priori argument, the usability argument and the changing user argument. Systems which seek to tailor their responses to the needs of individuals - from intelligent tutoring systems through natural language, intelligent support and explanation systems to autonomous agents - h a v e to be adaptive. They are constrained by the bandwidth of the dialogue record, but they must conform to the architecture presented here. We began this article by questioning how successful the concept of interface agents could be. These miniature knowledge-based systems would take on some of the more mundane tasks for us. We may dream of agents attending meetings for us, organising our diaries, guiding us through large information spaces, explaining the complexities of some device or filtering the excesses of beaurocratic detail, but can they be a reality? We believe that the architecture and methodology presented here is highly applicable to developing interface agents. The concept of a user model is generalised to the concept of an agent model, but otherwise the architecture remains the same. The experimental work described the development of an interface selection agent. Natural Language (NL) interfaces are agents which can translate the human's natural language into the pedantic tongue of the computer. In section 2 we discussed critical agents, helpful agents, tutoring agents and explanation agents. Following Totterdell, et al., we have discussed levels of agency in respect of the models possessed by the agent. Simple agents, Learning agents, Self-mediating agents and Self-modifying agents. These are meta-agents since they can change the agent, interaction and domain models. The methodology presented is applicable to developing agent-based systems. The designer has to consider how the usability of system can be improved by including agents. The explicit nature of the models possessed by the agents can help to overcome problems of users expecting too much from their software. They can display (and edit if this is deemed desirable) the knowledge possessed by the agent. This provides the sort of security described in (Maes and Kozierok, 1993) where users have control over a threshold below which the agent will advise them of the decision rather than acting independently.

page 43

The application of knowledge-based techniques to interface issues still has a long way to go. However, the provision of a reference model is at least a start on the road to considered and understandable adaptive systems. ACKNOWLEDGEMENTS Much of the work reported here was sponsored by the National Physical Laboratory, Teddington, UK under extra-mural research agreements NPL 860436 and NPL 82-0486. Special thanks is due to all the people who were involved in the projects at NPL, Leicester Polytechnic and the Open University, in particular, Nigel Bevan, Steve Howard, Peter Innocent, Frances Jennings, Steve Milan, Richard O'Keefe, David Schofield, Jaginder Shergill, and Cathy Thomas. Much of this paper was compiled during David Benyon's visiting fellowships where opportunities and help were provided by Judy Kay at the University of Sydney and David Hill and the University of Calgary. The Royal Society of Great Britain supported the visit to the University of Calgary. REFERENCES Alty, J.L. and Mullin, J. (1987) The role of the dialogue system in a User Interface Management System., in: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Alty, J.L. and McKell, P. (1986) Application modelling in a User Interface Management System, in: M.D. Harrison and A.F. Monk (Eds.), People and Computers: Designing for Usability (Cambridge: Cambridge University Press). Barnard, P. (1987) Cognitive Resources and the Learning of HumanComputer Dialogues. In Carroll, J. (ed.) Interfacing thought: Cognitive aspects of human-Computer Interaction. MIT Press, Cambridge, Mass Benyon, D.R. (1984) Monitor: a self-adaptive user interface. In: B. Shackel (Ed.), Proc. INTERACT 84, First IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Benyon, D.R. (1988) User models: what's the purpose, In: M. Cooper and D. Dodson (Eds.) Alvey Knowledge-Based Systems Club, Intelligent Interfaces Special Interest Group, Proc. of the Second Intelligent Interfaces Meeting, May 28-29th 1987, London (London: Department of Trade and Industry Information Engineering Directorate). Benyon, D.R. (1990) Information and Data Modelling. Blackwell Scientific Publishers, Oxford Benyon, D.R. (1992a) The Role of Task Analysis in Systems Design, in: Interacting with Computers, vol. 4 no. 1, 102 - 121 Benyon, D.R. (1992b)Task Analysis and Systems Design: The Discipline of Data. In Interacting with Computers, 4(2) 246 - 259

page 44

Benyon, D.R. (1993a) Adaptive Systems; a solution to usability problems, in: User Modelling and User Adapted Interaction, Benyon, D. R. (1993b ) Accommodating Individual Differences through an Adaptive User Interface. In Schneider-Hufschmidt, M., Kuhme, T. and Malinowski, U. (eds.) Adaptive User Interfaces - Results and Prospects, Elsvier Science Publications, North-Holland, Amsterdam. 1993 Benyon, D. R. (1993a) Adaptive Systems; a solution to usability problems. User Modelling and User Adapted Interaction 1993 Benyon, D. R. and Murray, D. M. (1993) Applying user modelling to humancomputer interaction design AI Review (6) pp 43 - 6993 Benyon D.R., Innocent P.R., Murray, D.M. (1987)System adaptivity and the modelling of stereotypes. In: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Benyon D.R., Milan, S. and Murray, D.M. (1987) Modelling users cognitive abilities in an adaptive system, In: J. Rasmussen and P. Zunde (Eds.), Proc. 5th Symposium EFISS, Ris National Laboratory, Denmark, November 1987 (New York: Plenum Publishing). Benyon, D.R. and Murray, D.M. (1988) Experience with adaptive interfaces, The Computer Journal, Vol. 31(5). Benyon, D.R., Jennings, F. and Murray, D.M. (1990) An adaptive system developers toolkit. In: Diaper, D. Cockton, G., Gilmore, D. and Shackel, B. (Eds.), Proc. INTERACT 90, Third IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Bosser, T. (1987) Learning in Man-Computer Interaction, Springer-Verlag Branjik, G., Guida, G. and Tasso, C. (1987) User modelling in intelligent information retrieval, Information Processing and Management, Vol. 23(4). Branjik, G., Guida, G. and Tasso, C. (1990) User Modelling in Expert ManMachine Interfaces: A case study in Information Retrieval, IEEE Trans. Systems, Man and Cybernetics, Vol. 20(1) Bronisz, D., Grossi, T. and Jean-Marie, F. (1989) Advice-giving dialogue: an integrated system. In: Proc. 6th Annual ESPRIT Conference, Brussels, November Kluwer Academic Publishers. Browne, D.P., Totterdell, P.A. and Norman, M.A. (1990) Adaptive User Interfaces (London: Academic Press). Bundy, A. (1983) (ed.) Alvey 1983 Intelligent Front End Workshop 26-27 Sept. 1983 Coseners House, Abingdon, England. DTI, London Burton, R. R. (1982) Diagnosing Bugs in a Simple Procedural Skill in D. H. Sleeman and J. S. Brown (eds.) Intelligent Tutoring Systems. (Academic Press, New York)
page 45

Carberry, S. (1989) Plan recognition and its use in understanding dialog. In: W. Wahlster and A. Kobsa (Eds.) op. cit. Card, S., Moran, A.P., and Newell, A. (1983) The Psychology of HumanComputer Interaction (Hillsdale, NJ: Lawrence Erlbaum Associates). Carroll, J.M. and McKendree, J. (1987) Interface design issues for advice-giving systems, Communications of the ACM, Vol. 30(1). Carroll, J.M. (1991) The Nuremberg Funnel. MIT Press, Cambridge, Mass Chignell, M.H. and Hancock, P.A. (1988) Intelligent Interfaces in M. Helander (Ed.) Handbook of Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Chin, D.N. (1986) User modelling in UC, the Unix consultant. In: M. Mantei and P. Orbiton (Eds.), Proc. CHI 86, Human Factors in Computing Systems (New York: ACM). Chin, D.N. (1989) KNOME: Modelling what the user knows in UC. In: W. Wahlster and A. Kobsa (Eds.) op.cit. Computational Linguistics, (1988) Vol. 14(3). Coutaz, J. (1987) PAC: An object orientated model for implementing user interface. In H.J. Bullinger and B. Shackel (Eds.) Human- Computer Interaction, Proceedings of INTERACT 87 (Amsterdam: Elsevier Science Publishers B.V.). Dede, C. (1986) A review and synthesis of recent research in intelligent computer-assisted instruction, International Journal of Man-Machine Studies, Vol. 24(?). Dennett, D. (1989) The Intentional Stance MIT press, Cambridge, Ma. Diaper, D. (1989) Task Analysis for Human-Computer Interaction (Chichester: Ellis Horwood). Edmonds, E.A. (1981) Adaptive Man-Computer Dialogues. In Coombs, M. J. and Alty, J. L. Computing Skills and the user Interface. Academic Press, London Edmonds, E.A. (1987) Adaptation, Response and Knowledge, KnowledgeBased Systems, Vol. 1(1), Editorial. Egan, D.E. (1988) Individual differences in Human-Computer Interaction. In Helander, M. (Ed.) Handbook of Human-Computer Interaction(Amsterdam: Elsevier Science Publishers B.V.). Elkerton, J. (1987) A framework for designing intelligent human-computer interfaces, in: G. Salvendy (Ed.) Cognitive Engineering in the Design of Human-Computer Interaction and Expert-Systems (Amsterdam: Elsevier Science Publishers B.V.).

page 46

Finin, T.W. (1989) GUMS - A general user modelling shell. In: W. Wahlster and A. Kobsa (Eds.) op.cit. Fischer, G. (1987) Making computers more useful. In: G. Salvendy (Ed.) Cognitive Engineering in the Design of Human-Computer Interaction and Expert-Systems (Amsterdam: Elsevier Science Publishers B.V.). Fischer, G., Lemke, A.C. and Schwab, T. (1986) Knowledge-based help systems. In: M. Mantei and P. Orbiton (Eds.), Proc. CHI 86, Human Factors in Computing Systems (New York: ACM). Fischer, G., Morch, A. and McCall, R. (1989) Design environments for constructive and argumentative design. In: ????. (Eds.), Proc. CHI 89, Human Factors in Computing Systems (New York: ACM). Fischer, G. (1989) HCI Software: Lessons learned, challenges ahead, IEEE Software, January 1989. Fowler, C. and Murray, D.M. (1987) Gender and cognitive style differences at the human-computer interface. In: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on Human-Computer Interaction(Amsterdam: Elsevier Science Publishers B.V.). Furnas, G. (1985) New Jersey experience with an adaptive indexing scheme. In: A. Janda (Ed.), Proc. CHI 85, Human Factors in Computing Systems (New York: ACM). Gray, W., Hefley, W. and Murray, D. M. (eds.) (1993) Proceedings of 1st international Workshop on Intelligent User Interfaces, ACM Publications, New York Green, M. (1985) Report on Dialogue Specification Tools in UIMS, in Pfaff, G.E. (Ed.),User interface Management Systems, Springer Verlag, Heidelberg. Green, T.R.G., Schiele, F. and Payne S. J. (1988) Formalisable models of user knowledge in human-computer interaction, In: C.C. van der Veer, T.R.G. Green, J.M. Hoc and D.M. Murray (Eds.) Working with Computers: Theory versus Outcome, (London: Academic Press). Greenberg, S. and Witten, I.H. (1985) Adaptive personalised interfaces -a question of viability, Behaviour and Information Technology, Vol. 4(1). Hancock, P.A. and Chignell, M.H. (1988) Mental workload dynamics in adaptive interface design, IEEE Trans. SMC, Vol. 18(4). Hancock, P.A. and Chignell, M.H. (1989) (eds.) Intelligent Interfaces; Theory, Research and Design. North-Holland, New York Hansen, S.S., Holgaard, L. and Smith, M. (1988) EUROHELP: intelligent help systems for information processing systems. In: Proc. 5th Annual ESPRIT Conference, Brussels, November 1988 (Kluwer Academic Publishers).

page 47

Hartson, H.R. and Hix, D. (1989 ) Toward empirically derived methodologies and tools for HCI development in International Journal of Man Machine Studies 31, 477-494 IEEE Software , (1989) January. Innocent, P.R. (1982) A self-adaptive user interface, International Journal of Man Machine Studies, Vol. 16(3) 287 - 300 Jennings F., Benyon D.R. and Murray D.M. (1991) Adapting Systems to individual differences in cognitive style, Acta Psychologica, December. Jennings, F. and Benyon, D.R. (1992) Database Systems: Different Interfaces for different users in press Jerrams-Smith, J. (1985) SUSI -A smart user-system interface,. In: P. Johnson and S. Cook (Eds.), People and Computers: Designing the Interface (Cambridge: Cambridge University Press). Johnson, P. (1989) Supporting System Design by analysing current task knowledge in Diaper, D. (ed.) Task Analysis for Human-Computer Interaction. Ellis-Horwood Kass R. (1989) Student modelling in intelligent tutoring systems. In: W. Wahlster and A. Kobsa (Eds.) op. cit. Kass, R. and Finin, T. (1988) The need for user models in generating expert system explanations, International Journal of Expert Systems, Vol. 1(4). Kass, R. and Finin, T. (1989) The role of user models in co-operative interactive systems, International Journal of Intelligent Systems, Vol. 4(1). Kay, A. (1989) User interface: A personal view, In: B. Laurel (Ed.) The Art of Human-Computer Interface Design, Addison Wesley, Wokingham. Kay, J. (1991) UM: a toolkit for user modelling, In Schneider-Hufschmidt, M., Kuhme, T. and Malinowski, U. (eds.) Adaptive User Interfaces - Results and Prospects, Elsvier Science Publications, North-Holland, Amsterdam. Kieras, D. and Polson, P.G. (1985) An approach to the formal analysis of user complexity, International Journal of Man Machine Studies, Vol. 22 (?). Kobsa, A. (1987)A taxonomy of beliefs and goals for user modelling in dialog systems, Memo Nr. 28, Universitat des Saarlandes, Saarbrucken. Kobsa, A. (1988) A bibliography of the field of user modelling in artificial intelligence dialog systems, Memo Nr. 23, Universitat des Saarlandes, Saarbrucken. Kobsa, A. and Wahlster, W. (1989) User models in dialog systems (Berlin: Springer-Verlag) Kok (19xx) Laurel, B. (1990) Interface Agents. In: B. Laurel (Ed.) The Art of HumanComputer Interface Design, Addison Wesley, Wokingham.
page 48

Lehner, P.E. (1987) Cognitive factors in user/expert-system interaction, Human Factors, Vol. 29(1). Maes, P. and Kozierok, R. (1993) Learning Interface Agents AAAI '93 Conference on Artificial intelligence, Washington Mason, M.V. and Thomas, R.C. (1984) Experimental adaptive interfaces, Information Technology: Research & Development, Vol. 3. Mason, M.V. (1986) Adaptive command prompting in an on-line documentation system, International Journal of Man-Machine Studies, Vol. 25 (?). Mason, J. and Edwards, J.L. (1988) Surveying projects on intelligent dialogue, International Journal of Man-Machine Studies, Vol. 28 (?). Moore, J. and Swartout, W.R. (1988) Planning and reacting, Proc, AAAI Workshop on text planning and generation, August 25, 1988, St. Paul, Minnesota. Moran, T.P. (1981) Command language grammar: a representation for the user interface of interactive computer systems, International Journal of ManMachine Studies, Vol. 15(3). Moran, T.P. (1983) Getting into a system: external-internal task mapping analysis, In: R.N. Smith and R.W. Pew (Eds.) Proceedings CHI'83: Human Factors in Computing Systems (ACM Press). Morik, K. (1989) User models and conversational settings: modelling the user's wants. In: W. Wahlster and A. Kobsa (Eds.) op. cit. Murray, D.M. (1987a) A survey of user modelling definitions and techniques, NPL DITC Report 92/87. Murray, D.M. (1987b) Embedded user models. In: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on HumanComputer Interaction (Amsterdam: Elsevier Science Publishers B.V.) Murray, D.M. (1988) Building a user modelling shell. in: P. Zunde (Ed.), Proc. 6th Symposium EFISS, Georgia Tech., Atlanta, Georgia, USA, October 1988 (New York: Plenum Publishing). Murray, D.M. (1989) Modelling for adaptivity, Proceedings of 8th Interdisciplinary Workshop, Informatics and Psychology, Scharding, Austria, May 1989 (Amsterdam: North Holland). Murray, D.M. and Benyon, D. R. (1988) Models and designers tools for adaptive systems, presented at 4th European Conference on Cognitive Ergonomics (ECCE-4), Cambridge, U.K., September 1988. McCoy, K.F. (1989) Highlighting a user model to respond to misconceptions. In: W. Wahlster and A. Kobsa (Eds.) op. cit.

page 49

Negroponte, N. (1989) Beyond the desktop metaphor, International Journal of HCI, Vol. 1(1). Newell, A. (1982) The Knowledge Level Artificial Intelligence 18(1) 87 - 127 Nielsen, J. (1986) A virtual protocol model for computer-human interaction in International Journal of Man-Machine Studies, Vol. 24. Noah, W, and Halpin, S.M. (1986) Adaptive user interfaces for planning and decision aids in CHI systems, IEEE Trans. SMC, Vol. 16(6). Norcio, A.F. and Stanley, J. (1989) Adaptive human-computer interfaces: a literature study and perspective, IEEE Trans. Systems, Man and Cybernetics, Vol. 19(2). Norman, D. (1986) in Norman, D. and Draper, S. (eds.) User Centred System Design. Paris, C.L. (1989) The use of explicit user models in a generation system. In: W. Wahlster and A. Kobsa (Eds.) op. cit. Payne, S.J. (1988) Complex problem spaces: modelling the knowledge needed to use interactive devices. In: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Payne, S.K. and Green, T.R.G. (1989) Task Action grammar: The Model and Its developments. In Diaper, D. (ed.) Task Analysis for Human-Computer Interaction. Ellis-Horwood Pollack, M.(1985) Information Sought And Information Provided. An Empirical Study of User/Expert Dialogues. In Proceeding of CH1'85. Human Factors in Computing Systems, ACM: New York Pylyshyn, Z. W. (1984) Computation and Cognition MIT press, Cambridge, Ma. Rasmussen, J. (1986) Information processing and human-machine interaction (Amsterdam: Elsevier North-Holland). Rasmussen, J. (1987) Mental models and their implications for design, In: Austrian Computer Society 6th Workshop on Informatics and Psychology, June 1987. Rich, E. (1979) User modelling via stereotypes, Cognitive Science, Vol. 3. Rich, E. (1983) Users are individuals: individualising user models, International Journal of Man Machine Studies, Vol. 18(?). Rich, E. (1989) Stereotypes and user modelling. In: W. Wahlster and A. Kobsa (Eds.) op. cit. Rivers, R. (1989) Embedded User Models. Where next? Interacting with Computers, Vol. 1(1)

page 50

Rouse, W.B. et al (1989) Intelligent Interfaces, Human-Computer Interaction, Vol. 3(?). Seel, N. (1990) From here to Agent Theory. AISB Quarterly, no. 72 Spring Self J. (1986) Applications of machine-learning to student modelling, Instructional Science, Vol. 14. Self, J. (1987) User modelling in open learning systems. In: J. Whiting and D. Bell (Eds.), Tutoring and Monitoring Facilities for European Open Learning (Amsterdam: Elsevier Science Publishers B.V.). Shackel, B. (1990) A Framework for usability in Preece, J. and Keller, L. (eds.) Readings in Human-Computer Interaction Prentice-Hall Sleeman, D. (1985) UMFE: A user modelling front-end system, International Journal of Man Machine Studies, Vol. 23(?). Steels, L. (1987) The deepening of Expert Systems, AICOM, No 1, 9-16 Stehouwer, M. and van Bruggen, J. (1989) Performance interpretation in an intelligent help system. In: Proc. 6th Annual ESPRIT Conference, Brussels, November 1989 (??: Kluwer Academic Publishers). Schneider-Hufschmidt, M., Kuhme, T. and Malinowski, U. (eds.) (1993) Adaptive User Interfaces - Results and Prospects, Elsvier Science Publications, North-Holland, Amsterdam Sullivan and Tyler, 1988; Tang, H., Major, N., and Rivers, R. (1989) From users to dialogues. In: L. Macauley and A. Sutcliffe (Eds.) People and Computers V (Cambridge: Cambridge University Press). Thimbleby, H. (1990a) Youre right about the cure - dont do that, Interacting with Computers, Vol. 2(1). Thimbleby, H. (1990b) User Interface Design (Wokingham: Addison Wesley). Thomas, R.C., Benyon, D.R., Kay, J. and Crawford, K. (1991) Monitoring Editor Usage: The Basser Data Project, in proceedings of NCIT '91 Penang. Totterdell, P.A, Norman, M.A. and Browne, D.P. (1987) Levels of adaptivity in interface design. In: B. Shackel and H-J. Bullinger (Eds.), Proc. INTERACT 87, Second IFIP Conference on Human-Computer Interaction (Amsterdam: Elsevier Science Publishers B.V.). Totterdell, P.A. and Cooper, M. (1986) Design and evaluation of the AID adaptive front-end to Telecom Gold. van der Veer, G.C. (1990) Human-Computer Interaction. Learning, individual differences and design recommendations Offsetdrukkerij Haveka B.V., Alblasserdam

page 51

Vicente, K.J. and Williges, R.C. (1987). Assaying and isolating individual differences in searching a hierarchical file system., Human Factors, Vol. 29, 349-359. Vicente, K.J. and Williges, R.C. (1988). Accommodating individual differences in searching a hierarchical file system., International Journal of Man-Machine Studies, Vol. 29. Vicente, K. and Williges, R.C., (1988) Visual Momentum as a means of accommodating individual differences among users of a hierarchical file system. In: J. Rasmussen and P. Zunde (Eds.), Proc. 5th Symposium EFISS, Ris National Laboratory, Denmark, November 1987 (New York: Plenum Publishing). Vicente, K., Hayes, B.C. and Williges, R.C. (1987) Assaying and isolating individual differences in searching a hierarchical file system. Human Factors, 29(3), 349-359 Wahlster W. and Kobsa A. (1987) Dialogue-based user models, Proc. IEEE Vol. 74(4). Wilensky, R., Arens, Y. and Chin D. (1984) Talking to Unix in English: an overview of UC, Communications of the ACM , Vol. 27(6). Williges, R.C. (1987) The use of models in human-computer interface design, Ergonomics, Vol. 30(3). Wilson, M.D., Barnard, P. J, Green, T.R.G. and Maclean, A. (1988) Task Analyses in Human-Computer Interaction. In: C.C. van der Veer, T.R.G. Green, J.M. Hoc and D.M. Murray (Eds.) Working with Computers: Theory versus Outcome, (London: Academic Press). Young, R. M. and Hull, A. (1982) Categorisation structures in hierarchical menus in Proceedings of 10th International Symposium on Human Factors in Telecommunications, Helsinki. pp 111 - 118 Zissos, A. and Witten, I., I. (1985) User modelling for a computer coach: a case study, International Journal of Man-Machine Studies, Vol. 23.

page 52