0% found this document useful (0 votes)

7 views82 pages

Visual Languages For Interactive Computing Definitions and Formalizations Fernando Ferri Download

The document discusses 'Visual Languages for Interactive Computing' edited by Fernando Ferri, which explores the syntax, semantics, and ambiguities of visual languages. It presents methodologies for defining and formalizing these languages, as well as their interpretation in interactive computing contexts. The book includes contributions from various authors on topics such as visual query languages, user interface design, and the management of ambiguities in visual communication.

Uploaded by

bslhxxf027

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

7 views82 pages

Visual Languages For Interactive Computing Definitions and Formalizations Fernando Ferri Download

Uploaded by

bslhxxf027

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 82

Visual Languages For Interactive Computing

Definitions And Formalizations Fernando Ferri

download

https://ebookbell.com/product/visual-languages-for-interactive-
computing-definitions-and-formalizations-fernando-ferri-4109616

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Visual Languages For Interactive Computing Definitions And

Formalizations 1st Edition Fernando Ferri

https://ebookbell.com/product/visual-languages-for-interactive-
computing-definitions-and-formalizations-1st-edition-fernando-
ferri-2104462

Tuning In Music Book Sixtyfour Songs For Children With Complex Needs
And Visual Impairment To Promote Language Social Interaction And Wider
Development Adam Ockelford

https://ebookbell.com/product/tuning-in-music-book-sixtyfour-songs-
for-children-with-complex-needs-and-visual-impairment-to-promote-
language-social-interaction-and-wider-development-adam-
ockelford-12199492

Visual Language For Designers Principles For Creating Graphics That

People Understand Connie Malamed

https://ebookbell.com/product/visual-language-for-designers-
principles-for-creating-graphics-that-people-understand-connie-
malamed-4676828

Visual Language For Designers Connie Malamed

https://ebookbell.com/product/visual-language-for-designers-connie-
malamed-170516128
Visual Language For Designers Principles For Creating Graphics That
People Understand Connie Malamed

https://ebookbell.com/product/visual-language-for-designers-
principles-for-creating-graphics-that-people-understand-connie-
malamed-5160564

Foundations Of Ccli The Visual C Language For Net 35 1st Edition

Gordon Hogenson

https://ebookbell.com/product/foundations-of-ccli-the-visual-c-
language-for-net-35-1st-edition-gordon-hogenson-4101410

The Visual Language Of Spatial Planning Exploring Cartographic

Representations For Spatial Planning In Europe 1st Edition Stefanie
Duhr

https://ebookbell.com/product/the-visual-language-of-spatial-planning-
exploring-cartographic-representations-for-spatial-planning-in-
europe-1st-edition-stefanie-duhr-1752334

From Child Art To Visual Language Of Youth New Models And Tools For
Assessment Of Learning And Creation In Art Education Andrea Krpti

https://ebookbell.com/product/from-child-art-to-visual-language-of-
youth-new-models-and-tools-for-assessment-of-learning-and-creation-in-
art-education-andrea-krpti-5603190

Thai Picture Dictionary Learn 1500 Key Thai Words And Phrases The
Perfect Visual Resource For Language Learners Of All Ages Jintana
Rattanakhemakorn

https://ebookbell.com/product/thai-picture-dictionary-learn-1500-key-
thai-words-and-phrases-the-perfect-visual-resource-for-language-
learners-of-all-ages-jintana-rattanakhemakorn-35341448
Visual Languages for
Interactive Computing:
Definitions and Formalizations

Fernando Ferri
Istituto di Ricerca sulla Popolazione e le Politiche Sociali
Consiglio Nazionale delle Ricerche, Italy

Information science reference

Hershey • New York
Acquisitions Editor: Kristin Klinger
Development Editor: Kristin Roth
Senior Managing Editor: Jennifer Neidig
Managing Editor: Sara Reed
Assistant Managing Editor:
Copy Editor: Larissa Vinci
Typesetter: Michael Brehm
Cover Design: Lisa Tosheff
Printed at: Yurchak Printing Inc.

Published in the United States of America by

Information Science Reference (an imprint of IGI Global)
701 E. Chocolate Avenue, Suite 200
Hershey PA 17033
Tel: 717-533-8845
Fax: 717-533-8661
E-mail: [email protected]
Web site: http://www.igi-pub.com/reference

and in the United Kingdom by

Information Science Reference (an imprint of IGI Global)
3 Henrietta Street
Covent Garden
London WC2E 8LU
Tel: 44 20 7240 0856
Fax: 44 20 7379 0609
Web site: http://www.eurospanonline.com

Copyright © 2008 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by
any means, electronic or mechanical, including photocopying, without written permission from the publisher.
Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does
not indicate a claim of ownership by IGI Global of the trademark or registered trademark.

Library of Congress Cataloging-in-Publication Data

Visual languages for interactive computing : definitions and formalizations / Fernando Ferri, editor.

p. cm.

Summary: "This book presents problems and methodologies related to the syntax, semantics, and ambiguities of visual languages. It
defines and formalizes visual languages for interactive computing, as well as visual notation interpretation"--Provided by publisher.

Includes bibliographical references and index.

ISBN 978-1-59904-534-4 (hardcover) -- ISBN 978-1-59904-536-8 (ebook)

1. Interactive computer systems. 2. Visual programming languages (Computer science) 3. Visual programming (Computer science) I.
Ferri, Fernando.

QA76.9.I58V59 2007

005.1'18--dc22

2007007281

British Cataloguing in Publication Data

A Cataloguing in Publication record for this book is available from the British Library.

All work contributed to this book set is new, previously-unpublished material. The views expressed in this book are those of the authors, but
not necessarily of the publisher.
Table of Contents

Detailed Table of Contents................................................................................................................... vi

Preface . ................................................................................................................................................ xii

Section I
Visual Languages Theory

Chapter I
The Theory of Visual Sentences to Formalize Interactive Visual Messages / Paolo Bottoni,
Maria Francesca Costabile, Stefano Levialdi, and Piero Mussio........................................................... 1

Chapter II
Visual and Diagrammatic Languages: The Logic Perspective /
Bernd Meyer and Paolo Bottoni............................................................................................................ 22

Chapter III
A Transformation-Based Metamodel Approach to the Definition of Syntax and Semantics of
Diagrammatic Languages / Paolo Bottoni, Dino Frediani, Paolo Quattrocchi, Luigi Rende,
Goran Sarajlic, and Domenico Ventriglia............................................................................................. 51

Chapter IV
Meta-Modelling and Graph Transformation for the Definition of Multi-View Visual Languages /
Esther Guerra and Juan de Lara........................................................................................................... 74

Chapter V
Extended Positional Grammars: A Formalism for Describing and Parsing Visual Languages /
Gennaro Costagliola, Vincenzo Deufemia, and Giuseppe Polese....................................................... 102

Chapter VI
Visual Notation Interpretation and Ambiguities / Arianna D’Ulizia, Patrizia Grifoni,
and Maurizio Rafanelli........................................................................................................................ 117
Chapter VII
The Management of Ambiguities / Maria Chiara Caschera, Fernando Ferri,
and Patrizia Grifoni............................................................................................................................. 129

Section II
Approaches and Methods for Specific Domains

Chapter VIII
Visual Query Languages, Representation Techniques, and Data Models / Maria Chiara Caschera,
Arianna D’Ulizia, and Leonardo Tininini............................................................................................ 142

Chapter IX
Toward a Visual Query System for Spatio-Temporal Databases / Valéria M. B. Cavalcanti,
Ulrich Schiel, and Claudio de Souza Baptista..................................................................................... 158

Chapter X
Multi-Facet Design of Interactive Systems through Visual Languages / Daniela Fogli, Andrea Marcante,
Piero Mussio, Loredana Parasiliti Provenza, and Antonio Piccinno.................................................. 174

Chapter XI
Designing a Visual Language for Interaction Representation Based on
Aspects of Human Movement / Kristine Deray and Simeon J. Simoff................................................ 205

Chapter XII
Sketch Understanding: Issues and State of the Art / Vincenzo Deufemia............................................ 232

Section III
Visual Languages for the Semantic Web

Chapter XIII
User Interface Formalization in Visual Data Mining / Tiziana Catarci, Stephen Kimani,
and Stefano Lodi.................................................................................................................................. 247

Chapter XIV
Visual Development of Defeasible Logic Rules for the Semantic Web / Efstratios Kontopoulos,
Nick Bassiliades, and Grigoris Antoniou............................................................................................. 273

Chapter XV
A Visual Programming Tool for Designing Planning Problems for Semantic Web
Service Composition / Dimitris Vrakas, Ourania Hatzi, Nick Bassiliades,
Dimosthenis Anagnostopoulos, and Ioannis Vlahavas........................................................................ 302
Section IV
Visual Interfaces for Standard Languages

Chapter XVI
User Interaction and Interface Design with UML / Jesus M. Almendros-Jimenez
and Luis Iribarne................................................................................................................................. 328

Chapter XVII
XQBE: A Visual Language for XML Data Management / Alessandro Campi, Davide Martinenghi,
and Alessandro Raffio.......................................................................................................................... 357

Section V
Visualization, Aesthetic Computing, and Usability

Chapter XVIII
GeoVisualization and GIS: A Human Centered Approach / Vlasios Voudouris
and Stephanie Marsh........................................................................................................................... 389

Chapter IXX
A New Constraint-Based Compound Graph Layout Algorithm for Drawing Biochemical Networks /
Sabri Skhiri dit Gabouje and Esteban Zimānyi................................................................................... 407

Chapter XX
Customized Visual Computing: The Aesthetic Computing Method / Paul Fishwick.......................... 425

Chapter XXI
Documentation Methods for Visual Languages / Eduardo Costa, Alexandre Grings,
and Marcus Vinicius Santos................................................................................................................. 436

Chapter XXII
Questioning Usability / Marco Padula and Amanda Reggiori............................................................ 455

Compilation of References................................................................................................................ 468

About the Contributors .................................................................................................................... 502

Index.................................................................................................................................................... 514
Detailed Table of Contents

Preface . ................................................................................................................................................ xii

Section I
Visual Languages Theory

This chapter introduces the theory of visual languages providing characteristics of interactive processes
and the modelling of visual transformations by visual rewriting systems. The dynamics of visual sentences
are discussed considering the enabling and disabling mechanisms and coordination of transformations.
Finally the chapter gives, before concluding, some elements for the integration of the theory’s components
to provide a framework for the design of interactive visual systems.

This chapter discusses visual and diagrammatic languages in logic perspective. The grammatical
approach to visual language specification, highlighting its shortcomings as a basis for this research
plan, is presented. The different logic approaches to diagrammatic languages and advantages and
disadvantages of the different types of logic formalization are discussed too. An original approach, that
is based on linear logic and completely subsumes the grammatical approach, is proposed to avoid most
of these shortcomings.

This chapter proposes an abstract view of the semantic of visual elements with respect to a transforma-
tion-based metamodel approach. It introduces and discusses an integrated framework and interactive
environment, which uses a collection of metamodels to express both syntactical characterisations and
semantic interpretations of diagrammatic sentences.

The chapter presents an approach for the definition of multi-view visual languages (MVVLs). It introduces
two techniques to define MVVL environments: meta-modelling and graph transformation. The former
is used to describe the syntax of the language as a whole. Consistency between views is ensured by
translating each one into a unique repository model, which conforms to the meta-model of the language
as a whole. The translation is performed by automatically generated graph transformation rules.

This chapter gives an overview of extended positional grammars (XPG), an extension of context-free
grammars used for modelling visual notations. The chapter describes the XpLR parsing algorithm
(Costagliola, Deufemia, & Polese, 2004) based on the well-known LR parsing technique. The benefits of
the formalism of a visual language include easier customisation and modification as well as maintenance
and debugging, code and report generation by definition of suitable semantic productions, implementation
of visual and textual languages within a common framework.

This chapter classifies ambiguities that can arise in visual sentences into lexical and syntactic ambi-
guities. When an image associated with a visual sentence is unable to exactly express (be completely
faithful to) the user’s intentions, the system may produce an erroneous interpretation. In particular, a
visual sentence can assume more than one meaning, or incorrect/imprecise user’s drawing actions does
not permit an unequivocal interpretation of the image produced by the.

Chapter VII
The Management of Ambiguities / Maria Chiara Caschera, Fernando Ferri,
and Patrizia Grifoni............................................................................................................................. 129

This chapter proposes a classification of methods to resolve ambiguities that can arise in visual sentences
related to the system’s interpretation function. These methods are grouped in three classes: prevention
of ambiguities, a-posteriori resolution, and approximation resolution methods. Prevention methods are
based on a predefined set of possible system’s configurations. A-posteriori resolution methods are based
on mediation techniques. Approximation resolution methods are based on theories such as Fuzzy Logic,
Markov Random Field, and Bayesian Networks.
Section II
Approaches and Methods for Specific Domains

This chapter discusses the main characteristics of VQLs analysing visual languages to query conventional
relational databases and information systems with a less rigid structure such as Web resources storing
XML documents. It classifies VQLs according to the adopted visual representation technique (e.g.,
based on forms and tables, diagrams, icons, sketches, or combinations thereof) and the underlying data
model, (e.g., visual languages to query relational databases, object-oriented databases, collections
of XML documents, and languages specifically designed for particular data such as geographic and
multidimensional data).

This chapter focuses on visual query languages for spatio-temporal databases. It presents a spatio-
temporal visual query environment (S-TVQE) and a VQS that allows the formulation of conventional,
spatial, temporal, and spatio-temporal database queries in an integrated environment. With S-TVQE,
the user, instead of querying the database by textual query languages, interacts with the system by visual
operators to state the query conditions.

This chapter presents a multi-facet design of interactive systems considering that the knowledge relevant
to the design of an interactive system is distributed among domain experts, software engineers, and
human-computer interaction experts. Each community describes an interactive system through visual
sentences of a visual language (VL). Each VL permits user-system interaction processes to be specified
from a different point of view and for a different audience.

Chapter XI
Designing a Visual Language for Interaction Representation Based on
Aspects of Human Movement / Kristine Deray and Simeon J. Simoff................................................ 205

The chapter presents a framework for creating visual languages to represent interactions using human
movement as a source for the language’s core concepts. It starts from the assumption that interaction is
the core of interactive computing, but poorly understood.
Chapter XII
Sketch Understanding: Issues and State of the Art / Vincenzo Deufemia............................................ 232

This chapter presents a state of the art sketch of understanding techniques and tools. Sketch recognition
is a particularly difficult task as the symbols of a sketched diagram can be drawn with different stroke
orders, numbers, and directions.

Section III
Visual Languages for the Semantic Web

This chapter discusses user-interface formalisation in visual data mining and provides an approach
for the formalisation of the visual interface of a core data mining system. The description of the system
properties is given without touching on implementation details, and enables fundamental design issues
to be detected before they are manifested in the implementation.

This chapter discusses the visualisation of defeasible logic rules in the semantic Web domain. Logic
plays an important role in the development of the semantic Web and defeasible reasoning seems a very
suitable tool. The proposed approach uses direct graphs to assist the user. It appears highly applicable
to the representation of rule attacks and superiorities in defeasible reasoning.

This chapter discusses of the knowledge representation for AI planning problems, especially those related
to semantic Web service composition. It discusses current approaches in encoding planning problems
and presents ViTAPlan, a user-friendly visual tool for planning.
Section IV
Visual Interfaces for Standard Languages

This chapter shows UML diagrams for describing the user interfaces introducing the user-interaction,
user-interface, and GUI-class diagrams. It also examines code generation to implement the system’s
user interfaces through GUI-class diagrams and user-interaction diagrams. A case study of an Internet
book shopping system is introduced to test and illustrate the proposed user interaction and interface
design technique.

This chapter describes XQBE, a visual framework for XML data management, enabling the visualisation
of XML documents, query formulation, the representation and specification of document schemata, the
definition of integrity constraints, the formulation of updates, and the expression of reactive behaviours
in response to data modifications.

Section V
Visualization, Aesthetic Computing, and Usability

This chapter analyses the relationships among geovisualisation, human computer interaction (HCI),
geographic information systems (GIS), and cartography as a means of supporting decision making. It
discusses the importance of data modelling and associated visualisations in terms of what the user can
do by way of analysis and the methods by which he can undertake the analysis.

This chapter presents an algorithm for the generic representation of biochemical graphs in which users
can present knowledge about how to draw graphs in accordance with the biochemical semantics. The
visualisation tools must be able to cope with graphs and take account of the particular semantics of all
kinds of biochemical subgraphs.
Chapter XX
Customized Visual Computing: The Aesthetic Computing Method / Paul Fishwick.......................... 425

This chapter introduces aesthetic computing for the multimedia representation of formal structures. The
range of aesthetics within the arts is broader than in mathematics and computing, where aesthetics is
often synonymous with optimality criteria.

This chapter analyses the use visual programming languages (VPL) for the design and documentation
of real world applications and proposes new tools and approaches for VPL whose aim is the documen-
tation of data flow.

Chapter XXII
Questioning Usability / Marco Padula and Amanda Reggiori............................................................ 455

This chapter discusses usability, introducing concepts, aspects, and potentialities, which must nowadays
be taken into account when detailing usability and design suited systems.
Systems must be used by a community of users in their working activity as a whole to process material
or information, modifying not only rough material but also the working environment and methods. This
requires them to be considered as tools in a social context, which expects ever greater technological
progress.

Compilation of References................................................................................................................ 468

About the Contributors .................................................................................................................... 502

Index.................................................................................................................................................... 511
xii

Preface

Introduction from the Editor

This book gathers the contributions of authors working in the area of visual languages from countries
as diverse as Spain, Italy, United Kingdom, Australia, Greece, Canada, Brazil, and the USA.
It consists of 22 chapters written by a total of 54 different authors to provide an in-depth investiga-
tion of new approaches and methods for visual language theory, visualisation techniques, aesthetic
computing, and usability.
I hope that academics, researchers, technicians, and students in computer science will find this book
a valuable contribution to their knowledge of this important research area.
In normal life, people organise a complex structure of signs to communicate with the world around
them. The brain acts as an interface between the sign system and the environment, orienting the individual
in space. Visual languages systemise the visual representation of concepts in a formalised language
using visual objects such as images or pictorial objects and possibly formal visual expressions. They
are based on simple visual elements, which construct symbols that can be grouped to form structured
visual sentences.

THE OVERALL OBJECTIVE OF THIS BOOK

The main objective and mission of this book is to present problems and methodologies concerning the
syntax, semantics, and ambiguities of visual languages. Various syntactic techniques can be used to define
visual languages, which differ significantly in the way they conceptualise a visual notation and describe
its syntactic structure. From the semantic point of view, the precise meaning of a visual language is re-
quired in order to use it effectively. Finally, ambiguity is one of the main problems of visual languages.
A visual configuration expressing a sentence or a visual action may be interpreted in a number of ways
producing ambiguity. Visual languages offer an intuitive and incremental view of sentences, but may
give different interpretations of the same sentence.

ORGANISATION

The book contains 22 chapters split into five sections. Section I examines the theory underlying visual
languages. Section II discusses different approaches to and methods for visual languages on specific
domains. Section III describes visual languages for the semantic Web. Section IV examines visual
interfaces for standard languages. Finally, Section V considers several topics closely related to visual
languages such as visualisation, aesthetic computing, and usability.
xiii

Section I. Visual Languages Theory

Section I opens with a chapter by Bottoni, Costabile, Levialdi, and Mussio, introducing the Theory of
Visual Sentences by highlighting their main concepts, presenting some problems central to the defini-
tion of a theory of visual languages, and describing current developments. After an informal overview
of the main components of the theory, the authors explore the characteristics of interactive processes
and the modelling of visual transformations by visual rewriting systems. They go on to examine the
dynamics of visual sentences, focusing on the relationship between constancy and variability in visual
sentences, enabling and disabling mechanisms and coordination of transformations. The chapter ends
with a discussion of how the theory’s components can be integrated to provide a framework for the
design of interactive visual systems before drawing some conclusions.
Chapter II, by Meyer and Bottoni, traces progress in Visual and Diagrammatic Languages: The Logic
Perspective. The chapter starts by outlining the grammatical approach to visual language specification
highlighting its shortcomings as a basis for this research plan. It then revisits the history of logic ap-
proaches to diagrammatic languages and details the comparative advantages and disadvantages of the
different types of logic formalisation. Finally, it develops a new approach based on linear logic, which
avoids most of these shortcomings and completely subsumes the grammatical approach.
Chapter III, by Bottoni, Frediani, Quattrocchi, Rende, Sarajlic, and Ventriglia, proposes a Transfor-
mation-Based Metamodel Approach, which enables an abstract view of the semantic roles that visual
elements can play with respect to the process being described. More specifically, the chapter proposes
an integrated framework and interactive environment based on a collection of metamodels in which it
is possible to express both syntactical characterisations of diagrammatic sentences and their semantic
interpretations.
Chapter IV, by de Lara and Guerra, presents an approach for the definition of Multi-View Visual
Languages (MVVLs). These are made up of a set of different diagram types, which are used to specify
the different aspects of a system. The chapter introduces two techniques to define MVVL environments:
Meta modelling and graph transformation. The former is used to describe the syntax of the language as a
whole. A meta-model for each of the language’s diagram types (viewpoints) is defined as a restriction of
the complete MVVL meta-model. Consistency between views is ensured by translating each one into a
unique repository model, which conforms to the meta-model of the language as a whole. The translation
is performed by automatically generated graph transformation rules.
Chapter V, by Costagliola, Deufemia, and Polese, presents an overview of eXtended positional
grammars (XPG), a grammar formalism for modelling visual notations, which represents an extension
of context-free grammars and describes the XpLR parsing methodology. XPG and XpLR extend po-
sitional grammars (PG) and the associated pLR parsing methodology. These extensions have enabled
the modelling and efficient parsing of a wide class of notations. The associated parsing algorithm is the
XpLR parser based on the well-known LR parsing technique. The benefits of the formalism of a visual
language include easier customisation and modification as well as maintenance and debugging, code
and report generation by definition of suitable semantic productions, and implementation of visual and
textual languages within a common framework.
Chapter VI, by D’Ulizia, Grifoni, and Rafanelli, classifies the Different Kinds of Ambiguities that
can arise in visual sentences, distinguishing between lexical and syntactic ambiguities. When an im-
age associated with a visual sentence is unable to express exactly (be completely faithful to) the user’s
intentions, the system may produce an erroneous interpretation. Ambiguities are generally produced by
(1) the language, which can produce such one-to-many relationships, (2) imprecision introduced by the
xiv

interaction behaviour producing the visual sentence. In the first case, an image can assume more than
one meaning. The second case is connected with incorrect/imprecise information that does not permit
an unequivocal interpretation of the image produced by the drawing actions.
The final chapter in this section, by Caschera, Ferri, and Grifoni, proposes Different Kinds of Solu-
tions to Ambiguities that can arise in visual sentences. This chapter deals with ambiguities related to the
system’s interpretation function and methods to resolve them. These methods can be grouped in three
classes: prevention of ambiguities, a-posteriori resolution, and approximation resolution methods. Pre-
vention methods consider only a predefined set of possible system configurations, avoiding ambiguous
configurations. A-posteriori resolution methods are based on mediation techniques, which enable the
user to disambiguate his or her intention by dialogue. In contrast, approximation resolution methods
are based on theories such as Fuzzy Logic, Markov Random Field, and Bayesian Networks and do not
require user disambiguation.

Section II. Approaches and Methods for Specific Domains

Section II opens with the chapter by Caschera, D’Ulizia, and Tininini, which analyses the main char-
acteristics of VQLs, concentrating on visual languages to Query Conventional Relational Databases
but also examining information systems with a less rigid structure such as Web resources storing XML
documents. It considers two main VQL classifications: the adopted visual representation technique
(e.g., based on forms and tables, diagrams, icons, sketches, or combinations thereof) and the underlying
data model (e.g., visual languages to query relational databases, object-oriented databases, collections
of XML documents, and languages specifically designed for particular data such as geographic and
multidimensional data).
Chapter IX, by Cavalcanti, Schiel, and de Souza Baptista, focuses on the specific category of Visual
Query Languages for Spatio-Temporal Databases, which enable formulation of queries involving both
spatial and temporal dimensions. Current papers treat these dimensions separately with only a few
integrated proposals. This chapter presents a VQS called spatio-temporal visual query environment (S-
TVQE), which allows the formulation of conventional, spatial, temporal, and spatio-temporal database
queries in an integrated environment. With S-TVQE, the user, instead of querying the database by textual
query languages, interacts with the system by visual operators to state the query conditions.
Chapter X, by Fogli, Marcante, Mussio, Provenza, and Piccinno, considers that the knowledge rel-
evant to the design of an interactive system is distributed among several stakeholders: domain experts,
software engineers, and human-computer interaction experts, and presents a Multi-facet Design of In-
teractive Systems. Each community describes an interactive system through visual sentences of a visual
language (VL). This view results in an approach to VIS design based on the definition and use of three
visual languages. Each VL permits user-system interaction process to be specified from a different point
of view and for a different audience.
The chapter by Deray and Simoff starts from the assumption that although interaction is the core of
interactive computing, its mechanisms remain poorly understood. The tendency has been to examine
interactions in terms of the results they produce rather than to provide mechanisms explaining how
interactions unfold in time. The authors present a framework for creating visual languages to represent
interactions using human movement as a source for the language’s core concepts. The approach is moti-
vated and supported by evidence from research on kinaesthetic thinking that constructs based on human
movement support higher-level cognitive processes and can be intuitively recognised by humans.
The final chapter of Section II, by Deufemia, considers Sketch Understanding. This is a particularly
difficult task as the symbols of a sketched diagram can be drawn with different stroke orders, numbers,
xv

and directions. The recognition process is often made even harder by lack of precision and ambiguities
in messy, hand-drawn sketches. The chapter presents a brief survey of sketch understanding techniques
and tools.

Section III. Visual Languages for the Semantic Web

Chapter XIII, by Catarci, Kimani, and Lodi, examines User-Interface Formalisation in Visual Data
Mining. A formal specification facilitates the description of the system properties without touching on
implementation details, and enables fundamental design issues to be detected before they are manifested
in the implementation. An approach for the formalisation of the visual interface of a core data mining
system is given.
Chapter XIV, by Kontopoulos, Bassiliades, and Antoniou, discusses the Visualisation of Defeasible
Logic Rules in the semantic Web domain. Logic plays an important role in the development of the semantic
Web and defeasible reasoning seems a very suitable tool. However, it is too complex for end users who
often need graphical traces and explanation mechanisms for the derived conclusions. The chapter proposes
an approach that uses directed graphs to assist the user by offering the notion of direction, which appears
highly applicable to the representation of rule attacks and superiorities in defeasible reasoning.
The last chapter in the section, by Vrakas, Hatzi, Bassiliades, Anagnostopoulos, and Vlahavas, is
concerned with Knowledge Representation for AI Planning Problems, especially those related to semantic
Web service composition. It discusses current approaches in encoding planning problems and presents
ViTAPlan, a user-friendly visual tool for planning.

Section IV. Visual Interfaces for Standard Languages

Chapter XVI, by Almendros-Jimenez and Iribarne, shows how to use and specialise UML Diagrams for
Describing the User Interfaces of a software system considering three specialised UML diagrams called
user-interaction, user-interface, and GUI-class diagrams. It also examines code generation to implement
the system’s user interfaces through GUI-class diagrams and user-interaction diagrams. A case study of
an Internet book shopping system is introduced to test and illustrate the proposed user interaction and
interface design technique.
Chapter XVII, by Campi, Martinenghi, and Raffio, describes a visual framework, XQBE that covers
the most important aspects of XML Data Management, spanning the visualisation of XML documents,
query formulation, the representation, and specification of document schemata, the definition of integrity
constraints, the formulation of updates, and the expression of reactive behaviours in response to data
modifications.

Section V. Visualisation Aesthetic Computing, and Usability

Chapter XVIII, by Voudouris and Marsh, discusses how cartography, GIS, and HCI shape aspects of
Geovisualisation can support decision-making. The chapter analyses the relationships among geovisu-
alisation, human computer interaction (HCI), geographic information systems (GIS), and cartography as
a means of supporting decision-making. It emphasises the importance of data modelling and associated
visualisations in terms of what the user can do by way of analysis and the methods by which he or she
can undertake the analysis.
xvi

The following chapter, by Skhiri and Zimanyi, presents a Graph Layout Algorithm for Drawing
Biochemical Networks. Due to the huge amount of information available in biochemical databases, bi-
ologists need sophisticated tools to accurately extract the information from such databases and interpret
it correctly. Those tools must be able to dynamically generate any kind of biochemical subgraph (i.e.,
metabolic pathways, genetic regulation, signal transduction, etc.) in a single graph. The visualisation
tools must be able to cope with such graphs and take account of the particular semantics of all kinds
of biochemical subgraphs. The chapter presents an algorithm designed for the generic representation
of biochemical graphs, in which users can present knowledge about how to draw graphs in accordance
with the biochemical semantics.
Fishwick discusses the Aesthetic Computing Method. The purpose of aesthetic computing is to apply
the theory and practice of art and design to the field of computing. The range of aesthetics within the
arts is broader than in mathematics and computing where aesthetics is often synonymous with optimal-
ity criteria. This chapter introduces aesthetic computing for the multimedia representation of formal
structures.
Chapter 21, by Costa, Grings, and Santos, presents Documentation Methods for Visual Languages.
Visual programming languages (VPL) are generally self-documenting; this chapter analyses their use
for the design and documentation of real world applications. Finally, the chapter proposes new tools and
approaches for the documentation of data flow for VPL.
The book ends with the chapter by Padula and Reggiori, discussing Usability. This does not propose
a specific viewpoint on usability but rather examines numerous concepts, aspects, and potentialities,
which must nowadays be taken into consideration when detailing usability and design suited systems.
This topic is currently of great interest in the field of human-computer interaction as it is highly depen-
dent on the user interface. Systems must be used by a community of users in their working activity as
a whole to process material or information, modifying not only rough material but also the working
environment and methods. This requires them to be considered as tools in a social context, which expects
ever-greater technological progress.
xvii

Section I
Visual Languages Theory
xviii

Chapter I
The Theory of Visual Sentences
to Formalize Interactive Visual
Messages
Paolo Bottoni
Università “La Sapienza” di Roma, Italy

Maria Francesca Costabile

Università di Bari, Italy

Stefano Levialdi
Università “La Sapienza” di Roma, Italy

Piero Mussio
Università di Milano

ABStrAct

This chapter introduces an approach to the theory of visual languages based on the notion of visual
sentence as defined by the integration of pictures and descriptions. The chapter proceeds firstly by track-
ing the history of the ideas that stemmed from the initial IEEE Workshop held at Hiroshima (Japan)
during 1984 and then gradually progressing toward the formalisms that build up the theory of visual
languages. The theory of visual sentences allows a coherent view of both static and dynamic aspects of
human-computer interaction, as well as of the relations between the user and the machine during the
interaction.

IntroductIon that graphical and pictorial information (as op-

posed to simple textual information) could better
The official inception of visual languages as an represent both data and computations. This was
independent research field can be dated to the motivated by the development of the first com-
First International Workshop on Visual Languages puters with a visual user interface based on the
(IEEE, 1984) held in Hiroshima in 1984, where desktop metaphor, which in the 80s started to be
a number of researchers subscribed to the idea commercially available after years of developing

Copyright © 2008, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.
The Theory of Visual Sentences to Formalize Interactive Visual Messages

prototypes in research laboratories. Shneiderman regarded as images in that they are represented
coined the term “direct-manipulation” to refer to on a computer screen, which is structured as a
the way users were interacting with the elements rectangular array of pixels; the concept of image
of a computer system including data and functions is thus generalized to any arrangement of signs
operating on such data (Shneiderman, 1983). The appearing on the whole screen. Humans act on
key elements of direct manipulation interface are: these images to steer the computation and/or the
(a) visibility of objects of interest, (b) substitution communication. Computer systems act on them
of textual commands by physical actions and but- to respond to human requests and to synthesise
ton clicks, and (c) rapid and reversible incremental the state of the interactive computation.
actions with immediate feedback. The theory of visual sentences formalizes the
The workshop in Hiroshima was the first of a way the computer associates a computational
series that continued in the following years, spon- meaning with an image shown on the computer
sored by IEEE, and held annually since 1988; it is screen and, conversely, the way it generates an
now called IEEE Symposium on Visual Languages image on the screen from a computation. The
and Human-Centric Computing to indicate a visual sentence is defined as an interpreted im-
broader spectrum of interest. The scientific re- age and a visual language is viewed as a set of
search areas of the participants at the Hiroshima visual sentences in a user-computer dialogue.
workshop were image processing and pattern This theory emphasizes the algebraic structure of
recognition, database querying, and program visual languages, characterised by the relations
visualization (especially in the form of structured between the structures present in the images on
flowcharts, data and control flow diagrams, etc.), the computer screen, by the formal description
thus showing a vast field of applications. Formal of each image (its meaning) managed by the
tools were available from pattern recognition, computer or held by the user, and by the relation-
where systems based on rewriting of strings or ships between image and description. The theory
even of non-linear structures such as pictures or models the semantic level of visual interaction,
graphs, had become in current use already in the pointing out the existence of two semantics, one
70’s. Hence, a mix of practice and theory set the of the user and one of the computer, which can
scene for the following developments. Several explain many of the difficulties of human-com-
formalisms for visual languages have been pro- puter interaction. It also provides a formalism to
posed in the literature ever since. model the transformations of the visual sentences
The so-called theory of visual sentences has that occur during the interaction between a human
been developed with the goal to formalize visual and a computer, through a visual interface (i.e.,
languages used in human-computer interaction a visual language).
(HCI), also called interaction visual languages. In this chapter, we summarize the theory of
Since the 80’s, the availability on the market of visual sentences by highlighting its main concepts,
graphical workstations has placed more and more how it models interaction visual languages of
emphasis on human-computer interaction (HCI) current WIMP (Window, Icon, Menu, Pointer)
via pictorial representations including images, interfaces (Dix, Finlay, Abowd, & Beale, 1998),
sketches, diagrams, and forms, as well as text. Pic- and briefly describe its current developments.
torial representations play two fundamental roles: The chapter has the following organization. We
communication between humans and programs first present some problems central to the defini-
and communication among humans through com- tion of a theory of visual languages, and which
puters. In both cases, pictorial representations are provide the starting impulse for our work. After
the exchanged messages. Such messages may be an informal overview of the main components

The Theory of Visual Sentences to Formalize Interactive Visual Messages

of the theory, we explore characteristics of in- to the provinces to which they belong. This also
teractive processes, and the modeling of visual implies that any visual specification of a visual
transformations by visual rewriting systems. In language must be interpreted so as to identify
the following three sections, we deal with dynam- which parts of its concrete specification can be
ics of visual sentences, focusing, respectively, on abstracted from and which retained. Of course,
the relation between constancy and variability in this wide scope of problems is doubled by the
visual sentences, with mechanisms of enabling and fact that any visual sentence, besides having to be
disabling, and with coordination of transforma- managed by some programmed tool, is ultimately
tions. We then discuss how the components of the designed for being interpreted, and possibly in-
theory can be integrated to provide a framework teracted with, by some human user. Hence, the
for designing interactive visual systems before relations between the interpretation designed
drawing conclusions. into the tool by its programmer and those of the
possible users of the visual sentence, must be
considered. This involves cognitive aspects which
ProBLemS In the theorY of are remote from the studies on formal languages,
VISuAL LAnGuAGeS which the first approaches to the theory of visual
languages had turned to.
The foundation of a theory of visual languages The theory of visual sentences tries to provide
presents specific problems that only to a point can a reference framework to account for the many
be faced with traditional tools from the theory aspects of visual languages, in particular those
of formal, programming, or natural languages. exploited for interaction between human and
While we can still employ the traditional distinc- computer. Along this line, it starts by studying
tion between syntax, semantics, and pragmatics three fundamental aspects: the algebraic struc-
defined by studies on natural languages, the avail- ture underlying the possibility of bidimensional
ability of a second dimension marks a substantial arrangements of structures in an image, the rela-
difference from the one-dimensional setting in tion between these pictorial structures and their
which most frequently studied languages have interpretation, and the notion of visual alphabet
traditionally been formalized. In particular, in as a system of visual types abstracting from, but
absence of a predefined order in generating or constraining, the admissible concrete representa-
interpreting the actual deployment of 2-D symbols tions of instances of the type.
in the 2-D plane a wealth of spatial relations can
be arbitrarily imposed on any arrangement of
bidimensional elements, so that the identification An InformAL oVerVIeW of the
of those semantically significant must be care- theorY of VISuAL SentenceS
fully conducted. Finally, visual languages have
an aspect of concreteness (Goldberg, Burnett, & Starting from the previous considerations, the
Lewis, 1995), so that different arrangements of theory of visual sentences has been developed
the same elements can be interpreted in different with the goal to formalize visual languages used
ways, even when restricted to lie on a single strip. in human-computer interaction (HCI), also called
Consider, for example, a linear representation of interaction visual languages. The first paper that
a highway where nodes represent its exits. Nodes gives the definition of visual sentence is Bottoni,
can be shown at the same distance from one an- Costabile, Levialdi, and Mussio (1995). That paper
other or at distances proportional to the actual was the initial seed of the theory; it formalized
distance on the highway, or clustered according the way the computer associates a computational

The Theory of Visual Sentences to Formalize Interactive Visual Messages

meaning with an image (including the whole of descriptions as multisets of attributed symbols
screen image) and, conversely, the way it gener- was therefore soon adopted. Operations on visual
ates an image on the screen from a computation. sentences were defined, which preserve the struc-
The visual sentence is defined as an interpreted ture of operations on the separate components.
image and a visual language is viewed as a set of As a consequence, pictorial and visual alphabets
visual sentences in a user-computer dialogue. The were defined.
notion of interpretation supports the possibility of In order to allow users to interact with a com-
having different descriptions associated with the puter system and to correctly perform their tasks,
same image, or of relating different structures to it is crucial that: (1) both humans and computers
a same symbol, or, vice versa, of reading a same assign the same meaning to each message (i.e.,
structure in different ways. Several situations to the displayed image, and (2) possible human
are thus modelled, which can arise during an errors be trapped before they cause any undesired
interaction process, and which may either be the consequence. The objective of the theory of vi-
cause of errors and misunderstandings or support sual sentences is to provide a formal framework
complex cognitive processes, or allow the use of and an approach for designing and implement-
multiple representations for visualising a given ing visual interactive systems, which guarantee
set of data as shown through various examples these properties. The formal definition of visual
provided in that paper. sentence emphasizes the fact that in visual inter-
More specifically, a visual sentence is defined active systems a complex relation exists between
as a three-component structure consisting of the images and descriptions. Within the computer,
image on the screen, its meaning (i.e., its descrip- such descriptions result into structured sets of
tion with respect to the computational process data or program code. The definition of visual
defining the interaction), and the relations between sentence allows the dual and coordinated specifi-
program and image components. In Chang and cation and management of images and underlying
Mussio (1996) and Bottoni, Costabile, Levialdi, computational specifications. Moreover, it allows
and Mussio (1997), a set of visual sentences in a a systematic study of the characteristics of current
human-computer interaction is defined as visual visual interactive systems and the specification of
language and it is shown how such a language the requirements for the design of such systems.
may be constructed from a finite generator set. In particular, by adopting this point of view, it
This provides the framework to formalize the becomes possible to study visual sentences and
relations between images and meanings. Meaning visual languages with respect to some basic
description was first given in the form of strings of requirements for interaction, namely that users
attributed symbols (Tsai & Fu, 1980), generalizing cannot cause unexpected events and should not
the seminal work of Fu in pattern recognition, become disoriented by the system behaviour
and following a then widespread approach to (Shneiderman, 1992). Hence, a visual interaction
description formalization, fundamentally aimed process is seen as a sequence of visual sentences
at generalizing parsing techniques from context- in which, for each visual sentence, only a limited
free string grammars to the bi-dimensional world set of legal user actions is possible (Bottoni,
of images and structures therein. It was soon Costabile, Levialdi, & Mussio, 1997; Bottoni,
recognized that the absence of any privileged Costabile, & Mussio, 1999). In particular, visual
reading order in bi-dimensional structure makes sentences within an interaction process are not
it difficult to deal with visual processes other than produced arbitrarily. Each image is generated on
parsing by imposing on the picture the linear order the computer screen from the previous one as the
inherent to strings. A more adequate definition result of a system computation, performed either

The Theory of Visual Sentences to Formalize Interactive Visual Messages

in reaction to a user action or autonomously by visual sentences, some transformations are actu-
the system. ally activated by user actions, and only certain
This process of visual sentence transformation actions can be considered legal. Within the theory
is modelled through visual conditional attributed of vCARWs, mechanisms have therefore been
rewriting systems (vCARWs), which are seen as defined to constrain the possible actions. In par-
a general tool for the formal description of visual ticular, control automata have been defined whose
languages (Bottoni, Costabile, Levialdi, & Mus- automatic generation insures that only correct
sio, 1996; Mussio, Pietrogrande, & Protti, 1991). interactions can occur, leading to the generation
Compared to other formalisms for specifying of visual sentences in the language defined by a
visual languages, vCARWs are characterised by vCARW. Moreover, vCARWs can be enriched
dealing directly with elements of visual alphabets. with enabling mechanisms to prevent the user
That is to say that they do not involve the use of to try and activate rules, which are not coherent
non-terminal symbols, and that they simultane- with the current state. This leads to the defini-
ously rewrite both the pictorial and the description of a new family of vCARWs, called enabling
tion components of visual sentences. A typical visual Conditional Attributed ReWriting systems
vCARW rule states that if a (multi)set of visual (evCARWs) (Bottoni, Chang, Costabile, Levialdi,
elements are present, they will be replaced by & Mussio, 2002).
a different (multi)set, provided some condition Finally, the antecedent-consequent model
holds. Some elements can be present both in the of vCARW rules only allows the expression of
antecedent and in the consequent of a rule, thus local effects through a rule application. In order
indicating that they will be modified in some way, to achieve the effect of global actions, forms of
or that they simply provide context for the rule coordination among rules have been defined, so
application. In Bottoni, Costabile, and Mussio that different rules can be applied concurrently
(1999) it was shown how vCARWs have at least the on different matches, leading to the definition of
same expressive power of conditional set rewriting group evCARWs (gevCARWs) (Bottoni et al.,
systems (Najork & Kaplan, 1993) (the underly- 2002).
ing algebra being one of multisets rather than A moving force in the definition of the theory
of sets). However, the absence of non-terminals of vCARWs was the possibility of reflecting these
requires specific techniques to embody some con- concepts in the implementation of effective visual
straints on the desired language in the rewriting environments. Hence, attention was placed on the
systems. These constraints can be made explicit possibility of automatically generating parts of
by considering a visual language as defined by a the interaction control, starting from their formal
pair (vCARW, VALID), where VALID defines a description in terms of vCARWs.
predicate to be satisfied by the visual sentences
in the language. It is also possible to introduce
different kinds of constraints on the form or on InterActIon VIA
the application of rules in order to achieve spe- VISuAL LAnGuAGeS
cific effects, for example by imposing a certain
sequence on rule application, or preventing the When a user interacts with a WIMP interface
execution of some rules in certain configurations of a computer system, a set of visual sentences
of a visual sentence. belonging to the interaction visual language of
In particular, the specification of interac- that system is produced. In the theory of visual
tive visual languages must consider that, while sentences, the interaction between users and com-
vCARWs define the whole set of generatable puter systems is modelled as a cyclic process in

The Theory of Visual Sentences to Formalize Interactive Visual Messages

which users and systems communicate by mate- one, according to the user’s actions and/or the
rializing and interpreting a sequence of messages computer’s activities.
at successive times. These messages are subject The fundamental cognitive process, which
to two interpretations: one performed by the user, enables the whole visual interaction activity is
depending on his or her role in the task, as well that of recognising sets of pixels on the screen
as on his or her culture, experience, and skills, as functional or perceptual units (Bottoni et al.,
and the second, internal to the system, associating 1997; Bottoni et al., 2002). Examples of css (char-
the message with a computational meaning, as acteristic structures) are letters in an alphabet,
determined by the programs implemented in the symbols, icons, etc. Users associate each cs with a
system. In the case of WIMP interfaces, the mes- meaning. This association depends on the capabil-
sages exchanged between user and system are the ity of the user to interpret the implicit information
whole images represented on the screen display, conveyed by the image on the screen. For example,
formed by texts, pictures, icons, etc. As shown in a radiologist looking at an x-ray image shown on
Figure 1, the human user and the computer visual a Web page recognizes and correctly interprets
interactive system communicate by materialising it but, if not acquainted with Web tools, may not
and interpreting images i(ti) visible on the computer understand how to interact with it. On the other
screen at successive times t,…,tn. hand, a Web surfer looking at the same page may
The image on the screen is the materialisation understand how to operate on the shown buttons
of the meaning intended by the sender (human and menus, but will not be able to interpret the
or computer) and must be interpreted by the x-ray image. Users recognize complex css formed
receiver (computer or human), which associates by simpler ones (words formed by letters, plant
with it a (possibly different) meaning. The screen maps formed by icons, etc.) and attribute a mean-
is the communication surface between human ing to them, which stems from the meaning of
and computer, thus acting as a bi-directional the component css.
channel in the transmission of messages. The The visual interactive system, on its side,
interaction process is modelled as the genera- captures every gesture of the user, and interprets
tion of a sequence of images, each interpreted by it with respect to the image i(t) using a descrip-
both the human and the computer. Each image tion d of it. The system updates the description
derives from the transformation of the previous

Figure 1. The model of visual interaction at the basis of the theory of visual sentences. The image i(ti)
materialises the meaning intended by the sender (either the human H or the computer C) and must be
interpreted by the receiver (either H or C).

The Theory of Visual Sentences to Formalize Interactive Visual Messages

and the image and accordingly changes the set geometrical invariant of the original one; accord-
of pixels in the screen. ingly, it may change the int and mat functions.
In the theory of visual sentences, a character- This view allows the identification of visual
istic structure is a set of pixels to which either the languages as sets of visual sentences without tak-
human or the computer associates a meaning. In ing into account the possible relations between
the adopted model, such a meaning is expressed them. Their definition can thus be expressed in
as an attributed symbol. The association of a terms of visual rewriting systems, by consider-
characteristic structure with a description forms ing the whole set of visual sentences that such a
a characteristic pattern, which is modelled as system can generate.
a triple formed by the characteristic structure Moreover, interest in the so-called interaction
itself (the pictorial component), an attributed VL leads to considering not only the set of all the
symbol (description component), and a relation visual sentences, which can be produced during
between the two. The characteristic structure is user interaction with a visual interactive system,
the materialisation of the attributed symbol and but also the process itself by which visual sen-
the attributed symbol is the interpretation of the tences are produced according to some admissible
characteristic structure. sequence. Hence, the interaction between user
In an image, many structures can be simultane- and computer is formalised as a transformation
ously identified. The set of characteristic patterns process of these visual sentences. As vCARWs are
formed by such structures, their meaning, and the pure rewriting systems (i.e., without the distinc-
relations between them uniquely characterise the tion between terminals and non-terminals), the
message and its meaning. The set of attributed derivation process of a sentence must be restricted,
symbols associated with the characteristic struc- so as to reflect the sequences of interactive steps
tures in an image, form the image description. through which the sentence was produced.
A visual sentence is thus defined as a triple
vs = <i,d,<int,mat>> where i is an image, d is a
description (i.e., a set of attributed symbols), int VISuAL trAnSformAtIonS
a function mapping a structure of i into a sym-
bol describing it, and mat a function mapping a Within an interaction process, every visual sen-
symbol in d into a structure in i. i and d are called tence produced during an interaction derives from
pictorial and description component of the visual a direct parent and all the visual sentences in the
sentence, respectively. interaction process are derived from a common
In general, a visual sentence is built from a predecessor, the initial visual sentence, charac-
finite set of generator elements, called the visual terising the initial state of the visual interactive
alphabet K (Bottoni, Costabile, & Mussio, 1999). system. The pictorial component of this initial vi-
Given a characteristic pattern k, a set Θk={θ…,θn} sual sentence is the image appearing on the screen
of transformations can be associated with it, such when the user starts the interactive session.
that the application of a transformation θi∈Θk to As visual sentences are finite objects, a visual
k results into a new characteristic pattern which sentence produced during the interaction and its
maintains the same type as k. A typical trans- parent differ for a finite set of characteristic pat-
formation θi may change attribute values in the terns, while maintaining a finite set of common
description component, assigning new values in characteristic patterns. Either set can be possibly
the corresponding domains; or it may change the empty. To the user, this is reflected by the obser-
pictorial component, by defining a characteristic vation that some characteristic structures were
structure, which maintains some topological or created in the image, some were deleted, some

The Theory of Visual Sentences to Formalize Interactive Visual Messages

changed in various ways (e.g., changing colour, characteristic patterns in vs are found unchanged
size, orientation, location) and some remained in vs; (e) no other characteristic pattern is in
unchanged. The identification of what changes vs. We say that vs0 generates vsn (vs0⇒C * vsn) if
and what remains constant in the process is the a sequence of visual sentences, vs0, vs,…, vsn-,
key to its understanding from the cognitive point vsn, exists such that vsi⇒ C vsi+ by some rule in P,
of view. The conception of a visual interactive for i=0,…,.n-. Given a vCARW = <K,P, ⇒C> and a
system consists of the design of the set of visual set of visual sentences Ax, called the set of axi-
sentences, which can be produced during an oms, the generated language is the set of visual
interaction and of the algebraic structure to be sentences L(vCARW, Ax) = {vs | ∃vs0∈Ax, vs0⇒*C vs}
imposed on it, taking into account what changes (Bottoni et al., 1999).
and in which way. Different subtypes of vCARWs have been
The rule-based approach adopted in the introduced with different goals. For example, k-
theory of visual sentences and realised through increasing vCARWs are such that they admit rules
the definition of vCARWs, stresses both locality whose effect can only be the insertion of some
aspects—by prescribing what has to be changed— new characteristic pattern and the modification
and coordination aspects—by indicating which of some already existing one, but not the dele-
related transformations must be simultaneously tion of any pattern already present in the visual
executed. Hence, the definition of the admis- sentence. For the 1-increasing case, this allows
sible sequences of rules’ applications is a way the automatic construction of a control automaton
to specify the dynamics of the visual sentence driving the interaction, so that the user is guided
as determined by the interaction. In particular, to produce sentences in an incremental way. The
the designer can specify pairs of sets of char- automaton can be extended with states and transi-
acteristic patterns, in the form of before-after tions to manage user errors. The construction of
rules, to indicate what is required to change as a the automaton can be readily generalised to the
consequence of the interaction. The components increasing case by associating special transitions
of rules are sets of characteristic patterns built on with the request to apply a rule.
a common alphabet. In order to illustrate the transformation process
This approach has its first realisation in the of the visual sentences during user interaction
definition of visual conditional attributed rewrit- with a visual interactive system, let us consider
ing system (vCARW). A vCARW is a triple <K,P, the visual interface of the Corporate Digital
⇒C>, where K is a visual alphabet; P is a set of Library (CDL), a prototype of a visual interac-
rewrite rules, which are pairs of visual sentences tive system developed at the University of Bari
(antecedent and consequent) on K, guarded by a (Costabile, Esposito, Semeraro, & Fanizzi, 1999).
condition on the antecedent that must be satisfied The WWW interface of CDL visualizes in a win-
for the rule to be applicable (a rule is written in dow opened by a Web browser the topics of the
the form r=(ant,cons,cond)); and ⇒C is the rewriting documents stored in the digital library through
relation specifying how rules in P are applied to a “geographic” metaphor where cities, represent-
visual sentences. We say that vs directly generates ing topics of the CDL thesaurus, are connected
vs (vs⇒ C vs) if and only if ∃ r=(ant,cons,cond)∈P by roads, representing links between topics, and
such that: (a) vs contains an instance A of ant; (b) a region represents a set of topics (i.e., the topics
the condition cond evaluates to true on A; (c) vs in a certain class). We abstract from interaction
contains an instance C of cons such that its attri- with the browser and focus on the CDL visual
bute values are equal to those computed from A interface. A colour-based technique codes a topic
according to specifications in cons; (d) all other relevance depending on the number of documents

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Figure 2. Topic map giving the overview of the topics in a library of CDL

the topic has been assigned to. Therefore, the clicking on a coloured rectangle, thus eliminating
rectangle used to represent a topic (city on the less relevant topics (and all their links) in order to
map) is drawn in an appropriate color (appearing obtain a less cluttered map (Costabile, Esposito,
in the grey level figures of this chapter as different Semeraro, & Fanizzi, 1999).
shades of grey). Figure 2 gives an overview of all For example, the transformation of the rel-
topics in the current CDL prototype by showing evance columns in the Topics widget determined
the topic map. A rectangle representing a topic by the user selecting the third rectangle can be
may be partially overlapped by other rectangles. represented by a pair such as the one in Figure
A mouse click on a visible part of an overlapped 3. The arrow below the third rectangle does not
rectangle will bring it in the foreground making appear in the actual visual sentence, but is here
it completely visible. used to indicate that this transformation occurs
Topics are visualised in ten colours ranging as effect of the user’s action of selecting this
from light blue to red, where light blue is used rectangle, abstracting from the concrete realiza-
to represent the less relevant topics, and red the tion of this action.
most relevant ones. The user can interact with This transformation must be coordinated
the widget with the colour scale and with label with the one depicted in Figure 4b to set the ap-
Topics, appearing on the right of the topic map, pearance of less relevant topics to “not visible”
to quickly filter the information on the map by as stated by the condition indicated under the

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Figure 3. The transformation of the relevance columns in the Topics widget (only the pictorial component
of the transformation is shown)

Figure 4. The coordinated transformation for Figure 5. The coordinated transformation forces
disappearance of non-relevant topics links to disappear

Figure 6. The transformation to force irrelevant links to disappear

arrow. At the same time, this transformation is threshold. In this case, a transformation such as
also coordinated with the one depicted in Figure the one in Figure 6 would be synchronized with
5b, which also makes links connected to non transformations of the characteristic pattern
relevant topics disappear. In these specifications, relative to the LINKS relevance columns and the
a reminder of the transformation, which starts associated selection rectangle.
the set of coordinated actions, is placed close to
each pair (in parts a) of Figures 4 and 5. In the
figures, only some attributes of the characteristic conStAncY ASPectS durInG
pattern are shown, namely those significant for InterActIon
the transformation.
All these transformations must concurrently The transformation from a visual sentence into the
occur in parallel when the user performs the ac- subsequent one during human-computer interac-
tion of filtering the topics whose relevance is less tion must be visible in the sense that the variation
than three. Note that this set of transformations is of the image component of the visual sentence
different from the one causing link disappearance must indicate to the user what has changed, thus
when links are filtered according to their own providing both syntactic and semantic feedback
relevance. The specification of such a transfor- to the user. Moreover, the context must be identifi-
mation is shown instead in Figure 5, where the able. These requirements extend the requirement
occurrence of the transformation is conditioned of visibility (Dix, Finlay, Abowd, & Beale, 1998)
to the relevance of the link being below a given to the dynamic case, which can be restated in

0
The Theory of Visual Sentences to Formalize Interactive Visual Messages

our terminology by saying that the characteristic into several sequences of visual sentences with
structure present on the screen must indicate to the a common frame in which each visual sentence
user which is the current state of the interaction. is derived from the previous one according to
Moreover, those characteristic structures must be the current instruction fed by the user. When a
preserved allowing users to keep their bearings in user switches to a new environment, for example
the interaction space. These considerations lead from Word to Netscape™, the previous frame is
to the formulation of the notion of frame to define cancelled and substituted by the frame of the VL
the emergent property of remaining constant in a for the new environment.
sequence of steps, and to the proposal of the notion An example of frame is the menu bar in a
of scaffold as a set of characteristic patterns, which Macintosh application. In Microsoft Word™,
the designer exploits to communicate to the users the frame can also include the external icon bar
the context in which they are operating. if it is positioned at the right end of the screen
When considering interaction, it may hap- and the document window is not moved during
pen that a visual sentence in an interaction VL the interaction. Hence, this icon bar is not part of
is produced several times as a consequence of the frame for the whole dynamic visual language
different actions performed on different visual defined by Word™ since in some cases it may
sentences. In particular, sequences of actions not be visible, but it can be part of the frame for
may produce a closed path between two visual the set of visual sentences generated during a
sentences (e.g., a sequence of undo and redo ac- specific interaction.
tions). In general, some characteristic structures in The designer must provide a way for the system
the image component of a visual sentence remain to communicate the state of the interaction and the
unchanged or change according to a limited set available functionality to the user. Moreover, the
of transformations, thus identifying the frame of formal definition of the set of sequences of visual
reference for the interaction. Moreover, it is usually sentences and of user actions that must be allowed
required that the interpretation of these charac- and those that must be forbidden, requires the
teristic structures remains consistent throughout identification of the inter-dependencies between
the visual sentences so that two occurrences of a local states of individual characteristic patterns
same symbol are associated, via int and mat, to and the global state of the interaction process.
identical characteristic structures appearing in The scaffold is informally defined as a set of
the frame in different visual sentences. characteristic patterns in a visual sentence facili-
As a consequence, the pictorial component of a tating the understanding of the overall strategy for
visual sentence is made of a frame and a variable performing a task and for recording the history
part. The frame is a set of characteristic structures, of the interaction. It consists of a set of icons, text
which maintain a constant support in all the visual lines, and other widgets used to denote:
sentences generated during the interaction. The
variable part is a set of characteristic structures, 1. The activities that can be performed such
which are generated, disappear, or change their as select a characteristic structure launch
support according to the performed user actions a program, terminate the current activity,
or computer activities. The frame identifies the save current results;
context in which a user action or a computer 2. The cornerstones, which allow users to
activity is performed for the user; the variable establish orientation during the task execu-
part identifies the results of any action/activity, tion.
as well as the indication of the current state of
the computation. The whole interaction results

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Examples of cornerstones are window titles, dependencies among rules and the role of trig-
status bar, etc. gering in the rewriting process.
In general, the characteristic structures in An enabling visual conditional attributed
a scaffold must belong to the frame for some rewriting system (evCARW) is a construct
sequence of visual sentences in an interaction D=<K,P,⇒e,Ac,ε,φ,µ> where K is a visual alphabet,
process. More precisely, each state of the scaf- P is a set of visual conditional rewriting rules, Ac
fold defines a sub-visual language for the visual is a set of actions, ε and φ are two mappings ε,φ:
interactive system. All the visual sentences in P→℘(Ac), called enabling and forbidding map-
such a sub-visual language refer to a common ping respectively, µ is a mapping µ:Ac→℘(P). The
context, and the user can recognise this context rewriting relation ⇒e ⊂(DVL×℘(Ac)) is defined
by the persistence of the appearance of the char- on a pair (visual sentence, set of enabled actions) as
acteristic patterns in the scaffold. follows: (vs,en) ⇒e (vs’,en’) if and only if ∃p∈P, ∃
ac ∈ en⊂ Ac, such that p∈µ(ac), vs⇒Cvs’, where
en’=((en-φ(p)) -ac)∪ε(p).
modeLInG InterActIon
dYnAmIcS In words, every rule can enable or forbid the
subsequent application of rules from some set. The
The sentences of an interaction VL are derived pictorial representation of these rules may also
in sequence starting from an initial visual sen- indicate the associated enabled and disabled rules,
tence x. Each visual sentence xi in this sequence as in Figure 7. The fact that a rule is enabled does
is generated from a sentence xi- by the computer not imply that it will be applied, or that it can be
performing a computation or interpreting the applied at all. In particular, it is possible that no
user input and the description part of xi-. This set of characteristic patterns in the current visual
transformation is described as a derivation in a sentence matches the antecedent of a currently
proper visual rewriting system. enabled rule. Conversely, the fact that a rule is
By devising a rule, the designer expresses what currently not enabled implies that it cannot be
may change in an atomic transaction concern- applied, even if an instance of its antecedent can
ing a given set of characteristic patterns without be found in the current visual sentence.
making assumptions on how this transaction is The language generated by an evCARW D from a
going to be realised, provided that its atomicity set of axioms Ax with initial enabled actions A is:
is guaranteed. The user action is thus seen as a
L(D,Ax,A) = {vs | ∃ax∈∈Ax, ∃en∈∈℘(Ac), (ax,A)⇒C(vs,en)}
*
trigger for rule application. The local character
of rule application is such that it may not affect
arbitrary sets of characteristic patterns. Depen- Note that in this case we implicitly indicate the
dencies among sets of characteristic patterns must set of sequences of (enabled) actions, which
therefore be identified and made explicit as rule may cause the transition from a sentence in the
dependencies, by the designer, who has to define language to any other reachable sentence in the
models of coordination to prescribe simultaneous language. In order to model an actual interac-
application of different rules, or of the same rule tion, it is necessary to couple the evCARW with
to several instances of the antecedent. a process able to provide correct sentences in a
The considerations above are at the basis of the language of actions. Hence, we assume that the
proposal of a specific family of rewriting systems, user produces actions in accordance with their
called evCARWs, which take into consideration enabling, and that incorrect actions are captured
and rejected by the system.

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Figure 7. A rule for composing queries out of topics and logical operators. The pictorial representation
also specifies the enabling and disabling of some components according to the ε and φ mappings.

As an example, consider the rule sketched in The evCARWs previously presented allow the
Figure 7, which is part of the specification of a specification of VLs produced as the reaction to
visual system for query composition integrated sequences of correct actions (i.e., enabled ones,
within the prototype CDL. The identifier attribute since disabled ones cannot contribute to a deriva-
is reported for all characteristic patterns as the tion). In most cases, however, it is of interest in
first element in the tuple, either as a constant or visual interaction to trap user incorrect actions
as a variable, with capitalised initial (e.g., TopId and produce warning messages. To model such
for topic, b for a button, etc.). For a topic char- situations, we augment the previous specification
acteristic pattern, only two more attributes have by a VL Trap, whose visual sentences can be com-
been considered: nameOfTopic and selection. Each posed via superimposition on the current visual
occurrence of the variable String in the rule must sentence. The reader may refer to Bottoni (1997)
be unified with a corresponding value found in for the formal definition of superimposition and
the visual sentence for attribute nameOfTopic for to Bottoni (2002) for more detail on VL Trap.
a match of the antecedent. For the button charac-
teristic pattern, the attributes nameOfButton and
enabling are considered. For the text characteristic coordInAtIon In ruLe
pattern, designating the query string, the value APPLIcAtIon
of the attribute contentOfQuery changes from the
constant null to the current value of nameOfTopic It is often the case that a user action provokes ef-
written between quotes. This rule is associated fects that are not only local, but extend over several
with the action “select topic.” The set of rules as- characteristic patterns. For examples, several
sociated with this action also comprises a rule for windows can be linked to react in a consistent
the case in which the query is already partially way to a single zoom command, or to opening
formed, so that the buttons for operators and or closing a window (Bianchi, Bottoni, Mussio,
clearing are enabled. These rules specify the & Protti, 1993). Even in a same dynamic VL, it
way in which the CDL prototype allows users to can often occur that simultaneous modifications
request documents by direct manipulation of the of several characteristic patterns are started by
map shown on the screen. The reader may refer the same action. For example, when using radio
to Bottoni (2002) for more details. buttons in a dialogue window or selecting a tex-

The Theory of Visual Sentences to Formalize Interactive Visual Messages

tual item from a menu, the selection of an item In other words, this means that several rules, all
disables any previously selected item. Moreover, associated with a same action, can be simultane-
the corresponding characteristic structures are ously activated on different sets of characteristic
also constrained to appear or disappear simul- patterns in the current visual sentence. This
taneously. The definition of these coordination definition also accounts for the case when several
aspects requires an extension to the basic model actions can be concurrently activated, resulting
of evCARWs bringing us into the formalism of in several groups of characteristic patterns being
parallel rewriting systems where we admit that active. Again, one can augment this definition to
several rules can be applied at a same time. In par- consider trapping and recovery actions.
ticular, we assume that all characteristic patterns For example, in the case presented in Section
not explicitly affected by a rule are simultaneously 4.1, we have that all topic characteristic patterns are
rewritten through the application of an identity in a same group, together with the Topics widget,
rule. The following definition is an extension to the all link characteristic patterns are in a same group
case of visual rewriting systems of the concept of together with the Links widget, and a topic and a
group rewriting for attributed 0L-systems, which link characteristic patterns are in a same group if
was applied to coordination problems in Bottoni, the link is connected to the topic. Formally, let αi =
Mauri, and Mussio (1998a). link(IdL,(Top,Top)) and αj = topic(IdT,LinkList), where
Let Π(K) define the set of characteristic patterns we have presented only the attributes relevant for
that can be generated from the elements of an alpha- group assessment. Then ρ({αi,αj}) = true if and only
bet K. An evCARW with groups, or gevCARW, is if IdL∈LinkList ∧ (IdT=Top ∨ IdT=Top).
a construct gD=<K,P,⇒g,Ac,ε,φ,µ,ρ,ν>, where K,
P, Ac, ε, φ, µ are the same as in evCARWs, ρ is
a symmetrical predicate of the form ρ:℘(Π(K))→ deSIGnInG VISuAL
{true, false} assessing whether a set of charac- enVIronmentS
teristic patterns form a group, ν a mapping ν:
Π\Iden→℘( P), where Iden is the set of identity The formal specification of the intended be-
rules, with the property that if pi∈ν(pj), pi∉Iden, haviours expressed in terms of vCARWs allow
then pj∈ν(pi). Moreover, the rules in ν(p) are such the automatic generation of significant parts of
that they are all associated with the same action. interactive environments, in particular as regards
If ν is extended to Iden, the property holds also dialogue management and coordination of behav-
in Iden. To sum up, ρ defines group formation, ν iours. Such specifications are typically platform-
defines for each non-identical rule the associated independent and must therefore be adapted to the
set of rules. specific intended realisations, which may involve
The direct generation relation is as follows, use of different interaction devices, platforms, and
under the constraint imposed by the set of en- programming languages. Hence, Bottoni, Costa-
abled actions, acting as before. vs⇒g vs’ iff vs is bile, Fogli, Levialdi, & Mussio (2001) propose the
characterised by the union of sets of characteristic adoption of a design space, depicted in Figure 8,
patterns {α, α, …, αn}, vs’ is characterised by in which each dimension represents types of lan-
the union of sets of characteristic patterns {β, β, guages: (1) user activity languages to specify users
…, βn}, αi=βi or αi→βi∈P for each i=,…,n, and if activities; (2) programming languages to specify
ρ(αi ∪αj)=true, and a rule αj→βj∈P in the domain system computations; (3) pictorial languages to
of ν has been applied, a rule αi→βi∈P has also specify the images appearing on the computer
been applied, where αi→βi∈ν(αj→βj). screen. The coordinates of this space indicate the

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Figure 8. The 3-D interaction modeling space. (i.e., which programming languages, toolkits, I/O
Each point in the space identifies the abstraction devices, are available).
levels of the system model. For example, a system specification may be
composed of programs in some high-level lan-
guage for the computational axis, references to
Activity Language icons for the pictorial axis, and an alphabet of
Tasks
actions such as “select icon” without consider-
Actions
ing the gestures by which they are performed as
Gestures activity language. A mapping to concrete realisa-
Pixel Cs Screen level
tions of these activities as sequences of gestures
Machine Language
Pictorial Language will have to be constructed in order to construct
Instruction Set Architecture
Operating System such a system.
High Level Language
In general, the design of a concrete system is
High Level Customized Language
detailed as the design of the interaction visual lan-
Programming Language
guage, which will allow users to interact with the
system. In this process, the designer moves back
level of abstraction for each dimension and each and forth along the different axes to define several
point represents a virtual interaction machine visual languages at different levels of abstraction
(i.e., a hypothetical interactive system defined and with increasing levels of complexity.
by the user activity language, the programming In particular, starting from user and task
language, and the pictorial language). analysis and from the observation of the produced
The consistent design of an interactive system documents, an explicit but informal characteri-
requires that precise mappings be defined both sation of the domain is produced in the form of
intra- and inter-dimensions. For example, a com- a visual alphabet and a set of rules expressed in
plex user action must be reducible to a sequence the user language, defining a first task visual
of elementary gestures issuing commands to the language TVLo. The adequacy of TVLo with
computer (intra-dimension mapping). On the other respect to the user significant constructs requires
hand, a user gesture must be managed through a verification phase to demonstrate that all the
appropriate sequences of instructions and must sentences in the TVLo can be produced and that
result into the modification of identified sets of no sentence violating the constraints required by
pixels (inter-dimension mapping). the users can be created. Moreover, a validation
To design an interactive computer application, phase is also necessary to check that the visual
the set of admissible user activities, as well as alphabet and the rules can be properly understood
the set of rules, which determine the modelled and managed by the user.
interaction process, have to be specified. This From the observation of the produced docu-
specification process starts from the construction ments and of the users’ activities, we abstract the
of the set of the css and the set of computational definition of the task visual language (TVL) (i.e.,
constructs used to build the rules, and arrives at we identify the visual alphabet and rules implicitly
the definition of: (1) a language to specify user used by experts in building and transforming
activities; (2) a language to specify css; and (3) their documents). This process can go through
a language to specify computational constructs. the definition of several tentative generalisations
These specifications may occur at different levels of the TVLo before an agreement is reached. In
of abstraction. The choice of the abstraction level particular, it is possible that the typical graphical
is determined by the tools available to the designer constructs employed by the users are not suf-

The Theory of Visual Sentences to Formalize Interactive Visual Messages

ficiently expressive or non-ambiguous to build a The validation of the augmented TVL (ATVL)
formal notation. Several steps are then required requires a partial implementation or mock-up of
to reach a form adequate to be formalised by the system so that the consistency of the dynam-
eliminating the identified ambiguities and in- ics can be checked with the one expected by the
consistencies. user. In particular, the interaction dynamics can
Once an agreement is achieved on what has be simulated with mock-ups and simple proto-
to belong to the TVL, a formal definition of this types to ensure that the behaviours of the cps are
language is needed. In Bottoni, Chang, Costabile, understandable by the user and that the user has
Levialdi, and Mussio (1998b), details on this are access to all the needed details made implicit by
provided. A semi-formal verification is possible the synthetic notations. The verification of ATVL
by checking that every observed document can be requires that all the vss in the TVL remain acces-
produced with the use of the rewriting system that sible and obtainable through ATVL.
specifies the language, that incorrect documents Finally, the interaction visual language (IVL)
cannot be composed, and that the legal sentences is specified by defining how the vss of the ATVL
correspond to acceptable intermediate steps can be integrated into visual sentences with
during document construction and transforma- additional support for the interaction, such as
tion. The formal notation thus obtained must be frames and scaffolds providing context for user
evaluated with respect to the user semantics. In orientation in the interaction, or giving access to
particular, it is necessary to verify that the users the system functionalities, tools to operate on the
correctly understand the definition of the derived vss themselves, etc. The definition of the IVL also
TVL and of its use. encompasses the constraints posed by the strate-
The TVL thus defined is then augmented to gies of document construction identified during
take a communicational advantage from the in- the task analysis. These may imply sequences of
teractive capabilities of the system. Indeed, the actions to be performed in the production of the
sequences of TVL vss describe the development document, checklists, levels of access to func-
of experts’ activity. The TVL is an intermediate tionalities, etc.
step in the design of IVL, which proceeds by Hence, each step of the IVL design consists
enriching the TVL with the definition of the cps of: (1) design and implementation of abstract or
through which the visual system communicates to concrete tools; (2) verification of their soundness,
the user the state of the interaction and provides completeness, and consistency; and (3) validation
access to its functionalities. Moreover, the TVL of their adequacy with respect to human-com-
itself can be augmented to exploit the possibili- puter communication, using tools according to
ties of interaction and animation available in a the technique chosen with respect to the users’
computer system. The designer may augment semantics.
TVL by introducing new css to make the vss in During this process, each step or sequence of
the language more compact and expressive. The steps may be iterated several times if the verifica-
designer can also use attention holders, feedback tion or the validation of the results of some steps
mechanisms, animation effects, detail exhibition, in the procedure so requires. Each step in the IVL
and focus modification. The notation must also be design requires the preliminary choice of tools
adapted--hence often changed and approximated- necessary to execute the step and of a technique
-to the digital technologies by which the image of assessment of the obtained results: new tools or
on the screen is generated as well as to the fixed, new sets of data. In the tool choice and use, prob-
limited dimensions of the screen. lems arise from the existence of the two semantics,

The Theory of Visual Sentences to Formalize Interactive Visual Messages

the user, and the program ones. Evaluation of the as related to a mismatch between the perceived
results must be performed with respect to both and real absence or presence of resources dur-
semantics and require verification and validation ing the interaction process (Bottoni & Levialdi,
through experimental evaluation. 2005, 2006). The notion of resource has also been
related to an abstract view of visual languages
as defined by the existence of spatial relations
concLuSIon between identifiable elements with semantic
relevance (Bottoni & Costagliola, 2002; Bottoni
This chapter has illustrated the theory of visual & Grau, 2004). This approach, considering both
sentence developed with the aim of formally syntactical and semantical aspects of the defini-
specifying visual interactive system. Sets of tion of visual languages can be specified through
visual sentences are specified through the use of the identification of the specific roles, played by
visual rewriting systems called vCARWs. Other concrete instances with respect to the metamodels
families of vCARWs have been defined whose defining a syntactic family or a semantic variety
visual rewriting rules, augmented with regulation (Bottoni, Frediani, Quattrocchi, Rende, Sarajlic,
mechanisms, coordinate transformations of sets & Ventriglia, 2006a; Bottoni, Quattrocchi, &
of visual elements so that the complete dynamic Ventriglia, 2006b). A metamodel for interaction
behaviour of a visual interface can be specified. can also be defined so that abstract actions can
An important aspect of the interpretation be associated with different syntactic or semantic
process is that it is not simply compositional, roles (Bottoni, de Lara, & Guerra, 2006c).
as context can play a role, and often emergent A second line of research was motivated by
phenomena lead to revisions or enrichments of the use of visual expressions as boundary objects
the current description and of the interpretation (Arias, Eden, Fischer, Gorman, & Scharff, 2000)
function itself. Hence, the process by which the (i.e., shared objects to talk about and think with
description and the interaction process itself are in Web-based collaborative activities and in end-
created can only partially be modelled through user development (EUD)) (Carrara, Fogli, Fresta,
a local analysis of the image. The theory takes & Mussio, 2002; Costabile, Fogli, Mussio, &
this into account by allowing the definition of Piccinno, 2006a; Sutcliffe & Mehandjiev, 2004).
attributed symbols associated with the whole Visual interactive systems for collaborative ac-
image and by allowing the possibility of defining tivities allow users that have different stances on
structures of arbitrary complexity. the problems at hand to cooperate to a common
From this theoretical core, different lines of de- activity interacting among them and with systems
velopment have been pursued. In a first line, some via visual sentences (which are at this level the
works by Bottoni and Levialdi have seen char- boundary objects) (Mussio, 2003). The design
acteristic patterns as visual resources, consumed and development of such systems is recognised
and produced during visual transformations. The as a complex activity, which requires collab-
WIPPOG language (Bottoni, Mussio, Olivieri, & orative and participatory approaches that include
Protti, 1998) has therefore been adopted as a way to representatives of users in the design team, thus
specify resource transformations, and uniformly performing activities of EUD (Costabile, Fogli,
employed to define both visual rewriting systems Fresta, Mussio, & Piccinno, 2003; Costabile,
and interactive control mechanisms (Bottoni, De Fogli, Lanzilotti, Mussio, & Piccinno, 2006c).
Marsico, Di Tommaso, Levialdi, & Ventriglia, The visual interactive systems, the widgets, and
2004). The adoption of a resource-based perspec- the other visual entities are described as open
tive also supports modeling of interaction errors virtual systems called virtual entities (Costabile,

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Fogli, Fresta, Mussio, & Piccinno, 2003; Fogli, Proceedings of the International Conference on
Mussio, Celentano, & Pittarello, 2002;), whose Advanced Visual Interfaces (AVI 2002) (pp. 177-
dynamics are described starting from the tools 189). Trento, Italy: ACM Press.
introduced by the theoretical core described in
Bianchi, N., Bottoni, P., Mussio, P., & Protti, M.
this chapter.
(1993). Cooperative visual environments for the
Finally, the interaction model at the basis of the
design of effective visual systems. Journal of Vis-
theory of visual sentences has been recently ex-
ual Languages and Computing, 4(4), 357-381.
tended by considering an important phenomenon
occurring during the use of interactive systems, Bottoni, P., Costabile, M.F., Levialdi, S., & Mus-
which is called co-evolution of users and systems sio, P. (1995).Formalising Visual Languages. In
(Costabile, Fogli, Marcante, & Piccinno, 2006b). Proceedings of Visual Language ‘95, (pp. 45-52).
It is based on the following two observations: (1) IEEE Computer Society Press.
once people gain proficiency in system usage,
Bottoni, P., & Costagliola, G. (2002). On the defi-
they would like to use the system in different
nition of visual languages and their editors. In M.
ways and need different interfaces than those
Hegarty, B. Meyer, & N. Hari Narayanan (Eds.),
they required when they were novice users (user
Diagrammatic representation and inference: Se-
evolution) (Nielsen, 1993; Norman, 2005); and
cond International Conference, Diagrams 2002,
(2) designers are then forced to evolve the system
Proceedings, LNAI 2317 (pp. 305-319). Berlin,
to meet the new users’ needs (system evolution)
Germany: Springer.
(Arondi, Baroni, Fogli, & Mussio, 2002; Bourguin,
Derycke, & Tarby, 2001; Carroll & Rosson, 1992; Bottoni, P., & Grau, A. (2004). A suite of meta-
Costabile, Fogli, Mussio, & Piccinno, 2006a). In models as a basis for a classification of visual
Costabile, Fogli, Marcante, and Piccinno (2006b) languages. In P. Bottoni, C. Hundhausen, S.
and Costabile, Fogli, Marcante, Mussio, and Levialdi, & G. Tortora (Eds.), Proceedings of
Piccinno (2006d), a model of the interaction and VL/HCC 2004, (pp. 83-90). Roma, Italy: IEEE
co-evolution processes (ICE model) occurring CS Press.
between users and system is discussed. It leads
Bottoni, P., & Levialdi, S. (2005). Resource-based
to re-examine the way interactive system are
models of visual interaction: Learning from errors.
designed and to define new theoretical founda-
Proceedings of the VL/HCC 2005 (pp. 137-144).
tions on which new design approaches should
Dallas, USA: IEEE Computer Society Press.
be based.
Bottoni, P., & Levialdi, S. (2006). Entities, sub-
stances, and time: Resources for interaction. In
referenceS Proceedings of VL/HCC 2006 (pp. 121-124).
Brighton, UK: IEEE Computer Society Press.
Arias, E., Eden, H., Fischer, G., Gorman, A., &
Bottoni, P., Chang, S. K., Costabile, M. F., Levialdi,
Scharff, E. (2000). Transcending the individual
S., & Mussio, P. (1998b). On the specification of
human mind--creative shared understanding
dynamic visual languages. In Proceedings of IEEE
through collaborative design. ACM Trans on
Symposium Visual Languages (pp. 14-21). Halifax,
CHI, 7(1), 84-113.
Canada: IEEE Computer Society Press.
Arondi, S., Baroni, P., Fogli, D., & Mussio, P.
Bottoni, P., Chang, S. K., Costabile, M. F., Levialdi,
(2002). Supporting co-evolution of users and
S., & Mussio, P. (2002). Modeling visual interac-
systems by the recognition of Interaction Patterns.
tive systems through dynamic visual languages.

The Theory of Visual Sentences to Formalize Interactive Visual Messages

IEEE Trans. on Systems, Man, and Cybernetics- Bottoni, P., Mauri, G., & Mussio, P. (1998a). Coor-
-Part A, 32(6), 654-669. dination through group rewriting. In A. Salomaa
& G. Paun (Eds.), Grammatical models of multi
Bottoni, P., Costabile, M. F., Fogli, D., Levialdi,
agent systems (pp. 226-246). IEEE Computer
S., & Mussio, P. (2001). Multilevel modeling and
Society Press: Gordon & Breach.
design of visual interactive systems. In Proceed-
ings of HCC 2001 (pp. 256-263). Stresa, Italy: Bottoni, P., Mussio, P., Olivieri, B., & Protti,
IEEE Computer Society Press. M. (1998). A completely visual environment
for agent-based computing. In T. Catarci, M. F.
Bottoni, P., Costabile, M. F., & Mussio, P. (1999).
Costabile, G. Santucci, & L. Tarantino (Eds.),
Specification and dialogue control of visual in-
Proceedings of AVI’98 (pp. 261-263). L’Aquila,
teraction through visual rewriting systems. ACM
Italy: ACM Press.
Trans. on Programming Languages and Systems
(TOPLAS), 21(6), 1077-1136. Bottoni, P., Quattrocchi, P., & Ventriglia, D.
(2006b). Constraining concrete syntax via meta-
Bottoni, P., Costabile, M. F., Levialdi, S., &
model information. Proceedings of VL/HCC 2006
Mussio, P. (1996). Visual conditional attributed
(pp. 85-88). Città, stato: IEEE Computer Society
rewriting systems in visual language specification.
Press.
Proceedings of VL 1996 (pp. 156-163). Boulder:
IEEE Computer Society Press. Bourguin, G., Derycke, A., & Tarby, J. C. (2001).
Beyond the interface: Co-evolution inside interac-
Bottoni, P., Costabile, M. F., Levialdi, S., &
tive systems: A proposal founded on the Activity
Mussio, P. (1997). Defining visual languages for
theory. In Blandford, A., Vanderdonckt, J., & Gray,
interactive computing. IEEE Trans. on Systems,
P.H. (Eds), Proceedings of the of IHM-HCI 2001
Man, and Cybernetics--Part A, 27(6), 773-783.
Conference (pp. 297-310). Lille, France, People
Bottoni, P., de Lara, J., & Guerra, E. (2006c). and computer XV – Interactions without Frontiers:
Metamodel-based definition of interaction with Springer Verlag.
visual environments. Proceedings of MDDAUI
Carrara, P., Fogli, D., Fresta, G., & Mussio, P.
2006. In press.
(2002). Toward overcoming culture, skill, and
Bottoni, P., De Marsico, M., Di Tommaso, P., situation hurdles in human-computer interaction.
Levialdi, S., & Ventriglia, D. (2004). Definition International Journal Universal Access in the
of visual processes in a language for expressing Information Society, 1(4), 288-304.
transitions. Journal of Visual Languages and
Carroll, J. M., & Rosson, M. B. (1992). Delib-
Computing, 15(3), 211-242.
erated evolution: Stalking the view matcher in
Bottoni, P., Frediani, D., Quattrocchi, P., Ren- design space. Human-Computer Interaction, 6(3
de, L., Sarajlic, G., & Ventriglia, D. (2006a). A and 4), 281-318.
transformation-based metamodel approach to the
Chang, S. K., & Mussio, P. (1996). Customized
definition of syntax and semantics of diagram-
visual language design. Proceedings of Inter-
matic languages. In this volume.
national Conference on Software Engineering
Bottoni, P., Mariotto, M., Mussio, P., & Biella, G. and Knowledge Engineering (SEKE 96) (pp.
(1995). The design of anthropocentric cooperative 553-562). Lake Tahoe, NV: IEEE Computer
visual environments. Proceedings of VL 1995 (pp. Society Press.
334-341). Darmstadt, Germany: IEEE Computer
Society Press.

The Theory of Visual Sentences to Formalize Interactive Visual Messages

Costabile, M. F., Esposito, F., Semeraro, G., & specification of virtual reality environments. Pro-
Fanizzi, N. (1999). An adaptive visual environ- ceedings of the Multimedia Software Engineering
ment for digital libraries. International Journal (MSE2002) (pp. 148-155). Newport Beach, CA:
on Digital Libraries, 2(2 and 3), 124-143. IEEE Computer Society Press.
Costabile, M. F., Fogli, D., Fresta, G., Mussio, P., Goldberg, A., Burnett, M., & Lewis, T. (1995).
& Piccinno, A. (2003). Building environments What is visual object-oriented programming? In
for end-user development and tailoring. In Pro- M. M. Burnett, A. Goldberg, & T. G. Lewis (Eds.),
ceedings of the 2003 IEEE Symposia on Human Visual object-oriented programming: Concepts
Centric Computing Languages and Environments and environments (pp. 3-20). Greenwich, CT:
(HCC’ 03) (pp. 31-38). Auckland, New Zealand: Manning Publications Co.
IEEE Computer Society Press.
Ichikawa, T., Chang, S.K. (eds.) (1984). IEEE
Costabile, M., Fogli, D., Lanzilotti, R., Mussio, Computer Society Workshop on Visual Lan-
P., & Piccinno, A. (2006c). Supporting work guages. Hiroshima, Silver Spring, USA: IEEE
practice through end-user development environ- Computer Society Press.
ments. Journal of Organizational and End User
Mussio, P., Pietrogrande, M., & Protti, M. (1991).
Computing, 18(4), 43-65.
Simulation of hepatological models: A study in
Costabile, M. F., Fogli, D., Marcante, A., Mussio, visual interactive exploration of scientific prob-
P., & Piccinno, A. (2006d). Interactive environ- lems. Journal of Visual Languages and Comput-
ments supporting user and system co-evolution. ing, 2(1), 75-95.
In C. Hochberger & R. Liskowsky (Eds.), Lecture
Mussio, P. (2003). E-documents as tools for the hu-
Notes in Informatics (LNI). Proceedings of In-
manized management of community knowledge.
formatik 2006: Vol. P93 (pp. 588-591). Dresden,
In H. Linger et al. (Eds.), Constructing the infra-
Germany: Köllen Druck+Verlag GmbH.
structure for the knowledge economy: Methods
Costabile, M. F., Fogli, D., Marcante, A., & Pic- and tools; theory and practice. Keynote Address,
cinno, A. (2006b). Supporting interaction and ISD 2003. Melbourne, Australia: Kluwer.
co-evolution of users and systems. Proceedings
Najork, M., & Kaplan, S. M. (1993). Specifying
of AVI 2006 (pp. 143-150). Venezia, Italy: ACM
visual languages with conditional set rewrite sys-
Press.
tems. Proceedings of the 1993 IEEE Workshop on
Costabile, M. F., Fogli, D., Mussio, P., & Piccinno, Visual Languages (pp. 12-18). Bergen, Norway:
A. (2006a). End-user development: the software IEEE Computer Society Press.
shaping workshop approach. In H. Lieberman,
Nielsen, J. (1993). Usability engineering. San
F. Paternò, & V. Wulf (Eds.), End user develop-
Diego: Academic Press.
ment (pp. 183-205). Dordrecht, The Netherlands:
Springer. Norman, D. A. (2005). Human-centered design
considered harmful. Interactions of ACM, 12(4),
Dix, A., Finlay, J., Abowd, G., & Beale, R. (1998).
14-19.
Human computer interaction. London: Prentice
Hall. Shneiderman, B. (1983). Direct manipulation:
A step beyond programming languages. IEEE
Fogli, D., Mussio, P., Celentano, A., & Pittarello,
Computer, 16(8), 57-69.
F. (2002). Toward a model-based approach to the

0
The Theory of Visual Sentences to Formalize Interactive Visual Messages

Shneiderman, B. (1992). Direct manipulation.

In Designing the User Interface. Reading, MA:
Addison-Wesley.
Sutcliffe, A., & Mehandjiev, M. (2004). End-user
development. Communications of the ACM, 47(9),
31-32.
Tsai, W. H., & Fu, K. S. (1980). Attributed gram-
mar: A tool for combining syntatic and statistical
approaches to pattern recognition. IEEE Trans. on
Systems, Man, and Cybernetics, 10(12), 873-885.

Chapter II
Visual and Diagrammatic
Languages:
The Logic Perspective

Bernd Meyer
Monash University, Australia

Paolo Bottoni
University of Rome “La Sapienza”, Italy

ABStrAct

In this chapter we investigate a new approach to formalizing interpretation of and reasoning with visual
languages based on linear logic. We argue that an approach based on logic makes it possible to deal
with different computational tasks in the usage of visual notations, from parsing and animation to rea-
soning about diagrams. However, classical first order logic, being monotonic, is not a suitable basis for
such an approach. The chapter therefore explores linear logic as an alternative. We demonstrate how
parsing corresponds to linear proofs and prove the soundness and correctness of this mapping. As our
mapping of grammars is into a subset of a linear logic programming language, we also demonstrate
how multi-dimensional parsing can be understood as automated linear deduction. We proceed to discuss
how the same framework can be used as the foundation of more complex forms of reasoning with and
about diagrams.

IntroductIon term is ambiguous: we often speak of the “visual

language” of an artist to describe the artist’s
Like written textual languages and their spoken particular way of expressing himself or herself.
equivalents, many other types of symbolic nota- This almost never means that we could define a
tions, for example musical scores, can be regarded formal system of expression that the artist uses and
as languages. By “languages” we mean that they through which we could mechanically derive an
constitute communication systems based on “interpretation” of a work. In contrast, the syntax
symbol combinations and that they are, at least in and semantics of more technical visual languages
principle, formalizable. Where such systems are such as organizational charts, dance notation,
visual or graphical in nature, it is thus justified electrical circuit diagrams, and mathematical
to speak of “visual languages.” However, this notations can often be formalized rigorously. A

whole spectrum of semi-formal visual languages frameworks. However, as diagrammatic notations

exists between these two extremes, ranging from are inherently two- or three-dimensional rather
draft sketches in architectural design over maps than linear (as textual languages are), these gram-
to sign languages such as ASL or Auslan. mar frameworks have to be extended. This has
In general, visual languages do not only differ given rise to the field of multi-dimensional gram-
in the degree to which they are strict and can be mars. The various forms of multi-dimensional
formalized, but also in the degree to which they grammars, such as relational grammars, multiset
use distinct symbols as their fundamental means grammars, and constraint multiset grammars, are
of expression. For example, a valve in a mechani- now relatively mature formalisms that are widely
cal diagram is clearly a distinct symbol, whereas used to specify diagram syntax and to build com-
it is not so clear whether shading in a map can or putational tools for diagrammatic languages.
should always be understood in the same way. For The majority of computational methods for
the sake of clarity, we will use the term “diagram- the automatic interpretation of diagrams take a
matic languages” for visual languages that are syntax-based two-step approach consisting of
based on individual symbols and form expressions parsing and interpretation proper, and multi-
through spatial arrangement of these symbols. dimensional grammar frameworks are well
As is evident from the previous examples, such suited to cover the first phase. However, when
diagrammatic languages are central in almost all it comes to the interpretation proper (i.e., the
technical domains, and they are also relevant to treatment of semantics) they provide little support.
many aspects of everyday life. Thus, this phase is most often covered by ad-hoc
The research field of “diagrammatic rea- approaches defined on a case-by-case basis for
soning” investigates such languages from the individual diagram languages.
perspective of computer science and cognitive Grammar-based methods also have some other
science. The central questions for this field are shortcomings: Firstly, it is important to acknowl-
how we understand such languages and how we edge that diagrams are rarely used in isolation, but
employ them as tools in reasoning processes. To rather in combination with textual language. Thus,
investigate these questions in a rigorous way, we the interpretation of diagrammatic languages
need formal frameworks that allow us to define needs to take into account non-diagrammatic
diagrammatic languages and to reason about them. contextual information, for example for disam-
From the perspective of computer science, we are biguation. In a grammatical framework, the inte-
particularly interested in how interpretation and gration of such contextual information is difficult.
reasoning can be performed automatically on the Secondly, while multi-dimensional grammars are
basis of formal specifications of diagrammatic relatively mature computational tools, their theory
languages. It seems obvious that the treatment is not well developed, meaning that it is difficult,
of diagrammatic languages should distinguish for instance, to prove formal properties such as
between syntax and semantics, just as we do for the equivalence of grammars.
textual languages, and that a general framework In the light of these shortcomings, logic
needs to bridge between these two aspects. approaches present themselves as a valuable
The treatment of diagram syntax has been alternative to grammatical frameworks. Logic
investigated for close to three decades and is now approaches, by their very nature, are semantic.
reasonably well understood. The vast majority of A logic calculus for reasoning with and about
methods that are employed are essentially gen- diagrammatic languages is thus a more adequate
eralizations of approaches from computational approach to treating semantics. Interestingly, the
linguistics for textual languages, mostly grammar increased power of a logic specification over a

Visual and Diagrammatic Languages

grammatical one does not come at a price, as a forms of multi-dimensional grammars that have
well-designed logic approach can fully subsume been presented in the literature is beyond the scope
the grammatical approach. of this article, but the interested reader can find
The remainder of the chapter will trace out a comprehensive survey in Marriott et al. (1998),
the progress that the last two decades of research which also contains the “fossil record” of visual
have made in this direction. In the next section, language specification.2
we briefly outline the grammatical approach to For the purpose of this chapter, it will suffice
visual language specification and highlight its to divide the class of multi-dimensional gram-
shortcomings as a basis for this research plan. We mars broadly into two categories: relation-based
will then revisit the question: Why do we want grammars and attribute-based grammars.
reasoning with diagram languages? to motivate We will briefly sketch an example of relation-
an alternative approach, and outline what forms based grammars to highlight the basic charac-
of reasoning will be required. Following this, teristics of this approach and point out why this
we will outline the history of logic approaches approach is insufficient as a basis of more general
to diagrammatic languages and detail the com- forms of diagrammatic reasoning. We will then
parative advantages and disadvantages of the turn our attention to attribute-based formalisms,
different types of logic formalization. Based on which provide a better basis for this, and discuss
this, the core of this chapter will develop a new these in more detail.
approach based on linear logic, which avoids most
of the shortcomings and completely subsumes relation-Based Grammars
the grammatical approach. We will show this
subsumption formally. We will then briefly outline Almost all specification formalisms model a
existent software systems based on this approach. diagram as a set of primitive graphical objects
The concluding section reflects on what has been (tokens) that are classified into primitive types.
achieved so far and gives recommendations for The terms terminal (or primitive) and non-termi-
future research direction. nal (or derived) are used analogously to the case
of textual languages. The only difference lies in
the fact that terminal types in diagrams refer to
the cLASSIcAL APProAch: graphic primitives such as line and circle, instead
GrAmmAr SPecIfIcAtIon of textual tokens. A symbol is an instance of a
symbol type.
The vast majority of approaches to visual lan- As a running example, consider the language
guage specification interpretation are based on of state transition diagrams such as the one shown
various forms of multi-dimensional grammars, in Figure 1. To represent such a diagram, we would
which are essentially either extensions of string use sets of symbols with types such as circle, line,
grammars or graph grammars.1 The main dif- arrow, and text label. Such a set structure by itself
ferences between grammars for diagrammatic does, of course, not reflect the exact appearance
languages and grammars for textual languages of the diagram or the spatial relations between
is that the former must allow us to capture the these symbols. The relation-based approach
multi-dimensional (usually 2-dimensional) dispenses with representing the exact geometry
spatial arrangement of symbols instead of just and instead just represents the spatial relations
a sequential (1-dimensional) concatenation. We between individual symbols that are essential to
therefore call such grammars multi-dimensional the interpretation.
grammars. A full review of the many different

Visual and Diagrammatic Languages

Figure 1. State transition diagram

a
S2 a
b
S1

b
a S4
a b

Consider the state chart: for the interpreta- principle, tokenization is the only phase during
tion of such a state chart, we would need to which the actual geometry of the diagram needs
model whether a text label is inside a circle or to be inspected. After this, the explicitly modeled
attached_to an arrow, whether an arrow starts_at spatial relations are processed only in symbolic
or ends_at a circle, etc. For example, the diagram form. The obvious question is, how are rela-
fragment in Figure 2 could be represented with tions between derived (non-terminal) symbols
the token set: computed in this approach? The key to this is to
explicitly re-write not only the set of tokens, but
T = {circle1, circle2, text1, text2, text3, arrow1} also the set of spatial relations, so that derived
spatial relations are computed in parallel with
and the relation set: non-terminal symbols.
We outline Relation Grammars (Crimi et al.,
R = {arrow1, starts_at circle1, arrow1, ends_at 1990) as a simple example of the relational ap-
circle2, text1 inside circle1, text2, inside proach. Consider the following grammar from
circle2, text3 attached_to arrow1} Crimi et al. (1990), which essentially describes
diagrams that are directed graphs. There are two
In principle, the relation-based approach thus terminal token types: nodes (n), and edges (e), and
models a diagram as a set of graphical tokens one non-terminal token type: graph (G). Only two
and as a set of primitive spatial relations between types of relations are used: start, which holds if a
these tokens.3 node (or graph) is at the start of an edge and end,
The relations between the primitive tokens if it is at the end of an edge. The productions for
can be extracted at the time of tokenization. In the token set are:

1: G → {n}
Figure 2. Fragment of an NFA diagram
2: G → {n, G'}
3: G → {n, G'}{start (e, G'), end (e, G')}
b
S1 S2
which formalize that a primitive graph consists of
a single node, and that a graph can be augmented

25
Visual and Diagrammatic Languages

by either adding an unconnected node to it or by during the tokenization phase). This is because
adding a node that is connected to the rest of the the disambiguation of the vague concept “close
graph by an edge. To derive the spatial relations to” requires the context to be taken into account.
for the non-terminal graph tokens, the following For example, if two edges are running close to
derivation rules for the relation set are given: each other, each of their two labels may well be
close to both edges if a simple threshold is ap-
start(e, G) [1] ⇒ start(e, node) plied. The decision must be made by deciding
start(e, G) [2] ⇒ start(e, node) which text is closest to which edge. This, however,
start(e, G) [2] ⇒ start(e, G') requires inspection of the concrete geometry in
start(e, G) [3] ⇒ start(e, G') the context of both edges and labels. As it may
also require that some parts of the diagram have
Evaluation rules are associated with productions been parsed already, for example if the edges
(the number of the production is shown in brackets) themselves are derived tokens, it is impossible
and indicate how the relations have to be rewritten to determine these relations before parsing. In-
when this production is applied. Thus for example, stead, to perform such disambiguation effectively,
the first evaluation rule states that, whenever a inspection of the concrete geometry during the
node is reduced to a non-terminal graph G by parsing process is required. This is not possible
production 1, all edges that start at this node have in a purely symbolic approach.
to be rewritten to start at G. The evaluation rules The second problem stems from the require-
for end are analogous to those for start. ment to write the production rules for relations.
Pure graph-grammars, which were among the Consider a production for a final state (depicted
earliest methods for diagram specification, also by two concentric circles) in this approach:
fall into the relation-based category. Here, the
embedding function takes over the explicit rewrit- 1: final → {circle1, circle2}{inside(circle1, circle2)}
ing of symbolic spatial relations. The interested inside (circle1, text)[1] ⇒ inside (final, text)
reader is referred to Marriott et al. (1998), Tucci,
Vitiello, and Pacini (1992), and Tucci, Vitiello, Effectively, these rules do nothing but emulate the
and Costagliola (1994) for a detailed discussion natural laws of geometry: If the text is in a circle
of this aspect.4 that is inside another circle and we consider the
The clear advantage of the relation-based two circles as a final state token, the text is obvi-
approach is that it allows reasonably efficient ously inside it. As we are generally dealing with
parsing in the context-free case.5 However, there a concrete, drawable geometry, the whole task of
are a couple of conceptual as well as technical these rules usually is to model (some part of) basic
disadvantages to this seemingly simple approach: geometric inference. However, since these rules
Firstly, all spatial relations are purely symbolic. are tied in with the grammar for the symbols, we
This makes it very hard to model vague relations cannot state them once for all as the axioms of our
such as “close to” in a meaningful way. Consider geometry, but we have to tailor them to every new
the case of a text labeling an arrow in a transition grammar. Writing such rules is complex, tedious
diagram. The correspondence of the arrow and and error prone at the best of times.
the text is established by writing the text close Both problems can be overcome if we have
to the arrow. However, it is hard (and in general access to the concrete geometry of the diagram
impossible) to reduce this spatial relationship to a during the parsing process and if we allow to
binary symbolic relation that can be determined compute derived geometric attributes, such as
locally (by only inspecting the edge and the text bounding boxes, explicitly during parsing. This
is where the attribute-based approach comes in.

Visual and Diagrammatic Languages

Attribute-Based Grammars This specifies that the symbols U1, ..., Un can be
rewritten into the symbol U using this produc-
The attribute-based approach, in contrast to the tion if and only if the context symbols Un+1, ...,
relation-based approach, does not ignore the Um exist in the current multiset and the constraint
details of the spatial information during the pars- expression C holds. The symbols Ui can be at-
ing process. Instead of just using un-attributed tributed and C is a predicate on the attributes of
tokens6, it attributes all tokens with their concrete U1, ..., Um. During rewriting, the attributes of U
geometry. The parsing formalism based on this are computed from the attributes of U1, ..., Um
approach uses calculations embedded into the using the assignment expression E. Including
grammar productions to compute derived at- the attributes in the formal definition, a CMG
tributes of non-terminal symbols. For example, production takes the form:
assume that an arrow in the state diagram is not
    
modeled as a primitive token, and instead we only U(x) ::= U1(x1), ..., Un(xn) exists Un+1 (xn+1), ..., Um (xm)
    
have line and triangle as primitives. A production where C(x1, ..., xm) {x = E(x1, ..., xm)} (2)
for an arrow would specify that it is composed of
a line and a triangle attached to it, and it would In each grammar, there is a distinguished non-ter-
calculate the coordinates etc. of the arrow from minal symbol type called the start type. Attributes
the spatial attributes of the line and the triangle. can be associated with tokens and are typically
The attribute-based approach thus deals with used to describe their geometric properties, but
concrete geometries explicitly. This allows it to they can also be used to capture the semantics
dispense with rewriting a set of relations and to of the interpretation, similar to the way textual
just rewrite a set of attributed tokens instead. grammars process semantic attributes. As produc-
Arguably the most advanced attribute-based tions are applied in the parsing process, semantic
grammar approach to visual language specifica- functions in E can compute the attribute values for
tion is constraint multiset grammars (Helm et the generated non-terminal, up to the construction
al., 1991), which have been used by a number of a global interpretation in a designated attribute
of researchers for reasonably complex tasks of the start symbol.
such as the interpretation of state diagrams and We show a production for recognizing a final
mathematical equations. We will review CMGs state in a state transition diagram. This is made
in more formal detail than the relation-based ap- up of two circles C1 and C2 and a text T satisfying
proach, as we will later use CMGs to formally three geometric relationships: the mid-points of
demonstrate how the logical approach to diagram the circle C1 and the circle C2 are the same; the
specification subsumes the grammatical approach. mid-points of the circle C1 and the text T are the
Full detail, in particular parsing algorithms for same; and C2 is the outermost circle. Note the ad-
CMGs are given in Marriott (1994), and an in- ditional attributes kind and label used to construct
depth analysis of the relation between CMGs and an “interpretation.” The production is:
other multidimensional grammar frameworks is
given in Marriott and Meyer (1998). S: state ::= C1: circle, C2: circle, T:text where (
A diagrammatic sentence to be parsed by a C1.mid = C2.mid and C1.mid = T.mid and
CMG is just an attributed multiset of graphical C1.radius ≤ C2.radius){
tokens. CMG productions rewrite multisets of S.mid = C1.mid and S.radius = C2.radius and
attributed symbols and have the form: S.label = T.label and S.kind = final}

U ::= U1, ..., Un exists Un+1, ..., Um where (C) {E} (1) We are using the dot notation to refer to named
attributes (e.g., C1.mid refers to the attribute

Visual and Diagrammatic Languages

named mid of the token C1). Written in the form OnCircle(A.end, S2.mid, S2.radius)) {T.start
of Equation 2 the previous production is therefore = S1.label ∧ T.tran = A.label ∧ T.end =
equivalent to: S2.label}

state (Smid, Sradius, Slabel, Skind) ::= The unrestricted form of CMG productions (type
circle (C1mid, C1radius), circle(C2mid, C2radius), 0) allows us to rewrite an arbitrary number of
text (Tmid, Tlabel) where ( symbols simultaneously and is thus given by:
C1mid = C2mid ∧ C1mid = Tmid ∧ C1radius ≤ C2radius) {
   
(Smid, Sradius, Slabel, Skind) = (C1mid, C2radius, Tlabel, U1(x1), ..., Un(xn) ::= V1( y1), ..., Vm( ym)
     
final)} where C( y1, ..., ym) {(x1, ..., xn) = E( y1, ..., ym)} (3)

It is now easy to see that we do not need to write Corresponding to the standard structural
extra rules to emulate the laws of geometry as classification of string grammars, we define a
in the relation-based approach. All that we do is CMG production as shown in Equation 3 to be
to transfer the relevant spatial attributes to the context-sensitive if 1 ≤ n ≤ m and context-free if
derived symbol using semantic rules. CMGs also additionally n = 1. For a detailed analysis of the
allow us to use context-sensitive productions. As expressiveness of the different classes of CMGs,
indicated in the general form (1) of a production see Marriott and Meyer (1997).
above, context symbols are denoted by existential
quantification. For example, the following context-
sensitive production recognizes a transition: the GrAnd chALLenGe:
reASonInG WIth And ABout
T:transition ::= A:arc exists S1:state, S2:state dIAGrAmS
where (
OnCircle (A.start, S1.mid, S1.radius) ∧ Grammar formalisms like CMGs provide us with
OnCircle (A.end, S2.mid, S2.radius)) a powerful method for parsing and interpretation
{T.start = S1.label ∧ T.tran = A.label ∧ T.end = of diagrammatic notations, but are grammars an
S2.label} adequate framework for all aspects of diagram
specification? The answer is that they are not,
where an arc is simply a labeled arrow we believe. Syntax-directed interpretation of
diagrams is often straightforward enough: it is,
R:arc ::= A:arrow, T:text where (A.mid = T.mid) for example, not difficult to define a grammar that
{R.start = A.start ∧ R.end = A.end ∧ R.mid extracts the regular expression described by the
= A.mid ∧ R.label = T.label} NFA in Figure 1. But syntax-directed interpreta-
tion (by means of parsing) is only a small part of
Note that this existentially quantified form is the reasoning tasks that we would like to perform
equivalent to using the more conventional (but with diagrams, and grammars are ill-suited for
lengthier) notation, which puts context symbols other kinds of reasoning.
on both sides of the production: Arguably the most striking examples of com-
plex formal reasoning with diagrams arise from
T:transition, S1:state, S2:state ::= A: arc, S1:state, the “diagrammatic calculus” movement that is
S2:state where ( rooted in the pioneering works of Shin (1995)
OnCircle (A.start, S1.mid, S1.radius) ∧ and Hammer (1996) and subsequent extensions
of these by other authors, in particular Howse,

Visual and Diagrammatic Languages

Figure 3. A diagrammatic proof of n2 = ∑ i=1 (2i – 1)

Molina, Taylor, Kent, and Gil (2001). Diagrams in the right diagram clearly is c2. As we have
have long been used as a tool of thought in math- treated the objects in the diagram as rigid (i.e.,
ematical investigations, and examples of elegant only translation and rotation were applied), this
diagrammatic proofs are plentiful (see Nelsen, might be considered as a proper proof. However,
1993, for an inspiring collection). there are several kinds of problems with this type
Consider, for example, Figure 3, which gives a of reasoning. Firstly, this “proof” uses only one
simple and elegant argument for n2 = ∑ i=1 (2i – 1)
n
particular diagram. It is not clear or explicit for
. However, such diagrammatic reasoning contains which other instances the proof would remain
dangerous pitfalls. The question, whether these valid (i.e., what we quantify over): all right tri-
“diagrammatic proofs” actually constitute valid angles, all triangles, or even all polygons? The
proofs or mere illustration had been a matter of a second problem is that the inference rules that
long-standing debate. The core problem is that the have been used to derive the second diagram
inference rules in such proofs are not clearly for- from the first one are not formalized. They have
malized. For example, it is not unambiguous what just been informally derived from the analogy to
the ellipsis in Figure 3 means. To further clarify physical objects. This is neither rigorous enough
this point let us look at the classic example of the nor sufficient for more abstract types of reasoning.
proof of Pythagoras’ theorem in Figure 4. These considerations have led many researchers
This sequence of diagrams appears to be a clear to question whether visual formalisms can ever be
and unambiguous proof of Pythagoras’ theorem valid tools for formal mathematical reasoning.
a2 + b2 = c2.7 The white area in the left diagram Luckily this debate has (at least in principle)
clearly is a2 + b2 and the corresponding surface been settled by the ground-breaking work of Shin,
who fully formalized the system of (extended)
Venn Diagrams into a visual set calculus with
clearly stated inference rules. Others, in par-
Figure 4. Pythagoras’ theorem
ticular Howse (2001) have followed this lead and
have later established similarly rigorous visual
calculi. As an illustration of how such calculi
work, consider the following inference rule from
Shin’s calculus:

29
Visual and Diagrammatic Languages

Figure 5. Reasoning with a Venn Diagram Figure 6. Transition in an NFA Diagram

If an element of a chain is located in a shaded “bla.” The current state is identified by placing the
region, this element may be removed provided residual input string under it and the automaton’s
that the chain is reconnected. behavior can diagrammatically be simulated by
truncating and moving this residual input string.
To clarify the meaning of this rule we recall that Obviously, such animation can be used as a simple
shaded regions in Venn Diagrams denote empty form of diagrammatic reasoning, for example to
sets and that chains of elements postulate the ex- test whether a certain input is accepted by a given
istence of an element in at least one of the regions automaton. Diagram notations that can be used in
in which the chain’s elements are located. this form have therefore been termed “executable
Consider the diagram in Figure 5. The complete graphics” (Lakin, 1986).
translation of the leftmost Venn Diagram in this The crucial issue with the syntactic transfor-
figure is the proposition ∃x: x ∈ A ∩ B ∨ x ∈ A mations used in both examples is that they must
– B ∨ x ∈ B – A (from the chain). Now imagine be formally well defined. Unfortunately, the
adding shading to the right set B (middle Venn inference rules of Shin’s system and other syn-
Diagram). This is equivalent to adding the tactic calculi for diagrammatic notations (Shin,
proposition B = ∅ to the interpretation of the 1996; Hammer, 1996; Howse et al., 2001) have
diagram. Repeated application of the above been stated only in natural language.8 What is
transformation rule now allows us to reduce the required to fully specify these calculi is a formal
diagram to the form given on the right-hand side, meta-language that allows us to specify diagram
which is equivalent to inferring the proposition transformations on the level of the actual visual
x ∈ A – B. tokens.
It should have become clear that a diagram- In the case of executable graphics, this seems
matic calculus, such as Shin’s Venn calculus, can straightforward enough: If we move from context-
rightfully be given the same status as a reasoning free to context sensitive multi-dimensional gram-
device that a syntactic calculus for a textual logic mars, we can directly write down such animation
has. As in the latter case, its basic ingredients rules.9 For example, using the CMG formalism,
are syntactic transformations, but in this case, the rule that is the basis for the transformation
the transformations act on a multi-dimensional step illustrated in Figure 6 is:
visual notation.
It is interesting to note that this form of syn- L1:text, S1:state, S2:state, T:transition ::= L2:text,
tactic calculus closely corresponds to a more S1:state, S2:state, T:transition where T.start =
“natural” form of reasoning with diagrams, S1.label ∧ T.end = S2.label ∧
namely the idea that the animation of diagrams JustBelow (L2.mid, S1.mid, S1.radius) ∧ T.label
(again, syntactic transformation) can be used as = first (L2.label))
a simple form of reasoning. For example, Figure {L1.mid = PointJustBelow(S2.mid, S2.radius) ∧
6 provides an illustration of a single step in the L1.label = rest(L2.label)}
animation of an NFA that is processing the string

0
Visual and Diagrammatic Languages

Note that the production in this example is grouped into two distinct main categories: term-
context-sensitive. It is straightforward to give the based models and predicate-based models.
remaining rules that complete the specification for The first approach encodes a diagram as a
the visual execution of a state automaton such as single term modeling a hierarchical collection
the one for ε-transitions. Thus, unrestricted CMGs of the graphical objects in the diagram. This is
allow us to formalize the concept of “executable closely related to the way linear sentences are
graphics.”10 Such specifications will allow us, for modeled in DCGs (Pereira & Warren, 1980) and
example, to prove that a concrete word w (say, w other computational logic grammars. In fact,
= ababa) is accepted by a given automaton or to one of the first approaches to use this modeling
extract the regular expression that describes this was a straightforward extension of DCGs by set
automaton, say L = a+ (ba | ab+)*. However, we must structures called definite clause set grammars or
always reason using a finite sequence of animation DCSGs (Tanaka, 1991). Various other forms of
steps and we must actually execute the anima- such encodings have been demonstrated (Mar-
tion to perform any inference. This constitutes a riott et al., 1998), some modeling a diagram as a
very weak mechanism for reasoning. It would, hierarchical term (Helm et al., 1991), others using
for example, by no means be straightforward to a term to represent a graph-based model of the
devise an animation to prove that two automata diagram (Meyer, 1992, 1997).
are equivalent (in particular, in the presence of The key idea is the same in both cases, namely
attributed transitions). Such reasoning would to model a diagram as a single term. The key predi-
require a general calculus for grammars, which cate is reduce(D, D'), which holds if diagram D
does not exist for CMGs or for any other type of can be reduced to D' using one of the productions
multi-dimensional grammar. We can execute a in the grammar. To handle spatial conditions, we
grammar specification and parse with it, but we assume a first-order theory C, which models the
cannot mechanically reason about a grammar relevant aspects of geometry. C is assumed to be
specification. an additional set of axioms in our inferences.
The need to identify a framework that sup- For the state transition diagram grammar we
ports this kind of automatic (deductive) reasoning can define reduce(D, D') , which holds if D' can be
with and about diagram specifications is the main obtained from D by recognizing a final state:
rationale behind the logic approach to diagram
specification. reducefs (D, D') ↔ ∃{D, D '}
circle(Cmid1 1
, Cradius )∈D
∧ circle(Cmid , Cradius
2 2
)∈D∧
A WAY forWArd: LoGIc text(Tmid, Tlabel) ∈ D ∧
SPecIfIcAtIon of dIAGrAmS 1
Cmid 1 2
,=CC ∧ radius
mid , CC
radius
2 1
mid ,=CT1 1
C
radius ∧, Cradius
midmid
1 2
≤, Cradius
Cmid 2
∧
Smid = Cmid ,∧Cradius
1 1
Cmid=, Cradius ∧ Slabel = Tlabel ∧
Sradius2 2

Logic formalisms for the specification of visual Skind = final ∧

1 1
and diagrammatic languages have been investi- D' = D \ {circle(Cmid , Cradius ),
2 2
gated before, but only for first-order logic and for C , C
circle( mid radius), text(Tmid, Tlabel)}
logic programming based on first-order logic. An  {state(Smid, Sradius, Slabel, Skind)}
overview of the different techniques can be found
in Marriott et al. (1998). For the context of this where we use the Prolog convention of using
chapter, it will suffice to summarize the funda- identifiers starting with an upper-case letter for
mental properties of these methods, which can be variables and those starting with a lowercase

Another Random Scribd Document
with Unrelated Content
burn, 74
burn blue, 148
cotton, not easily blown out, 79
easily blown out, 51, 78
and rekindled, 51, 55
extinguished, 79
flame hot, 75
flame hollow, 76
flame pointed, 77
flame purple below, 76
flame tends upwards, 77
flame yellow, 76
gas of, 281
held at a door, 290
hottest above the flame, 79
give light, 76
make glass damp, 78
need snuffing, 81
Palmer’s, 80
prevent our seeing abroad, 378
reflected in a window, 379
rush, easily go out, 79
smoke, 81
spirt, 155
suddenly introduced give pain, 365
wax, need no snuffing, 81
Candlestick rags catch fire spontaneously, 58
Capillary veins, 84
Captain Ross, 414
Carbon, 33, 74
Carbonate of lime, 426
of soda, 426
Carbonic acid gas, 37, 108, 249, 264
deleterious, 250, 264
in human bodies, 84
its presence detected, 264
Carburetted hydrogen gas, 279, 280
Carpets warm, 169
Carriage wheels catch fire, 99
Carriage windows misty, 213
Casks charred, 73
Cart grease, 100
Cathedral aisles famous for echoes, 417
Cats in wet weather, 147
prowl by night, 367
rub their ears, 150
see in the dark, 367
wink before a fire, 367
Cattle uneasy in wet weather, 148
Caverns famed for echoes, 416
Ceilings sooty, 71
Cellars cold in summer, 256
warm in winter, 256
Cerebellum, 424
Cerebrum, 424
Chalk, 426
Charcoal, 72
bad conductor, 166
fire, 72
fire deleterious, 265
purifies water, 72
removes the taint of meat, 72
Charring bread, 73
casks, 73
wood, 73
Chemical action, 30
Chestnuts crack when roasted, 104
not if slit, 105
Chimney pots, 71
Chimneys smoke, if a room be too close, 61
remedy, 62
in vestries, valleys, 66
remedy, 67
in wind, 66, 111
if too long, 62, 116
if too short, 62
if too large, 69
remedy, 70
when the draught is slack, 63, 69
when the door is on the same side, 68
remedy, 68
when it needs repairing, 68
sweeping, 68
when two fires are in one room, 65
remedy, 65
China broken by hot water, 125
Choke damp, 264, 279
Church bells heard at a distance, 151
Churchyards smell offensively, 283
Chyle, 242
Cider, 269
Cinders, 43
iron, 43
will not blaze, 48
Cirro-cumulus clouds, 136
Cirro-stratus clouds, 134
Cirrus clouds, 134
Citizens pale, 243
Citric acid, 426
City air unhealthy, 253
Clean kettles, 186
Cleanliness connected with the dietary, 93
Clear day overcast, 304
Clear nights exhilarating, 144
Clocks heard, at a distance, 151, 413
Close rooms unhealthy, 253
Cloth collects but little dew, 208
Clothes gather damp in summer, 211
wet, 157
Clothing for workmen, 164
promotes warmth, 176
Clouds, 127
cause of, 129
classes of, 134
colour of, 132
compound, 136
compound simple, 134
differ from fog, 128, 227
dissipated, 304
distance from the earth, 129
edges most luminous, 399
electrical, 131
fall in rainy weather, 337
float, 128
height of, 4, 120
highest and lowest, 130
intermediate, 135
light, 120
motion of, 133
red, 132, 399
round mountain tops, 137
thickness of, 130
thickness how ascertained, 130
vary in shape, 129, 130
vary in colour, 133
where most abundant, 129
where least, 129
use of, 137
velocity of, 314
wind affects them, 129, 131
Coal gas, 280
mines explode, 281
Coals black, 403
Cold weather affects the barometer, 328
makes us love fat, 90
makes activity, 91
out of doors, 312
promotes hunger, 91
Collapsing, 289
Colour of clouds, 132, 133
Colours vary, 400
some warm, some cold, 187
Combining not mixing, 25
Combustion, see fire, 33, 85
cause of, 36
elements of, 36
heat of, 37
increased by wind, 58
in the veins, 84
Communication of heat, 164
Compound clouds, 136
Compression, 102
Condensation, 98, 102
Condensed air, 289
Conduction, 164
not absorption, 184
Conductors, best, 165
worst, 166
not absorbers, 185
of lightning, 22
dangerous, 24
Convection, 219, 231
Convective currents, 245
cool broth, 247
cool iron, 247
Cooking vessels with wooden handles, 166
Cooper applies hot hoops, 122
Copper sonorous, 410
tarnishes, 259
Copper-hole, 60
roars, 60
roars not when the door is open, 60
Cornea, 388
Corns ache in wet weather, 256
Corpse cold, 95
Corrosive sublimates, 426
Cotton bales catch fire spontaneously, 57
handkerchiefs hot, 184
Countrymen ruddy, 243
Cowls, 67, 111
Crowds produce drowsiness, 251
head-ache, 249
vitiate air, 249
unhealthy, 250
Culinary vessels have wooden handles, 166
should be sooty, 71, 201
Cultivation promotes dew, 210
warmth, 160
Cumulo-stratus clouds, 136, 137
Cumulus clouds, 134, 135
Cup in a pie, 120
why full of juice, 124

Damp banisters, 217

beds, 158
house, 217
Danger in a storm, 12
in attics and cellars, 16
a crowd, 18
theatre, &c., 19
before a fire, 15
near a tree, 12
river, 14
flocks and herds are exposed to, 19
those who bar shutters, 18
lean against a wall, 17
or carriage, 20
who ring bells, 15, 17
run, 15
Dark colours warm, 186
radiate heat, 197
Davy, Sir H., 281
Day-light produces hunger, 88
Dead bodies cold, 95
taller than living, 423
Deal snaps in fire, 107
Decanting liquor, 274
spirting, 274
Depression of spirits, 145
Descent in a diving bell painful, 146
Deserts hot and dazzling, 405
Dew deleterious, 220
differs from rain, 226
distilled after a hot day, 219
especially if the wind is westerly, 218
unequally, 207
in open places, 204
in valleys and hollows, 207
on clothes, 211
cultivated lands, 210
grass, 209
leaves, &c., 208
none beneath a tree, 204, 205
a flower awning, 205
hedge or wall, 205
none in a cloudy night, 204
in a windy night, 207
especially if easterly, 218
none on stones, cloth, 208
deserts, gravel, metal, rocks, wool, 209
Dew-drops round, 223
flattened, 223
roll on cabbages, 223
roses, 224
Digging promotes warmth, 161
Digestion, 91
Dinner covers, 202
Dirt warm, 93
Distant bells heard, 413
clocks, 151
objects appear small, 381, 382
sight, 390
spectacles for, 391
Distance makes things invisible, 384
Distilled vinegar, 426
Divers suffer pain, 255
Diving bell, 146
Doors swell and shrink, 151
Dogs uneasy in wet weather, 147
Dough, 274
set before a fire to rise, 276
Double concave glasses, 389
convex glasses, 391
Draining lands promote warmth, 160
Draught at a door, 314
key-hole, 314
window, 315
slack, 63, 64, 69
Dreams, 424
foolish things, 424
Drops of rain roll on dust, 155
Drowned men restored, 101
Drums, 411
Dry wood for kindling, 44
burns best, 107
snaps about, 107
Dryest months, 328
Dublin, rain of, 340
Duck dry in water, 224
Dunghills hot, 277
Dusty shoes hot, 194
Dyeing changes a colour, 401

Ear-trumpets, 415
Earth, bad conductor, 181
cool in summer, 182
cracks by frost, 357
crumbles in spring, 358
warm in whiter, 131
Earth-fog, 221
Earthen tea-pots, 197, 198
set on a hob to draw, 198
East wind cold, 302
dry, 303
prevents dew, 218
Eat more in cold, 90
less in warm weather, 91
Echo, 416
Echoes, two or more, 418
Effervescence, 269
soon subsides, 275
Egg cracked when boiled, 239
tested, 178, 239
Electricity affects the clouds, 131, 133
excited by friction, 29
felt at the elbow joints, 27, 29
hot, 3
of clouds, 4
positive & negative, 16
England grows warmer and warmer, 160
winds of, 300
east dry, 302
morning at watering places, 310
most prevalent, 300
north cold, 312
north-east dry, 305
south rainy, 303
warm, 303
south-west rainy, 304
west rainy, 304
when highest, 301
lowest, 301
See March wind.
Epsom salts, 426
Esquimaux love blubber, 92
Equatorial current, 298
Ether, 47
boils, 119
used for freezing, 360
used for inflammation, scalds, burns, 157
European skin white, 191
Evaporation, 156
freezes, 360
Evening clouds, 132
grey, 140
red, 138, 399
Evening rainbow, 141
Evergreens frost-bitten, 230
Ewers broken by frost, 349
Expansion by heat, 103
Extinguishers, 79
made of paper, 79
Eyes, two, 368
see single, 369
affected by blue glasses, 407
fire-light, 407
the sun, 407

Face soon scorched, 196

Fanning, 179, 313
Far sight, 390
spectacles for, 391
Farriers apply hot shoes, 123
Fat men swim best, 362
Feathers warm, 176
Feeling, 425
Feet cold before a fire, 53
wet dangerous, 157
Fermentation, 269
of dough, 276
Fender and fire-irons cold, 185
Fiddle-strings musical, 411
snap from wet, 339
Fine weather braces, 147
Finger feels cold when wet, 157
Fire black and red, 38, 40
blazes, 39
not in frost, 48
burns blue, 148
cause of, 36
charcoal, 72
damp, 280
effect upon the eyes, 407
extinguished, 54
by water, 107
fiercest in winter, 49
out of doors, 50
grotesque figures in, 40
heat of, 280
how increased, 63
hot, 37
kindled at the bottom, 42
light dazzles, 366
lighted with paper and wood, 41
luminous, 46
poker draws up, 52
radiates heat, 196
red hot, 38
reflected on windows, 379
sun dulls it, 49
thaw dulls it, 50
wind intenses it, 51
See combustion.
Fire-irons cold, 185
rust, 257
prevented, 258
Fishes ascend and dive, 363
seem nearer than they are, 388
cold, 94
Flame of a candle, 76
described, 77
blown out easily, 78
damp, 78
hollow, 76
hottest above, 79
pointed, 77
purple and yellow, 76
smokes, 81
Flame of a fire between the bars of a grate blue, 46
green, 45
yellow, 46, 47
Flannel warm, 94
used for foot-warmers, 173
Flash, see lightning
Flat beer, 273
improved, 273
Flavour discerned by the taste, 425
Flint and steel, 97
Flowing water freezes slowly, 354
makes rough ice, 354
oscillates, 348
pure, 347
Flower awnings arrest dew, 205
purify air, 253
Flowers smell sweetest at night and before rain, 152
Flues blacked, 238
See chimney.
Flutes, 412
Fly-poison, 422
Fogs, 225, 226
arrest sound, 413, 414
cause of, 129
differs from cloud, 128, 227
mist, 227
dispersed by wind, 228
by sun, 227
frozen, 231
in autumn, 228
in marshes, 226
valleys, 228
none in a frosty night, 227
Food converted to blood, 242
cooled by the breath, 312
Foot-prints frozen, 350
Foot-warmers, 173
Forked lightning, 5
dangerous, 5
Forests catch fire spontaneously, 102
France warmer than of yore, 162
Freezing mixtures, 360
Friction (see rubbing), 98
excites electricity, 29
sets forests on fire, 102
Frogs cold, 94
Frost affects barometer, 329
sound, 414
braces, 147
breaks ewers, 349
tiles, stones, rocks, 350
pipes, 351
cracks earth, 357
expands water, 352
prevents fog, 227
warm, 356
Froth of beer, 269, 350
Frozen ruts and footprints, 350
Fruits cool the blood, 92
fall to the earth, 419
pleasant in summer, 92
Fuel for the body, 87
Fumigation for sick rooms, 421
Fur, bad conductor, 166
for clothing, 175
warm, 171
Furnaces of brick, 174
lined with clay, 174
Furr of kettles, 262
steam engines dangerous, 263
Gallery hot, 249, 316
Gas, 112
of candles, 281
Gauze wire of safety lamps, 282
prevents explosion, 282
German silver tarnishes, 260
German tinder, 102
Germany warmer than of yore, 162
Ghosts, 286
Gideon’s miracle, 211
Ginger pop, 268
acid, 269
Glasses broken by hot water, 125
covered with mist, 214
which soon subsides, 215
dulled by a hot hand, 215
by breath, &c., 215
See spectacles.
Glass a reflector, 194
soon cools, 212
See looking-glass.
Glauber salts, 426
Gloves, black kid, 188
Lisle thread, 188
Glow-worms glisten by night, 368
God’s wisdom 182, 208, 210, 302, 308, 352, 357
Gold never tarnishes, 260
Goulard, 426
Grapes never ferment, 272
Grass promotes warmth, 161
collects dew, 209
Grate, see stove
Gravel collects no dew, 209
Gravity, 419
Gray morning, sign of a fine day, 140
Grease liked in cold weather, 90
loathed in hot, 92
prevents rust, 258
used for wheels, 100
Green colour, 401
wood does not burn, 107
Grottoes famous for echoes, 416
Ground frost, 229
Growth promoted by moonlight, 220
Gulls fly to sea, 154
to land, 154
Gusty weather makes a smoky house, 71

Hail, 331, 334

cause of, 335
accompanied with thunder, 335
falls in summer, 335
Hair, bad conductor, 166
covered with dew, 217
Halls famous for echoes, 417
Halo round the moon, 143
Hard work promotes hunger, 88
Hat covered with dew, 217
turned red at the seaside, 344
Hawks see near and far, 392
Hay-stacks catch fire spontaneously, 58
Haze round the sun, 143
moon, 143
affects sound, 414
Head aches in a crowd, 249
itches in wet weather, 150
Hearth-rug warm, 169
Hearth-stone cold, 169
hot, 170
Heat, 1
and light, 47
affects barometer, 328
sound, 414
animal, 83, 85, 277
applied to the bottom of boilers, 234
effects of, 2, 4, 103
expands water, 352
from beaten iron, 95
latent, 31, 75, 96
of candles, 75
dunghills, 277
fire, 37, 277
lime, 278
radiates, 196
sources of, 2
Heavy bread, 160
Hills larger in a fog, 148
Hoar frost, 228
not found on trees, 230
under shrubs, &c., 230
of frozen fog, 231
on clear nights only, 229
on grass, 229
tombstones, 189, 230
very partial, 229
Honey-dew, 220
ants fond of, 221
effects of, 221
injures plants, 221
Hot cloudy night oppressive, 144
Hot weather abates activity, 93
appetite, 90
love for grease, 92
Hottest place at church, 316
Hoops used red hot, 122, 123
Horse shoes fitted hot, 123
Horses strike fire, 98
snuff up air, 152
uneasy in dull weather, 148
Houses catch fire spontaneously, 56
smoke in valleys, 66
Hunger, 88
promoted by cold, 90
day-light, 88
singing, speaking, work, 88
See appetite.
Hydrogen gas, 34, 74
Ice, 349
lighter than water, 349
grows thicker, 353
dissolved by snow, 357
melted by sun, 126
Ice-bergs famous for echoes, 417
Idiots, 424
Ignis fatuus, 285
cause of, 286
Impure water purified, 72
Indian mode of striking fire, 99
Inflammable air, 34, 74
Insensible perspiration, 213
Intermediate clouds, 135
Iron, bad conductor, 185
cinders, 43
contains latent heat, 96
cooled by air, 246
convection, 246
radiation, 246
matches, 96
rust, 257
when most common, 258
prevented, 258
sonorous, 410
stoves, 174
Ironing-box, 155
Islands equable in temperature, 311
subject to wind, 309, 311
Itching in wet weather, 150
Jack o’lanthorn, see ignis fatuus
Jet of flame through bars, 45
Juice of lemons, 426
Jungle of Hindostan fatal, 266

Kendal rainy, 340

Keswick rainy, 340
Kettle boils over, 115
quickly when covered with soot, 186
slowly when clean, 186
when new, 186
bottom should be sooty, 200
cold when water boils, 200
inside white, 200
lid hot, 201
furr, 262
holder, 167
not full after boiling, 115
runs over, 233
through the spout, 115
sings, 113, 233
top bright, 200
Kindling wet, 43

Lakes which never freeze, 355

Lamps, 74
Argand, 83
smoke, 82, 83
spirit, 155
See candle.
Lamp-glasses, 83
Land air cold, 224, 311
breeze unhealthy, 309
Laplanders clad in skin, 183
Latent heat, 31, 75, 96
Laziness promoted by want of food, 89
by heat, 93
Lead tarnished, 259
Leaves collect dew unequally, 208
green, 404
light green in spring, 404
in a pond, 419
pale in dark places, 403
promote cold, 161
yellow in autumn, 408
Lid of kettles, &c. hot, 201
Light bread, 276
Light, 363
composed of various colours, 400
divided by a prism, 393
from a flint, 97
of candles, 76
fire, 46
sun, 363
reflected, 364
speed of, 364
sudden painful, 365
Lightning, 3
balls, 6
barks and snaps trees, 27
comes from clouds, 16
from earth, 16
conductors, 22
dangerous, 24
follows dry weather, not wet, 28
forked, 5
fuses metal, 27
kills animals, 7
knocks down churches, 24
maims, 8
passes down the outside of a tree, 14
passes through the inside of animals, 14
purifies air, 27
rare in winter, 28
sheet, 6
summer, 11
common, 28
straight, 6
turns beer sour, 27
not old beer, 27
milk sour, 25
See danger, safety.
Lilac steel rusts, 259
prevented, 259
Lime and water, 26, 30
burned, 278
hot, 277
purifies bins, 266, 421
sewers, 267, 421
Lime-wash for rooms, 422
Lincoln, rain of, 340
Linen cool wear, 183
dried, 160, 316
Linseed oil boils, 119
Liquefaction, 126
Liquids, 112
bad conductors, 172, 232
cooled, 235
Lisle thread gloves, 188
Liverpool, rain of, 340
London fog, 225
rain of, 340
Long flues, 110
grass promotes cold, 161
Log of wood, 2
two burn better than one, 44, 52
Looking-glass, 369
Lucifer matches, 284
Lunar caustic, 426

Mackarel scales, &c., 136

Macintosh prevents cold, 158
Madness from starvation, 86
Malt, 270, 272
Man a swimmer, 362
no bigger than a crow, 382
Manchester, rain of, 340
Magpies indicate weather, 153
Marble, 426
March comes in like a lion, 306
goes out like a lamb, 306
dry good, wet bad, 306
flowers undesirable, 307
wind dry, 305
Marsh damp, 280
Marsupium, 392
Meat-covers, 202
Meat liked in cold weather, 90
loathed in hot, 92
taint removed, 422
tainted by moonlight, 220
Mechanical action, 95
Mercury of barometer bright, 260
concave, 325
convex, 325
its rise and fall, 325
Metal collects no dew, 209
feels colder than wood, 168
hotter than wood, 167
fused by fire, 127
by lightning, 27
good conductor, 165
handles burn, 166
reflectors, 193
tea-pots, 197
Milk soured by lightning, 25
Miners’ danger, 283
prevented, 265
Mirror, 369
Mist arrests sound, 414
black, 144
cause of, 224
differs from cloud, 128
dew, 222
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com

Audit Objectives Procedures Evidences and Documentation
100% (4)
Audit Objectives Procedures Evidences and Documentation
35 pages
ServerAdmin v10.6
No ratings yet
ServerAdmin v10.6
197 pages
PT Mathematics-6 Q2
No ratings yet
PT Mathematics-6 Q2
7 pages
Seminar On Schedule U: Presented by
No ratings yet
Seminar On Schedule U: Presented by
21 pages
Logiq 500 GE
No ratings yet
Logiq 500 GE
407 pages
(U. Lang, H. Aichele, H. Pöhlmann, R. Rühle PDF
No ratings yet
(U. Lang, H. Aichele, H. Pöhlmann, R. Rühle PDF
221 pages
Steps - TR 2007 008
No ratings yet
Steps - TR 2007 008
50 pages
YAMAHA OUTBOARD LZ200NETO, LZ200TR Service Repair Manual X 100101 PDF
No ratings yet
YAMAHA OUTBOARD LZ200NETO, LZ200TR Service Repair Manual X 100101 PDF
60 pages
Visual Programming and Program Visualization - Towards An Ideal Visual Software Engineering System
No ratings yet
Visual Programming and Program Visualization - Towards An Ideal Visual Software Engineering System
17 pages
OOPM UNIT 1 (Cse)
No ratings yet
OOPM UNIT 1 (Cse)
64 pages
Graphics As Visual Language: Handbook of Statistics, Vol. 9
No ratings yet
Graphics As Visual Language: Handbook of Statistics, Vol. 9
31 pages
Unit 2
No ratings yet
Unit 2
15 pages
MK PT en
No ratings yet
MK PT en
308 pages
Business Research
No ratings yet
Business Research
44 pages
Experiment 106: Uniform Circular Motion
No ratings yet
Experiment 106: Uniform Circular Motion
7 pages
Book Reviewsiscience Oj Computer Programming 32 (1998) 211-220
No ratings yet
Book Reviewsiscience Oj Computer Programming 32 (1998) 211-220
3 pages
Leeb Hardness Tester
No ratings yet
Leeb Hardness Tester
4 pages
Bab III
No ratings yet
Bab III
22 pages
Leg en D: Construction Project Schedule
No ratings yet
Leg en D: Construction Project Schedule
6 pages
Visual Programming Programming by Example and Prog
No ratings yet
Visual Programming Programming by Example and Prog
9 pages
Magnetostriction and Applications of Ultrasonic Waves: 15Z204 - Materials Science
No ratings yet
Magnetostriction and Applications of Ultrasonic Waves: 15Z204 - Materials Science
17 pages
Synthesis of Ethanol by Fermentation
No ratings yet
Synthesis of Ethanol by Fermentation
10 pages
Supported Upgrade Paths For FortiOS Firmware 5.2
0% (1)
Supported Upgrade Paths For FortiOS Firmware 5.2
20 pages
Chapter 3 Abstract
No ratings yet
Chapter 3 Abstract
19 pages
978 3 540 76414 4
No ratings yet
978 3 540 76414 4
595 pages
Dis. Sensor
No ratings yet
Dis. Sensor
3 pages
Efficacy of Integrated Pest Management in Reducing Cockroach Allergen Concentrations in Urban Public Housing
No ratings yet
Efficacy of Integrated Pest Management in Reducing Cockroach Allergen Concentrations in Urban Public Housing
6 pages
The Visual Programming Paradigm
100% (1)
The Visual Programming Paradigm
4 pages
Lesson 5
No ratings yet
Lesson 5
2 pages
ĐỀ THI THỬ SỐ 10 - Khóa Đề
No ratings yet
ĐỀ THI THỬ SỐ 10 - Khóa Đề
6 pages
Roy T. Fielding Thesis
100% (3)
Roy T. Fielding Thesis
7 pages
MOD1
No ratings yet
MOD1
36 pages
VA Pilot Competencies
No ratings yet
VA Pilot Competencies
1 page
Conservation Strategies and Plannings of Pench Tiger Reserve
No ratings yet
Conservation Strategies and Plannings of Pench Tiger Reserve
5 pages
Zoology Non Chordata
No ratings yet
Zoology Non Chordata
525 pages
Visual Languages A New Way of Programmin
No ratings yet
Visual Languages A New Way of Programmin
6 pages
Adding and Subtracting Integers Lesson Plan
No ratings yet
Adding and Subtracting Integers Lesson Plan
3 pages
On The Wine-Dark Sea
No ratings yet
On The Wine-Dark Sea
1 page
Chemical Burn
No ratings yet
Chemical Burn
32 pages
National Guidelines For Management of DR TB - 27 3 2025
No ratings yet
National Guidelines For Management of DR TB - 27 3 2025
82 pages
CSD 04 1368
No ratings yet
CSD 04 1368
28 pages
The Design Thinking Playbook: Mindful Digital Transformation of Teams, Products, Services, Businesses and Ecosystems
From Everand
The Design Thinking Playbook: Mindful Digital Transformation of Teams, Products, Services, Businesses and Ecosystems
Michael Lewrick
5/5 (1)
Natural Language Understanding: Fundamentals and Applications
From Everand
Natural Language Understanding: Fundamentals and Applications
Fouad Sabry
No ratings yet
Infographics: The Power of Visual Storytelling
From Everand
Infographics: The Power of Visual Storytelling
Jason Lankow
4/5 (20)
Intellectual Property Strategy
From Everand
Intellectual Property Strategy
John Palfrey
4/5 (5)
Designing Neuroinclusive Workplaces: Advancing Sensory Processing and Cognitive Well-Being in the Built Environment
From Everand
Designing Neuroinclusive Workplaces: Advancing Sensory Processing and Cognitive Well-Being in the Built Environment
Kay Sargent
No ratings yet
Prompt Engineering ; The Future Of Language Generation
From Everand
Prompt Engineering ; The Future Of Language Generation
Michael Ferguson
3.5/5 (3)
Technical Writing for Software Developers: Enhance communication, improve collaboration, and leverage AI tools for software development
From Everand
Technical Writing for Software Developers: Enhance communication, improve collaboration, and leverage AI tools for software development
Chris Chinchilla
No ratings yet
Now the Chips Are Down: The BBC Micro
From Everand
Now the Chips Are Down: The BBC Micro
Alison Gazzard
3.5/5 (1)
AI Assistants
From Everand
AI Assistants
Roberto Pieraccini
No ratings yet
Don't Teach Coding: Until You Read This Book
From Everand
Don't Teach Coding: Until You Read This Book
Stephen R. Foster
No ratings yet
The Construction Technology Handbook
From Everand
The Construction Technology Handbook
Hugh Seaton
No ratings yet
Machines We Trust: Perspectives on Dependable AI
From Everand
Machines We Trust: Perspectives on Dependable AI
Marcello Pelillo
No ratings yet
Arduino Android Blueprints
From Everand
Arduino Android Blueprints
Marco Schwartz
No ratings yet
Coding in the Classroom: Why You Should Care About Teaching Computer Science
From Everand
Coding in the Classroom: Why You Should Care About Teaching Computer Science
Ryan Somma
No ratings yet
Designing the Internet of Things
From Everand
Designing the Internet of Things
Adrian McEwen
3/5 (6)
Embedded Linux Development with Yocto Project
From Everand
Embedded Linux Development with Yocto Project
Otavio Salvador
No ratings yet
Learning IPython for Interactive Computing and Data Visualization, Second Edition: Get started with Python for data analysis and numerical computing in the Jupyter notebook
From Everand
Learning IPython for Interactive Computing and Data Visualization, Second Edition: Get started with Python for data analysis and numerical computing in the Jupyter notebook
Cyrille Rossant
2/5 (3)
Pattern-Oriented Software Architecture, A System of Patterns
From Everand
Pattern-Oriented Software Architecture, A System of Patterns
Frank Buschmann
No ratings yet
Kivy – Interactive Applications and Games in Python - Second Edition
From Everand
Kivy – Interactive Applications and Games in Python - Second Edition
Roberto Ulloa
No ratings yet
Intermarket Trading Strategies
From Everand
Intermarket Trading Strategies
Markos Katsanos
No ratings yet
Handover of Power - Innovation: Global Version - Volume 13/21
From Everand
Handover of Power - Innovation: Global Version - Volume 13/21
Andreas Seidl
No ratings yet
Learning Elixir: Unveil many hidden gems of programming functionally by taking the foundational steps with Elixir
From Everand
Learning Elixir: Unveil many hidden gems of programming functionally by taking the foundational steps with Elixir
Kenny Ballou
No ratings yet
Android Programming: Pushing the Limits
From Everand
Android Programming: Pushing the Limits
Erik Hellman
No ratings yet
Learning VirtualDub: The complete guide to capturing, processing and encoding digital video
From Everand
Learning VirtualDub: The complete guide to capturing, processing and encoding digital video
Sohail Salehi
No ratings yet
Programming with CodeIgniter MVC
From Everand
Programming with CodeIgniter MVC
Eli Orr
No ratings yet
Information Bombardment: Rising Above the Digital Onslaught
From Everand
Information Bombardment: Rising Above the Digital Onslaught
Nick Bontis
No ratings yet
Learning Embedded Linux Using the Yocto Project
From Everand
Learning Embedded Linux Using the Yocto Project
Alexandru Vaduva
No ratings yet
Building Wireless Sensor Networks Using Arduino: Leverage the powerful Arduino and XBee platforms to monitor and control your surroundings
From Everand
Building Wireless Sensor Networks Using Arduino: Leverage the powerful Arduino and XBee platforms to monitor and control your surroundings
Matthijs Kooijman
4/5 (1)
Translation-mediated Communication in a Digital World: Facing the Challenges of Globalization and Localization
From Everand
Translation-mediated Communication in a Digital World: Facing the Challenges of Globalization and Localization
Minako O'Hagan
No ratings yet
Digital Experience Design: Ideas, Industries, Interaction
From Everand
Digital Experience Design: Ideas, Industries, Interaction
Linda Leung
No ratings yet
Designing Machine Learning Systems with Python
From Everand
Designing Machine Learning Systems with Python
David Julian
No ratings yet
History of Computer Science: Technology, Application and Organization
From Everand
History of Computer Science: Technology, Application and Organization
Georg E. Schäfer
No ratings yet
The Handbook of Online and Social Media Research: Tools and Techniques for Market Researchers
From Everand
The Handbook of Online and Social Media Research: Tools and Techniques for Market Researchers
Ray Poynter
No ratings yet
The Information Process: A Model and Hierarchy
From Everand
The Information Process: A Model and Hierarchy
Victor Yang
No ratings yet
C++ Windows Programming
From Everand
C++ Windows Programming
Stefan Björnander
No ratings yet
Design for Software: A Playbook for Developers
From Everand
Design for Software: A Playbook for Developers
Erik Klimczak
4/5 (2)
Computational Intelligence: Synergies of Fuzzy Logic, Neural Networks and Evolutionary Computing
From Everand
Computational Intelligence: Synergies of Fuzzy Logic, Neural Networks and Evolutionary Computing
Nazmul Siddique
No ratings yet
Microsoft System Center Endpoint Protection Cookbook - Second Edition
From Everand
Microsoft System Center Endpoint Protection Cookbook - Second Edition
Nicolai Henriksen
No ratings yet
Iconic Communication
From Everand
Iconic Communication
Philip Barker
No ratings yet
Building a BeagleBone Black Super Cluster
From Everand
Building a BeagleBone Black Super Cluster
Andreas Josef Reichel
No ratings yet
BeagleBone Essentials
From Everand
BeagleBone Essentials
Rodolfo Giometti
No ratings yet
Semantic Computing
From Everand
Semantic Computing
Phillip C.-Y. Sheu
No ratings yet
Innovation Engineering: The Power of Intangible Networks
From Everand
Innovation Engineering: The Power of Intangible Networks
Patrick Corsi
No ratings yet
IPython Notebook Essentials
From Everand
IPython Notebook Essentials
L. Felipe Martins
No ratings yet
Voice Application Development for Android
From Everand
Voice Application Development for Android
Michael F. McTear
1/5 (1)
NW.js Essentials
From Everand
NW.js Essentials
Alessandro Benoit
No ratings yet
Kinect in Motion – Audio and Visual Tracking by Example
From Everand
Kinect in Motion – Audio and Visual Tracking by Example
Clemente Giorio
No ratings yet
Computing with Windows 7 for the Older and Wiser: Get Up and Running on Your Home PC
From Everand
Computing with Windows 7 for the Older and Wiser: Get Up and Running on Your Home PC
Adrian Arnold
No ratings yet
Distinctive Design: A Practical Guide to a Useful, Beautiful Web
From Everand
Distinctive Design: A Practical Guide to a Useful, Beautiful Web
Alexander Dawson
No ratings yet

Visual Languages For Interactive Computing Definitions and Formalizations Fernando Ferri Download

Uploaded by

Visual Languages For Interactive Computing Definitions and Formalizations Fernando Ferri Download

Uploaded by

Visual Languages For Interactive Computing

Definitions And Formalizations Fernando Ferri

Explore and download more ebooks at ebookbell.com

Visual Languages For Interactive Computing Definitions And

Visual Language For Designers Principles For Creating Graphics That

Visual Language For Designers Connie Malamed

Foundations Of Ccli The Visual C Language For Net 35 1st Edition

The Visual Language Of Spatial Planning Exploring Cartographic

Information science reference

Published in the United States of America by

and in the United Kingdom by

Library of Congress Cataloging-in-Publication Data

Includes bibliographical references and index.

ISBN 978-1-59904-534-4 (hardcover) -- ISBN 978-1-59904-536-8 (ebook)

British Cataloguing in Publication Data

Detailed Table of Contents................................................................................................................... vi

Preface . ................................................................................................................................................ xii

Compilation of References................................................................................................................ 468

About the Contributors .................................................................................................................... 502

Preface . ................................................................................................................................................ xii

Compilation of References................................................................................................................ 468

About the Contributors .................................................................................................................... 502

Introduction from the Editor

THE OVERALL OBJECTIVE OF THIS BOOK

Section I. Visual Languages Theory

Section II. Approaches and Methods for Specific Domains

Section III. Visual Languages for the Semantic Web

Section IV. Visual Interfaces for Standard Languages

Section V. Visualisation Aesthetic Computing, and Usability

Maria Francesca Costabile

IntroductIon that graphical and pictorial information (as op-

Figure 6. The transformation to force irrelevant links to disappear

Shneiderman, B. (1992). Direct manipulation.

IntroductIon term is ambiguous: we often speak of the “visual

whole spectrum of semi-formal visual languages frameworks. However, as diagrammatic notations

Figure 1. State transition diagram

Figure 3. A diagrammatic proof of n2 = ∑ i=1 (2i – 1)

Figure 5. Reasoning with a Venn Diagram Figure 6. Transition in an NFA Diagram

Logic formalisms for the specification of visual Skind = final ∧

Damp banisters, 217

Face soon scorched, 196

Hail, 331, 334

Kendal rainy, 340

Lakes which never freeze, 355

Mackarel scales, &c., 136

More than just a book-buying platform, we strive to be a bridge

Join us on a journey of knowledge exploration, passion nurturing, and

You might also like