XVI Congreso Internacional de Interacción

Persona-Ordenador
7 al 9 de SepƟembre de 2015
Vilanova i la Geltrú. Barcelona. España

Actas del XVI Congreso Internacional de  
Interacción Persona‐Ordenador. 
Interacción 2015

EDITORES
Pere Ponsa, José Antonio Román, Daniel Guasch
 XVI Congreso Internacional de Interacción Persona-Ordenador

Del 7 al 9 de Septiembre de 2015, Vilanova i la Geltrú, Barcelona, España

EDITORES

Pere Ponsa
José Antonio Román
Daniel Guasch
Escola Politècnica Superior d’Enginyeria Vilanova i la Geltrú

Universitat Politècnica de Catalunya

Maquetación: Yolanda Guasch

Copyright: ACM
ENTIDADES ORGANIZADORAS

Asociación Interacción Persona Ordenador

ICE PhD Erasmus Mundus Joint Doctorate

in Interactive and Cognitive Environments
 PATROCINADORES

KUKA Robots Ibérica

Hewlett Packard

Universitat Pompeu Fabra.

Barcelona School of Management

Tobii

ASAC group, Universitat Autònoma de Barcelona

 COMMITTEE

HONORARY CHAIR

Enric Fossas
President of the Universitat Politécnica de Catalunya BarcelonaTech, Spain

Conference Chair

Pere Ponsa
Technical School of Vilanova, Spain.

Conference Co-Chair

José Antonio Román

Technical School of Vilanova, Spain

Program Chair

Daniel Guasch
Technical School of Vilanova, Spain.

Special Track Chairs

Cristina Manresa-Yee
University of Balearic Islands

Ann Morrison
Aalborg University

Workshop Chairs
Ana M. González Ramos
GENTIC, Internet Interdisciplinary Institute, IN3, Open University of Catalonia

Carina González
Department of Computer Engineering, University of La Laguna

Concurso AIPO TFG/TFM Chair

Asociación Interacción Persona Ordenador, AIPO
Student Design Challenge Chairs
Núria Ferran
Universitat Oberta de Catalunya, Open University of Catalonia

Muriel Garreta
Universitat Oberta de Catalunya, Open University of Catalonia

XVI Congreso Internacional de Interacción Persona-Ordenador

 PROGRAM COMMITTEE

Abascal, Julio Bravo, Crescencio

Universidad País Vasc, Spain Universidad de Castilla-La Mancha,
Spain
Abdulrahman H. Altalhi.
King Abdulaziz University,Arabia Saudí Carro, Rosa Maria
Universidad Autónoma de Madrid,
Aciar, Silvana Vanesa Spain
Universidad Nacional de San Juan,
Argentina Carron, Thibault
University of Savoie, France
Acuña, Silvia
Universidad Autónoma de Madrid, Ceres, Ramón
Spain Instituto de Automática Industrial
(CSIC), Spain
Alamán, Xavier
Universidad Autónoma de Cerezo, Eva
Madrid,Spain Universidad de Zaragoza, Spain

Albedawi, Ibrahim Castro, Silvia

King Abdulaziz University, Arabia Universidad Nacional del Sur,
Saudí Argentina

Arellano, Diana Cetina, Carlos

Filmakademie Baden-Wuerttemberg, Universidad de San Jorge, Spain
Germany
Clunie , Clifton
Azorín, José M. Universidad Tecnológica de Panamá,
Universidad Miguel Hernández, Spain Panama

Baldassarri, Sandra Clunie, Gisela

Universidad de Zaragoza, Spain Universidad Tecnológica de Panamá,
Panama
Bez, Marta
Universidad de Rio Grande do Sul, Cole, Ian
Brazil University of York, United Kingdom

Borges, Marcos Collazos,César

Universidad Federal de Rio de Janeiro, Universidad del Cauca, Colombia
Brazil
Coto, Mayela
Botella, Federico Universidad Nacional de Costa Rica,
Universidad Miguel Hernández, Spain Costa Rica

XVI Congreso Internacional de Interacción Persona-Ordenador

 PROGRAM COMMITTEE

De Castro, Carlos Garrido, José Luis

Universidad de Córdoba, Spain Universidad de Granada, Spain

Deco, Claudia Gil, Rosa M.

Universidad Nacional del Rosario, Universitat de Lleida, Spain
Argentina
Giraldo, William
Díaz, Antonio Universidad del Quindío, Colombia
Universidad de Málaga,Spain
Giulianelli, Daniel
Fardoun, Habib Universidad Nacional de La Matanza,
King Abdulaziz University, Arabia Argentina
Saudí
González, Carina
Fabregat, Ramón Universidad de la Laguna, Spain
Universidad de Girona, Spain
González, José Mariano
Fernández, Eduardo Universidad de Sevilla, Spain
Universidad de la República, Uruguay
González, Juan Manuel
Ferré, Xavier Universidad Autonoma de Puebla,
Universidad Politécnica de Madrid, México
Spain
González, Mª Paula
Frizera Neto, Anselmo CONICET, Argentina
Federal University of Espírito Santo
(UFES), Brazil
González, Pascual
Universidad de Castilla-La Mancha,
Gallud, José A. Spain
Universidad de Castilla-La Mancha,
Spain
González, Víctor
Instituto Tecnológico Autónomo de
Garay, Néstor México ITAM, Mexico
Universidad País Vasco, Spain

González-Sánchez, José L.
García, Roberto Universidad de Granada,Spain
Universitat de Lleida, Spain

Granollers, Toni
García-Pañeda, Xabiel
Universitat de Lleida,Spain
Universidad de Oviedo, Spain

Guerrero, Josefina
García-Peñalvo, Francisco Universidad Autonoma de Puebla,
Universidad de Salamanca, Spain
México

XVI Congreso Internacional de Interacción Persona-Ordenador

 PROGRAM COMMITTEE

Guerrero, Luís Macías, José A.

Universidad de Costa Rica, Costa Rica Universidad Autónoma de Madrid,
Spain
Gutiérrez, Francisco
Universidad de Granada, Spain Manresa, Cristina
Universidad de las Islas Baleares,
Haya, Pablo Spain
Instituto de Ingeniería de
Conocimiento, Spain Marco, Javier
Universidad de Zaragoza, Spain
Hernán, Isidoro
Universidad Rey Juan Carlos, Spain Marcos, Mari-Carmen
Universidad Pompeu Fabra, Spain
Hersh, Marion A.
University of Glasgow, Scotland Martín, Estefanía
Universidad Rey Juan Carlos, Spain
Hoppe, Ulrich
University Duisburg-Essen, Germany Marty, Jean-Charles
University of Savoie, France
Jaén, Fco. Javier
Universidad Politécnica de Valencia, Mashat, Abdulfattah
Spain King Abdulaziz University, Saudi
Arabia
Kafure, Ivette
Universidade de Brasilia, Brazil Mendes, Antonio José
Universidad de Coimbra, Portugal
Latorre, Pedro
Universidad de Zaragoza,Spain Molina-Massó, J. Pascual
Universidad de Castilla-La Mancha,
Llamas Nistal, Martín Spain
Universidad de Vigo,Spain
Montero, Francisco
Universidad de Castilla-La Mancha,
López, Juan Miguel
Spain
Universidad País Vasco, Spain

López, Víctor Mor-Pera, Enric

Universidad Oberta de Catalunya,
Universidad de Castilla-La Mancha,
Spain
Spain

Lozano, María Morán, Alberto

Universidad Autónoma de Baja
Universidad de Castilla-La Mancha,
California (UABC), Mexico
Spain

XVI Congreso Internacional de Interacción Persona-Ordenador

 PROGRAM COMMITTEE

Moreno, Lourdes Pastor, Óscar

Universidad Carlos III de Madrid, Universidad Politécnica de Valencia,
Spain Spain

Moreno, Mario Penichet, Víctor

Universidad Tecnológica de la Mixteca, Universidad de Castilla-La Mancha,
México Spain

Moreno-Ger, Pablo Peñalver, Antonio

Universidad Complutense de Madrid, Universidad Miguel Hernández, Spain
Spain
Plaza, José Antonio
Moriyón, Roberto Instituto de Microelectrónica de
Universidad Autónoma de Madrid, Barcelona IMB-CNM (CSIC), Spain
Spain
Pons, Claudia
Muñoz-Arteaga, Jaime Universidad Nacional de La Plata,
Universidad Autónoma de Argentina
AguasCalientes, México
Pow, José A.
Oliva, Marta Pontificia Universidad Católica del
Universitat de Lleida, Spain Perú, Perú

Ortega, Manuel Puerta, Angel

Universidad de Castilla-La Mancha, RedWhale Software,USA
Spain
Redondo, Miguel A.
Padawreski, Patricia Universidad de Castilla-La Mancha,
Universidad de Granada, Spain Spain

Padilla Zea, Natalia Reyes, Arcadio

Universidad de Granada, Spain Universidad de Málaga, Spain

Palanque, Philippe Ribera, Mireia

Universidad Paul Sabatier, France Universitat de Barcelona, Spain

Panach, José I. Rodríguez, Inmaculada

Universidad Politécnica de Valencia, Universitat de Barcelona, Spain
Spain
Rodríguez, Juan José
Paredes, Maximiliano Bankinter, Spain
Universidad Rey Juan Carlos, Spain

XVI Congreso Internacional de Interacción Persona-Ordenador

 PROGRAM COMMITTEE

Rojas, Luis A. Velázquez, Angel

Universidad Autónoma de Madrid, Universidad Rey Juan Carlos, Spain
Spain
Van der Veer, Gerrit
Rossi, Gustavo Universidad de Vrije, The Netherlands
Universidad Nacional de la Plata,
Argentina Vera, Pablo
Universidad Nacional de La Matanza,
Rusu, Cristian Argentina
Pontificia Universidad Católica de
Valparaiso, Chile Vicari, Rosa
Universidad de Rio Grande do Sul,
Sánchez, J. Alfredo Brazil
Universidad de las Américas Puebla,
Mexico Vilanova, Ramón
Universitat Autònoma de Barcelona,
Sanz, Cecilia Spain
Universidad Nacional de la Plata,
Argentina Winckler, Marco
Sendín, Montserrat Université Paul Sabatier, France
Universitat de Lleida, Spain
Xu, Diana
Serrano, Marcos University of Central Lancashire,
Institut de Recherche en Informatique England
de Toulouse (IRIT), France
Zepeda, Joaquín Sergio
Sevilla Moróder, Joaquin Universidad Autónoma Metropolitana,
Universidad de Navarra, Spain Mexico

Sigut, José F.
Universidad de La Laguna, Spain

XVI Congreso Internacional de Interacción Persona-Ordenador

 ORGANIZING COMMITTEE

Local Committee

Víctor Agramunt
Maria Hortènsia Álvarez
Bernardino Casas
Andreu Català
Yolanda Guasch
José María Merenciano
Montserrat Mestres
Luis Miguel Muñoz
Helena Querol
David Raya
Montserrat Solsona
Frederic Vilà

Technical School of Vilanova, Spain

XVI Congreso Internacional de Interacción Persona-Ordenador

PRESENTATION
 Presentación

En el siglo XXI, el ordenador de sobremesa está siendo desplazado por tecnología que

se mueve con nosotros. El cambio en la forma de interactuar supone revisar los

paradigmas clásicos existentes y avanzar en una interacción natural entre la persona, la

interfaz y el entorno. Interacción 2015 es un congreso internacional fomentado por la

Asociación de Interacción Persona-Ordenador (AIPO) que tiene como objetivo principal

promover y difundir los avances recientes en el área de la Interacción Persona-

Ordenador (IPO), tanto a nivel académico como empresarial. Este congreso es punto de

encuentro de profesionales y académicos en accesibilidad, usabilidad, ergonomía,

robótica, experiencia de usuario, realidad aumentada, etc. La IPO abarca un espectro

multidisciplinar como por ejemplo el diseño de productos-servicios, la sociología,

comunicación audiovisual, ciencias empresariales, bellas artes, psicología, ingeniería

industrial, etc.

La organización de Interacción 2015, XVI edición del congreso, está a cargo de la

Escuela Politécnica Superior de Ingeniería de Vilanova i la Geltrú y cuenta con la

colaboración de la Cátedra de Accesibilidad de la Universidad Politécnica de Catalunya

y el Centro de Estudios Tecnológicos para la Dependencia y la Vida Autónoma

(CETpD) y el apoyo de diversas entidades.

La presente edición del congreso ha recibido un total de 105 trabajos. En concreto, 94

contribuciones científicas procedentes de 22 países, de las cuales se han seleccionado 66

en forma de 41 artículos largos, 21 artículos cortos y 4 trabajos para el coloquio

doctoral. A estas contribuciones se suman los envíos de 9 propuestas para el Concurso

AIPO TFG/TFM 2015 (los finalistas (2) y vencedores (2)) y las propuestas recibidas

(2) del Student Design Challenge. A partir de los tópicos iniciales del congreso, los

trabajos seleccionados se han agrupado en las sesiones: natural user interfaces,

methodologies and models, enGendering Technology, child computer interaction,

XVI Congreso Internacional de Interacción Persona-Ordenador

 Presentación

interface design, interaction devices, interacton for people with disability, games,

usability and user experience, software-architecture and interaction, accessibility and

semantic web.

El congreso cuenta con la presencia de conferencias plenarias de Panos Markopoulos

(Interaction Design for Rehabilitation), Els Rommes (Including Gender in the

Interaction) y José Antonio Plaza (Towards Intracellular Computer-Human Interaction:

a microelectronic perspective), la conferencia de Manel Garrido (Robots Sensitivos) y el

tutorial de Antonio Miguel Baena (Desarrollo de aplicaciones de realidad aumentada

para tabletas y smartphones). Finalmente el panel de discusión sobre el presente y

futuro de IPO a cargo de José Antonio Macías (AIPO) y Marina Talavera (Hewlett

Packard).

En esta edición del congreso se presenta dos concursos en su primera edición: Concurso

AIPO al mejor Trabajo Fin de Grado y Trabajo Fin de Màster TFG/TFM en el ámbito

de la interacción y el Concurso de Diseño Student Design Challenge para estudiantes de

diseño y con el reto de interacción entre ciudadanos y ciudad mediante productos

digitales interactivos.

En Interacción 2015 se ha querido establecer un equilibrio entre el programa científico y

el social, en este sentido la colaboración del Ayuntamiento de Vilanova i la Geltrú ha

estado clave para mostrar aspectos de la vida de la ciudad, el patrimonio cultural

presente en el Museo del Ferrocarril de Catalunya y el Museo Víctor Balaguer. Otra

muestra es la de torres humanas castelleras a cargo de LLunàtics UPC Vilanova.

El programa científico y el programa social se complementan con exposición de

producto en el hall de la Escuela. Un agradecimiento a todas aquellas entidades que han

patrocinado y dado soporte al congreso: KUKA Robots Ibérica, UPF Barcelona School

XVI Congreso Internacional de Interacción Persona-Ordenador

 Presentación

of Management, Tobii, grupo investigación Automatización y Sistemas Avanzados de

Control ASAC 1 UAB, Hewlett Packard.

El congreso agradece a todas las personas que han colaborado en las diversas

actividades programadas. Gracias a José Antonio Macías, presidente de AIPO el soporte

para la confección del Congreso, así como a Víctor M. Penichet, CHISPA, y a todo el

equipo del capítulo español de ACM SIGCHI, por la suma de esfuerzos entre el

congreso y Association for Computing Machinery ACM. Finalmente, in Memoriam

Jesús Lorés por sus esfuerzos en la fundación de AIPO y el impulso de la disciplina IPO

desde el grupo de investigación GRIHO de la Universitat de Lleida.

Pere Ponsa Conference Chair

José Antonio Román, Conference Co-Chair
Daniel Guasch Program Chair

1
Grupo con el que se colabora en programa CiCYT DPI2013-47825-C3-1-R.

XVI Congreso Internacional de Interacción Persona-Ordenador

 Presentation

Interacción 2015 is the International Conference promoted by the Spanish Human

Computer Association (In Spanish: Asociación para la Interacción Persona-Ordenador,

AIPO), whose main objective is to promote and disseminate the recent advances in the

field of Human-Computer Interaction. This conference provides a forum for discussion

and exchange of ideas on design and application of techniques and methodologies with

a multidisciplinary approach (from engineering to human factors, human-robot

interaction, accessibility, interface design, usability, natural interaction, etc.),

The 16th International Conference Interacción 2015 is organized by the Technical

School of Vilanova I la Geltrú (Universitat Politècnica Catalunya Barcelona Tech) and

has the support of the Accessibility Chair, the Interactive Systems Design Lab and the

Technical Research Center of Dependency Care and Autonomous Living.

This edition has received 105 contributions: 94 scientific contributions from 22

countries, 9 papers into the AIPO Challenge best final project (degree, master) and 2

papers into the Student Design Challenge. In the 94 scientific contributions, the 66

accepted papers in these categories: 41 full papers, 21 short papers and 4 papers for the

Doctoral Colloquium. The sessions in this conference are classified in: natural user

interfaces, methodologies and models, EnGendering Technology, child computer

interaction, interface design, interaction devices, interacton for people with disability,

games, usability and user experience, software-architecture and interaction, accessibility

and semantic web.

The conference structure is a set of sessions and a set of social and scientific activities.

In example, the plenary conferences of Panos Markopoulos (Interaction Design for

Rehabilitation), Els Rommes (Including Gender in the Interaction) and José Antonio

Plaza (Towards Intracellular Computer-Human Interaction: a microelectronic

perspective), the Spanish Conference of Manel Garrido (Robots Sensitivos and the

XVI Congreso Internacional de Interacción Persona-Ordenador

 Presentation

Spanish tutorial de Antonio Miguel Baena (Desarrollo de aplicaciones de realidad

aumentada para tabletas y smartphones). Finally, a discussion about the present and

future of HCI with José Antonio Macias and Marina Talavera (Hewlett Packard).

Thank you for the support of all the people, local committee, chairs, entities (Víctor

Penichet CHISPA, Spanish local Chapter of ACM SIGCHI), (José Antonio Macías,

AIPO) and sponsors: KUKA Robots Ibérica, UPF Barcelona School of Management,

Tobii, research group ASAC UAB, Hewlett Packard and Llunàtics UPC Vilanova

(Human Towers).

In memoriam Jesús Lorés for the effort and support of HCI activities through the

GRIHO Research Group from the Universitat de Lleida.

Pere Ponsa Conference Chair

José Antonio Román, Conference Co-Chair
Daniel Guasch Program Chair

XVI Congreso Internacional de Interacción Persona-Ordenador

TABLE OF CONTENTS
CONFERENCES, TUTORIALS AND PANEL DISCUSSIONS, 23
Plenary Conference: Interaction Design for Rehabilitation, 24
Panos Markopoulos
Tutorial: Desarrollo de Aplicaciones de Realidad Aumentada para Tabletas y Smartphones, 25
Antonio Miguel Baena
Conferencia: Kuka – Robots sensitivos, 26
Manel Garrido
Plenary Conference: Including Gender in the Interaction, 27
Els Rommes
Plenary Conference: Towards Intracellular Computer-Human Interaction: a micro-electronic
perspective, 28
José Antonio Plaza
Panel de discusión: Reflexiones sobre la Interacción Persona-Ordenador, 29
José Antonio Macías, Marina Talavera

NATURAL USER INTERFACES (I), 30

Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles, 31
Clara Bonillo, Javier Marco, Eva Cerezo, Sandra Baldassarri
Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users, 40
Alfredo Mendoza G., Francisco J. Álvarez R., Ricardo Mendoza G., Francisco Acosta E., Jaime Muñoz A.
Multi-touch Technology in Early Childhood: Current Trends and Future Challenges, 49
Vicente Nacher, Javier Jaen
Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”, 54
Ann Morrison, Cristina Manresa-Yee, Hendrik Knoche

NATURAL USER INTERFACES (II), 62

Exploración del busto humano en tiempo real mediante interacción natural con fines educativos, 63
Roi Méndez, Julián Flores, Rubén Arenas
Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law, 70
Diana Carvalho , Luís Magalhães, Maximino Bessa, Eurico Carrapatoso
Including multi-stroke gesture-based interaction in user interfaces using a model-driven method, 77
Otto Parra González, Sergio España, Oscar Pastor
Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos, 86
Elena de la Guía, María D. Lozano, Víctor Penichet, Luis Orozco, Vicente López

METHODOLOGIES AND MODELS, 93

Limitaciones del Modelo de Tareas del W3C para aplicaciones Post-WIMP, 94
Miguel A. Teruel, Arturo C. Rodríguez, Francisco Montero, Elena Navarro, Víctor López-Jaquero,
Pascual González
An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus
Interactions, 103
Maha Khemaja , Félix Buendía
SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation, 112
David Céspedes-Hernández, Jorge Luis Pérez-Medina, Juan Manuel González-Calleros, Francisco J.
Álvarez Rodríguez, Jaime Muñoz-Arteaga
Towards the Definition of a Framework for the Management of Interactive Collaborative Learning
Applications for Preschoolers, 121

XVI Congreso Internacional de Interacción Persona-Ordenador

 Liliana Rodríguez-Vizzuett, Jorge Luis Pérez-Medina, Jaime Muñoz-Arteaga, Josefina Guerrero-García,
Francisco J. Álvarez-Rodríguez

DOCTORAL COLLOQUIUM, 130

Intelligent Playful Environments for Animals, 131
Patricia Pons, Javier Jaen, Alejandro Catala
Multi-Display Environments to Foster Emotional Intelligence in Hospitalized Children, 133
Fernando Garcia-Sanjuan, Javier Jaen, Alejandro Catala
KINDERTIVITY: Using Interactive Surfaces to Foster Creativity in Pre-kindergarten Children, 135
Vicente Nacher, Javier Jaen
Facial Emotion Analysis in Down’s syndrome children in classroom, 137
Pablo Torres-Carrión, Carina González-González

CHILD COMPUTER INTERACTION, 140

Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos
activos, 141
Carina S. González-González, Vicente Navarro-Adelantado
The SARA Project: An Interactive Sandbox for Research on Autism, 152
Diana Arellano, Ulrich Max Schaller, Volker Helzle, Reinhold Rauh, Marc Spicker, Oliver Deussen
Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach,
157
Sandra Cano, Jaime Muñoz Arteaga, César A. Collazos, Viviana Bustos Amador
Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje, 166
Jaime Muñoz Arteaga, Dulce Morales Hernández, Ricardo Mendoza, Carina S. Gonzalez
Enseñando Emociones a Niños Mediante Videojuegos, 171
Noemí Marta Fuentes García, Rafael López Arcos, Francisco Luis Gutiérrez Vela, Patricia Paderewski
Rodríguez, Natalia Padilla Zea

INTERFACE DESIGN, 180

CamScan, an application to identify everyday objects for users with vision impairments, 181
Rodrigo Capa-Arnao, Cristina Manresa-Yee, Ramon Mas-Sansó
Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio, 189
Félix Albertos Marco, Víctor M.R. Penichet, José A. Gallud
Elaborating a Web Interface Personalization Process, 198
J. Eduardo Pérez, Xabier Valencia, Myriam Arrue, Julio Abascal
Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison, 202
A. L. Carrillo, J. Falgueras
Generating User Interface from Conceptual, Presentation and User models with JMermaid in a
learning approach, 207
Jenny Ruiz, Gayane Sedrakyan, Monique Snoeck

INTERACTION DEVICES (I), 216

Blind Source Separation Performance based on Microphone Sensitivity and Orientation within
Interaction Devices, 217
Navya Amin, Thomas Gross, Susanne Rosenthal, Markus Borschbach
DIY computer mouse for special needs people, 225
Lluís Ribas-Xirgo, Francisco López-Varquiel

XVI Congreso Internacional de Interacción Persona-Ordenador

 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
229
Ernesto de la Rubia, Antonio Diaz-Estrella
An investigation into the comprehension of map information presented in audio, 234
Feng Feng, Tony Stockman, Nick Bryan-Kinns, Dena AI-Thani

INTERACTION DEVICES (II), 242

InclineType – An Accelerometer-based Typing Approach for Smartwatches, 243
Timo Götzelmann, Pere-Pau Vázquez
Basketball Activity Recognition using Wearable Inertial Measurement Units, 248
Le Nguyen Ngu Nguyen, Daniel Rodríguez-Martín, Andreu Català, Carlos Pérez-López, Albert Samà,
Andrea Cavallaro
EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical
knowledge, 255
Guillermo Bernal, Dishaan Ahuja, Federico Casalegno

INTERACTION FOR PEOPLE WITH DISABILITIES, 263

Performing universal tasks on the Web: interaction with digital content by people with intellectual
disabilities, 264
Tânia Rocha, Maximino Bessa, Luís Magalhães, Luciana Cabral
Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and
Navigation, 271
Nam Tung Ly, Jörn Hurtienne, Robert Tscharn, Samir Aknine, Audrey Serna
Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos, 276
Héctor Cardona Reyes, Jaime Muñoz Arteaga, Francisco Acosta Escalante, Francisco J. Álvarez
Rodríguez, Ángel Eduardo Muñoz Zavala
Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con
discapacidad cognitiva, 281
Juan C. Torrado, Javier Gómez, Germán Montoro

ENGENDERING TECHNOLOGY (I), 289

An ICT experience in Computer Women Role promotion: WikinformáticA! in Aragón. Promoting the use
of Wiki tools and visualizing the role of women in ICT, 290
María Teresa Lozano, Raquel Trillo-Lado, María Villarroya-Gaudó, Ana Allueva, Eva Cerezo
Inclusion and promotion of women in technologies, 294
Beatriz Revelles-Benavente, Lidia Arroyo Prieto, Nùria Vergés Bosch
Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms,
298
Núria Vergés Bosch

ENGENDERING TECHNOLOGY (II), 306

Influencia del Género en el Pensamiento Computacional, 307
Elisenda Eva Espino Espino
Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas, 312
Adriana Gil-Juárez, Ester Conesa, Joel Feliu, Montse Vall-llovera
Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusión, 319
Patricia Paderewski, Carina González González, Maribel García Arenas, Eva M. Ortigosa, Rosa Gil
Iranzo, Natalia Padilla-Zea

XVI Congreso Internacional de Interacción Persona-Ordenador

GAMES, 327
Desarrollo de experiencias lúdicas interactivas geolocalizadas, 328
A. J. Soriano Marín, J. L. González Sánchez, F. L. Gutiérrez Vela
La importancia de las emociones en el diseño de historias interactivas, 337
José Rafael López-Arcos, Patricia Paderewski Rodríguez, F. L. Gutiérrez Vela, Natalia Padilla-Zea,
Noemí Marta Fuentes García
Agente virtual emocional para dispositivos móviles, 341
Sandra Baldassarri, Eva Cerezo
Second Mind: A System for Authoring Behaviors in Virtual Worlds, 345
Manish Mehta, Andrea Corradini
Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que
Requieren Necesidades Especiales, 351
Iván Durango, Alicia Carrascosa, José A. Gallud, Víctor M. R. Penichet

USABILITY AND USER EXPERIENCE (I), 361

ECUSI: una herramienta que apoya la Evaluación Colaborativa de la Usabilidad de Sistemas
Interactivos, 362
Andrés Solano , Juan Camilo Cerón, César A. Collazos, Habib M. Fardoun, José Luis Arciniegas
Towards an Integration of Usability and Security for User Authentication, 368
Paulo C. Realpe, Cesar A. Collazos, Julio Hurtado, Antoni Granollers
Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el
análisis de la interacción del usuario, 374
Rafael Duque Medina, Alicia Nieto-Reyes
Supporting Users Experience in a 3D eCommerce Environment, 383
D.Contreras, M. Salamó, I. Rodríguez, A. Puig, A. Yañez
La delgada línea roja entre la usabilidad y la experiencia de usuario, 387
Yuliana Puerta Cruz, Cesar A. Collazos, Toni Granollers

USABILITY AND USER EXPERIENCE (II), 391

Effect of Snippets on User Experience in Web Search, 392
Mari-Carmen Marcos, Ferran Gavin, Ioannis Arapakis
Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación, 401
Juan P. Moreno, Antonio Peñalver, Federico Botella
Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el
Usuario, 410
María Constanza Pabón, César A. Collazos

SOFTWARE, ARCHITECTURE AND INTERACTION, 419

An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive
Software, 420
Luis A. Rojas, José A. Macías
Hacia la Caracterización de la Calidad de Interacción, 429
Cristina Roda, Víctor López-Jaquero, Francisco Montero
Extending MOOC ecosystems using web services and software architectures, 438
Juan Cruz-Benito, Oriol Borrás-Gené, Francisco J. García-Peñalvo, Ángel Fidalgo Blanco, Roberto
Therón
Arquitectura para la interacción en un videojuego para el entrenamiento de la voz de personas con
discapacidad intelectual, 445

XVI Congreso Internacional de Interacción Persona-Ordenador

 Mario Corrales, David Escudero, Valle Flores, Cesar González, Yurena Gutiérrez
Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y
Accesibilidad, 449
Giankaris Moreno, Vanessa Castillo, Kaiser Williams, Nyder Menéndez

ACCESSIBILITY AND SEMANTIC WEB, 453

The Effects of Cross-modal Collaboration on the Stages of Information Seeking, 454
Dena Al-Thani, Tony Stockman, Anastasios Tombros
Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs, 463
Francisco Iniesto, Covadonga Rodrigo
Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer, 469
Jordi Roig, Mireia Ribera
Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines, 478
Lourdes Moreno, Paloma Martínez, Isabel Segura-Bedmar, Ricardo Revert
Building a unified repository of interaction patterns, 487
Alfons Palacios, Roberto García, Toni Granollers, Marta Oliva

1º CONCURSO 2015 AIPO DE TFG/TFM, 492

Desarrollo de una herramienta para el diseño y ejecución de actividades enfocadas a ancianos con el
tabletop NIKVision, 493
Clara Bonillo Fernández
Análisis de requerimientos y prototipado de una aplicación accesible para personas ciegas basada en
la API de Google Maps, 496
Rubén Alcaraz Martínez
Factores importantes para un sistema de recomendacion de una red social educativa, 499
Virginia del Castillo Carrero, Isidoro Hernán-Losada, Estefanía Martín-Barroso
Case study on mobile Applications UX: Effect of the usage of a cross-platform development framework,
502
Esteban Angulo

INDEX OF AUTHORS, 503

XVI Congreso Internacional de Interacción Persona-Ordenador

CONFERENCES, TUTORIALS AND PANEL
DISCUSSIONS
 Plenary Conference: Interaction Design for Rehabilitation
P. Markopoulos

Plenary Conference: Interaction Design for

Rehabilitation

Panos Markopoulos

Department of Industrial Design, TU/e Eindhoven University of Technology, Eindhoven. The

Netherlands

[email protected]

ABSTRACT

There is a growing demand for rehabilitation and this necessitates the development of
rehabilitation technology. Further, such technologies can not only improve the quantity of
rehabilitation that can be offered but also its quality. For a long time, this challenge has been
considered as a purely engineering and technical challenge. The prolific growth of such
technologies and the increasing relevance of tele-rehabilitation scenarios, mean have drawn the
attention of industry and academia to the challenge of designing such technologies to motivate
patients, to ensure compliance, good ergonomics and product design to support the correct
execution of training exercises, and of course the potential of generating immense amounts of
data that can help monitor and provide feedback regarding patient progress. This talk presents a
few cases of such technologies and discusses some general challenges such as designing
appropriate feedback, including patients in participatory design processes, acceptance of
technologies, evaluation in the field, and interaction design issues relating to the implementation
of innovations in healthcare.

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 24 |
 Tutorial: Desarrollo de Aplicaciones de Realidad Aumentada para Tabletas y Smartphones
A. M. Baena

Tutorial: Desarrollo de Aplicaciones de Realidad

Aumentada para Tabletas y Smartphones

Antonio Miguel Baena

AMB Piensa
[email protected]

RESUMEN

Este tutorial se centra en la realidad aumentada y en cómo puede ser utilizada para coeditar y
desarrollar aplicaciones para tabletas y smartphones. El tutorial contempla estudio de casos
como los llevados a cabo recientemente en el Museo Thyssen Bortnemisza y la Central Nuclear
José Cabrera.

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 25 |
 Conferencia: Kuka – Robots sensitives
M. Garrido

Conferencia: Kuka – Robots sensitivos

Manel Garrido

KUKA Robots Ibérica

RESUMEN

La nueva era en la robótica pasa por transferir a los robots capacidades sensitivas. KUKA ha
conseguido que su robot LBR iiwa sienta y gestione con gran precisión los esfuerzos que se
generan en su entorno de acción. Por lo tanto nos permite automatizar procesos tanto
industriales como fuera de ese entorno, donde la sensibilidad de las personas es imprescindible
para conseguir el objetivo de la acción. Como consecuencia de esa virtud sensitiva, además, este
robot permite trabajar de forma segura en entornos colaborativos.

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 26 |
 Plenary Conference: Including Gender in the Interaction
E. Rommes

Plenary Conference: Including Gender in the

Interaction

Els Rommes

Institute for Gender Studies, Radboud University Nijmegen

The Netherlands

ABSTRACT

What does it mean to pay attention to gender in the design of computer systems? Should
everything be pink, or all content be about fashion? In this lecture, Rommes will discuss various
notions of what ‘inclusion of gender’ can mean, and which advantages and disadvantages the
various way of ‘including gender’ may have. In addition, she will discuss why it could be
relevant to pay attention gender and how this can be done through the use of various design
methodologies.

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 27 |
 Plenary Conference: Towards Intracellular Computer-Human Interaction: a micro-electronic perspective
J. A. Plaza

Plenary Conference: Towards Intracellular

Computer-Human Interaction: a micro-electronic
perspective

José Antonio Plaza

Micro and Nano-tools research group

Institute of Microelectronics of Barcelona
IMB-CNM (CSIC)
[email protected]

ABSTRACT

Success of the semiconductor industry has been driven in part by the miniaturization process, as
approximately every three years we see a new generation of memory chips and microprocessors,
in which the size of their fundamental elements, the transistors, is reduced 33%. The sizes of
these transistors are several orders of magnitude smaller in relation typical human body cell
(tens of microns of diameter). The microfabrication techniques of the electronics industry are
being also routinely adapted to fabricate systems which are able to integrate mechanical,
thermal, optical, magnetic, chemical, or even fluidics components in the same silicon chip, in
addition to combining them with electronic components. These silicon chips inside human
living cells could provide endless possibilities, beyond the scope of our imagination. In this talk,
we present our research in the bases of this incipient future field. We believe that the study of
this field will open a new line of research based on Human-Computer Interactions to investigate
the relationship between chips and human living cells, human organs or human actors.

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 28 |
 Panel de discusión: Reflexiones sobre la Interacción Persona-Ordenador
J. A. Macías, M. Talavera

Panel de discusión: Reflexiones sobre la

Interacción Persona-Ordenador

José Antonio Macías Marina Talavera

Presidente Asociación Interacción Persona- Experience Design Strategist

Ordenador Hewlett Packard Española
Universidad Autónoma de Madrid [email protected]
[email protected]

OBJETIVO

En el siglo XXI, el ordenador de sobremesa está siendo desplazado por tecnología que se mueve
con nosotros. El cambio en la forma de interactuar supone revisar los paradigmas clásicos
existentes y avanzar en una interacción natural entre la persona, la interfaz y el entorno.
Interacción 2015 es un congreso internacional fomentado por la Asociación de Interacción
Persona-Ordenador (AIPO) que tiene como objetivo principal promover y difundir los avances
recientes en el área de la Interacción Persona-Ordenador (IPO), tanto a nivel académico como
empresarial. Este panel de discusión es un punto de encuentro de profesionales y académicos
para reflexionar conjuntamente sobre el presente de la interacción persona-ordenador y
tendencias de futuro. El panel lanza las siguientes preguntas, ¿Hacía donde avanza el diseño de
la interfaz de usuario?, ¿Cómo mejorar la experiencia de usuario?, ¿Cuáles serán los siguientes
paradigmas clásicos?, ¿Qué retos deben afrontarse en la sociedad ante los cambios
tecnológicos? , ¿Cómo potenciar la sinergia entre profesionales y académicos?

XVI Congreso Internacional de Interacción Persona-Ordenador | Conferences, Tutorials and

Panel Discussions
| 29 |
NATURAL USER INTERFACES (I)
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri

Diseño de actividades de mejora de capacidades

cognitivas para tabletops tangibles
Clara Bonillo Javier Marco Eva Cerezo,
GIGA Affective Lab GIGA Affective Lab Sandra Baldassarri
Dep. Informática e Ingeniería de Dep. Informática e Ingeniería de GIGA Affective Lab
Sistemas, Sistemas, Dep. Informática e Ingeniería de
Universidad de Zaragoza, España. Universidad de Zaragoza, España. Sistemas, Instituto de Investigación en
[email protected] [email protected] Ingeniería de Aragón (I3A)
Universidad de Zaragoza, España.
{ecerezo, sandra}@unizar.es

ABSTRACT Además de las actividades tradicionales de mejora o

El objetivo de este trabajo es explorar la potencialidad del uso de mantenimiento, durante los últimos años se ha empezado a
tabletops tangibles en el desarrollo de actividades terapéuticas de explorar el uso de dispositivos tabletop para trabajar la
mejora o mantenimiento de capacidades cognitivas deterioradas estimulación cognitiva, centrándose principalmente en ancianos
por la edad, enfermedades o tratamientos. Para ello se ha llevado a [2] [7]. Un tabletop es un dispositivo, con un aspecto más o menos
cabo un análisis de las áreas cognitivas y de las actividades más cercano al de una mesa convencional, cuya superficie está
habituales utilizadas por los terapeutas en el trabajo de dichas aumentada mediante la proyección de imagen y sonido procedente
áreas. Ello ha llevado a la definición de un lenguaje de marcas de una aplicación informática y en el que la interacción con dicha
basado en XML destinado a la definición de las actividades, y de aplicación se lleva a cabo mediante movimientos de los dedos en
un reproductor que permite la ejecución de dichas actividades contacto con la superficie de la mesa (multitáctil). Esta forma de
sobre el tabletop tangible NIKVision del grupo GIGA Affective interacción presenta muchas ventajas, ya que la superficie del
Lab de la Universidad de Zaragoza. Nueve actividades han sido tabletop supone un espacio amplio para las capacidades visuales y
desarrolladas con dichas herramientas y evaluadas por una motrices de los usuarios. La estimulación audiovisual les resulta
terapeuta y su grupo de pacientes, pertenecientes a una asociación motivante y se les ofrece un mayor rango de actividades que
local de adultos con enfermedad mental. Los resultados de la pueden abarcar uno o más aspectos de la estimulación cognitiva.
evaluación son muy prometedores y animan a continuar Sin embargo, este tipo de interacción presenta una importante
trabajando en la línea. desventaja y es que la complejidad y precisión de muchos gestos
táctiles puede hacerla dificultosa y frustrante para usuarios con
Categorías and Términos Descriptores problemas de motricidad, como los ancianos o personas con
H.5.2. Interfaces de usuario: estilos de interacción, prototipado. trastornos mentales graves. Por ello, otros modelos de Interacción
D.2.2 Herramientas de diseño y técnicas. Interfaces de Usuario. Natural, como la Interacción Tangible, pueden ofrecer una
alternativa más adaptada a las características de dichos usuarios
Términos Generales [4]. La Interacción Tangible plantea que la interacción entre el
Diseño, Lenguajes. usuario y la aplicación informática sea a través de objetos físicos
de uso cotidiano. Técnicamente, un dispositivo tabletop tangible
Palabras clave es capaz de identificar distintos objetos colocados en su
Interacción tangible, interacción natural, estimulación cognitiva, superficie, así como seguir las distintas manipulaciones que los
mesa interactiva, tabletop, lenguaje de marcas. usuarios realizan con ellos, mostrando información proyectada
sobre la misma superficie donde se manipulan los objetos.
1. INTRODUCCIÓN
El envejecimiento de la población en los últimos años ha supuesto El grupo GIGA Affective Lab cuenta con NIKVision [8], un
una revolución demográfica: se estima que actualmente hay unos tabletop Tangible en el que la interacción se lleva a cabo mediante
600 millones de personas mayores de 60 años y parece que esta objetos colocados sobre la superficie de una mesa. El objetivo de
cifra va a ir aumentando. Este hecho supone un reto para el resto este trabajo ha sido explorar las potencialidades de la Interacción
de la sociedad, puesto que hay que considerar el declive tanto Tangible en un dispositivo tabletop como NIKVision en la
físico como cognitivo que se produce a medida que una persona estimulación cognitiva.
envejece. Por otra parte, se estima que el 14% de la carga global El artículo está organizado de la forma siguiente: en la sección 2
de las enfermedades a nivel mundial puede ser atribuida a se realiza un estado del arte en el que se estudian tabletops
trastornos neuropsiquiátricos [10], los cuáles también producen un destinados a la estimulación cognitiva; en la sección 3 se realiza
declive en las capacidades cognitivas ya sea por la propia un estudio de las áreas cognitivas que más se suelen trabajar y del
enfermedad o por efectos secundarios de la medicación que dichas tipo de actividades más frecuentes en cada una de ellas; así
enfermedades conllevan. Una forma de ayudar a estas personas a mismo, se presentan las herramientas generadas para la definición
mejorar dichas capacidades, o por lo menos evitar su deterioro, es y ejecución de actividades: el lenguaje de marcas y el reproductor.
a través de actividades que potencien el uso de dichas capacidades En la sección 4 se presentan las actividades desarrolladas y en la
[9].

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 31 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri
sección 5 su evaluación con usuarios. Finalmente en la sección 6 3. ESTUDIO DE ACTIVIDADES
se presentan conclusiones y trabajo futuro.
COGNITIVAS
2. ESTADO DEL ARTE Nuestro objetivo es llevar a cabo una selección de actividades que
A continuación se va a hacer un repaso del estado del arte en cubran el máximo número de áreas cognitivas y que además
cuanto a experiencias de trabajo de capacidades cognitivas aprovechen al máximo el uso del tabletop y de la Interacción
haciendo énfasis en aquellas que hacen uso de tabletops. Tangible.

En Kown et al. 2013 [5] se presenta el sistema E-CoRe para la 3.1 Áreas cognitivas
mejora de la capacidad cognitiva a través de un dispositivo Antes de realizar la selección de actividades se explicará en qué
tabletop. La una aplicación llamada “Making Cookies” utiliza consiste cada una de las áreas cognitivas que suelen trabajarse [9],
distintos objetos para simular que se hacen galletas y su objetivo junto con ejemplos de actividades que se suelen desarrollar para
es mejorar tres áreas cognitivas: la memoria, el razonamiento y la potenciar dichas áreas.
atención.
• Memoria: es una de las funciones cerebrales cuyo
En Gamberini et al. 2006 [2] se ha desarrollado un tabletop con deterioro es más patente con la edad. Un ejemplo típico
una aplicación similar al juego de encontrar parejas. La de juego de memoria es el juego de encontrar las
interacción se lleva a cabo con unos lápices especiales en lugar de parejas.
con el dedo a través de diversos minijuegos, en los que se busca
mejorar funciones cognitivas como la memoria, el razonamiento, • Atención: dentro de la atención podemos distinguir a su
la atención selectiva y dividida, y la clasificación. vez tres subtipos [1]: selectiva (aquella que se basa en
reaccionar ante estímulos concretos de entre todos los
En Leitnet et al. 2007 [6] se ha desarrollado un tabletop Tangible que se presentan), dividida (aquella en la que hay que
para la rehabilitación física y cognitiva. En él se plantea trabajar centrarse en varios estímulos a la vez, perteneciendo
con distintas actividades tipo puzle manipulando cubos con generalmente estos estímulos a sentidos diferentes como
distintos patrones dibujados sobre ellos, siendo el objetivo de la por ejemplo la vista y el oído), y sostenida (aquella que
tarea alinearlos para que coincidan con el patrón que se muestra requiere concentración continua). Un ejemplo de
en la mesa. actividad de atención es el juego de encontrar las
Por último, Sociable [11] es un proyecto europeo para el diferencias o los puzles, que se utilizan para trabajar la
entrenamiento cognitivo de personas mayores. Para ello se hace atención sostenida.
uso de un tabletop táctil para ejecutar actividades organizadas en • Cálculo: esta área cognitiva se refiere a la capacidad de
diversas categorías: memoria, atención, razonamiento, lenguaje y una persona de realizar cálculos matemáticos sin
orientación. instrumentos adicionales. Ejemplos de actividades de
Del estudio de los ejemplos anteriores se puede concluir que tanto este tipo se pueden encontrar en juegos de mesa como el
el tabletop como las actividades desarrolladas para éste acercan a Parchís (hay que ir contando los puntos cuando te
las personas con dificultades cognitivas a las nuevas tecnologías y comes una ficha), o el Dominó (se tienen que estar
les permiten seguir trabajando sus capacidades para evitar (o por contando constantemente los puntos que se llevan).
lo menos disminuir) su deterioro. Sin embargo, no se está • Lenguaje: capacidad de los seres humanos para
aprovechando todo lo que se podría las ventajas que ofrecen la comunicarse por medio de signos. Actividades que
combinación de la Interacción Tangible con los tabletop. potencian esta área pueden ser emparejar sinónimos y
Por ejemplo, a pesar de que Gamberini et al. 2006 [2] trabaja antónimos, u ordenar frases desordenadas.
diferentes áreas cognitivas, los lápices que se utilizan • Orientación espacio-temporal: es la toma de
simplemente como sustitutos de los dedos, por lo tanto no se conciencia de los movimientos en el espacio y el tiempo
tendría que considerar como Interacción Tangible. de forma coordinada. Se relaciona con el conocimiento
Con los cubos de Leiner et al. 2007 [6], o los objetos para hacer que tiene cada persona del tiempo en el que está (fecha
galletas de Kown et al. 2013 [5], sí que se está utilizando la y hora) y del lugar en el que está. Para trabajar la
Interacción Tangible. Sin embargo, ambos se han centrado en el orientación se suelen utilizar preguntas del tipo: ¿qué
desarrollo de una única actividad con juguetes ya predefinidos, hora es?, ¿en qué momento del día estamos?, ¿dónde
por lo que es muy limitada en los aspectos cognitivos cubiertos. estamos?

Por último, en el proyecto Sociable [11] se menciona un tabletop • Razonamiento: facultad que permite resolver
con una gran variedad de actividades cognitivas pero todas ellas problemas, extraer conclusiones y aprender de manera
están basadas en la interacción táctil y por lo tanto no aprovecha consciente de los hechos, estableciendo conexiones
las ventajas que tendrían esas actividades usando Interacción causales y lógicas necesarias entre ellos. Un ejemplo
Tangible. Basándonos en la variedad de actividades de proyecto claro de juego de razonamiento es el Sudoku.
Sociable el presente trabajo pretende explorar las potencialidades
Una vez detectadas la áreas y actividades más habituales, se llevó
que ofrecen los dispositivos tabletop y la Interacción Tangible en
a cabo un análisis de requisitos que se explican en la siguiente
el tratamiento de problemas cognitivos.
sección.
A continuación se presenta el estudio que se ha realizado de las
distintas actividades y juegos que se utilizan habitualmente para el 3.2 Análisis de requisitos
tratamiento de problemas cognitivos. Se llevó a cabo un análisis exhaustivo de las diferentes
actividades cognitivas expuestas en el anterior apartado, con
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)
| 32 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri
objeto de extraer los elementos comunes a todas ellas y poder determinado, durante el cual el jugador debe memorizarlo (R4 de
elaborar un conjunto de requisitos (Tabla 1) a cumplir por una la Tabla 1), por ejemplo, en el juego Simon™.
aplicación informática capaz de ejecutar cualquiera de estas
actividades en un dispositivo tabletop. A menudo, las actividades y juegos cognitivos se juegan como
una secuencia de tareas (R7 de la Tabla 1) con un nivel de
Tabla 1. Requisitos de la herramienta dificultad variable, o gradual. Conforme el jugador completa
tareas fáciles, se vuelve a plantear la misma actividad con un
Requisito Actividad grado de dificultad mayor. Por ejemplo, de nuevo el juego
R Se han de tener áreas interactivas en las Simon™, se juega como una sucesión de tareas en el que la
Todas
1 que poder poner uno o más objetos. secuencia a memorizar va aumentando gradualmente.
R En algunas actividades se ha de tener en
Una vez recopilados los requisitos comunes a las actividades
2 cuenta la orientación de los objetos Atención cognitivas y juegos de mesa, se ha creado una herramienta
colocados sobre el área informática que permite definir y ejecutar actividades basadas en
R Las áreas tienen una lista de objetos Interacción Tangible en un dispositivo tabletop. Para ello, primero
3 correctos y una lista de objetos Todas ha sido necesario crear un lenguaje de definición de actividades y
incorrectos juegos que cubra todos los requisitos anteriormente expuestos
R La imagen de fondo mostrada en pantalla usando una sintaxis entendible por una aplicación informática.
4 podrá cambiar pasado un cierto tiempo Memoria
que será configurable
R Las áreas interactivas pueden estar fijas
5 en una posición en pantalla o ir asociadas Todas
a objetos
R Se tiene que poder ofrecer algún tipo de
6 realimentación cuando se coloquen
Todas
objetos en las áreas dependiendo de si
están bien o mal
R Se tiene que poder definir tareas dentro Figura 1: Diferentes juegos de mesa. Izquierda: tablero de
7 de una misma actividad de modo que Parchís. Derecha: Dominó, un juego de mesa sin tablero
Todas
cuando se complete una tarea de la
misma, se pase a la siguiente 3.3 Lenguaje de modelado de actividades
R Las actividades pueden requerir objetos cognitivas
8 concretos diferenciados o fichas del Todas Partiendo del análisis de requisitos previo se ha extraído un
mismo tipo
vocabulario o conjunto de términos que siempre aparecen en los
juegos y actividades cognitivas. Un juego de tabletop es modelado
En el análisis se vio que en realidad los elementos comunes eran como una secuencia de tareas (ver fig. 2). Cada tarea es un
coincidentes con los de otros juegos puramente lúdicos que se objetivo que el jugador ha de alcanzar para poder avanzar en el
pueden jugar sobre la superficie de una mesa. Así, un juego típico juego. Cada tarea está compuesta por:
de tablero está compuesto por un conjunto de áreas fijas en la
superficie (R1 de la Tabla 1), un conjunto de piezas de juego (R8 • Un fondo: el tablero.
de la Tabla 1) y un conjunto de reglas que definen el significado
• Varias áreas: zonas del tablero o de una pieza de juego
de colocar piezas en cada área (R3 de la Tabla 1) y su efecto
en las que posicionar unas determinadas piezas de juego
asociado (R6 de la Tabla 1). Además de la colocación de la ficha
tiene un significado.
dentro del área, la orientación puede también tener un significado
concreto (R2 de la Tabla 1), por ejemplo, en el juego del • Varias piezas de juego: objetos utilizados en el juego.
Tangram. Juegos de tablero clásicos como el parchís (ver fig. 1 Se colocan en las áreas; fuera de ellas la pieza no tiene
izquierda) pueden describirse como un tablero dividido en varias ningún significado para el juego.
celdas en las que se colocan las piezas de juego siguiendo unas
determinadas reglas. • Retroalimentación: elementos gráficos o sonoros que
muestran las consecuencias de las acciones del jugador.
Por otro lado, hay muchos juegos populares de mesa que no usan
tablero y por tanto no tienen áreas predefinidas en la superficie de Siguiendo el R3 de la Tabla 1, cada área lleva asociada una lista
la mesa; p.e. Dominó (ver fig. 1 derecha) o juegos de cartas. En de piezas de juego correctas e incorrectas o, por defecto, todas las
este tipo de juegos cada pieza de juego se relaciona con las otras piezas no correctas son consideradas incorrectas. Cuando una
piezas por “proximidad”, la orientación puede tener un significado pieza de juego se sitúa dentro de un área, esta acción es
(R2 de la Tabla 1) y las reglas de juego se aplican cuando un pieza interpretada en el juego como correcta o incorrecta dependiendo
de juego está al lado de otra; en otras palabras, cada pieza juego de la lista en la que esté contenida la pieza de juego. El
tiene un área asociada a ella (R5 de la tabla 1), y cuando otras componente de retroalimentación informará al jugador de si la
piezas entran en dicha área se produce un efecto en el juego (R6 acción es correcta o incorrecta a través de una imagen, una
de la Tabla 1). animación y/o un sonido. Para completar una tarea del juego,
todas las áreas definidas en esa tarea han de contener todas las
En los juegos de memoria, es necesario mostrar inicialmente al piezas de juego correctas (y sólo las correctas). En ese caso, el
jugador una información que se oculta tras un tiempo componente de retroalimentación informará también de que la
tarea está completa y el juego avanzará a la siguiente tarea.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 33 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri
Cuando se ejecuta, el intérprete lee todos los ficheros XML
almacenados en una carpeta específica del sistema informático del
tabletop. Cada fichero XML contiene un juego modelado con la
sintaxis detallada en la sección anterior. El reproductor muestra en
la superficie del tabletop una pantalla de menú con todos los
juegos de la carpeta (ver fig. 4 izquierda). De esta manera, el
jugador puede ejecutar un juego tocando su icono correspondiente
Cuando un juego ha sido elegido, el intérprete carga e interpreta el
fichero XML, y recupera todos los gráficos y ficheros de audio
requeridos para renderizar el juego sobre el tabletop. El intérprete
es capaz de cargar los formatos de imagen (BMP, GIF, JPG,
PNG…), y de audio (WAV, MP3…) más comunes. En caso de los
recursos de animación, el intérprete acepta ficheros SWF.
Finalmente, el tablero se muestra en el tabletop y el jugador puede
jugar situando las diferentes piezas de juego sobre la superficie
Figura 2: Jerarquía de los elementos
(ver fig. 4 derecha).
La estructura jerárquica de los diferentes elementos hace que la
forma más adecuada de modelar las actividades y juegos sea
mediante un lenguaje de marcas como el XML, ya que este tipo
de lenguajes son muy adecuados para representar jerarquías. La
figura 3 muestra la especificación en sintaxis XML del lenguaje
de modelado.
<game>
<task>
<background>
<color rgb="0xrrggbb"/>
<image path="path/filename"/>
<audio path=”path/filename”/> Figura 4: Intérprete del lenguaje de marcas. Izquierda:
</background>
Pantalla de menú del intérprete. Derecha: Intérprete
<area[associated_fid=”id”][orient=”yes/no”]>
<pos x="n" y="n" width="n" height="n"/> ejecutando un juego
<image path="path/filename"/>
<imageCorrect path="path/filename"/> 4. ACTIVIDADES DESARROLLADAS
<imageWrong path="path/filename"/> Se decidió comenzar con actividades de memoria, atención y
<fid id="1,2,…" correct="yes/no" razonamiento, en las que las potencialidades del uso de objetos
sound="path/filename" orient="n"/> eran más palpables. A continuación se detallan las actividades
<fid id="*” correct="yes/no" agrupadas por área de trabajo cognitivo.
sound="path/filename" orient="n"/>
</area>
…
4.1 Actividades de memoria
<feedback> Como actividades de memoria se desarrollaron las actividades
<pos x="n" y="n" width="n" height="n"/> “Lista de la compra”, “Viajes” y “Kraken”.
<imageCompleted path="path/filename"/>
<soundCompleted path="path/filename"/> En “Lista de la compra” se muestra en pantalla una imagen con
<imageCorrect path="path/filename"/> una lista de la compra con alimentos durante cinco segundos.
<soundCorrect path="path/filename"/> Pasado ese tiempo, se quita la imagen de la lista poniéndose una
<imageWrong path="path/filename"/> imagen con una bolsa de la compra y el usuario ha de seleccionar
<soundWrong path="path/filename"/> los alimentos correctos y situarlos en la bolsa. Los objetos que se
</feedback> utilizan en esta actividad son doce juguetes de comidas y bebidas:
</task>
agua, cebolla, guisantes, huevo, leche, manzana, naranja,
…
</game> pimiento, tomate, zanahoria, zumo de naranja y zumo de uva (ver
fig 5 arriba). La retroalimentación de esta actividad consiste en
Figura 3: Sintaxis XML del lenguaje de modelado una carita que se pone sonriente cuando se coloca un alimento
Una vez definido el lenguaje de marcas, se ha implementado una correcto, y triste si el alimento colocado no estaba en la lista.
aplicación informática capaz de interpretar ficheros XML que En la actividad “Viajes”, al usuario se le muestra un mapa de
siguen la sintaxis previamente expuesta para modelar los juegos, y Europa y se reproduce una grabación con un itinerario a seguir
ejecutarlos en un dispositivo tabletop. con distintos medios de transporte. El usuario ha de recordar el
itinerario y situar los transportes correspondientes en los países
3.4 Intérprete del lenguaje de modelado de correctos. Los objetos que se utilizan en esta actividad son tres
actividades cognitivas juguetes de medios de transporte: tren, avión y barco (ver fig 5
El intérprete es la aplicación encargada de cargar y ejecutar los abajo-izquierda). La retroalimentación de esta actividad consiste
juegos en el dispositivo tabletop. en una carita que se pone sonriente cuando se pone el medio de
tranporte en su correspondiente país, o triste en caso contrario.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 34 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri
En la actividad “Kraken”, al usuario se le muestra una imagen de roja encima del número dependiendo de si este es correcto o
un mar dividido en casillas. Durante unos segundos, el usuario incorrecto.
puede ver en qué casillas están varios monstruos justo antes de
que la imagen inicial se oculte y sólo queden las casillas En “Marca los símbolos” se presenta un mapa con símbolos de
mostrando mar. El objetivo del juego es que el usuario (que gasolineras, restaurantes, hoteles, farmacias...y se pide al usuario
empieza en la casilla de la esquina inferior derecha) llegue a la que marque todos los que sean de un tipo concreto. Los objetos
isla del tesoro (situada en la esquina superior izquierda) evitando que se utilizan en esta actividad son fichas que el usuario tendrá
las casillas en las que había monstruos. El objeto que se utiliza en que colocar en los símbolos correspondientes (ver fig 6 abajo-
esta actividad es un juguete de un barco (ver fig 5 abajo-derecha). izquierda). La retroalimentación de esta actividad consiste en la
La retroalimentación de esta actividad consiste en la animación aparición de un tick verde o de una cruz roja encima del símbolo
del monstruo que aparece cuando el jugador pisa una casilla dependiendo de si este es correcto o incorrecto.
incorrecta. La actividad “Tangram” tiene dos modos: el fácil, en el que se ve
en qué posición está cada una las piezas que conforman la figura,
y el difícil, en el que solo se muestra el contorno de la figura. En
la modalidad difícil también se trabaja el razonamiento, al tener
que deducir dónde va cada pieza. Los objetos con los que
trabajamos en esta actividad son las siete piezas del Tangram (ver
fig 6 abajo-derecha). La retroalimentación de esta actividad
consiste en una carita que se pone sonriente cuando una pieza se
coloca de forma correcta, o triste si se coloca una pieza del puzle
en un lugar que no le corresponde.

Figura 5: Actividades de memoria. Arriba: Lista de la compra

Abajo-Izquierda: Viajes. Abajo-Derecha: Kraken

4.2 Actividades de atención

Como actividades de atención, se desarrollaron las siguientes:
“¿Cuántos hay?” (atención dividida), “Marca los símbolos”
(atención selectiva), y “Tangram” (atención sostenida).
En “¿Cuántos hay?” al usuario se le muestra una imagen con
números entre el 0 y el 9. Nada más empezar la actividad suena
una grabación que le dice al usuario qué número ha de buscar. El
usuario tendrá que ir situando fichas sobre todos los números que
encuentre de ese tipo, mientras que a la vez se estará
reproduciendo una grabación con una secuencia de golpes: por
ejemplo, sonarán cuatro golpes, habrá una pausa, sonarán ocho Figura 6: Actividades de atención. Arriba: ¿Cuántos hay?
golpes, habrá una pausa... de modo que el usuario, a la vez que Abajo-Izquierda: Marca los símbolos. Abajo-Derecha:
marca los números, durante la pausa tendrá que decir cuántos Tangram
golpes han sonado. Los objetos que se utilizan en esta actividad
son fichas que el usuario tendrá que colocar en los números 4.3 Actividades de razonamiento
correspondientes (ver fig 6 arriba). La retroalimentación de esta Como actividades de memoria, se desarrollaron las actividades
actividad consiste en la aparición de un tick verde o de una cruz “Relaciones sintagmáticas”, “Analogías”, y “Completa la
secuencia”.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 35 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri
En “Relaciones sintagmáticas”, el usuario ha de seguir unas
instrucciones que guardan relaciones entre sí para rellenar una
composición de figuras geométricas con las figuras de la forma y
color adecuados. Esta actividad tiene dos modos: el fácil, con una
composición de cuatro figuras geométricas (dos cuadrados y dos
círculos) y el difícil, con una composición de seis figuras
geométricas (dos cuadrados, dos círculos y dos triángulos). Los
objetos que se utilizan son cuadrados, círculos y triángulos de
colores rojo, azul, verde o amarillo (ver fig 7 arriba). La
retroalimentación de esta actividad consiste en una carita que se
pone sonriente o triste dependiendo de si la acción del jugador es
correcta o incorrecta.
En “Analogías” se dispone de un conjunto de fichas con dos
imágenes cada una. A cada imagen le corresponde una pareja, que
estará en otra ficha diferente. El objetivo de la actividad es que
todas las fichas formen una cadena de modo que todos los
extremos de las fichas estén con su correspondiente pareja. Los
objetos que se utilizan en esta actividad son las diferentes fichas
(ver fig 7 abajo-izquierda). La retroalimentación de esta actividad
es únicamente sonora, diferenciando cuando dos fichas están bien
unidas y cuando no.
En “Completa la secuencia” se usan fichas de dominó para
mostrar una secuencia en la que faltan algunas piezas. El usuario
ha de seleccionar la pieza que falta para completar la secuencia.
Los objetos que se utilizan en esta actividad son las fichas de
dominó (ver fig 7 abajo-derecha). La retroalimentación de esta
actividad consiste en una carita que se pone sonriente cuando se
coloca una pieza en su lugar correcto, o triste en caso contrario.

Figura 7: Actividades de razonamiento. Arriba: Relaciones

sintagmáticas. Abajo-Izquierda: Analogías. Abajo-Derecha:
Completa la secuencia

5. EVALUACIÓN DE LAS ACTIVIDADES

Nuestro objetivo al plantear la evaluación fue responder a la
pregunta: ¿Cómo perciben los pacientes las nuevas actividades en
un tabletop Tangible?
La evaluación se llevó a cabo como resultado de la colaboración
con ASAPME, una asociación local que trabaja con adultos con
enfermedades mentales. La terapeuta de esta asociación usa
juegos tradicionales (sin ordenador) con sus pacientes para
trabajar y mejorar sus habilidades cognitivas, que pueden estar
afectadas como resultado de su enfermedad o como efecto
secundario de su medicación. Las áreas cognitivas que trabajan
son: atención, concentración, memoria, percepción, lenguaje,
cálculo, razonamiento y pensamiento lógico, y motivación entre
otras. El grupo de terapia estaba compuesto por cuatro adultos de
edades entre los 25 y 65 años con diferentes enfermedades
mentales: esquizofrenia, esquizofrenia y retraso mental, síndrome
frontal, psicosis maniaco depresiva, trastorno mental orgánico con
descontrol de impulsos y retraso mental, trastorno de ansiedad y
trastornos físicos.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 36 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri

5.1 Metodología Además, en relación con el cuestionario AttrakDiff, la terapeuta

Después de enseñarle a la terapeuta las actividades expuestas en la nos dijo que este tipo de cuestionario resultaría difícil de rellenar
sección 4, ella mostró gran interés en aplicarlas con sus pacientes para los pacientes, así que se realizó una simplificación del
en sus sesiones de trabajo. Además, la actividad de razonamiento mismo: en primer lugar se eliminó la pregunta “predecible –
“Relaciones sintagmáticas” es una actividad que la terapeuta impredecible” por considerar la terapeuta que sus pacientes no
aplica habitualmente con sus pacientes sin uso de ordenador (ver iban a comprender a qué se refería este par de adjetivos; en
fig 8). segundo lugar, en vez de tener que puntuar utilizando dos
adjetivos antónimos como indica el cuestionario AttrakDiff
Se planearon dos sesiones de evaluación. Durante la primera original, la terapeuta eligió de entre los dos adjetivos
sesión, los pacientes probaron las actividades desarrolladas contrapuestos el más comprensible (por ejemplo, entre el par de
exceptuando “Relaciones Sintagmáticas”. Con esta evaluación se adjetivos “aislante” e “integrador” se eligió el segundo) de modo
quería valorar la experiencia del usuario a la hora de probar las que el paciente sólo tuviera que calificar dicho adjetivo entre 1 y
actividades en la mesa. Para ello se utilizó el cuestionario 7, significando el 1 “totalmente en descuerdo” y el 7 “totalmente
AttrakDiff [3], el cual es ampliamente usado para evaluar la de acuerdo” (ver fig. 9).
percepción de la usabilidad que los usuarios tienen al utilizar un
dispositivo o aplicación. El cuestionario AttralDiff considera tanto
dimensiones pragmáticas como hedónicas de usabilidad.
Durante la segunda sesión de evaluación, los pacientes probaron
la actividad “Relaciones Sintagmáticas”. El objetivo de esta
sesión era comparar la preferencia de los pacientes entre la
actividad con tabletop y sin tabletop.
Durante ambas sesiones estaban presentes la terapeuta, que
ayudaba a los pacientes mientras estos probaban las actividades, y
dos evaluadores, que eran los encargados de tomar nota de los
comentarios de los participantes. Ambas sesiones duraron dos
horas cada una y se llevaron a cabo en la sala donde los pacientes
realizaban normalmente sus actividades. Fue en esa misma aula
donde se instaló un tabletop NIKVision para que los pacientes
probaran las actividades. Figura 9: Cuestionario AttrakDiff adaptado que rellenaron
los pacientes
Durante esta sesión los participantes probaron por turnos algunas
de las actividades de memoria, atención y razonamiento
desarrolladas en el tabletop. Uno de los pacientes se negó a jugar
con el tabletop ya que comentó que no le gustaba el dispositivo.
Los demás completaron las actividades sin problemas aunque los
tiempos de completitud de las actividades variaron dependiendo
de la enfermedad mental del paciente, en un rango de quince
minutos y media hora.
Finalmente, los usuarios rellenaron su cuestionario AttrackDiff.
Uno de los pacientes necesitó para ello la ayuda de la terapeuta
porque no acababa de entender lo que había que hacer.

5.3 Sesión 2: Comparativa de la actividad con

tabletop y sin tabletop
Durante esta sesión, todos los participantes trabajaban en papel
con la versión no tecnológica de la actividad (ver fig. 8) y por
turnos también jugaban con la versión de tabletop.
El mismo paciente que se había negado a jugar con el tabletop
durante la sesión anterior tampoco quiso en esta sesión probar la
versión de tabletop de la actividad, pero el resto de pacientes
completaron el juego sin problemas, aunque necesitaron diferente
tiempo: los pacientes más afectados cognitivamente requirieron
Figura 8: Versión no tecnológica de la actividad “Relaciones media hora mientras que los menos afectados completaron el
sintagmáticas” juego en diez minutos.

5.2 Sesión 1: Uso de actividades por los Al finalizar todos los usuarios sus actividades de “Relaciones
Sintagmáticas”, se realizó una discusión en común de la opinión y
pacientes percepción que tienen sobre cada una de las versiones. Todos
Previamente a la sesión, la terapeuta decidió qué actividades iba a (menos el que no jugó), participaron en dicha discusión, durante
realizar cada uno de los pacientes, así como el grado de dificultad. 15 minutos.
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)
| 37 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri

5.4 Resultados Respecto a las notas que se tomaron, pudimos extraer algunas
En cuanto a los resultados de la primera sesión, en el cuestionario conclusiones para mejorar las actividades, como se comenta a
AttrakDiff cada adjetivo se engloba dentro de una de las cuatro continuación:
dimensiones siguientes: Casi todos los participantes encontraron las últimas tareas de la
• Cualidades pragmáticas (PQ): mide el producto como actividad de razonamiento “Completa la secuencia” demasiado
herramienta utilizada para realizar una tarea. difíciles, lo cual indica que es necesario bajar la dificultad de
dicha actividad.
• Cualidades Hedónicas-Estimulantes (HQ-S): mide en
También nos dimos cuenta de que en la actividad “Lista de la
qué medida el producto apoya la estimulación.
compra” el dibujo de la bolsa que aparecía en el área destinada a
• Cualidades Hedónicas-Identificación (HQ-I): mide en poner los alimentos podría no ser el más adecuado, ya que uno de
qué medida el producto permite al usuario identificarse los pacientes interpretó que al ser una bolsa, los alimentos tenían
con él. que meterse por la parte de arriba y por lo tanto concentraba todos
los alimentos únicamente en la parte del arriba del área en vez de
• Escala de atractivo global (ATT): mide el valor global en toda la zona de la bolsa. En esa misma actividad también
del producto basado en la percepción de sus cualidades descubrimos que el tiempo que se muestra la lista con los
hedónicas y pragmáticas. alimentos al principio de la actividad era demasiado corto para
A pesar de haber hecho una simplificación del cuestionario, uno de los pacientes.
analizando las respuestas dadas por los pacientes se han Por último, observamos que la posición de la retroalimentación
observado contradicciones a la hora de responder, puesto que un visual en determinadas actividades tenía que mejorarse, ya que en
paciente calificó el adjetivo “Integrador” con un 0 pero después actividades como la de “Viajes” o “Tangram” la retroalimentación
calificó “Me acerca a la gente” con un 5, otro paciente calificó (carita que se pone triste o contenta) está situada demasiado
“Ingenioso” con un 4 y “Soso” con un 6, y otro paciente calificó alejada de las áreas interactivas, lo que provoca que muchas veces
“Engorroso” con un 6 y “Claro” con otro 6. Por lo tanto, se puede el paciente no la perciba. En otras actividades, como “Busca los
llegar a la conclusión de que para este tipo de pacientes sería símbolos”, la retroalimentación sí que era adecuada ya que ésta
recomendable cambiar algunos de los adjetivos por otros más aparecía directamente sobre el área en la que jugaba el paciente y,
sencillos para que les resulten más fáciles de comprender y los por tanto, éste siempre lo veía.
resultados sean más fiables. A la hora de calcular los resultados,
los adjetivos contradictorios previamente mencionados no se En cuanto a los resultados de la segunda sesión, el grupo discutió
tuvieron en cuenta en ningún caso. durante 15 minutos con los evaluadores su valoración del juego de
tabletop. La discusión empezó preguntándoles qué actividad les
En la figura 10 se pueden ver las medias de los resultados de los gustaba más: la no tecnológica o la versión del tabletop. La
tres usuarios que participaron en la sesión agrupados en las cuatro actividad de tabletop fue, en general, bien recibida por los
dimensiones. Aunque todas las notas superan el 50%, se aprecia pacientes que jugaron en el dispositivo tabletop, y todos ellos
que la nota más baja es la pragmática (PQ), indicando que los eligieron la versión de tabletop de la actividad en vez de la de
pacientes no consideran que la herramienta destaque por su papel. Sin embargo, también expresaron algunas matizaciones
utilidad en la realización de actividades. Sin embargo, las notas de personales. Uno de los participantes apuntó que, aunque era más
las dos dimensiones hedónicas (HQ-S y HQ-I) son mayores, divertido hacer la actividad en el tabletop, prefería leer las
indicando que los usuarios destacan más del tabletop su capacidad instrucciones en papel en vez de en el tabletop ya que mirar a la
motivadora y atrayente. Por último, se puede ver que la dimensión pantalla de tabletop por mucho tiempo le cansaba la vista. Otro
con mejores resultados es la última (ATT), con un valor de casi el paciente dijo inicialmente que prefería hacer la actividad en papel
75%, lo cual indica que el valor global del producto es bastante porque cuando jugaba con la versión del tabletop, a veces le daba
satisfactorio. Por tanto, se puede extraer como conclusión que los retroalimentación errónea diciendo que la pieza de juego estaba
usuarios están satisfechos con el uso de las actividades sobre el mal puesta e inmediatamente después decía que estaba bien (este
tabletop, ya que aunque no consideran que éstas les aporten suceso era debido a un problema técnico que fue posteriormente
mucho más que el hacer sus actividades tradicionales, sí que se solucionado). Cuando se le preguntó al paciente qué elegiría en
sienten atraídos y motivados a la hora de realizarlas. caso de que el problema estuviera solucionado, dijo que elegiría la
actividad de tabletop en vez de la de papel porque era más
divertida y porque era más fácil saber si habías elegido la figura
correcta gracias al sonido, mientras que en la de papel tenía que
preguntar a la terapeuta además de borrar y colorear de nuevo
cuando fallaba. Finalmente, otro paciente nos dijo que prefería la
actividad de tabletop pero que no le gustaban los sonidos
incorrectos: él prefería que un sonido sólo se reprodujera cuando
hacía algo bien y que nada sonara cuando hacía algo mal. La
terapeuta nos dijo después que algunos pacientes se motivan con
retroalimentación positiva pero que se frustran fácilmente con la
retroalimentación negativa; por lo tanto, es un aspecto importante
a considerar en el futuro a la hora de diseñar actividades de
tabletop personalizadas según las preferencias de los pacientes.

Figura 10: Resultados del cuestionario AttrakDiff adaptado

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 38 |
 Diseño de actividades de mejora de capacidades cognitivas para tabletops tangibles
C. Bonillo | J. Marco | E. Cerezo | S. Baldassarri

6. CONCLUSIONES Y TRABAJO FUTURO [8] Marco, J., Cerezo, E., Baldassarri, S., Mazzonne, E., Read, J.
Tras analizar las áreas cognitivas más tratadas por los terapeutas 2009. Bringing Tabletop Technologies to Kindergarten
en pacientes mayores o con enfermedad mental se han Children. 23rd BCS Conference on Human computer
desarrollado las herramientas necesarias para el diseño y Interaction. Cambridge University.
ejecución de actividades de ese tipo en tabletops Tangibles [9] Muñoz-Marrón, E., Blázquez-Alisente, J.L., Galparsoro-
basados en visión como NIKVision: un lenguaje de marcas Izagirre, N., González-Rodríguez, B., Lubrini G, Periáñez-
flexible basado en XML y un intérprete de dicho lenguaje. Morales J.A., et al. 2009. Estimulación cognitiva y
Ambas han sido utilizadas en el desarrollo de nueve actividades rehabilitación neuropsicológica. Barcelona: UOC.
que han sido probadas por una terapeuta y su grupo de pacientes
[10] Rocha, K.B., Pérez, K., Rodríguez-Sanz, M., Borrell, C., y
adultos con enfermedad mental. La evaluación ha servido para
Obiols, E.J. 2010. Prevalencia de problemas de salud mental
reafirmar las potencialidades de este tipo de actividades
y su asociación con variable socieconómicas, de trabajo y
desarrolladas sobre un tabletop Tangible y también para encontrar
salud: Resultados de la Encuesta Nacional de Salud de
algunos problemas de usabilidad y de la propia metodología de España. Psicothema, 22, pp. 389-395.
evaluación con ese tipo de pacientes. Así, se ha detectado la
importancia de diseñar una correcta y adaptada retroalimentación [11] Sociable http://www.cognitivetraining.eu/?q=services-0
de las actividades, así como la necesidad de una mayor adaptación
de los cuestionarios de evaluación con objeto de obtener
resultados más fiables de los mismos.
Como trabajo futuro, se está trabajando en el desarrollo de una
interfaz gráfica que permita el desarrollo de actividades por parte
de los propios terapeutas.

7. AGRADECIMIENTOS
Agradecemos a la asociación ASAPME Huesca su participación
en las sesiones de evaluación. Este trabajo ha sido parcialmente
financiado por el Gobierno de España a través del contrato
DGICYT TIN2011-24660.

8. REFERENCIAS
[1] Drake, M.A. 2007. Evaluación de la atención. In Burin, D.I.,
Drake, M.A., and Harris, P. Evaluación neuropsicológica en
adultos. Buenos Aires, Barcelona, México: Paidós.
[2] Gamberini, L., Alcaniz, M., Barresi, G., Fabregat, M.,
Ibanez, F., and Prontu, L. 2006. Cognition, technology and
games for the elderly: An introduction to ELDERGAMES
Project. PsychNology Journal, vol. 4, pp. 285-308.
[3] Hassenzahl, M., Burmester, M., Koller, F. 2003. AttrakDiff:
A Questionnaire for Measuring the Perceived Hedonic and
Pragmatic Quality. Human & Computer 2003. Pp. 187-196.
[4] Kern, D., Stringer, M., Fitzpatrick, G., and Schmidt, A. 2006.
Curball - A Prototype Tangible Game for Intergenerational
Play. IEEE Press.
[5] Kwon, G.H., Kim, L., and Park, S. 2013. Development of a
cognitive assessment tool and training systems for elderly
cognitive impairment: A case study of a successful
development of a cognitive assessment tool and training
systems for the elderly people in South Korea. 7th
International Conference on Pervasive Computing
Technologies for Healthcare and Workshops.
[6] Leitner, M., Tomitsch, M., oltringer, T. K., Kappel, K., and
Greshenig, T. 2007. Designing tangible tabletop interfaces
for patients in rehabilitation. In Proceedings of Conference &
Workshop on Assistive Technologies for People with Vision
and Hearing Impairments: Assistive Technology for All
Ages. M. Hersh (Eds.), Spain. pp. 1-7.
[7] Loureiro, B., and Rodrigues, R. (2011) Multi-touch as a
Natural User Interface for elders: A survey. Information
Systems and Technologies, 6th Iberian Conference. IEEE.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 39 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz

Analyzing Learnability of Common Mobile Gestures used

by Down Syndrome Users
Alfredo Mendoza G. Francisco J. Alvarez R. Ricardo Mendoza G.
Universidad Juarez Universidad Autónoma Instituto Tecnológico
Autónoma de Tabasco de Aguascalientes de Aguascalientes
Av. Universidad S/N, Av. Universidad 940, Cd. Av. A. López Mateos Ote. No. 1801,
Magisterial, Villahermosa, Tab. Universitaria, Aguascalientes, Ags. Bona Gens, Aguascalientes, Ags
(52) 993 312 4680 (52) 449 910 7400 (52) 449 910 5002
[email protected] [email protected] [email protected].

Francisco Acosta E. Jaime Muñoz A.

Universidad Juarez Universidad Autónoma
Autónoma de Tabasco de Aguascalientes
Av. Universidad S/N, Av. Universidad 940, Cd.
Magisterial, Villahermosa, Tab. Universitaria, Aguascalientes, Ags.
(52) 993 312 4680 (52) 449 910 7400
[email protected] [email protected]

ABSTRACT considered in the design of any user interface in order to be

The objective of this research was to analyze the learnability of friendly-used, since these limitations may affect the use of any
the 8 most common mobile gestures used by first-timers Down computational application when it has not been well designed.
Syndrome (DS) users. The study was performed by testing mobile The quality and mount of knowledge, which designers had
gestures usage by 10 DS teenagers that had never interacted with considered to the final product about users, will benefit the user
a mobile/gestural interface. They developed 18 tasks over a tablet experience. Understanding people, their characteristics,
computer. Results were measured by the Task Performance capabilities, commonalities and differences allow designers to
Learnability Metric which focuses in factors such as success, create more effective, safer, efficient, and enjoyable systems
optimality, and error decreasing. Although gestural interaction (Ritter et al., 2014).
was thought to be intuitive by design, DS users face limitations in This study focuses in the learnability of the most common mobile
fine-motor and visual skills, so as in eye-hand coordination, which gestures used by DS users. Learnability is one of the five
difficult manipulation of particular mobile gestures. This vein, components of Usability (Learnability, Efficiency, Memorability,
findings of this research helped to determine the most adequate Errors and Satisfaction) and is defined as how easy for users is to
interactions for mobile gestural interfaces used by DS users. complete basic tasks the first time they encounter with some
design (Nielsen, 2012). This means that the main goal of this
Categories and Subject Descriptors study is to analyze how easy to learn are the most common mobile
H.5.2 [Information Interfaces and Representation]: User gestures by the DS users in the very first encounter with mobile
Interfaces - Graphical user interfaces, Input devices and devices.
strategies
2. BACKGROUND
General Terms There are significant characteristics of DS individuals that may
Documentation, Performance, Design, Human Factors affect their performance as computers users. Based on the
complete list of characteristics found in (Champman & Hesketh,
Keywords 2000, Lauteslager, 2000), the most related with mobile gestures
UX, Learnability, Usability, Interaction, Disabilities are shown next:
1. INTRODUCTION • Short hands and broad fingers.
Down Syndrome (DS) is caused by a genetic anomaly where • Difficulties in fine and gross motor skills.
subjects have an extra copy of chromosome 21; such genetic
condition brings particular characteristics in learning, • Low muscle tone
communicating and perceiving skills. Worldwide, the estimated • Difficulties in vision capacities
incidence of DS is between 1 in 1,000 to 1 in 1,100 live births. • Poor eye- hand coordination
About 6,000 babies are born in the United States with DS each • Anxiety
year (Parker et al, 2010). The prevalence of Down syndrome is • Difficulties in comprehension of abstract concepts
about 8 people with DS per each 10,000 people in the United
States (Presson, 2013), 7 per 10,000 in England and Wales (Wu & Drs. Jinjuan Feng and Jonathan Lazar of Towson University had
Morris, 2013), and 8 per 10,000 in Spain (FESD, 2010), to quote leaded some important researches on Human-Computer
some data. The particular characteristics of DS people might be Interaction that involved DS users. In (Lazar, 2010), they applied

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 40 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz
a survey about general usage of Computers to 561 DS children.
Among their most important findings, they show that 93% of the
children use mouse without much difficulty, they also remark that
they applied the survey only to children that already use
computers, but it is a remarkable finding because their Figure 1. Tap gesture
expectations based on the clinical information was overcome. Dr.
Lazar also analyzed the work-place related skills of adult expert 3.2 Double Tap
users with DS; he found that many of the participants were able to The double tap (figure 2), also known as double touch, consists in
efficiently use different programs, systems and devices at work rapidly touching twice the surface of the device with one fingertip
(Lazar et al, 2011). These papers leave an open question about in the same point (Apple, 2014; Android, 2015). The most
how easy to use were input techniques for DS users. In (Hu, et al, common functionality of this gesture is to zoom in (or out when is
2011), through an empirical study that involves eight users with already zoomed) a picture.
DS, the efficacy of keyboard-mouse, word prediction, and speech
recognition, where examined. All of them already were familiar
with the keyboard-mouse input technique, and seven of the eight
participants completed all the tasks, although none of them where
used neither word prediction nor speech recognition.
Figure 2. Double tap gesture
Within their findings, they confirm the fact that Dr. Lazar and
their team visualized previously: One design is unlikely to fit the 3.3 Swipe
needs of all DS users, instead, offering different design options Swiping is the action of pressing, moving and lifting at some point
targeting for specific user profiles might be a more effective of the screen. One particular characteristic of this gesture, in order
solution (Hu, et al, 2011). They mention two factors that influence to avoid misunderstandings with others, is that the fingertip that
the user’s performance level: the computer experience (number of triggers the gesture will never lose contact with the surface until
years using computers) and number of hours per week of the gesture is done. Figure 3 presents the graphic representation of
computer usage. swipe gesture.
In 2012, the research team of Towson University made an
empirical study about how usable were the tablets computers used
by expert adults with DS (Kumin et al, 2012). They found that
adults with DS are able to use multi-touch devices when already
had received formal computer training. Involving 10 participants, Figure 3. Swipe gesture
Social networking, Email, Calendar, Price comparison, and Text
entry/note-taking activities made in a tablet computer were tested. 3.4 Flick
It is important to remark that all users were experts and use Flick is defined as quickly brush the surface of the device with
computers in their everyday work activities. one fingertip (Villamor et al., 2010). The gesture finishes with the
One interesting finding in this study was that some users had fingertip away from the surface, that and the speed of doing it, are
problems with the sensitive nature of the device, mixing and the main differences between flick and swipe. Figure 4 is the most
confusing gestures, this gives the idea that mobile gestures are not common graphical representation of flick gesture. The most
as self-evident as it was thought. known action of this gesture is to advance (forwards or
backwards) rapidly in a list of elements.
One of the main advantages of the gestural interfaces used by
mobile devices over traditional keyboard-mouse interfaces is the
direct relationship with ordinary-living gestures; this implies that
users must not necessarily pass through a capacitation program for
learning the manipulation of such devices; being natural, last Figure 4. Flick gesture
generation of mobile devices interfaces, allows an intuitive
learning. Thus, the question discussed in this study is how natural 3.5 Hold and drag
are the mobile interfaces gestures for DS novice users? This gesture is composed of two elements: long touch and swipe.
By holding the touch with one fingertip over a specific element of
3. COMMON MOBILE GESTURES the screen, the user can drag it to another position. Commonly, by
This section describes the 8 most commonly used mobile gestures. holding and dragging an object of the screen, user may change its
It is important to mention that, with the exception of figure 5, all position, send it to trash or close it. In order to represent the hold
gestures representations are based on (Villamor et al., 2010). and drag gesture, authors has develop the figure 5, based on those
shown in (Villamor et al., 2010).
3.1 Tap
Tap gesture is the most basic one; it consists in pressing briefly
the surface of the device with one fingertip. The result could be
just the selection of an item of the screen or the trigger of a
predefined action. (Apple, 2014; Android, 2015), the typical
symbol to express this gesture is shown in figure 1. Figure 5. Hold and Drag gesture

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 41 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz

3.6 Pinch
The pinch gesture consists in pressing the surface of the device Table1. Specs of device
with two fingers, move them inwards and lifting (Google, 2015).
The graphical representation shown in figure 6, presents the pinch Characteristic Samsun Galaxy
gesture made with thumb and index fingers. Display size 10.1’’
Screen resolution 1280x800
Internal memory 16GB
Processor 1.2 GHz Qd-Core
Figure 6. Pinch gesture Platform Android™ 4.4

3.7 Spread 4.3 Metrics

Spread gesture implies the inverse movement of pinch gesture; i. In (Grossman et al., 2009) a taxonomy of learnability evaluation
e. the gesture begins with pressing the device’s surface with the metrics was presented; there, six different categories group the
two fingers close-together, spreading and lifting. Figure 7 presents metrics according with their objective. For this study, the next
a graphical representation. points, taken form task performance metric, was chosen:
• Percentage of users who completed the task optimally
• Percentage of users who completed the task without any help
• Ability to complete task optimally after certain time frame
• Decrease in task errors made over certain time interval
Figure 7. Spread gesture • Time until user completes a certain task successfully

3.8 Rotate In order to clarify the way the metrics will be measured, an
The rotate gesture consists in touching the device’s surface with explanation for the key terms is presented next:
two fingers and move them in a clockwise or counterclockwise • A task is considered as optimally completed when the user
direction (Villamor, et al., 2010). A graphical representation is completes all the instructions at the first try, without any help.
shown in figure 8. • A task is considered as completed without help when the user
completes the task successfully, regardless the number of tries.
• A task is considered as completed successfully when the user
triggers the current gesture action without errors.
• An error occurs when the actions of the users do not trigger the
Figure 8. Rotate gesture
current gesture’s actions. A hesitation is not considered as
4. EMPIRICAL STUDY error.
• All the time measured starts when the first explanation is
4.1 Participants
completed, and stops when the activity is successfully
The study involves 10 DS teenagers between 15 and 20 years old;
all of them enrolled in a special education center where no completed.
computational instruction is given to them. All 10 participants
Thus, data were computed as next:
had no experience with mobile devices, touchscreens or
smartphones. It is worth mentioning that all parents of the 10 • Percentage of users who completed the task optimally
users involved in this research gave their consent by signing the • Percentage of users who completed the task without any help
institutional consent document based on the Informed Parental
• Percentage of users that acquired the ability to complete task
Consent for Research Involving Children of the World Health
Organization. optimally after certain time frame
• Percentage of users that decrease in task errors made over
4.2 Instruments certain time interval
The device selected for this study was the Samsung Galaxy Tab • Average time until user completes a certain task successfully
10.1; specifications are shown in table1.
4.4 Experiment design
28 users’ tasks were developed in order to test all 8 gestures. The
main premise was to define tasks where the cognitive load would
not affect the user performance of any gesture. From this fact, the
next rules where followed by each user task:
• Use the minimum number of colors as possible
• Use simple images without too many realism
• Avoid abstract images
• Avoid ambiguity

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 42 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz
• Avoid words inside the images. Intermediate: Use flick and stop it.
• Use white background when possible. Advanced
• Use the most typical forms of all objects. (a) Use flick forwards and stop it to reach certain page.
(b) Use flick backwards and stop it to reach certain page.
Figure 9 shows an example of the application of the rules, both
images represent a house; left side picture is overloaded with 6. Rotate:
graphical information (trees, ground, fence, bench, etc.), right Basic: Rotate image
picture is a simple representation of a house with minimum Intermediate: Rotate and resize an image
colors. Advanced: Rotate 180° and zoom in an image.
7. Pinch:
Basic: Zoom out a picture
Intermediate: Users must reach the minimum zoom of a picture
as fast as they can.
Advanced: Users must zoom-out a picture and rotate it 180° in
clockwise at the same time.
8. Spread: All tasks will be developed using only one hand.
a) b)
Basic: Zoon in a picture
Figure 9. a) Overloaded Picture b) Simple picture Intermediate: Users must reach the maximum zoom of a
Every gesture was tested in three levels: basic, intermediate, and picture as fast as they can.
advanced; basic level implies the simplest tasks where the user Advanced: Users must zoom-in a picture and rotate it 180° in
only makes the gesture. Levels intermediate and advanced involve clockwise at the same time.
more complicated tasks that involve repetitions, combinations, or
Next table shows an overview of all the 28 tasks:
specified conditions of gestures. Next are shown the tasks defined
for each gesture: Table2. Overview of user tasks
1. Tap: In a grill given to user, he/she must tap the indicated Gesture Level Task
cell. The size of each cell is decreasing with each success. Basic Tap the indicated cell, cell
The three levels are tested with the cell size.
sizes:7’’and 4’’
2. Double tap: In a grill given to user, he/she must tap the Intermediate Tap the indicated cell, cell size:
indicated cell. The size of each cell is decreasing with each Tap
2’’and 1’’
success. The levels of difficulty are evaluated same way as
Advanced Tap the indicated cell, cell size: .5’’
the first task.
and .25’’
3. Hold and drag: For all levels of difficulty, a group of left- Basic Double-tap the indicated cell, cell
aligned elements is given to users; they must drag all of them size: 7’’ and 4’’
to their final position, the difference between levels is the way
the user must follow with the object. Double Intermediate Double-tap the indicated cell, cell
Basic: Objects must be moved in a straight line. Tap size: 2’’and 1’’
Intermediate level: Objects must be moved in an “L” line. Advanced Double-tap the indicated cell, cell
Advanced: size: .5’’and .25’’
a) Objects must be moved in a straight line, in certain
Basic Move objects in a straight line.
time. Hold and
Intermediate Move objects in an “L” line.
b) Objects must be moved in an “L” line, in certain time. drag
Advanced Move objects in an “S” line.
4. Swipe: All the next tasks will be done by side to side swipe. Basic Go to the next picture
Basic: Go to the previous one
Swipe
(a) Being in certain picture, user goes to the next one. Intermediate Use swipe continuously.
(b) Being in certain picture, user goes to the previous one. Advanced Use swipe with speed.
Intermediate: Users must go forward in a document, sheet by
Basic Flick forward
sheet until reach some pre-established page, then they must Flick backwards
return to the first one. Flick Intermediate Use flick and stop it.
Advanced: Users must reach the final page of a document as Advanced Use flick and stop it to reach
fast as they can, then they must return to the first one as the certain page.
same way. Basic Rotate image
Rotate Intermediate Rotate and resize an image
5. Flick Advanced Rotate 180° and zoom in an image
Basic: Use flick forward and backwards. Pinch Basic Zoom out a picture

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 43 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz
Intermediate Reach the minimum zoom 4.5.2 Double tap
Advanced Zoom-out a and rotate 180° Double tapping involves the same windows and cells of single
tap, but users have to touch twice the surface in order to advance
Basic Zoom in a picture in the activity.
Spread Intermediate Reach the maximum zoom
Common mistakes involved accidental activation of other gestures
Advanced Zoom-in a and rotate 180° (such as swipe), multiple tapping. For some users were difficult to
make only two taps in the tiny cells.
4.5 Study development The same consideration for time measurement made with single
Although, all activities were planed and designed with a multi- tapping, were done for double tapping: only advance level were
disciplinary team involving psychologists, clinicians and experts measured. Corresponding results are shown in table 4.
in special education, all tests were made with the presence of the
psychologist in order to supervise that users understand the Table 4. Results for double tap
instructions and the activities procedure. Level Basic Intermediate Advanced
All 10 individual tests were videotaped in order to be analyzed Factor
and data were tabulated. All results obtained in the study are
Percentage of users who 90% 60% 30%
presented arranged by gesture. A table presents the corresponding
completed the task
value involving all 10 users’ information using the metric
optimally
presented in 4.3. Some data were note as N. M. (Not Measured)
Percentage of users who 100% 60% 30%
due to the low significance of measuring that factor.
completed the task
4.5.1 Tap without any help
The goal of all activities for tap gesture was tapping the indicated Percentage of users that 100% 90% 80%
(in green color) cell of a table. The size of all cells reduces at each acquired the ability to
success, whilst the number of cells increases; thus, the difficulty complete task
also increases. successfully after certain
time frame
For tap gesture, since this is a very easy task, the time was not Percentage of users that 100% 100% 90%
measured in basic and intermediate level, due to users tap the decrease in task errors
corresponding cell almost instantly. In advance level, the time made over certain time
was taken until user complete the tasks, and involves time for interval
locating the cell and time expended in all tries. The results for tap Average time until user N. M. N. M. 2.5
are shown in table 3. completes a certain task
As it is shown in the table, some users fail to complete tasks successfully (secs.)
where more precision were required; errors involved accidental
activation of swipe and double tap, keep holding and too much
force applied to make the tap (resulting in not detection of the
4.5.3 Swipe
gesture by the device). A document with several pages, each of them presenting only one
single basic image of fruits, vegetables, animals and other simple
Table 3. Results for single tap objects, was shown to the users; after a brief explanation of the
Level Basic Intermediate Advanced functionality to advance forward and backwards through the pages
of the document, user was ask to advance to the next, previous
Factor and certain image. Next, user was asked to go to certain
Percentage of users who 100% 60% 30% previously located image, so he/she must do the swipe gesture ten
completed the task times. Once user was familiar with this gesture, for advanced
optimally level, he/she was asked to go to certain object (10 pages away to
Percentage of users who 100% 80% 30% the current) as fast as he/she can. Single swiping did not represent
completed the task a challenge to any user, but when multiple swiping was required,
without any help users tend to make some mistakes. Among the most common
Percentage of users that 100% 100% 80% were: undetected swipe, due to incomplete or short gestures,
acquired the ability to diagonal swipe, that sometimes results in undetected gestures, and
complete task opposite swipe, due to no-lifting finger from the device’s surface
successfully after certain when swiping. All errors were most presented in advance level
time frame activities (See Table 5).
Percentage of users that 100% 100% 100% 4.5.4 Flick
decrease in task errors For flick gesture, the same document with simple images was
made over certain time used, after a brief explanation of its functionality users were asked
interval to do the activities. First, users made the flick to advance forwards
Average time until user N. M. N. M. 1.5 and backwards, in a preview-slide of the all document pages.
completes a certain task Then he/she was asked to stop the slide before stops itself. Once
successfully (secs.) user understands the gesture and its actions, he/she was asked to

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 44 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz
stop the slide in a certain page. For this activity, the objective
page was the only one in colors, the rest of all pages were in gray
scale. The most common behavior of users was trying to Table 6. Results for flick
accelerate the slide by doing multiple slides. The result of this Level Basic Intermediate Advanced
action was commonly to stop the slide. Factor
Percentage of users who 20% 10% 10%
Table5. Results for swipe completed the task
optimally.
Level Basic Intermediate Advanced Percentage of users who 40% 60% 10%
Factor completed the task
without any help
Percentage of users who 90% 90% 70%
Percentage of users that 80% 80% 20%
completed the task
acquired the ability to
optimally
complete task
Percentage of users who 100% 90% 70%
successfully after certain
completed the task
time frame
without any help
Percentage of users that 80% 80% 20%
Percentage of users that 100% 100% 100%
decrease in task errors
acquired the ability to
made over certain time
complete task
interval
successfully after certain
Average time until user 4.6 8.6 13
time frame
completes a certain task
Percentage of users that 100% 100% 100%
successfully (secs.)
decrease in task errors
made over certain time
interval 4.5.5 Hold and drag
Average time until user N. M. 12.25 7.4 This activity involves two steps: tap over one object of the screen
completes a certain task and hold it, and then, move the object to another place. It was
successfully (secs.) especially difficult to user to understand the moment when he/she
can move the object; that is why none of them complete the
The advanced level activity was very hard to achieve by all users. activity optimally. Once the user hold the tap over one object, it
At first, researchers thought it was caused of vision problems, but takes less than a second to presented a colored-transparent square
after a brief analysis of the visual capacities of users it was that covers the object, then it is enabled to be moved. At first,
discarded. Researchers and experts support that the difficulty was users were asked to move certain object in a straight line and
due to a coordination problem that in general affects DS place in a previously located square. In general, all three levels of
individuals. difficulty involve moving an object to a predefined place by
following the indicated trajectory. The most common mistakes
Table 6 shows the information tabulated that involve all 10 users’ made in hold and drag gesture were trying to move the object
tests for flick gesture. without hold, and stop holding in the way to the final position.
Sometimes, users lift the finger in the middle of the trajectory;
they were asked to do it again but without releasing the object.
Only those users that did not make this mistake were tabulated as
optimally completed. All results are shown in table 7.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 45 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz

Table 7. Results for hold and drag Table 8. Results for Rotate
Level Basic Intermediate Advanced Level Basic Intermediate Advanced
Factor Factor
Percentage of users who 0% 0% 0% Percentage of users who 80% 60% 40%
completed the task completed the task
optimally optimally
Percentage of users who 0% 0% 0% Percentage of users who 80% 80% 90%
completed the task completed the task
without any help without any help
Percentage of users that 50% 50% 40% Percentage of users that 100% 100% 100%
acquired the ability to acquired the ability to
complete task complete task
successfully after certain successfully after certain
time frame time frame
Percentage of users that 50% 50% 40% Percentage of users that 100% 100% 100%
decrease in task errors decrease in task errors
made over certain time made over certain time
interval interval
Average time until user 10.5 33.6 15.6 Average time until user N. M. 4.5 5.6
completes a certain task completes a certain task
successfully (secs.) successfully (secs.)

4.5.6 Rotate 4.5.7 Pinch

In general, users can do pinch, spread, and rotate with two fingers After a brief explanation of making the pinch and the spread
of one hand or with one finger of each hand. Both ways of doing gestures over a picture, user was asked to do it. Once he/she
these gestures was shown to users, when users seem to have understand both gestures in general, particular activities to each
troubles with one way; then were asked to do it in the other way. were asked to be done. Users were asked to zoom-out a picture as
Optimally data was record when users made it with one of the two it can be done in order to test the continuous pinching over one
options, even if fail with the first one. Users were asked to rotate a point, and then were asked to increase the speed: next picture,
picture with no restrictions, and then were asked to rotate 90° and more speed. Once the gesture was understood, it was easy for
180° a picture. Pictures involve single images of simple objects, users to make multiple gestures slowly; but when speed factor
animals and fruits that were easy understandable not only their entered, mistakes begin to happen. Common mistakes when users
contents but their position, this was verified by experts. The tried to do multiple pinch rapidly were: undetected gesture,
application used for these activities restricts the exact position of undecireble swipe, spread, and rotate gestures. Results are shown
the picture, no help is on positioning. Sometimes users achieve the in table 9.
goal of the activities but taking various steps, first rotating the Table 9. Results for Pinch
picture 90°, stopping, replacing hands and rotating another 90°;
these users were not taken into account as optimally completed. Level Basic Intermediate Advanced
All results for this gesture are shown in next table: Factor
Percentage of users who 60% 20% 0%
completed the task
optimally
Percentage of users who 70% 60% 40%
completed the task
without any help
Percentage of users that 90% 80% 80%
acquired the ability to
complete task
successfully after certain
time frame
Percentage of users that 90% 80% 80%
decrease in task errors
made over certain time
interval
Average time until user N. M. 15.5 5.4
completes a certain task
successfully (secs.)

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 46 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz

4.5.8 Spread 5. Conclusions

Pinch and spread gestures cannot be explain to users in separated People with DS commonly have problems with fine motor skills
activities, since they involves related movements of fingers and (coordination, manipulation), skills required to interact with any
produce very similar results in the test application. Nevertheless, mobile/gestural interface. It was found that there are some
activities for each gesture were applied separately. Activities gestures that are simple and very understandable by DS users, but
involved zooming-in basic pictures, reach the maximum size of it others are still hard to discover, learn and apply. Interfaces where
by making continuous spread gestures, and do it with speed. this fact is not considered on their design will diminish the user’s
Results shown in table 10 denote that it is a little more difficult to performance.
DS users to make, rapidly and continuously, the spread gesture
than the pinch gesture, probably because pinch movement is more Tap gesture, for example, is the most basic and simple gesture; for
commonly used in real life than it is spread. Users have the same DS users it was evident how to make a tap over an interface
errors at advanced level as they have in pinch: undetected gestures object with great performance and almost no trouble; but due to
and mistake gestures. the common visual problems in DS users, plus the characteristics
of hands and fingers (see part 2) it turns to be difficult for them to
Table 10. Results for Spread make tap gesture over the tiny ones. These factors affect the same
Level Basic Intermediate Advanced way the development of double-tap gesture.
Factor It was also evident the usage of swipe gesture when used in a
normal way. When DS users were asked to swipe continuously
Percentage of users who 70% 20% 0%
and rapidly, they tend to make mistakes.
completed the task
optimally It turned out that flick is one of the most difficult gestures to
Percentage of users who 70% 60% 20% make, especially when users were asked to stop the slide. It
completed the task requires great eye- hand coordination that becomes a real
without any help challenge to them.
Percentage of users that 90% 80% 40%
There were some troubles with Hold and drag gesture, but only
acquired the ability to
when the required time for hold was too long. The default time of
complete task
new systems such as Android 5.0.1 works just fine.
successfully after certain
time frame The last three gestures, rotate, pinch and spread, did not present
Percentage of users that 90% 80% 40% greatest troubles in the basic actions. Errors appeared when users
decrease in task errors start to mix them and repeat them continuously. Along with flick
made over certain time gesture, they required a more extended explanation than the first
interval four, to be completely understood by users.
Average time until user N. M. 17.6 6
completes a certain task Based on these findings, the next recommendations are proposed:
successfully (secs.) • For very-first-timers users, require only simple gestures that
involve one simple movement, trigger only one evident and
By calculating the joint probability between the three levels of distinguishable action, and that go according to real life, not
difficulty of all the first four factors, the probability that one only for their movement but also for their action that trigger.
individual succeed in at least one factor (independently of the
• Add a flexible error handling rule for gestures considering that
level) is shown in Figure 10. With this graphic, it is easy to
identify which gestures are difficult and which are not. users may accidentally make undesirable gestures.
• Avoid, as much as possible, gestures that involves great eye-
hand coordination such as flick, especially for novice users.
• Avoid little tiny selectable objects even for expert users.

These recommendations do not tend to asseverate that DS users

could not be able to use some robust mobile interfaces; with the
proper capacitation plan, users can understand, use, and even
master almost any activity, in fact, throughout the study, all users
increase their performance as they use the mobile devices.

6. ACKNOWLEDGMENTS
Authors want to thank all students, parents, instructors and
clinicians that collaborate with this research. Specially thanks to
Dr. Cristian Russu and Rodolfo Inostroza from the Pontificia
Universidad Católica de Valparaiso, to share their experience and
hard-work.

Figure 10. Probability of success on each gesture

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 47 |
 Analyzing Learnability of Common Mobile Gestures used by Down Syndrome Users
A. Mendoza | F., J. Alvarez | R. Mendoza | F. Acosta | J. Muñoz

7. REFERENCES [8] Lauteslager, P., 2000, Children with Down’s Syndrome:

[1] Apple Inc., Apple Developer Site, Motor Development and Intervention, Thesis University
https://developer.apple.com/, last visited February 2015. Utrecht, The Netherlands.
[2] Chapman, R., and Hesketh, L., Behavioral phenotype of [9] Lazar, J., Kumin, L., Feng, J., 2011, Understanding the
individuals with Down syndrome, Journal: Mental computer skills of adult expert users with Down syndrome:
Retardation and Developmental Disabilities Research an exploratory study, The proceedings of the 13th
Reviews - MENTAL RETARD DEV DISABIL RES , vol. 6, international ACM SIGACCESS, 51-58, ACM New York,
no. 2, pp. 84-95, (2000). NY, USA.
[3] FESD (Federación Española de Síndrome de Down), 2010, II [10] Parker SE, Mai CT, Canfield MA, Rickard R, Wang Y,
Plan de Acción para Personas con Síndrome de Down, 2009- Meyer RE, Anderson P, Mason CA, Collins JS, Kirby RS,
2013. Down España, Madrid (ISBN:9788469315651). Correa A; National Birth Defects Prevention Network.
Updated national birth prevalence estimates for selected birth
[4] Google Inc., Android Developers Site, defects in the United States, 2004-2006. Birth Defects Res A
http://developer.android.com/, last visited February 2015. Clin Mol Teratol. 2010;88:1008-16.
[5] Grossman, T., Fitzmaurice, G., Ramtin, A., A survey of [11] Persson, H., Ahman, H., Yngling, A., Gulliksen, J. Universal
software learnability: metrics, methodologies and guidelines, design, inclusive design, accessible design, design for all:
CHI '09 Proceedings of the SIGCHI Conference on Human different concepts one goal?, Universal Access in the
Factors in Computing Systems, 649-658, ACM New York, Information Society, Springer, (2014).
NY, USA, (2009).
[12] Strothotte, C., Strothotte, T., Seeing Between the Pixels:
[6] Hu, R., Feng, J., Lazar, J., Kumin, L., 2013, Investigating Pictures in interactive Systems, Springer-Verlag Berlin
input technologies for children and young adults with Down Heidelberg (1997).
syndrome, Universal Access in the Information Society,
March 2013, Volume 12, Issue 1, pp 89-104. [13] Villamor, C., Willis, D. and Wroblewski, L., Touch Gesture:
Reference Cards, LukeW Ideation + Design, (2011).
[7] Kumin, L., Lazar, J., Feng, J., Wentz, B., and Ekedebe, N.
(2012). Usability Evaluation of Workplace-Related Tasks on [14] Wu, J., Morris, J., 2013, The population prevalence of
a Multi-Touch Tablet Computer by Adults with Down Down’s syndrome in England and Wales in 2011, European
Syndrome. Journal of Usability Studies 7(4), 118-142. Journal of Human Genetics 21, p. 1016–10

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 48 |
 Multi-touch Technology in Early Childhood: Current Trends and Future Challenges
V. Nacher | J. Jaen

Multi-touch Technology in Early Childhood: Current

Trends and Future Challenges
Vicente Nacher, Javier Jaen
ISSI Group, Dpto. Sistemas Informáticos y Computación
Universitat Politècnica de València.
[email protected] [email protected]

ABSTRACT
The advantages of the direct manipulation style make the multi-
touch technology an ideal mechanism to support learning
activities for children. Moreover, although pre-kindergarten
children are becoming frequent users of the technology little work
has been done in the area to assess their actual abilities. This
paper goes over the state of the art of multi-touch technology
targeting pre-kindergarten children and its use for educational
purposes. In addition, in this work we present future challenges
that should be faced in the area in the near future to establish the
basis on which designers will develop educational applications for
children that fully exploit the multi-touch technology according to
the actual abilities of pre-kindergarten children.

Categories and Subject Descriptors Figure 11. Target age 2009 vs 2011 for Education cat. in the
H.5.2. [Information interfaces and presentation]: User Interfaces - Apple Store [22].
Interaction Styles. However, the increasing interest in multi-touch technology has
not given rise to studies on the design of multi-touch systems for
General Terms the youngest age range [10]. Moreover, the lack of standardized
Performance, Design, Experimentation, Human Factors, and universally accepted interactions for these challenging users
Standardization. makes the design of well-designed multi-touch interactions even
Keywords more crucial [11].
Multi-touch interaction; pre-kindergarten children; gestures; According to these facts, in this paper we carry out a review of the
future challenges current state of the literature of multi-touch technology with pre-
kindergarten children and provide a set of future challenges to be
1. INTRODUCTION addressed in the future. The end goal is to define a research
Multi-touch technology has rapidly evolved in recent decades and agenda to make this technology usable by pre-kindergarten
nowadays it has widespread acceptance [5] because it provides children and give application designers the necessary guidelines to
users with a more natural and intuitive way to interact [23]. As develop touch applications according to the actual skills of pre-
pointed out in [7] children between zero and eight years old are kindergarten children.
frequent users of digital media in the USA and they meet with
touch technology often before they can even speak. Supporting 2. STATE OF THE ART
these ideas the Horizon report [12] places mobile devices, such as Until very recently, there have been no research efforts addressing
tablets and smartphones, as one of the two emerging technologies multi-touch interaction with pre-kindergarten children (aged less
suitable for children aged under 2 years. than three years). Possibly this has been the case because age is a
limiting factor for experimental studies; young children do not
The three basic ideas behind the direct manipulation concept were
have the verbal and cognitive skills to express their likes and
listed by Shneiderman and Plaisant [21]: (1) the visibility of
preferences [15] and, they are not able to carry out tasks for long
objects and actions of interest; (2) the replacement of typed
periods or are easily distracted [8]. However, according to Piaget
commands by pointing-actions on objects of interest; and (3) the
[19], children are in a preoperational stage from 2 years old
rapid, reversible and incremental actions that help children to keep
onwards, i.e., they begin to think in terms of images and symbols,
engaged, giving them control over the technology and avoiding
and develop symbolic play with imaginary objects, which means
complex instructions.
they could be candidates for multi-touch technology. In addition,
A detailed analysis of applications in the Apple store made by being aware of children developmental abilities is critical when
Shuler [22] reveals the growing use of educational applications designing software for the very young [26].
for children based on touch devices. This is specially the case for
This has motivated recent studies that assess the abilities of pre-
preschool children (Figure 11) who were the target users of nearly
kindergarten children to perform basic touch gestures. The work
60% of these applications by 2011.
by Nacher et al [18] reveals that the advantages and features of
multi-touch technology is not being fully exploited in existing
commercial applications since only the drag and tap gestures are

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 49 |
 Multi-touch Technology in Early Childhood: Current Trends and Future Challenges
V. Nacher | J. Jaen
being used and no support for collaboration is given. Hence, the
work evaluates a set of basic touch gestures with children aged
from two to three years (Figure 12) concluding that even these
young users are able to perform a set of basic touch gestures such
as tap, drag, scale (up & down) and one finger rotation. In
addition, in this work a set of design guidelines is proposed to
deal with complex gestures such as double tap and long pressed.
These assisted strategies were evaluated later in [16] and results
show that even these more complex gestures can be successfully
performed by pre-kindergarten children with basic assistance.
Figure 13. Child performing double drag gestures (extracted
from [24]).
Usability is not the only dimension that has been addressed by
recent studies in the field. Another topic of interest is that of
communicability when pre-kindergarten children are considered.
The work in [17] report a first approach to evaluate mechanisms
for applications to communicate pre-kindergarten children the
expected multi-touch gestures at a given moment. In this study,
the authors present and evaluate two visual approaches (iconic
and animated) to communicate touch gestures (see Figure 14).
Three touch gestures are considered: the tap representing in-place
gestures (i.e. those in which the hand does not actually describe a
trajectory but taps at a very specific pace or in a specific way); the
drag representing one-contact point gestures that require a
movement following a specific trajectory; and the scale up gesture
representing two-contact point gestures that require movement.
The results show, firstly, that none of the evaluated languages is
Figure 12. Examples of scale up and drag tests (extracted
effective to communicate in-place gestures and, secondly, that the
from[18]).
animated approach overcomes the iconic one for gestures that
In the experimental studies in [1, 2] four applications are require movement of contacts reaching success rates above 85%.
considered and the interaction needed to play with them is This fact suggests that even pre-kindergarten children are able to
evaluated. The gestures under test are the tap, drag, rotate, drag interpret the direct mapping between the visual stimuli (i.e. a hand
and drop, pinch, spread and flick and the experiment involves sliding on the surface) and the gesture to be performed. Therefore,
children aged 2 to 4 years. According to the results, the authors this work suggests that visual languages could be an effective way
conclude that children aged four are able to perform all the to enable pre-kindergarten children autonomous interaction.
evaluated gestures. Those aged three only find problematic the
spread task. Finally, those aged two are able to perform the tap
and drag gestures properly, learn quickly the flick gesture but they
are less effective with the more complex gestures such as drag and
drop and pinch.
Another interesting study is the work of Vatavu et al [24] who
evaluate touch gestures with children between 3 and 6 years using
tablets and mobile devices. They conclude that despite there is a
significant performance growth with age, in general, children have
good performance with the tap, double tap and single hand drag
and drop gestures. However, in the case of the double hand drag
and drop gesture (see Figure 13) they do not reach these good Figure 14. Description of the animated visual (top) and the
results and the success rate drops to 53.7%. In addition, authors iconic (bottom) language for the scale up gesture (extracted
correlate the results with a sensorimotor evaluation based on from [17]).
children’s finger dexterity and their graphomotor and visuospatial Exploring the educational dimension and the suitability of multi-
processing abilities. The correlation shows that children with touch surfaces to support educational activities there are several
higher visuospatial skills (i.e. having better skills for works that strengthen the idea that this technology provides
understanding relationships between objects, as location and benefits for pre-kindergarten children education. For example, the
directionality) perform better in the drag and drop tasks. work by Bebell et al [4] shows a nine week study comparing the
Furthermore, the study also reveals that children with more improvement of kindergarten children taking early literacy lessons
developed visuospatial skills tap closer to the center of targets in when using tablets for learning or using a traditional non-
the tap and double tap tasks since they have a better understanding technological method. Their results show that the group that
of the targets’ geometries. learned with tablets scored higher on early literacy assessments,
particularly these higher scores are present in the ability to
recognize sounds and represent sounds and letters. Following the
same line, Chiong and Shuler [6] conduct an experiment involving
touch devices and audiovisual material adapted to children aged

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 50 |
 Multi-touch Technology in Early Childhood: Current Trends and Future Challenges
V. Nacher | J. Jaen
three to seven years and their results show that children obtain of work to do to assess the best approaches. This preliminary
remarkable gains in vocabulary and phonological awareness. study points out that animated languages can be effective to
Moreover, Knoche et al [14] point out that the interaction of communicate touch gestures to pre-kindergarten children and help
children aged between 16 and 33 months with interactive them to be autonomous when using the multi-touch devices.
elements in a tablet does not reduce their comments in dialogic However, as pointed out by the authors, only three gestures were
reading activities. Another example is provided by Zaranis et al evaluated, hence, it remains to be evaluated whether the inclusion
[27] who conduct an experiment to study the effectiveness of of additional gestures has an impact on the overall performance
digital activities on smart mobile devices (tablets) to teach and effectiveness of the languages. Moreover, the gestures were
mathematical concepts such as general knowledge of numbers, tested in isolation; therefore, languages should be studied when
efficient counting, sorting and matching with kindergarten the interaction area is cluttered with many touchable elements and
children. Their results show that the tablet-aided learning their corresponding visual cues or with elements that may be
provided better learning outcomes for the students than the manipulated with several different gestures. With respect to
traditional teaching method. Kammer et al [13] present three communicability of touch gestures, another possible future work
applications to foster the development of cognitive and motor can be the evaluation of other languages both iconic and animated
skills on a multi-touch tabletop with children aged from four to in order to find out which type of language fits better to
six years. The conducted experiment shows that even preschool communicate touch gestures to infants. In addition, an interesting
children are able to use this technology and they enjoy the task future work could be to design and evaluate a similar approach to
and collaborate in the multi-user activity. the method used by Balonian et al [3] for children between five
and six years old. In this work, each gesture was associated to a
The results of these works suggest that pre-kindergarten children
specific character in a way that the gestures were “recallable”.
are prepared to use multi-touch technology and the intuitive and Metaphors such as a walking ladybug for a drag gesture or a
natural interaction of direct manipulation style of the multi-touch jumping grasshopper for a double tap were used. Finding suitable
technology makes it ideal to support pre-kindergarten children metaphors that pre-kindergarten children can understand could be
interaction and, hence, educational activities targeted to them. very useful to develop autonomous educational applications.
Moreover, these works conclude that these particular users are
able to perform a set of touch gestures successfully and future The studies in the literature point out that pre-kindergarten
applications designed for them do not need to be restricted to only children have the necessary skills to make use of multi-touch
basic interactions such as the drag and tap gestures. However, technology. However, these works implement assistive techniques
these works reveal that there is no consensus or standardization of to deal with precision issues during the initiation and termination
the multi-touch interaction style for users in these early ages. phases of gestures since they assume that pre-kindergarten
children are no able to perform the gestures with high accuracy.
3. FUTURE CHALLENGES This causes that the implemented interaction styles do not allow
In this section we present a compilation of future works that we children to have the control over the termination of the gestures
consider interesting and necessary to complete the literature and despite they are in the process of developing their motor skills and
shed light on the specific needs of pre-kindergarten children when some of them may have already the proper cognitive abilities to
using multi-touch technology. perform the gestures with higher levels of precision. As a result,
existing applications designed under these assumptions do not
On the one hand, according to Hinrichs and Carpendale [9] who
benefit from the use of multi-touch technology to help children to
point out that there is evidence that some events are affected by
develop their precision-related cognitive and motor skills.
previous and subsequent ones and given that all the analyzed
According to this observation it would be interesting to evaluate
studies consider and evaluate the gestures in isolation, an
dynamic gestures (those that require movement of contacts over
interesting strand of future work would be to evaluate these same
the surface) demanding high levels of accuracy to users,
gestures that pre-kindergarten can do in isolation but when several
specifically, it would be interesting to evaluate how accurately
interactive elements are shown simultaneously. Users should
they can rotate an object, how close they can drag an object to a
perform different preset sequences of these gestures in order to
target and whether they are able to perform scaling gestures (up
determine whether the cluttering of elements in the interaction or
and down) with enough accuracy to stop the stroke of the gesture
the task chaining affect their performance. These results will help
in a specific moment to reach a desired size. This would certainly
to develop more complex applications which are not restricted to
help in understanding the limitations on accuracy that should be
only one possible interaction in each phase of the game.
fulfilled in applications targeting pre-kindergarten children. In
Another interesting issue that remains to be addressed is the addition, the data collected during the experimentation could be
processing of the unexpected touch events when children are used to develop assistive strategies to deal with precision issues in
holding the tablet with a finger resting on the display or when part an adaptive way for the users that actually need it and not in an
of the palm also touches the surface. This issue is difficult to exhaustive way for every child as current systems do.
address because children may not be aware of such unintentional
Finally, once known the actual capabilities and abilities of pre-
contacts with other parts of their body when their fingers approach
kindergarten children, the gestures that they can perform, the
the screen resulting in an unexpected effect. It would therefore be
accuracy that they can achieve and evaluated the communicative
interesting to explore potential improvements in multi-touch
strategies suitable for them; interesting future works could be
usability, for instance by determining and filtering out unexpected
developed for the definition, construction and evaluation of
blob contacts wherever applicable.
environments based on multi-touch technology that foster
Addressing the topic of the definition of effective mechanisms to creativity and allow collaboration between peers. These
communicate which actions are expected from the user; and environments could be integrated into classrooms and be used
taking as a starting point the work in [17], there is still a great deal with educational purposes to allow children to develop their

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 51 |
 Multi-touch Technology in Early Childhood: Current Trends and Future Challenges
V. Nacher | J. Jaen
creativity skills and allow educators to monitor the progress of [8] Egloff, T.H. Edutainment: a case study of interactive cd-rom
their students and create appropriate content for them. As Rushton playsets. Computers in Entertainment 2, 1 (2004), 13.
[20] points out, the creation of classroom environments that [9] Hinrichs, U. and Carpendale, S. Gestures in the wild:
enhance students expression and selection opportunities provides studying multi-touch gesture sequences on interactive
a proper atmosphere towards helping children to learn at their own tabletop exhibits. Proc. of CHI’11, 3023–3032.
pace. A starting point could be taking advance of (1) the [10] Hourcade, J.P. Interaction Design and Children.
portability of tablets that allow children and teachers to use Foundations and Trends® in Human-Computer Interaction
various locations in the classroom enabling the creativity and 1, 4 (2007), 277–392.
collaboration of small groups of children; and (2) the more [11] Ingram, A., Wang, X., and Ribarsky, W. Towards the
sophisticated interface of tablets that allows both individual and establishment of a framework for intuitive multi-touch
mutual interaction between two or more users [25]. interaction design. Proc. of AVI’12, 66–73.
[12] Johnson, L., Adams, S., and Cummins, M. The NMC
4. CONCLUSIONS Horizon Report: 2012 K-12. The New Media Consortium,
The contributions of this paper are twofold; firstly, a review of the Austin, Texas, 2012.
state of the art of multi-touch technology for pre-kindergarten [13] Kammer, D., Dang, R., Steinhauf, J., and Groh, R.
children was made. The literature suggests that the use of this Investigating interaction with tabletops in kindergarten
technology is an ideal way to develop applications for these very environments. Proc. of IDC’14, 57–66.
young users. In addition, the presented studies show that pre- [14] Knoche, H., Rasmussen, N.A., and Boldreel, K. Do
kindergarten children are able to perform more touch gestures Interactions Speak Louder than Words ? Dialogic Reading
than could be expected when analyzing existing commercial of an Interactive Tablet-based E-book with Children
applications. On the other hand, pre-kindergarten children are between 16 Months and Three Years of Age. Proc. of
developing their motor skills and, hence, in order to use the multi- IDC'14, 285–288.
touch technology, designers should be aware of the actual [15] Kremer, K.E. Conducting Game User Experience Research
capabilities of children when developing interfaces for them. The with Preschoolers. Workshop on Games User Research:
second contribution is a collection of future challenges to be faced practice, methods, and applications (collocated to CHI’12).
in the near future when building multi-touch technology for [16] Nacher, V., Jaen, J., Catala, A., Navarro, E., and Gonzalez,
preschool children. These challenges are related to P. Improving Pre-Kindergarten Touch Performance. Proc.
communicability, adaptability and usability of multi-touch of ITS ’14, 163–166.
applications designed for these challenging users. We hope that [17] Nacher, V., Jaen, J., and Catala, A. Exploring Visual Cues
the usability studies and design guidelines compiled from these for Intuitive Communicability of Touch Gestures to Pre-
works and the future challenges discussed in this paper will allow kindergarten Children. Proc. of ITS ’14, 159–162.
designers of future applications for pre-kindergarten children to [18] Nacher, V., Jaen, J., Navarro, E., Catala, A., and González,
fully exploit the potential of multi-touch technology to support P. Multi-touch gestures for pre-kindergarten children.
their cognitive and motor development. International Journal of Human-Computer Studies 73,
(2015), 37–51.
5. ACKNOWLEDGMENTS [19] Piaget, J.The Child and Reality. Grossman, New York,
Work supported by the MINECO (grants TIN2010-20488 and 1973.
TIN2014-60077-R) and from GVA (ACIF/2015/075). [20] Rushton, S. and Juola-Rushton, A. Classroom Learning
Environment, Brain Research and The No Child Left
6. REFERENCES Behind Initiative: 6 years Later. Early Childhood Education
[1] Abdul Aziz, N.A., Batmaz, F., Stone, R., and Paul, C. Journal 36, 1 (2008), 87–92.
Selection of touch gestures for children’s applications. Proc. [21] Shneiderman, B., Plaisant, C., Cohen, M., and Jacobs, S.
of SIC'13, 721–726. Designing the User Interface: Strategies for Effective
[2] Abdul Aziz, N.A., Mat, N.S., Batmaz, F., Stone, R., and Human-Computer Interaction. Prentice Hall, 2009.
Paul, C. Selection of Touch Gestures for Children’s [22] Shuler, C. iLearn II: An Analysis of the Education Category
Applications: Repeated Experiment to Increase Reliability. of the iTunes App Store. The Joan Ganz Cooney Center at
International Journal of Advanced Computer Science and Sesame Workshop, New York, 2012.
Applications 5, 4 (2014), 97–102. [23] Smith, S.P., Burd, E., and Rick, J. Developing, evaluating
[3] Baloian, N., Pino, J. a., and Vargas, R. Tablet gestures as a and deploying multi-touch systems. International Journal of
motivating factor for learning. Proc. of ChileCHI'13, 98– Human-Computer Studies 70, 10 (2012), 653–656.
103. [24] Vatavu, R., Cramariuc, G., and Schipor, D.M. Touch
[4] Bebell, D., Dorris, S., and Muir, M. Emerging Results From interaction for children aged 3 to 6 years : Experimental fi
The Nation’s First Kindergarten Implementation of iPads. ndings and relationship to motor skills. International
Auburn, 2012. Journal of Human-Computer Studies 74, (2015), 54–76.
[5] Buxton, B. Multi-touch systems that I have known and [25] Wakefield, J. and Smith, D. From Socrates to Satellites:
loved. 2013. iPad Learning in an Undergraduate Course. Creative
http://billbuxton.com/multitouchOverview.html. Education 03, 05 (2012), 643–648.
[6] Chiong, C. and Shuler, C. Learning: Is there an app for [26] Wolock, E., Ann Orr, E.D., and Buckleitner, W. Child
that? Investigations of young children’s usage and learning development 101 for the developers of interactive media.
with mobile devices and apps. New York, 2010. Active Learning Associates, Inc., 2006.
[7] Common Sense Media. Zero to Eight: Childrens Media Use [27] Zaranis, N., Kalogiannakis, M., and Papadakis, S. Using
in America 2013. 2013. Mobile Devices for Teaching Realistic Mathematics in

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 52 |
 Multi-touch Technology in Early Childhood: Current Trends and Future Challenges
V. Nacher | J. Jaen
Kindergarten Education. Creative Education 04, 07 (2013),
1–10.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 53 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche

Vibrotactile Vest and The Humming Wall: “I like the hand

down my spine”
Ann Morrison Cristina Manresa-Yee Hendrik Knoche
Department of Architecture, Design Department of Mathematics Department of Architecture, Design
and Media Technology, and Computer Science and Media Technology,
Aalborg University Universitat de les Illes Balears Aalborg University
Rendsburggade 14, Crta. Valldemossa km 7.5 Rendsburggade 14,
9000, Aalborg, DENMARK 07122, Palma, SPAIN 9000, Aalborg, DENMARK
[email protected] [email protected] [email protected]

ABSTRACT use and is placed in Utzon Park (Aalborg, Denmark), with seating
Vibrotactile information can be used to elicit sensations and at each end on both sides (that also act as wind protection).
encourage particular user body movements. We designed a For the vest wearer though, the interactive experience is unique.
vibrotactile vest with physiological monitoring that interacts with Several actions performed on the wall (knocks, a swipe action
a vibroacoustic urban environment, The Humming Wall. In this downwards, upwards; sideways left to right, right to left) are
paper, we describe the first field trial with the system held over a replicated in actuators vibrating in the vest. Further, people feel,
5-week period in an urban park. We depict the participants’ see and hear their own heartbeat in one zone of the wall and feel
experience, engagement and impressions while wearing the and hear their own breath in another.
vibrotactile vest and interacting with the wall. We contribute with
positive responses to novel interactions between the responsive The aim of this work is to describe participant experience and
environment and the vibrotactile vest, with a system designed to engagement with the paired tactile interaction between vest and
calm, activate, guide and warn the participants. wall, evaluated in field trials held over a 5-week period in an
urban park with 39 people. As this was the first execution of these
Categories and Subject Descriptors paired vibro-tactile-acoustic systems, we kept the interaction
H.5.2. Information interfaces and presentation: User interfaces. paradigms relatively simple.

General Terms The paper is structured as follows: Section 2 compiles works on

Design, Human Factors, Measurement vibrotactile interfaces related to activation, navigation and
calming purposes as well as responsive environments. Section 3
Keywords describes the design of the vest and The Humming Wall. Section 4
Wearable computing; Vibrotactile; The Humming Wall; Audio presents the evaluation and section 5, the results of the field trial.
vibrations; Haptic patterns; Sensations; Vibroacoustics Finally, we summarise the findings and discuss their implications.

1. INTRODUCTION
Touch is highly connected to our emotions [1] and vibrotactile
cues can cause different sensations (e.g. calming, pain) depending
on factors such as frequency or rhythm [2, 3]. Further, touch is a
powerful communication channel that is not used as frequently as
sight or hearing. By structuring vibrotactile patterns, we can build
a vibrotactile language to convey information for different
applications such as enhancing communication [4] or helping to
perceive the interlocutor’s facial expression [5]. We are especially
interested in those works using vibrotactile cues working with
whole body navigational cues and instigating body motion as well
as vibrations used for activation/calming purposes.
Working with a kinesiologist-neurophysiologist, we designed Figure 1. General public at the wall, hearing and feeling
novel placement and patterns for vibrotactile actuators on the the vibration and playing with the wall
body to activate, guide, warn and to emulate natural touch
sensations such as stroking up or down the back to relax a 2. RELATED WORK
person—‘there, there, it’s okay’.
Work related to the vibrotactile vest and The Humming Wall
In order to enrich and yield a positive experience for the vest includes (i) vibrotactile languages working with whole body
wearer, we built a responsive urban environment The Humming navigational cues and instigating body motion, (ii) vibrotactile
Wall, designed to enrich participants experience with the interfaces initiating activating/calming sensations and (iii)
environment and invite open-ended, embodied and playful responsive vibroacoustic environments in public spaces where
interactions [6-8]. The Humming Wall is a 12 x 3.5 x 2.7 meter visual is not the privileged sense. As we cannot cover all of these
(long x wide x high) vibroacoustic wall (see Fig.1) with a calming in detail here, we cover an abridged version.
low-level hum. The curving wall is designed for general public

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 54 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
In order to motivate and improve motor skills, Spelmezan et al. actuators are operated in overlapping patterns to provide different
[9] developed a tactile language to assist athletes to improve their haptic synesthesia sensations such as sense of movement, shiver
motor skills in snowboarding. In this line of work, McDaniel et al. and states of activation and/or calming as well as providing
[10] proposed a framework, MOVeMENT (Mapping Of navigational cues. We explored the positioning and combinations
Vibrations to moveMENT), that used vibrotactile stimulation to of the actuators with a kinesiologist trained in neurophysiology.
b
teach motor skills and to motivate fundamental movements
(flexion, extension, abduction, adduction and rotation). Further, c
Rosenthal et al. [11] presented a belt to provide vibrotactile
spatio-temporal patterns to teach choreographed dance. Morelli et
al [12] and Yuan and Folmer [13] also used vibrotactile cues to
inform users with sight loss or blindness to carry out certain
movements to play videogames.
Several research projects use vibrotactile cues for navigation.
Tsukada et al. [14] designed a belt that transmits directional
information. Van Erp et al. [15] also tested a directional belt but
with different distance-coding schemes. Other wearable supports
have been used, such as the Vibrotactile Glove [16] that offered
users with disabilities information by indicating warning signals,
spatial representation, and directional guidance using a glove.
Vibrations have been used for therapies to generate soothing and
calming sensations to users. Vaucelle et al. [17] presented Touch Figure 2. The vibrotactile vest: adjustable harness (a)
Me, a flexible vibrotactile motor array in a soft enclosure that can front and (b) back and (c) outer shell and skirt-apron.
be applied to large areas of the body and remotely actuated by a
care-giver through a remote switch array to provide soothing
touch simulation over an entire part of the body. Paredes and For the initial patterns, 29 actuators were used (see Fig. 3). These
Chan [18] employed two vibrotactile motors in a bracelet that patterns try to emulate the hands on work that a kinesiologist-
stimulates acupressure points in the wrists and the chest (when the neurophysiologist does in activating sequential points of the body.
wrist is held to the sternum) to reduce stress. And using the same In addition, we emulated the natural touching we give each other
bracelet, they coached participants to breathe according to well- for calming-comforting (e.g. stroking the back to calm or comfort
known deep-breathing techniques. Arafsha et al [19] designed a a person), guidance-navigation (e.g. placing hands on nape of
haptic jacket to enhance comfort with 6 affective haptic back and shoulder as if to support and guide an elder) or
components: chest and neck vibration, neck warmth, heartbeat information-instruction (e.g. stopping the body with pulses to the
simulation, arms vibration and shivering. solar plexus). The reader is directed to [22] for a more detailed
description of the patterns.
Prior work on responsive vibroacoustic environments in public
spaces for developing embodied, multi-user interactive
experiences include works such as MEDIATE, an environment
that uses real time visual, aural and vibrotactile stimuli both for
sensing and giving feedback for education with autistic children
[20] or ListenTree, an audio-haptic display embedded in a tree in
a natural environment. A visitor notices a faint sound that appears
to emerge from a tree, and feels a slight vibration under their feet
as they approach. Future versions will add touch input to achieve
a more responsive installation [21].

3. THE SYSTEMS
There are two systems. First, a vibrotactile vest which produces
vest-to-human interactions as felt sensations for the wearer.
Second, The Humming Wall produces two types of human-to-
Wall-to-human interactions, one type between the vest wearer and
the wall and the other enables interaction between non-vest Figure 3. Placement of actuators
wearers-to-vest wearers via the wall and also interactions for non-
vest wearers with the wall itself. In Table 1, examples of patterns are described. Ten patterns were
designed:
3.1 Vibrotactile vest • Calming: Up, Down,
The vest is made of two layers, an inner layer consisting of one-
size-fits all adjustable harness (see Fig. 2a, 2b) and an outer layer • FeelGood: WaistLeftToRight, WaistRightToLeft,
consisting of an enclosing vest (See Fig. 2c). The harness houses MidFrontToBack
32 actuators, moveable to ensure they are placed exactly on each
different shaped body. The lower harness fits around the legs, to • Activation: Shiver, Tarzan
safeguard that the harness stays pulled down and keeps the • Navigation: TurnLeft, TurnRight
actuators in place while the participants move around. The

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 55 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
• Warning: Stop d) Swipe—3 panels that respond to up, down, left and right
gestures. These played as audio and vibration at the wall with
The Zephyr BioHarness 3 [23] is integrated into the harness and a corresponding gesture played on the vest—e.g. swiped up or
vest system to read the physiological state of the wearer in real- down the back and swiped across the waist—left to right or
time, specifically their heartbeat and breath rate. Two custom- right to left—depending on direction of gesture (Fig. 4d).
made electronic boards are also included in the wearable system:
one controls communication and the other powers and controls the e) Knock & Swipe—5 panels that respond to knock and swipe at
actuators. the wall and on the vest. The swipe gestures run along the 5
sections with audio, vibration and lights. The knocks are
Table 1. Examples of patterns relayed alternatively to all actuators and felt all over the body.
Actions Pattern Knocking on two panels activated two actuators at once,
knocking on three activated three actuators (Fig. 4b).
Activating:
4x(FcL, FcR), pause, FcM
Tarzan The wall was placed in an outside environment at Utzon Park
Calming: (BsL6, BsR6), (BsL5, BsR5), (BsL4, BsR4), (Aalborg, Denmark) between two of Utzon’s architectural
down (BsL3, BsR3), (BsL2, BsR2), (BsL1, BsR1) structures, the Utzon Center and Utzon student housing, and
Navigating: Bw7, (BhR1, BhR2), (Fw3, Fw2), (Fw2, Fw1), encompasses Utzon’s aesthetics and design principles. The design
turn left (Fw1, Bw8) includes harmonious modular repeating segments with internal
Alert: Stop (FspL, FspR) voids large enough for arms to access electronics and cables to
enable interactivity, serving both the aesthetics and pragmatics.

The outer shell, made in 3 adjustable sizes, is a padded layered

stretchable vest designed to keep the actuators tightly in place
against the bowed areas of the body, especially the lower back and
chest, so the vibrations are evenly felt in all areas. We offered an
adjustable skirt-apron for the sake of modesty and/or aesthetics,
particularly for those wearing dresses.

3.2 The Humming Wall

A 12m long, 3.5m wide and 2.7m high was cut from EPS150
foam blocks with a hot wire and glued together to form 11
segments. Each segment is coated with a PolyUrea coating for
strength and durability. The wall is fitted with an array of 2-
channel DMX controlled warm and cold white lights. These lights
are used for ambient and indicative visual effects on the wall. An
Figure 4. a) Heartbeat: Vest transmits to Wall; Audio,
array of tactile transducers placed in the wall generates different
Vibration and Light emulate vest wearers heartbeat at
haptic sensations, audio cues and haptic feedback. Each segment
wall. (b) Knock and Swipe: Wall transmits to Vest; Audio,
has an 80W Bass-Shaker mounted with a frequency range suitable
Vibration and Light respond to knock and swipe. (c)
for haptic and audio feedback, and amplified so that each segment
Knock: Wall to Vest (d) Swipe: Wall to Vest (e) Breath
functions as a vibrating speaker. The Humming Wall has its own
Rate: Vest to wall
baseline vibration and audible hum. A combination of piezo- and
capacitive electronics to read and analyse touches, enables the
differentiation of knocking (or tapping) and swiping gestures. 4. EVALUATION
To evaluate the tactile interaction, people were invited to
The wall is divided into five zones that interact with different
participate in pairs asking a familiar, friend or partner to take part
input from the data registered in the vest or from gestures done on
with them, as early tests proved the interaction was best done in
the wall and trigger different output in the wall and on the vest
pairs and personal. The trial was held over an extended period of
simultaneously. In the two seating areas, the physiological data
5 weeks from June 23-July 25, 2014. The duration of the trials per
from the vest is relayed to the wall:
pair took between 1.5 to 4 hours, averaging 2.5 hours per pair.
a) Heartbeat—displayed as audio, vibration and light (at the The initial programmed 3-week period proved too short given
same beat) in the wall (Fig. 4a). emergent circumstances and the programmed 2 hours for each set
of paired participants was not enough time as we constantly had to
b) Breath rate—displayed as audio and vibration in the wall push-rush the participants to move onto the next zone.
(Fig. 4e). Subsequently, from participant 16 onwards, participants were told
Then, there are three zones where sensations relayed at the wall there were 7 activities and they could either choose their own
are transferred to the vest: timing or we could prompt. All chose to set their own timing.

c) Knock—3 panels respond to knocking with audio and 4.1 The Participants
vibration in the wall and knocks are felt in individual We enlisted 39 volunteers with ages from 12 to 65 years (average
actuators of the vest. Right panel knocks are transferred to the age 39), 20 females and 19 males. 19 participated in mixed gender
right side of the body—front and waist; middle panel groups, 11 in female/female and 9 in male/male groups (uneven
transferred to the back; and Left panel transferred to left side numbers are due to only one participant from a pair wearing the
of the body—front and waist (Fig. 4c). vest). Most people took part with close or good friends (11), their

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 56 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
partners (10), colleagues (10), family members (5), or social cut into manageable chunks and coded focusing on where
friends or acquaintances (3). 22 self reported basic or average IT activities and interactions occurred. We transcribed the talk aloud
skills with 17 advanced or above. 28 spent on average more than responses of the participants to each vibration and coded them
20 hours on a computer each week and 12 enjoyed playing a numerically. For each zone we measured the amount of time the
musical instrument regularly. 31 had tertiary level qualifications participants spent, coded to what degree they explored the
with 14 knowledgeable about wearable technology. interrelation of knocking and swiping on the wall and
corresponding sensations for the vest wearer.
4.2 Procedure and venues
Each pair of participants was accompanied throughout the trial by We used 5 level Likert-type scales (from completely disagree to
two researchers and all sessions were videoed. The trial was completely agree) to measure items adopted from IMI, FSS and
divided in 4 steps: MEC-SPQ. The levels of agreement were mapped to values from
1 to 5. The semantic differential scale items featured 5 numerical
(1) Utzon Library Fitting: fitting was done in a side room of the levels with the differential labels at the extreme values (e.g.
Utzon Library with a change screen for privacy. Demographic Impersonal 1,2,3,4,5 Personal).
info was collected and the BioHarness, harness and vest was fitted
and tested to place correctly actuators on the body (15-35 mins). 5. RESULTS
In this section we report on findings from questionnaires, vest
(2) Utzon Park Patterns: participants were guided by a researcher activity logging, video footage analysis, time spent at each section
and introduced to the ten vibration patterns 3 times while asked to of the wall and semi-structured interviews.
talk aloud their responses to the sensations. Then participants and
researchers stopped, each vibration pattern was played again and 5.1 Questionnaires
responses discussed again (10-15 mins). In Table 2 we find a summary of the results for a subset of the
(3) The Humming Wall Interaction: participants interacted with 5 questions. Most responses show that users enjoyed interacting
zones of activity (with 2 of the zones repeat visits = 7 total with The Humming Wall. Participants were relaxed when carrying
activities) at The Humming Wall. The participants moved out the tasks (Q19) and they found it pleasant to complete the
clockwise around the wall from one zone to the next. The different activities at the wall (Q2, Q16). They did not report
activities were: (1) heartbeat, (2) knock, (3) breath rate, (4) swipe discomfort associated with the vibrations from the wall (Q4), and
(5) 2nd visit to breath, (6) knock & swipe and (7) 2nd visit to most users felt that they lost track of time during the trial (Q15).
heartbeat (20-90 mins). When responding to the items regarding the vest, they did not feel
much discomfort associated with the vibrations in the vest (Q4),
(4) Utzon Library Evaluation: participants took off the vest, filled they felt it was possible to be active in the surroundings (Q6) and
in questionnaires and undertook semi-structured interviews (15-40 the vest vibrations made it easier to understand the tasks and
mins). responses (Q17). In general users stated that they were
concentrating on the sensations in the vest (Q1) and thought
4.3 Data Collection and Processing whether these sensations could be useful for other activities (Q7).
Data was gathered using quantitative and qualitative methods. Most participants found that some sensations held greater
Before the trial, the participants filled in informed consent forms meaning than others (Q3).
and a demographic questionnaire with questions on fitness levels,
their general IT level and experience with vibrotactile technology, Results summarising the overall experience with the vest and the
embodied interaction, large public displays and playing musical wall are shown in Fig. 5. Participants reported to be relaxed and
instruments. Activity data was logged for each participant from calmed even though they reported to be active as well. Further,
the actuators and from the BioHarness, the heartbeat and breath users felt that it was a personal experience, but at the same time,
rate. The actions done on the wall were also registered, knocking they felt sociable at the installation.
and swiping, together with the segment and direction and the time When crossing results of the questionnaires with the demographic
spent in each zone. data using non-parametric Mann-Whitney tests we found the
After walking around the wall, participants completed shortened following statistically significant differences. Males agreed
adapted versions of questions from MEC Spatial Presence significantly more than females that some sensations made more
Questionnaire (MEC-SPQ) [24], Flow State Scale (FSS) [25] and sense than others but enjoyed less learning how to do the tasks
Intrinsic Motivation Inventory (IMI) [26], to gauge reactions to suggested by the sensations. They felt less warm and personal
the sensations in the vest and the interaction with The Humming than the women. Participants who exercised 2 or more times per
Wall. The questionnaires comprised 21 Likert-type items to week reported significantly warmer and more relaxed experiences
analyse and cross-check users' perceptions. For Presence, we than those exercising less. Participants with average and lower IT
measured concentration, errors, activated thinking, and imagining knowledge found the experience significantly: easier to respond to
space. For IMI, we measured interest/enjoyment, perceived vibrations, found their understanding of the sensations improving
competence, pressure/tension, and effort/importance. For Flow, more as the trial progressed and felt warmer and more sociable
we measured challenge-skills balance, goals, concentration on than the participants with high IT knowledge. Those participants
task, and sense of control. The overall experience was measured with more experience at large displays agreed significantly more
through 10 semantic differential scale items (See Table 2). Lastly, that some of the vest vibrations were uncomfortable when
each participant described their experience, highlighting chosen compared to the less experienced. Musical participants who
aspects in semi-structured recorded interviews. enjoyed playing an instrument reported a different experience
from participants who did not. Non-instrument players had a
Data post-processing tasks included transcribing user interviews significantly more sociable and open experience. We found no
and translating them into English for further analysis. Videos were significant differences between those with experience with

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 57 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
embodied games, e.g. Wii, compared to those with little or none.
Those less knowledgeable with vibrotactile technology enjoyed
learning how to do the tasks suggested by the sensation
significantly more and had a more personal experience than the
savvy ones. Those knowledgeable in wearable technology
reported a significantly more relaxed experience compared to
those with little or no knowledge in this area.

Figure 6: Comparative Times (guided or not) for Wall

Figure 5: Overall experience results for 39 participants.

Semantic differential items

5.2 Logging of wall activity at zones

As commented before, from participant 16 onwards, participants
could choose their own pace. As a consequence, all interaction Figure 7: Comparative Time spent in Physiological and
durations from that point at the wall lengthened with an average Knocking and Swiping zones (guided or not)
of 9.8 extra minutes per participant when interacting at the wall.
In Fig. 6, the time spent at each zone is shown. The larger The total time spent in the physiology zones was significantly
differences can be seen at Heartbeat zone 1st visit, Breath rate predicted by the following three factors in the regression
zone 1st visit and Knock zone when participants could decide on analysis: participants spent more time in those zones when they
their timings. Participants spent more time in the physiology could advance at their own pace, were less knowledgeable about
zones when they decided their own pace (54.8%) than in the vibrotactile technology and if they had tried to find out the
Knocking and Swiping zones (45.2%) (Fig. 7). On average, mapping of the knocking to the vest. These three factors
participants deciding on their own timings stayed almost 6 extra accounted for 58% of the variance in time spent in the physiology
minutes in the physiology zones and around 4 minutes more in the zones.
action zones comparing with participants that were guided.
5.2.1 Knocking
In addition and regardless of this impact, there were two 56% of the participants tried to map knocking to the vest wearer’s
significant predictors of the time spent in the knocking and body. That is, they assumed a direct mapping between body and
swiping zones: participants spent more there, the stronger their wall, e.g. that knocking higher would map closer to the head,
relationship was with their pair and if they tried to find out the knocking in the middle section would trigger actuators around the
mapping of the knocking to the vest. Together these factors waist and lower would be felt in the lower back (see Fig. 8).
accounted for 32% of the variance in time spent in these zones.
5.2.2 Swiping
The main swipe movements were up, down, left and right.
However, participants also experimented with other swiping
directions. 64% of the participants made waving-swipe-circular
movements with one or two hands, as if massaging a back when
swiping at the wall (See Fig. 9).

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 58 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
feeling and it was stressful… Later I got used to it’ and ‘a bit
scary, intimidating at start, but then more natural when I
understood how it felt’. Some tried to compare to other
experiences: ‘front vibrations felt like standing in room with loud
music, connection vibrations going one side to another like being
in the ocean, and then stroking hand down back’. Others were
uncertain how to discuss it, especially the whole body sensation:
Figure 8. (a) Knocking at lower wall, (b) at mid wall. (c) at ‘haven’t felt before… single vibrations from mobile… but, not all
high wall and looking at partner to see response. over body and flowing… not sure how to describe… is new’. And
discussed: ‘all immersed, when all over body—could feel like me
vibrating all over, especially ones on stomach’. Others talked
about it acting like a real touch: ‘Putting up the same emotion as
touch—bringing out the same sense of well-being or happiness as
when somebody touches or pats or hugs you… the hug [Shiver,
Activating] was very surprising’.

Figure 9: Participant is making circular wave style swipes Some of the patterns made people ticklish, especially around the
waist [Waist, FeelGood], and at the sides: ‘It was like somebody
was tickling me’. And ‘I like the hand down my spine’ [Back,
5.3 Vibration Patterns Calming] or ‘The tickling on the side [Waist, FeelGood]… gave
The talk aloud responses to the 10 patterns that were registered in me goosebumps from the outside—like you didn't have the
the Utzon Park Patterns stage were coded numerically, i.e. a sensation but got the goosebumps anyway’. For most, the waist
response to each individual pattern was judged as either negative and the back sensations were the most pleasant: ‘I liked the ones
(-1), neutral (0) or positive (1) - see Fig. 10. We used the around the waist the most’. And ‘the waist was very pleasant and
numerically coded responses to the 10 patterns in a non- it tickles a bit’ [Waist, FeelGood] or ‘Those on the back were
parametric Friedman test rendering a Chi-square value of 51.9, pleasant… nothing was uncomfortable… it was all very gentle’
which was significant (p<.001). Post-hoc pairwise comparisons [Back, Calming] and ‘Preferred up and down—lifting you up with
showed that Up, Down, and WaistRightToLeft all were up swipe…Up more than down, felt more natural for me’. Yet
significantly more positively evaluated than all other patterns, there was differences in interpretations and preferences between
whereas the Tarzan-activation pattern was significantly evaluated elements of the patterns: ‘Solar plexus [Front, Warning], not so
more negatively than all others (see Fig. 10). bad once got used to it—at first didn’t like it—was heavier—then
saw it is a warning of don't do that’, whereas for another ‘Front—
not as pleasant—not as natural—vibrating there’. But others
preferred a different set of patterns again: ‘The feeling in the chest
was best, feel it better also at lower back---very nice’ and
different intensities ‘All could be stronger’ and ‘Tarzan [Front,
Activating], was pleasant. Just as pleasant for waist etc… Some
also suggested certain movements took longer to learn: ‘turn left
or right, took time to get more used to them–needed to do again a
few times at first—and was harder to figure out at the beginning,
but become almost normal quickly’.
The Humming Wall also provided new sensations with responses
such as: ‘Knew touching wall, but no idea how it would feel,
exciting to know how it felt—big difference to having the feeling
of it all over body and not just on fingers or one part.’ Many
participants explored mapping the body to the wall: ‘I thought I
was able to knock high and feel high in body, middle to middle
Figure 10: Means of coded responses to the vibration and once or twice—there was a specific connection between the
patterns with error bars (standard error of the mean) two… it became difficult to find the good spots like you do with
massage’. Sensations felt better in certain spots, so participants
5.4 Semi-structured interviews kept seeking a logic, often ordering a partner to repeat or go back
In the semi-structured interviews we asked a series of questions, to find a sensation again.
for example ‘How did you find the sensations in the vest?’ ‘Were Many found the wall helped them relax with participants often
any unpleasant?’ in order to engage and to get participants talking referring to the wall as: ‘therapeutic’ or ‘good to help take care of
about their experience and express their views in their own terms. health’ and ‘Love the wall, soothing and relaxing listening to
People were generally very forthcoming and the interviews lively. humming sounds and feeling vibrations---also to play it when sat
Participants often describe the interactions as interesting, fun, there—calming humming sound—like it in my couch’. The wall
unusual or new and different. For example, participants described appeared to be a safe place where people felt protected and even
the experience as ‘Funny - different –a new experience’; ‘I like to though in a public space, somehow private. ‘Did not mind to hear
try new things, so it was new to me’. Others took some time to my physiology in a public space… Enjoyed to sit in each others
adjust before they felt comfortable: ‘In the beginning it was a new [physiology]…. tried to breathe in the same rhythm and heart

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 59 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
beats also’. Most participants seemed to relax well on the wall massaging a body. The time spent in the physiology zones was
(see Fig. 11). longer when we offered participants to walk around the wall at
their own pace; they were less knowledgeable with vibrotactile
technology and if they had explored the mapping of the actions
done in the wall to the vest.
From the interviews we found that people had very diverse
reactions to the patterns: some requested more activating (Tarzan
or Shiver) but most preferred the calming sensations (waist and
back). Overall, for many participants this was a new experience,
Figure 11. Participants relaxing and listening to their for some daunting at the beginning, others uncertain how to ‘name
heartbeat at the end of the interaction sequence. it’, but a whole body experience and for most an activating and
enjoyable one. There was an obvious connection to the wall as a
5.5 Observations whole body and playful experience and one that had put their
The participants enjoyed exploring and playing with the paired inside body available to the wall in a public space.
vibrotactile interaction. In the Breath rate zone, they played
around with their breathing by holding their breath or doing deep
7. DISCUSSION AND CONCLUSIONS
slow breathing. In the Heartbeat zone, they followed the We designed a vibrotactile vest with physiological monitoring that
suggestion of jumping to increase their heartbeat and then sitting interacts with a vibroacoustic urban environment, The Humming
down to ‘feel-hear’ the heartbeat rhythm slowing down. In the Wall. Different vibrotactile patterns were designed together with a
Swipe and Knock zones, participants played their own little kinesiologist-neurophysiologist for the vest to emulate calming
musical rhythms, running up and down, or trying to touch 4 and activating sensations and to guide or warn the vest wearer.
panels at once and/or generally experimenting with what was Further, actions on the wall are replicated on the vest, and in two
possible and playing together at the wall. areas of the wall the participants ‘feel’ (viboracoustically) and
hear their heartbeat and breath rate. We conducted a field trial
When interviewing the participants, we noted that for many of held over a 5-week period in Utzon Park, a public space in
them, it was difficult to describe the sensation experienced. Aalborg, Denmark. In this work we described the experience of
Additionally, sensations in more than just one area of the body at 39 participants interacting with the vest and the responsive
the same time generally confused people. environment and their engagement with the system.

6. SUMMARY OF FINDINGS The participants had a novel, unusual and generally activating, yet
The results of the first field trial with participants wearing the calming experience. It was very physical, tangible, embodied—
vibrotactile vest and interacting with The Humming Wall were not that they did particularly use those words—but they talked
very positive. Answers to the questionnaires denoted that about feelings of well-being, hunger, goose bumps on the skin and
participants found the experience enjoyable and pleasant. They touch as if it was real enough—‘I like the hand down my spine’.
were highly engaged and concentrated with this novel interaction Overall, they had a very personal experience and people were
with the vest and the environment, as they did not notice the ‘touched’ and often moved by it.
passing of time. In addition, they were activated, motivated but Allowing participants to control their own timing increased the
relaxed and understood what was required of them. As well, time spent and potentially the quality of ownership of the
participants found this a personal, warm, sensitive, sociable, experience. Participants built proprietorship from their speculative
active and open experience. play and experimentation—despite their personal heartbeat and
Crossing demographic information with the questionnaires, we breath rate being made publicly available in the public space.
observed that women reported to have a more personal and warm For this first implementation, delays and unforeseen factors meant
experience than men. Fitter people and particularly those with we pared back and simplified many of our initial planned
average or lower IT knowledge had the best experiences—the interactions in the interests of robustness. Our concerns then were
latter even understanding the sensations better as the trial that the experience would hold a short novelty factor and that
progressed. Further, participants less familiar with vibrotactile there would be ‘not enough to do’—breath rate, heartbeat, swipe,
technology had a more personal experience and enjoyed learning knock—people would be finished in 5 minutes! In addition, we
how to do the tasks suggested by the sensations more than experts had deliberately worked without the visual, (beyond minimal
in this domain. responsive lighting—which many did not notice) in order to
Regarding the patterns and the sensations they elicited from the investigate the potential of vibrotactile interaction. Given the
vest wearer, we found, particularly for males, that some sensations newness for many of our participants, their warm, social
made more sense than others, and this was confirmed across responses, and that they wanted to stay much longer than we had
genders in the video logging and semi-structured interviews. envisaged—or even allowed time for—anything more
Participants responded more positively to Up, Down, complicated could have undermined this initial investigation.
WaistLeftToRight, WaistRightToLeft, (calming patterns) than to We found the nature of the relationships—the intimacy—that the
e.g. Shiver, Tarzan (activating) or Turn Left (navigation) and Stop wall enabled was surprising. Pairs who play well generally found
(warning). a good place to play and discover at the wall. The experience was
For the participants’ relation to The Humming Wall interaction, abstract, there were no real tasks, no game as such, rather a place
most participants experimented with mapping the knocking to the to explore—which people did and made their own fun.
vest wearer’s body and with different types of swipes—emulating

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 60 |
 Vibrotactile Vest and The Humming Wall: “I like the hand down my spine”
A. Morrison | C. Manresa-Yee | H. Knoche
As future work, we will redevelop some of the internal electronics mapping vibrotactile stimulations to fundamental body
and place The Humming Wall in a public environment to conduct movements. In Proc. HAVE’10, IEEE, 13-18.
a longitudinal study including general use. We found a wide array [11] Rosenthal, J., Edwards, N., Villanueva, D., Krishna, S.,
of personal preferences and responses to the sensations in the McDaniel, T., and Panchanathan, S. 2011. Design,
vibrotactile patterns and we need to address these. Calming implementation, and case study of a pragmatic vibrotactile
sensations were clearly more popular and easily understood than belt. IEEE Trans. Instrum. Meas., 60, 1, 114–125.
the activating, warning or guiding ones—but understandably the
later would not necessarily be designed to be as pleasant. People [12] Morelli, T., Foley, J., Columna, L., Lieberman, L., and
were easily confused by sensations in more than one area of the Folmer, E. 2010. VI-Tennis: a vibrotactile/audio exergame
body, so we have work to do on uncovering best placements of for players who are visually impaired. In Proc. FDG '10.
actuators for guiding sensations on e.g. the shoulder and nape of ACM, New York, NY, USA, 147-154.
the back that are more easily understood—yet this is something [13] Yuan, B., and Folmer, E. 2008. Blind hero: enabling guitar
we see and use in everyday life when guiding elders or children. hero for the visually impaired. In Proc. Assets '08, ACM,
Further, it would be interesting to evaluate the system with blind New York, NY, USA, 169-176.
(or blindfolded) users to analyse their perceived sensations when
[14] Tsukada, K., and Yasumura, M. 2004. ActiveBelt: Belt-Type
using the system.
Wearable Tactile Display for Directional Navigation. In
This work demonstrates potential for responsive urban Proc. Ubicomp’04. LNCS 3205, Springer, 384-399.
environments that are nurturing for and assistive to the [15] Van Erp, J. B. F., Van Veen, H. A. H. C., Jansen, C., and
physiological states of their inhabitants. Dobbins, T., 2005. Waypoint navigation with a vibrotactile
waist belt. ACM Trans. Appl. Percept. 2, 2, 106-117.
8. ACKNOWLEDGMENTS
We thank the volunteers for the field trials and Aalborg Kommune [16] Uchiyama, H., Covington, M. A. and Potter, W. D. 2008.
for support. This work is supported by the EU funded project Vibrotactile Glove guidance for semi-autonomous
CultAR (FP7-ICT-2011-9 601139), and mobility grant wheelchair operations. In Proc. ACM-SE 46. ACM, New
CAS12/00199, Programa José Castillejo granted by the Spanish York, NY, USA, 336-339.
MECD. [17] Vaucelle, C., Bonanni, L. and Ishii H. 2009. Design of haptic
interfaces for therapy. In Proc. CHI '09. ACM, New York,
9. REFERENCES NY, USA, 467-470.
[1] Field, T. 2014 Touch. Second ed. MIT press.
[18] Paredes, P. and Chan, M. 2011. CalmMeNow: exploratory
[2] Réhman, S., Liu, L., and Li, H. 2008. Vibrotactile Rendering research and design of stress mitigating mobile interventions.
of Human Emotions on the Manifold of Facial Expressions, In Proc. CHI EA '11. ACM, New York, NY, USA, 1699-
J. of Multimedia, 3,18-25. 1704.
[3] Seifi, H., and MacLean, K. E. 2013. A first look at [19] Arafsha, F., Alam, K. M., and El Saddik, A., 2012.
individuals' affective ratings of vibrations. In EmoJacket: Consumer centric wearable affective jacket to
Proc. WHC’13, 605-610. enhance emotional immersion. In Proc. IIT’12, IEEE, 350-
[4] Chang, A., O’Modharain, S., Jacob, R., Gunter, E. and Ishii, 355.
H. 2002. ComTouch: design of a vibrotactile communication [20] Pares, N., Carreras, A., Durany, J., et al. 2004. MEDIATE:
device. In Proc. DIS '02. ACM, New York, NY, USA, 312- An interactive multisensory environment for children with
320. severe autism and no verbal communication. In Proc.
[5] Krishna, S., Bala, S., McDaniel, T., McGuire, S. and IWVR’04, Lausanne, Switzerland.
Panchanathan, S. 2010. VibroGlove: an assistive technology [21] Dublon, G., and Portocarrerro, E. 2014. ListenTree: Audio-
aid for conveying facial expressions. In Proc.CHI EA '10. haptic display in the natural environment. In Proc. ICAD’14,
ACM, New York, NY, USA, 3637-3642. June 22-25, 2014, New York, NY.
[6] Alves Lino, J., Salem Bernard, B. and Rauterberg, M. [22] Morrison, A., Knoche, H., and Manresa-Yee, C. 2015.
2010. Responsive environments: User experiences for Designing a Vibrotactile Language for a Wearable Vest. In
ambient intelligence. Journal of Ambient Intelligence and Proc HCII’15, LNCS, Springer.
Smart Environments, 2 (4). 347-367.
[23] Zephyr Technology Corporation. 2015.
[7] Bullivant, L. 2006. Responsive Environments: architecture http://zephyranywhere.com/ Last accessed March 10, 2015.
art and desing. Ed. Victoria & Albert Museum.
[24] Vorderer, P., Wirth, W., Gouveia, F.R., et al. 2004. MEC
[8] Morrison, A., Viller, S., and Mitchell, P. 2011. Building spatial presence questionnaire (MEC-SPQ): Short
sensitising terms to understand free-play in open-ended documentation and instructions for application. Report to the
interactive art environments. In Proc. CHI’11, ACM, New European Community, Project Presence. (IST-2001-37661).
York, NY, USA, 2335-2344.
[25] Jackson, S.A., Marsh, H.W. et al. 1996. Development and
[9] Spelmezan, D., Jacobs, M., Hilgers, A. and Borchers, J. validation of a scale to measure optimal experience: The
2009. Tactile motion instructions for physical activities. In Flow State Scale. J. Sport Exerc. Psychol. 18, 17-35.
Proc. CHI’09. ACM, New York, NY, USA, 2243-2252.
Deci, E.L. and Ryan, R.M. 2000. The“ what” and“ why” of goal
[10] McDaniel, T., Villanueva, D., Krishna, S. and Panchanathan, pursuits: Human needs and the self-determination of
S. 2010. MOVeMENT: a framework for systematically behavior. Psychological inquiry 11, 4, 227-268

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (I)

| 61 |
NATURAL USER
INTERFACES (II)
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas

Exploración del busto humano en tiempo real mediante

interacción natural con fines educativos
Roi Méndez Julián Flores Rubén Arenas
CITIUS CITIUS CITIUS
Universidade de Santiago de Universidade de Santiago de Universidade de Santiago de
Compostela Compostela Compostela
[email protected] [email protected] [email protected]

ABSTRACT denominada Chispas compuesta por 13 módulos interactivos, que

En este trabajo se presenta el desarrollo de un sistema interactivo recorren cada una de las 8 inteligencias descritas por Gardner [4]
de visualización del interior del cráneo humano para La Domus y algunos aspectos de la anatomía y fisiología del cerebro. Se
(Casa del Hombre de A Coruña). La interacción del usuario con el plantea crear un contenido interactivo que permita al visitante
mismo se realiza a través de una interfaz natural e intuitiva observar las características anatómicas del cerebro, explorarlo y
(desarrollada ad-hoc) basada en sensores de movimiento Prime visualizar las diferentes partes del mismo, así como los elementos
Sense. Se presentan el ciclo de vida del desarrollo del proyecto orgánicos del busto humano. Por lo tanto, este módulo cubre la
basado en prototipos y por tanto los diferentes prototipos parte de la exposición centrada en la anatomía del cerebro.
implementados así como las pruebas realizadas sobre los mismos
hasta llegar a la implementación final que, a día de hoy, se
2. OBJETIVOS
El objetivo principal de este proyecto es desarrollar un sistema
encuentra operativa en el museo.
que permita la exploración anatómica del cerebro humano
Categories and Subject Descriptors mediante una interacción natural por parte del usuario.
I.3.6 [Computer Graphics]: Methodology and Techniques – Para lograr este objetivo se plantearon dos requisitos funcionales
Interaction Techniques principales:
General Terms • Implementar una herramienta de visualización de
Design, Human Factors. imagen médica (TAC, RMI o similar) que permita
explorar el modelo mediante un plano de corte en los
Keywords tres ejes cartesianos del espacio, pudiendo seleccionar la
Interacción, Kinect, Prime Sense, museo interactivo, anatomía posición del plano y el eje de corte del mismo en cada
humana, interacción natural. momento. De esta forma se podrá visualizar el interior
del cráneo desde una perspectiva cenital, frontal o
1. INTRODUCCIÓN lateral permitiéndole al usuario la exploración del
En los últimos años los museos, en especial los científicos, han modelo desde diferentes ángulos y puntos de vista.
sufrido una gran evolución pasando de meras salas de exposición
a espacios donde el usuario puede interactuar con los elementos • Implementar un sistema de interacción intuitivo, natural
que se muestran. Incluso museos clásicos como el Louvre han y atractivo del usuario con el sistema, evitando el uso de
incorporado elementos como la Nintendo 3DS o el sensor Kinect periféricos clásicos como teclados, ratones y, cada vez
para crear una experiencia más interactiva con el visitante. En este más, pantallas táctiles. Este sistema debe permitir que el
tipo de entornos tiene una importancia capital el hecho de usuario defina en todo momento el plano de corte
mantener la atención del visitante y transmitirle conocimientos de deseado y su posición en el espacio. Igualmente debe de
una forma clara, amena y entretenida, promoviendo un definirse la posición física del usuario respecto al
aprendizaje por experimentación que haga más atractiva la sistema así como las acciones que este debe realizar
información que se desea transmitir. Si, por otra parte, los para un correcto funcionamiento del mismo.
potenciales visitantes son niños debemos desarrollar paradigmas
educativos que permitan difuminar, en la medida de lo posible, la 3. DISEÑO DEL CONTENIDO
frontera entre el aprendizaje y el juego. El paradigma utilizado para el diseño del sistema se basa en la
arquitectura clásica de un expositor de museo. Estos expositores
El caso de estudio se centra en el diseño e implementación de un son urnas en cuyo interior se expone el elemento de interés. En
contenido interactivo para la Domus, Casa del Hombre de A nuestro caso se sustituye la urna por un monitor y se introduce un
Coruña [2], museo interactivo, encuadrado en el conjunto de componente de interacción del usuario con el contenido que
museos científicos de la ciudad, que centra su actividad en permita incrementar el valor expositivo y docente del mismo. De
diferentes aspectos del ser humano como especie, esta forma, el diseño debe tener una arquitectura similar, tal como
proporcionando, entre otras cosas, un recorrido completo a través se muestra en la Figura 1. Está formado por una pantalla en la que
de su anatomía. Se trata de un museo orientado principalmente a realiza la visualización de las imágenes del cráneo y uno o varios
un público infantil y familiar, por lo que ha de tenerse en cuenta el sensores para permitir la interacción natural del usuario con el
aspecto lúdico a la hora de desarrollar la instalación. Esta sistema.
instalación se enmarca dentro de una nueva exposición

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 63 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
motor gráfico orientado al render en tiempo real, decantándonos
por OSG (Open Scene Graph) [11] por tratarse de uno de los más
extendidos y potentes, así como por la posibilidad de integración
que ofrecía tanto con ITK (Insight Segmentation and Registration
Toolkit) [6] como con DCMTK (DICOM Toolkit) [10],
herramientas que permiten leer e interpretar variados formatos de
imagen médica y en especial DICOM, formato que se ha
convertido en un estándar en este campo y que presenta grandes
dificultades de lectura, interpretación y generación debido a su
complejidad interna [1]. Con esta herramienta se obtuvo una
visualización en tiempo real (sin variaciones en la calidad del
renderizado) y con un frame rate estable, condiciones
indispensables para el sistema planteado.
En este punto se planteó la substitución de las imágenes iniciales
en formato DICOM (en escala de grises) por unas equivalentes en
formato png (en color) ya que la información cromática permite
una mejor percepción de los diferentes tejidos y estructuras
presentes en el modelo además de aumentar la espectacularidad de
la visualización. Al disponer del dataset completo del Visible
Human Project [12] (incluidas las imágenes reales en color de los
cortes realizados) y utilizar OSG como motor de render el cambio
fue directo, ya que este motor gráfico permite la importación de
volúmenes como conjuntos de imágenes en diferentes formatos.
Figura 1. Esquema de diseño del módulo Este cambio permitió, además, aumentar la resolución de la
visualización al tener las imágenes en color un mayor detalle que
Puesto que se trata de un contenido interactivo se define una zona su equivalente DICOM. Se hubo de ajustar el tamaño del volumen
de control en la que el usuario puede moverse manteniendo el de entrada para mantener el frame rate dentro de los parámetros
control del sistema, fuera de la cual el usuario perderá el control del tiempo real ejecutando la aplicación sobre una tarjeta gráfica
del mismo. Esto es necesario debido a que, al estar situado el Nvidia Geforce 9600GT.
módulo en una zona con tránsito de visitantes se debe de discernir
cual de los posibles usuarios en la zona cercana al quiosco tiene el
control. Esta diferenciación se realizará mediante la restricción del
funcionamiento del sistema a la zona de control especificada.

4. TECNOLOGÍAS EMPLEADAS Y
DISEÑO HARDWARE
Se realizó un estudio previo de cuáles eran las mejores
alternativas para llevar el proyecto a cumplir sus objetivos de
forma eficiente y satisfactoria. Siguiendo las necesidades básicas
planteadas se realizó un estudio de las diferentes herramientas de
software libre tanto para la visualización como para la interacción
del usuario con el sistema. Como fue comentado con anterioridad
el sistema consta de dos partes: un entorno gráfico que permite la
visualización del modelo y un sistema de interacción para
controlar dicha visualización.
Figura 2. Imagen de un corte procesado.
4.1 Sistema de Visualización Las imágenes en color precisaron de un preprocesado ya que la
Se ha de construir una imagen tridimensional a partir de la zona exterior al cráneo estaba completamente cubierta por la
información contenida en 275 imágenes con una resolución de mezcla de gelatina y agua empleada para la congelación del
512x512 pixels pertenecientes al Visible Human Project [12]. sujeto. A efectos prácticos esto supone que al realizar la
Para realizar este proceso se planteó el uso de un motor gráfico visualización exterior lo que observa el usuario es un bloque azul.
especializado en esta tarea: VTK (The Visualizaton Toolkit)[7]. Para su procesado se definió una zona de interés (en la que estaba
No obstante, los resultados de rendimiento con las imágenes del contenido el busto) y descartar todos los elementos exteriores a
dataset desaconsejaron su uso, ya que los frame rates obtenidos no esta zona. Una vez realizado este proceso se procedió a la
eran estables y caían en ocasiones por debajo del tiempo real eliminación de la región exterior azul mediante un proceso de
necesario para toda aplicación interactiva. Asimismo, la propia umbralización. Al ser la parte anatómica de la imagen una región
implementación del visor de VTK realiza una simplificación del cerrada y al tener los tejidos de la misma una pigmentación bien
modelo cuando está siendo manipulado (cuando la imagen diferenciada de la región a eliminar bastó con seleccionar el color
visualizada no es fija) para evitar caídas en el frame rate, lo que azul y aplicarle una serie de umbrales. El resultado puede verse en
deriva en una visualización poco precisa y en un efecto de la Figura 2. En cuanto a las zonas que contienen mucosas en el
popping cuando se deja de manipular el modelo y se muestra el interior del cráneo (aparecen también azules debido a la
render sin simplificar. Teniendo esto en cuenta se optó por un penetración del líquido de congelación en su interior) se decidió

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 64 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
no procesarlas ya que la pigmentación resultado de la congelación necesidad de usar patrones ni marcadores de ningún
permite a un usuario no experto detectarlas con mayor facilidad. tipo. Emplea cámaras de color e infrarrojos para
El conjunto de datos final está formado por 275 imágenes con una detectar usuarios en su zona de acción y reconstruye
resolución de 1024x608 pixels. automáticamente un esqueleto del mismo, siguiendo sus
movimientos en el espacio.
El proceso seguido a la hora de construir un modelo 3D a partir de
una serie de cortes 2D es el siguiente: Primero se cargan las Teniendo en cuenta que las condiciones lumínicas del museo no
imágenes en memoria. Una vez cargadas se construye una textura podían ser controladas sin afectar a toda la exposición se descartó
3D que se visualizará a través de un proceso de Raycasting. Para el uso de cámaras RGB por los problemas que presentan ante
realizar los cortes requeridos en los tres ejes cartesianos se hace cambios de iluminación. Se descartó la manipulación de un objeto
uso de tres planos de clipping (uno por cada eje) que, combinados como interfaz de control (un mando de consola o elemento
con la textura 3D y el raycasting permiten renderizar todos los manipulable con captura de movimiento) por los problemas de
detalles tanto exteriores como interiores del volumen visualizado. mantenimiento que este podría presentar así como por la
posibilidad de hurto del mismo. Se descartó, por tanto, el uso de
4.2 Sistema de Interacción cámaras de infrarrojos o mandos de control. Se decidió, por tanto
Al tratarse de un museo principalmente dirigido a público infantil emplear sensores PrimeSense para crear una interacción natural
y familiar el diseño de una interacción intuitiva y al mismo tiempo entre el usuario y el sistema a través de los movimientos de su
atractiva resulta indispensable. cuerpo.
En los últimos años han aparecido una serie de sensores El esquema de interacción es el siguiente: El visitante del museo
avanzados que permiten enriquecer, a través una captura de llega al módulo, se sitúa de pie, en una posición indicada
movimiento de bajo coste, la interacción del usuario con el mediante un gráfico en el suelo, de forma frontal a la instalación y
ordenador, creando nuevas interfaces naturales y metáforas de el sensor lo detecta y obtiene de sus gestos la información
acción que añaden una tercera dimensión además de aumentar la necesaria para manipular el sistema. Finalmente, la imagen que se
implicación física del usuario en la interacción con la máquina muestra en pantalla varía siguiendo los gestos realizados por el
[3]. Persiguiendo un control más natural del sistema se plantea la usuario. Para la detección de gestos se decidió optar por un
utilización de estos sensores con el fin de evitar el uso de desarrollo ad-hoc en lugar de emplear gestos estándar ya que se
periféricos tradicionales como el ratón y el teclado buscando entendió que estos se adecuarían mucho mejor a una interacción
simplificar la realización de una serie de tareas por parte del natural y efectiva [9].
usuario. Los tipos de sensorización planteados fueron los
siguientes: 5. IMPLEMENTACIÓN
La implementación del sistema se planteó como un desarrollo por
• Sensorizaciones mediante cámaras de infrarrojos: Se prototipos, se crearon diferentes prototipos y se probaron en
planteó el uso de sistemas de cámaras de infrarrojos sujetos experimentales, hasta obtener una implementación
para sensorizar al usuario. El usuario tendría una intuitiva del sistema, de modo que este a penas necesitase de
herramienta que manejaría en el aire y a través de la instrucciones de uso y la interacción con el mismo surgiese de
cual podría interactuar con el sistema. Esta herramienta forma natural. También se tuvieron en cuenta diversos factores
sería seguida por el sistema y el renderizado de las para evitar la fatiga del usuario [5] como una posición natural y
imágenes 3D se realizaría consecuentemente a esta relajada del mismo durante la interacción, la no necesidad de
interacción. Los principales inconvenientes que llevaron mantener una tensión postural innecesaria en manos o cabeza para
a decartar este tipo de sensorización fue, por un lado, la tener una interacción efectiva, etc.
ausencia de librerías libres y fiables para el tipo de
hardware disponible y, por otro, la necesidad de que el 5.1 Prototipo 1
usuario manejase una herramienta, con las posibilidades La exploración tradicional de modelos 3D se basa en el zoom, la
de hurto y rotura que esto implica. rotación y la traslación [8] convirtiendo la interacción del usuario
• Sensorizaciones mediante cámaras RGB y marcadores: con el sistema (generalmente a través de interfaces 2D) en algo
El extenso uso que el grupo de investigación ha hecho complejo y que requiere de un aprendizaje. Sin embargo, el
de sensorizaciones con webcams empleando marcadores paradigma planteado reduce éste al desplazamiento de planos a
llevó a que se planteara una interacción basada en esta través de los ejes cartesianos, lo que simplifica la interacción al no
tecnología. El problema planteado era la dificultad de, ser dependientes las transformaciones en unos de las realizadas en
con un usuario manipulando un objeto en el aire, evitar otros. Por ser tres los planos de corte planteados, se definieron tres
oclusiones en los patrones, así como obtener la zonas de acción distintas que permiten al usuario operar en cada
configuración óptima de iluminación de la sala sin, con una sobre un plano de corte distinto.
ello afectar al resto de la exposición o, al igual que en el
caso de la sensorización con cámaras de infrarrojos la
presencia del elemento físico manipulable que puede ser
hurtado o que se puede deteriorar con el uso.
• Sensorización mediante el uso de sensores PrimeSense:
Se planteó el uso de sensores PrimeSense como Kinect
o Asus Xtion debido a las grandes posibilidades que
ofrecen y lo novedoso de la interacción que plantean. La
ventaja principal de esta tecnología es que trabaja sin la

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 65 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
• La indefinición entre zonas de acción: Cuando la mano
pasa de una zona de acción a otra la perspectiva del
modelo que visualiza el usuario cambia al modificar el
plano de corte sobre el que actúa, produciendo
confusión al no ser capaz de identificar los límites de las
diferentes zonas de acción, situación a la que contribuye
el tamaño reducido de las mismas.

5.2 Prototipo 2
Dados los problemas que planteaba el prototipo 1 se decidió
implementar dos cambios básicos en la concepción del sistema.
Por un lado se decidió aumentar el recorrido de la mano
ampliando las zonas de acción que pasaron del entorno de la
cabeza a ocupar todo el entorno del tronco (de 25 cm a 50 cm de
recorrido) para permitir al usuario tener un mayor control del
detalle que deseaba ver en cada momento (5,5 imágenes por cada
centímetro de recorrido de la mano). Al ser más grande el rango
Figura 3. Zonas de acción del prototipo 1.
de movimiento de la mano el plano se moverá más lentamente
Se definieron las zonas de interacción que pueden verse en la respecto al movimiento de la misma permitiendo observar detalles
Figura 3 y cada una de ellas se asoció con un eje. Este primer que, de otra forma, serían imposibles de visualizar y reduciendo,
prototipo restringe las zonas de acción al entorno de la cabeza del de esta forma, la intensidad de los temblores en la imagen debidos
usuario asumiendo una metáfora exploratoria basada en que si se al ruido de la señal del sensor.
explora el busto, la forma más simple de interacción para el
usuario era asumir que lo que está visualizando es su propio
cráneo y que, al desplazar las manos en torno a él, puede explorar
su interior. La zona 1 se situó frente al pecho (para evitar poner
las manos delante de los ojos y entorpecer la visión) y permite
realizar cortes en el modelo en el sentido horizontal (eje X). La
zona 2, a la derecha o izquierda de la cabeza, permite realizar
cortes en sentido vertical (eje Y) y la zona 3, situada delante de la
zona 2 permite explorar el interior del busto en profundidad (eje
Z).

El prototipo se probó con 6 usuarios experimentales (entre los que

se encontraban los organizadores de la exposición) que utilizaron Figura 4. Zonas de acción del prototipo 2.
el sistema en base a objetivos (detectar el nervio óptico, visualizar
el cerebelo, etc.). A pesar de las ventajas que presentaba esta Por otro lado, se decidió también hacer una mayor diferenciación
propuesta, como el ser válida tanto para zurdos como para diestros de zonas de trabajo, para que en cada zona espacial sucediese sólo
(las zonas de interacción estaban situadas a ambos lados de la una cosa, buscando simplificar, de esta forma, la comprensión de
cabeza) o lo acertado de la metáfora de interacción que facilitaba qué se manipulaba en cada zona. Se definieron las tres zonas
la comprensión del funcionamiento al usuario, empíricamente se como se muestra en la Figura 4. A la derecha del hombro derecho
pudieron detectar una serie de problemas operativos: se permitía al usuario interactuar con el eje vertical de corte
desplazando su mano en sentido vertical, en la zona delantera del
• Problemas derivados del poco recorrido que tiene el torso y cabeza se interactuaba con el eje horizontal de corte
movimiento de la mano entorno a la cabeza: Ya que el desplazando la mano en sentido horizontal y a la izquierda del
modelo 3D se construye a partir de 275 cortes y el hombro izquierdo, acercando y alejando la mano del mismo, se
desplazamiento de la mano para mover el plano a través realizaba la interacción del usuario con el eje de profundidad de
del mismo se reducía a 25 centímetros, lo que corte.
dificultaba la visualización de pequeños detalles al
avanzar el plano de corte una gran distancia con un Para probar el funcionamiento de este prototipo se realizaron
movimiento sutil de la mano (11 imágenes por pruebas a 7 usuarios no expertos. Estas pruebas estaban
centímetro de recorrido de la mano). compuestas por 9 tareas cronometradas (localizar el nervio óptico,
localizar el bulbo raquídeo, etc.) y un breve cuestionario para
• El ruido en la señal producida por el sensor: El sensor evaluar el nivel de frustración del usuario así como la usabilidad
introducía una gran cantidad de ruido, lo que derivaba del sistema. Las tareas fueron realizadas con éxito por la totalidad
en un movimiento continuo de los planos de corte, de los usuarios, que invirtieron un tiempo medio de 13 segundos
impidiendo una visualización clara de los mismos. Este en realizar cada una de ellas. De las respuestas al cuestionario se
ruido se veía incrementado, además, por la cercanía de dedujo que los usuarios consideraban el prototipo atractivo, así
las manos al cuerpo derivada del tipo de interacción como intuitivo una vez habían interiorizado la mecánica de
propuesta, que dificultaba la labor de seguimiento del funcionamiento del mismo. Sin embargo, en los momentos
sensor. iniciales, se sentían perdidos al no saber qué dirección exploraban
con cada mano, lo que les producía frustración al pasar de una
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)
| 66 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
tarea a otra. También salió a relucir la dificultad de algún usuario posicionamiento de los diferentes elementos del sistema (Figura
diestro para manejar el plano de profundidad (Z) con la mano 5).
izquierda, ralentizando de manera notable su desempeño en las
tareas que requerían el uso de esta vista.
Al ser la primera impresión y el manejo intuitivo del sistema de
capital importancia en un museo interactivo, así como la
accesibilidad y que la aplicación sea usable para el usuario
independientemente de su pericia con su mano menos hábil, se
decidió cambiar radicalmente el enfoque que se estaba dando a la
interacción, eliminando por completo las zonas de actuación
diferenciadas.

5.3 Prototipo 3
Ante los problemas planteados por las zonas diferenciadas de
interacción (frustración en el usuario al no saber que tarea realiza
en cada zona, dificultades de manejo de la aplicación con la mano
menos hábil, etc.) se cambió el enfoque de la interacción
planteada, centrando la misma en la dirección principal de
movimiento de la mano. Esta solución consiste en que la
visualización es modificada y el objeto explorado en la dirección
del eje en que la mano del usuario presente un mayor
desplazamiento. Esta solución plantea tres ventajas principales
respecto a las dos anteriores:
• Permite una diferenciación más intuitiva del eje de corte
que se está utilizando en cada momento y de por qué es
ese el eje sobre el que se actúa al no importar en que Figura 5. Diseño final del sistema con medidas.
parte del espacio se encuentre la mano, sino la dirección
en la que se mueve. 6. PRUEBAS
En la fase de prototipado también se observó la necesidad de dar
• Permite la utilización de ambas manos
un punto visual de referencia para que el usuario supiese en todo
indiferentemente, solo hay que seleccionar cual tiene
momento que plano de corte estaba visualizando y en qué
preferencia en caso de que ambas estén interactuando.
posición respecto al busto se encontraba. Para esto, en una
• Ofrece la posibilidad de tener un mayor rango de primera aproximación se optó por replicar el modelo en
movimiento en los tres ejes, al no estar este limitado por perspectiva y situarlo en el interior de un cubo transparente a
su correspondiente zona acción. través del cual se desplazaban los planos de corte de forma visual
y solidaria con el plano de corte real. Esta visualización, a pesar
Se repitieron las pruebas que se habían realizado en el prototipo de lo vistoso no resultó efectiva para conocer la posición exacta
anterior sobre los mismos 7 usuarios. De nuevo las 9 tareas fueron del plano respecto al modelo (por motivos de perspectiva) por lo
completadas con éxito por todos los participantes no observándose que se decidió optar por una solución más esquemática y que
diferencias notables en los tiempos empleados en uno y otro caso transmitiera la información de una forma más simple y directa. Se
(entorno a los 13 segundos de media) para completarlas. Sin emplearon dos siluetas (una de frente y otra de perfil) que se
embargo si se apreciaron diferencias en las respuestas al visualizan en paralelo a la imagen del modelo 3D. Según el plano
cuestionario. Se subrayaba la facilidad para el manejo del mismo, de corte que se esté explorando en cada momento varía la silueta
lo intuitivo que resultaba y lo preciso que era en el movimiento. que se está visualizando y, dependiendo de la posición del plano
Se señalaba también alguna dificultad a la hora de cambiar de tipo que actúa sobre el modelo, se desplaza una línea a través de la
de vista, pero la valoración general, tanto de usabilidad como de silueta indicándole al usuario qué zona del busto está
atractivo, fue muy positiva eliminando la frustración inicial que si visualizando. Dos flechas indican la dirección en la que se está
habían sentido en la utilización del anterior prototipo. desplazando el plano en cada momento simplificando el control
Se determinó, por tanto que este sería el prototipo final y se pasó a por parte del usuario así como la comprensión del funcionamiento
su implementación final para ser probado con una muestra del sistema (Figura 6).
poblacional más representativa de los futuros usuarios que el Aprovechando una serie de visitas de escolares al centro de
sistema tendría en el museo. investigación y dado el enfoque familiar del museo se realizaron
En esta implementación final se determinó que el sistema una serie de pruebas de uso del sistema en las que participaron 41
reaccionaría a movimientos de las manos de los usuarios cuando estudiantes de entre 8 y 9 años. La prueba consistió en dejarlos
estas estuviesen por encima de la cintura y que, en caso de no ser manipular libremente el sistema sin ponerles límite de actuación o
así, el sistema permanecería en reposo. Asimismo se decidió que tiempo y una puesta en común posterior de la experiencia. Se debe
en caso de interacción de las dos manos al mismo tiempo se le de subrayar que el tiempo de utilización no fue totalmente libre al
daría preferencia a la derecha por ser la de mayor frecuencia encontrarse la prueba en medio de una visita que debía de cumplir
poblacional. También se definieron las medidas y unos tiempos aunque sí que es un escenario que se puede asemejar
a una visita de escolares en grupo al museo en la que hay un
tiempo limitado y numerosas instalaciones para visitar.
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)
| 67 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
El tiempo medio que estuvieron interactuando con el sistema fue Más allá de los datos obtenidos, los escolares afirmaron sentir una
de 61 segundos de los cuales un 75% del tiempo lo hicieron mayor curiosidad por el sistema que si hubiese sido manejado a
utilizando la mano derecha, el 11% la mano izquierda y un 14% través de un periférico clásico y que la interacción les había
no realizando ninguna actividad. Estos datos confirmaron que ante resultado más amena. Insistiendo repetidamente en que les habría
la utilización del sistema los usuarios se mantenían activos y gustado probar un sistema similar a través del que se pudiese
probando sus funcionalidades. Destaca en este aspecto el uso de la observar el cuerpo humano al completo.
mano izquierda ya que solo un usuario era zurdo. El porcentaje
del 11% se explica a través del intento de los escolares de probar
todas las posibilidades del sistema “retándolo” a funcionar con las
dos manos.

Figura 7. Resultados del proceso de pruebas en porcentajes de

tiempo.
Desde el punto de vista de una supervisión visual exterior se
puede afirmar que, en general, este tipo de interacción atraía la
atención de los usuarios pero, en ciertos momentos esta
interacción se convertía más en una finalidad que en un medio,
ocupando el juego de “retar” a la máquina con movimientos
rápidos o ondulantes una mayor parte de la atención del niño que
la visualización propiamente dicha.

Figura 8: Instalación final en el museo

7. CONCLUSIONES Y TRABAJO FUTURO

Figura 6. Diferentes visualizaciones presentes en la aplicación Se ha desarrollado un sistema de visualización y exploración del
final. cerebro humano a partir de una visualización basada en texturas
Dentro de la fase de actividad (ya sea con la mano derecha o la 3D, controlado de una manera natural e intuitiva a través de una
izquierda) los usuarios interactuaron con la vista frontal (eje Z) un interfaz natural para su exhibición en el museo de divulgación
29% del tiempo, con la vista superior (eje Y) un 40% del tiempo y científica Casa Domus. El funcionamiento del sistema se ha ido
con la vista lateral (eje X) un 31% (Figura 7). Destaca probando a lo largo de su desarrollo por prototipos y se ha
ligeramente la vista superior al ser la que posee una mayor validado en una prueba final con usuarios potenciales del mismo.
resolución, al tratarse de los cortes originales a partir de los cuales El sistema ha sido instalado en el museo, con gran aceptación por
se realizó la reconstrucción del modelo 3D. Esta mayor nitidez se parte de los organizadores y visitantes de la exposición.
refleja en un mayor interés de los usuarios por los pequeños
detalles visibles en ese eje. Como trabajo futuro se plantea la mejora del cambio de plano de
corte utilizado en cada momento, ya que, debido al ruido de la

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 68 |
 Exploración del busto humano en tiempo real mediante interacción natural con fines educativos
R. Méndez | J. Flores | R. Arenas
señal del sensor se descartan los pequeños cambios en la dirección Consumer Electronics (ICCE), 2011 IEEE International
del movimiento para esta finalidad, dificultando, en ocasiones, la Conference on, 269–270, 2011.
interacción del usuario con el sistema. [6] Kitware. Itk - segmentation and registration toolkit, 2014.
También se plantea la necesidad de introducir información visual Retrieved January 15, 2013 from Kitware.
y auditiva sobre los diferentes órganos y tejidos que se encuentran http://www.itk.org/
en el busto, indicando su nombre, una breve explicación de su [7] K. Martin, W. Schroeder, and B. Lorensen. Vtk - the
funcionalidad etc. Esto incrementaría, tanto el valor divulgativo visualization toolkit, 2015. Retrieved November 13, 2012
de la instalación, como el interés del usuario. from Kitware. http://www.vtk.org/
8. AGRADECIMIENTOS [8] S. A. Lacolina, A. Soro, and R. Scateni. Natural exploration
Domus, Casa del Hombre de A coruña. of 3d models. In Proceedings of the 9th ACM SIGCHI Italian
Chapter International Conference on Computer-Human
9. REFERENCIAS Interaction: Facing Complexity, CHItaly, 118–121, New
[1] W. D. Bidgood and S. C. Horii. Introduction to the York, USA, 2011.
ACR_NEMA DICOM standard. RadioGraphics, 12(2):345– [9] M. Nielsen, M. Storring, T. Moeslund, and E. Granum. A
355, 1992. procedure for developing intuitive and ergonomic gesture
[2] Domus | Museos Científicos Coruñeses (=mc2), 2015. interfaces for hci. In Antonio Camurri and Gualtiero Volpe,
Retrieved April 15, 2015 from Museos Científicos editors, Gesture-Based Communication in Human-Computer
Coruñeses (Ayuntamiento de A Coruña). Interaction, volume 2915 of Lecture Notes in Computer
http://mc2coruna.org/es/ Science, pages 409–420. Springer Berlin Heidelberg, 2004.
[3] R. Francese, I. Passero, and G. Tortora. Wiimote and kinect: [10] Offis. Dicom software made by offis - dcmtk - dicom toolkit,
Gestural user interfaces add a natural third dimension to hci. 2014. Retrieved January 21, 2013 from Offis.
In Proceedings of the International Working Conference on http://dicom.offis.de/index.php.en
Advanced Visual Interfaces, AVI ’12, 116–123, New York,, [11] Openscenegraph, 2014. Retrieved January 15, 2013 from
USA, 2012. ACM. Openscenegraph. http://www.openscenegraph.org/
[4] H. Gardner. Frames of mind: The theory of multiple [12] V. Spitzer, M. J. Ackerman, A. L. Scherzinger, and D.
intelligences. New York: Basic Books, 1983. Whitlock. The visible human male: A technical report.
[5] Y. Kim, G.A. Lee, D. Jo, U. Yang, G. Kim, and J. Park. Journal of the American Medical Informatics Association,
Analysis on virtual interaction-induced fatigue and difficulty 3:118–130, 1996.
in manipulation for interactive 3d gaming console. In

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 69 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso

Performance evaluation of gesture-based interaction

between different age groups using Fitts’ Law
Diana Carvalho Maximino Bessa
INESC TEC and UTAD INESC TEC and UTAD
4200-465 Porto, Portugal and 4200-465 Porto, Portugal and
5000-801 Vila Real, Portugal 5000-801 Vila Real, Portugal
+351 {222094000, 259350000} +351 {222094000, 259350000}
[email protected] [email protected]

Luís Magalhães Eurico Carrapatoso

Centro ALGORITMI, UMinho INESC TEC and FEUP
4800-058 Guimarães, Portugal 4200-465 Porto, Portugal
+351 253510180 +351 222094000
[email protected] [email protected]

ABSTRACT Keywords
The recent advances made in human-computer interaction have HCI, natural user interfaces, gestural interaction, performance,
allowed us to manipulate digital contents exploiting recognition- Fitts' law, Microsoft Kinect, Leap Motion, target acquisition,
based technologies. However, no work has been reported that selection tasks.
evaluates how these interfaces influence the performance of
different user groups. With the appearance of multiple sensors and 1. INTRODUCTION
controllers for hand gesture recognition, it becomes important to For many years, the traditional mode of interaction with
understand if these groups have similar performance levels computers was based on a WIMP interface (Windows, Icons,
concerning gestural interaction, and if some sensors could induce Menus, Pointing device), which allowed us to interact with the
better results than others when dealing with users of different age machine via specific pointing devices, usually a computer mouse.
brackets. In this respect, it could also be important to realize if the With the paradigm shift to Post-WIMP interfaces [5], we have
device’s sensor accuracy in terms of hand / full body recognition turned towards a user-oriented and task-oriented approach that
influences interaction performance. We compare two gesture- attempts to simplify the usability of the interface [17], giving
sensing devices (Microsoft Kinect and Leap Motion) using Fitts’ preference to the users innate skills [16] and allowing us to take
law to evaluate target acquisition performances, with relation to advantage of recognition-based technologies that understand
users’ age differences. In this article, we present the results of an human behaviors, such as gestures or touch. However, there
experiment implemented to compare the groups’ performance seems to be a constant addition of new modes of interaction
using each of the devices and also realize which one could yield without the proper awareness as to which could be the most
better results. 60 subjects took part in this study and they were adequate for different user profiles (e.g., children, elderly users,
asked to select 50 targets on the screen as quickly and accurately people with different levels of digital literacy, people with
as possible using one of the devices. Overall, there was a disabilities) and also regarding which types of tasks (selection,
statistically significant difference in terms of performance insertion, manipulation).
between the groups in the selection task. On the other hand, users’
A previous study of ours [3] presented insights on how users
performance showed to be rather consistent when comparing both
interact using different input modalities and which interface holds
devices side by side in each group of users, which may imply that
the best results in terms of usability testing regarding three user
the device itself does not influence performance but actually the
groups with different age brackets. Our preliminary findings
type of group does.
indicated that the gestural interface presented worse results than
Categories and Subject Descriptors the other ones, and we thus acknowledged that the device itself
used for the gesture-recognition – the Kinect – could be
H.5.2 [Information Interfaces and Presentation (e.g., HCI)]: negatively influencing the user due to the accuracy required by the
User Interfaces – evaluation/methodology, input devices and task. Different optical sensors, which allow human body
strategies (e.g., mouse, touchscreen). acquisition with respected accuracy, have been released and
comparable controllers in the same price range include the Kinect
General Terms and Leap Motion, a sensor with declared sub-millimeter precision
Performance, Design, Human Factors. that claims to obtain higher levels of accuracy than the Kinect
[24]. Therefore we think that it may be important to understand if
another device could perform better than the one used in the
previous experiment. After all, when it comes to gestural
interfaces the precision of the sensor is said to be vital.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 70 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso
In this context, this study aims at throwing some light on the 3. CASE STUDY
indices of performance of three groups of users with different age- The aim of this work is to compare two different gestural input
brackets concerning the use of two different gestural sensors. We devices in order to understand if different age groups display
intended to understand: (1) if the gestural interaction presents similar indices of performance between them, and which device
significantly better results on specific groups opposed to others could lead to better target acquisition performance rates for
with regards to their age; (2) if the devices used and their declared different user groups, using Fitts’ law.
precision influence or not the reaction times and indices of
performance in each group. To accomplish this, we compared the use of Microsoft Kinect and
Leap Motion Controller for motion and gesture control with users
2. SCIENTIFIC BACKGROUND from three distinct age groups: children, young adults and older-
adults. A total of 60 participants who were naïve to the purpose of
2.1 Natural Interfaces the experiment took part in this case study, and were grouped by
Active and passive input modes of interaction [19] have been age and device: (1) 20 children from 9 - 12 years old (10 for each
implemented in countless fields. However, published studies have device); (2) 20 young adults from 20 - 35 years old (10 for each
not yet provided an understanding of how different user groups device); (3) 20 older-adults from 45 - 60 years old (10 for each
perceive distinct tasks and if their performance is directly device).
influenced by the interaction modality.
We intended to emphasize the distinct age groups and work with
Little is known about how the different interfaces affect one’s users that had noticeable differences concerning cognitive
performance when it comes to age-related issues. There are no performance levels and dexterity and, as such, we tried to select
transversal comparisons of different age groups in one same study distinct groups. First, we did not consider users younger than 9
where more than one natural interface is evaluated. Work has years old because: (1) they are not so used to the computer and do
been developed in this area, but not as a systematic approach. not use it on a daily basis, and also they do not have the dexterity
Sambrooks and Wilkinson compare gestural, touch and mouse capabilities as improved as older children; and (2) these ages are
interaction with 15 participants aged between 22 and 34 years old encompassed by the fifth and sixth year of primary school, being
[21]. They reached the conclusion that touch and mouse presented these school years the ones that show greater variance in terms of
better results, but this interaction performance was not compared reasoning by the children [12]. As this variance tends to
between other groups of users and thus it just clarified that the significantly fade towards high school, we selected graduate
gestural performance was indeed worse than the other interfaces students, being these users already at a different stage concerning
regarding that specific niche of participants. Other studies, as [2, cognitive performance when compared with the group of children.
6, 9], compared the interaction performance when using Finally, the group of older-adults consisted of active workers of a
traditional mouse inputs or touchscreens, but not between other secretariat department. This group could present worse dexterity
natural recognition-based interfaces as a hole, and they only or consider gestures to be more challenging.
compare at the most two groups.
Also, we ascertained that all of the participants had the same level
Concerning the gestural controllers, tests have been performed in of computer proficiency and used the computer on a daily basis.
order to understand the accuracy of any given device [20, 24], but However, when it comes to gestural interaction, this modality was
they have not been compared amongst different user profiles, such not familiar to the participants, who were conscious of this
as distinct age groups. gesture-based interface mainly due to having seen gaming
consoles with support to this modality. In fact, only seven users
2.2 Fitts’ Law: An Overview had already tried gestural interaction once.
Fitts’ law is a human performance model that has been widely
applied to describe the relationship between movement time, The majority of participants were right-handed (right-handed: 58;
distance and accuracy within specific pointing tasks, as a way to left-handed: 2), and the setup was calibrated to accommodate each
formally evaluate pointing devices [15] and compare their arm as needed: the left-handed participants used the left arm to
efficiency [23]. The original experimental paradigm Fitts’ Law [7] perform the tests, and the right-handed used the right arm.
measures the rate of transmission in “bits per second” of a pre-
established movement with different indices of difficulty. Also, 3.1 Apparatus
ISO 9241-9 “Ergonomic requirements for office work with visual We conducted the experiment in a closed room with artificial light
display terminals (VDTs) -- Part 9: Requirements for non- and the tests were performed in a specific setup assembled for the
keyboard input devices” [10, 11] provides a standard approach for purpose of this research. The system consisted on a 22’’ screen
input device evaluation based on Fitts’ law. Fitts’ performance placed on a desk in front of the user, with a resolution of
model can also be applied when comparing and evaluating 1280x800 pixels; a Microsoft Kinect sensor mounted on a tripod
pointing devices. Task differences, selection techniques, response behind the screen, about 25 cm above it and facing the user; a
irregularity and test conditions applied may influence Leap Motion sensor placed on top of the desk, between the user
experimental variations. Understanding these variables increases and the screen, and facing upwards. The distance between the user
the validity of across-studies comparisons regarding input and the Kinect sensor was about 75 cm, and was calibrated to see
techniques [15]. In this sense, researchers use this model to the user’s upper body only. The user was seated facing the setup
measure multiple movement times and then determine how the at all times, and this setup was not altered throughout the different
different conditions or devices affect the coefficients of the trials. Also, a purpose-built application based on Fitts’ law was
equation, that is, performance. developed in Python with the support of: the Kivy Framework,
TuioKinect and the Leap Motion Python API.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 71 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso

3.2 Experiment Design

The experiment made use of an independent-measures design and
the tasks were performed in a controlled environment. Aside from
the discrete data collected during the tests, we also gathered
qualitative observational analysis on the participants’ behavior. At
the beginning of each phase of this study, we performed a
questionnaire regarding the participants’ previous experience with
gestural interfaces. Also, at the end of each test we proposed a
questionnaire with qualitative Likert Scales [13], in order to Figure 1. Feedback for the selection strategy.
understand the users’ preferences and their views regarding: ease
of use of the device in question, fatigue effect, level of user
comfort / frustration, and users’ degree of concentration. 3.3 Procedure
We analyzed the users’ performance considering only movements
The tests were performed on different days and with different along the “x” axis and used a reciprocal one-dimensional pointing
users. As such, the participants only tried one of the devices, and task based on Fitts’ original experiments [7]. This consisted on a
not both of them, as we did not intend for the use of one device to horizontal movement between two vertical bars displayed on the
influence the movement times recorded on the other due to screen, representing an initial location and a target (as shown on
performance improvement over trials. Figure 2). According to MacKenzie [15], three-dimensional
In order to understand the users’ index of performance when movements may follow the same predictive model as a one-
considering target acquisition tasks using gestural interaction with directional task, as a possible substitute for target width when the
the Kinect sensor versus Leap Motion, we resorted to Fitts’ law angle of the approach varies can be the distance through the
evaluation paradigm, a frequently used model for measuring targets along the approach vector. However, we only intend to
movement performance. Fitts quantifies the movement tasks' understand the one-dimensional movement with homogeneous
difficulty, also known as Index of Difficulty (ID), by the metric shapes.
"bits" and calculates it by using the value of the distance, or
amplitude (A), between two specific targets and their width (W)
or tolerance area:
ID = log2(A/W + 1). (1)
Although not in its original form, this formulation is a more stable
model for the Fitts’ Law suggested by Mackenzie [15], since it is
resilient to negative values. The Index of Difficulty can be
obtained by varying the values of the width of the targets and their
distance, thus providing a range of task difficulties. We used six
different amplitudes: A = 200, 460, 750, 770, 930, 940 pixels; and
five different target widths: W = 16, 26, 50, 120, 200 pixels.
These choices attain seven levels of Fitts' Law’s index of
difficulty, from 1.72 bits to 5.20 bits. The order of appearance of
the seven indices of difficulty was randomized. We also included Figure 2. Layout of the targets displayed on the screen
two target directions (left, right) in the study because it could be
relevant to understand if the contactless motion was influenced by
different directions [4, 8]. Each test proceeded as follows. At the beginning of the test, the
participant would be required to select a “start” button shown on
In summary, the target acquisition data consisted of: 60 the screen, therefore maintaining a consistent original position.
participants divided in three groups; two input devices (Microsoft Afterwards, two vertical bars would be revealed: a grey bar
Kinect; Leap Motion); two target directions (left, right); 25 representing the initial position, and the green bar being the
repetitions per index of difficulty; and seven indices of difficulty intended target to be selected. Participants then selected the target
(1.72, 2.25, 3.12, 3.14, 4.04, 4.89, and 5.20 bits). Thereby, bar in each successive trial, which was at all times organized in
altogether there were 28 experimental conditions and an overall of the opposite direction (from left to right and from right to left).
42 000 trials. Also, there was a visual feedback to indicate the gesture position
The gestural-based cursor illustration was a target badge (Figure was on top of the target object: the bar changed its color to blue.
15) grounded on the validated “point and wait” strategy [22] for The start time was recorded from each target bar to the next and
selection. This cursor was shown on the screen when a pointing stopped when the next target was selected. At the end of all of the
gesture was interpreted by the system. The center of the target trials, the bars disappeared and a “successfully completed task”
badge represented the point from which the selection coordinates message appeared on the screen.
were referenced by the application. When the system detected the Participants were given a practice attempt before starting the test.
gesture to be motionless, a visual feedback was triggered to make This period consisted in completing a continuous task [15] with
the user aware of the selection progress: a green circle started 10 consecutive trials, in order for the participant to feel at ease
growing inside the cursor representation until it reached the whole with the interaction device.
target and thus the selection was made. Here, the user had to point
and wait for 0.8 seconds for the system to recognize the intention.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 72 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso

3.4 RESULTS AND DISCUSSION pronounced as the children, being the Kinect the one with the
In this section, we present the results of this case study. We highest movement times (Kinect, average rank of 13.20; Leap
believe that we may be able to throw some light on how or if: (1) Motion, average rank of 7.80). Also, we determined a statistically
the different user groups exhibit significant differences in terms of significant difference when comparing the older-adults’
interaction performance when compared between each other; (2) interaction using both sensors (U = 21, p = .028), where the Leap
the accuracy of the devices influences gestural interaction when it Motion remains faster (Kinect, average rank of 13.40; Leap
comes to target acquisition tasks, concerning users of different age Motion, average rank of 7.60). This may indicate that according to
brackets. the movement times all the groups display lower results when
interacting with the Leap Motion sensor than with the Kinect,
3.5 Movement Times and Errors which may imply that for gestural interaction concerning selection
We present the movement times recorded during each trial of the tasks, the Leap Motion could induce quicker reaction times
experimental tests with the Kinect and Leap Motion devices regardless of the users’ age.
(Figure 17). A boxplot analysis of the data revealed a presence of Overall, considering a comparison between the three groups, the
outliers from standardization failure [18] on some of the young adults presented the lowest mean time with both devices,
movement times regarding the calibration of the equipment, and which may indicate that when it comes to gestural interaction the
thus these results were removed in order to prevent distortion of young adults are quicker in completing target acquisition tasks
estimates. than the other groups. Since our group sample was limited to 10
In terms of movement times recorded, the device that registered participants for each group and each device and some data were
the fastest mean results throughout all the three groups was the not normally distributed, a non-parametric independent-samples
Leap Motion: children (1.94 s); young adults (1.76 s); and older- Kruskal-Wallis 1-way ANOVA was run to determine whether this
adults (2.24 s). On the other hand, the Kinect sensor registered difference in terms of mean times throughout the groups was
higher mean movement times: children (2.46 s); young adults indeed significant. The distribution of the times of all the groups
(2.03 s); and older-adults (2.64 s). using the Leap Motion was considered the same and thus not
statistically different (χ2(2) = 5.635, p =.060), presenting a mean
rank time of 15 for the group of children, 11.10 for young adults,
and 20.40 for older-adults. On the other hand, considering the use
of the Kinect, the Kruskal-Wallis test showed that except for the
children and older-adults (p = .875), there was a statistically
significant difference in mean times between the different groups
(χ2(2) = 12.742, p =.002), with a mean rank time of 17.30 for the
group of children, 7.75 for young adults, and 21.45 for older-
adults. Hence, the difference of movement times using of the
Kinect was statically significant between the group of children
versus young adults (p = .046), and the group of young adults
versus older-adults (p = .002).
In this sense, when using the Leap Motion, the groups did not
present statistically different results concerning movement times,
which may suggest that their reaction times are not that disparate.
However, when using the Kinect sensor the groups show
inconsistency with regard to movement times.
Another aspect that we looked at was error rate. We assumed a
Figure 3. Mean movement times of each group with both speed–accuracy tradeoff approach [25] when dealing with errors,
devices as the selection rested on the “point and wait” strategy and thus
did not trigger false selections caused by user mistakes. The more
precisely the task was performed, the longer it took to be
We assessed the normality of data with the Shapiro-Wilk Test in completed, and vice-versa. Here, the selection would only be
order to understand if the data were normally distributed and triggered when the system detected the gesture to be motionless
could thus be considered for statistical analysis. We elected this and precisely on top of the target bar. Otherwise, no selection was
test over the Kolmogorov-Smirnov due to our narrow sample. As made.
we found that the data were not normally distributed across all of
the groups and since we were working with small samples, we 3.6 Indices of Performance
performed a non-parametric Mann-Whitney U test to understand Figure 18 shows the Index of Performance achieved by the three
if the devices presented significantly different results for each groups for each interaction device. The chart clearly shows the
group in terms of movement time. Indeed, every group presented children as having the worst performance of all the groups, either
a statistically significant difference between the use of each with the Kinect (IP = 1.64 bits/s) or with the Leap Motion (IP =
device. In this regard, the group of children revealed slower 2.03 bits/s); followed by the young adults with an IP of 2.98 bits/s
movement times using the Kinect sensor, with an average rank of with the Kinect, and 2.47 bits/s with the Leap Motion. The group
13.80, while the Leap Motion had an average rank of 7.20, being of older-adults presented the highest throughput regarding
this difference significant (U = 17, p = .013). Concerning the gestural interaction for target acquisition tasks with both devices:
group of the young adults, there was also a statistically significant the Kinect with an IP of 3.03 bits/s and the Leap Motion with 3.25
difference between the devices (U = 23, p = .041), but not as bits/s.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 73 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso
A Shapiro-Wilk Test showed us that the data were not normally 3.7 Participants’ Behavior and Feedback
distributed and, as such, we performed a Mann-Whitney U test to During the experiment, participants shared the same behavior
understand if there were significant differences between each concerning both devices. They were attentive to the task: trying to
group’s performance concerning the device used. None of the maintain a certain pattern with the arm’s motion and trying to
groups presented a significant difference between the performance avoid any sudden movements. The most perceptible behavior we
using each device: children (U = 34, p = .226); young adults (U = noticed was that some of the participants would push the arm
38.50, p = .384); older-adults (U = 44, p = .650). foreword in order to try to elevate the target badge on the screen,
instead of lifting it upwards. This situation would lead the
participants to a higher level of frustration with either device
because the badge would not go up unless the arm was raised.
Also, at the end of the experiment the participants answered a
questionnaire about their preferences. We discuss below the
responses with the majority of the preferences. In terms of ease of
use, 50% of the children focused their choice in that the Kinect
was relatively difficult to use, being the other answers scattered
throughout the Likert scale. In contrast, the Leap Motion had 37%
of the children saying it was relatively easy. The older-adults also
shared this opinion (40% signed the Kinect as relatively difficult;
and 45% said the Leap Motion was very easy). Although the
results do not show this tendency, some participants felt that the
Leap Motion sensor seemed easier to use than the Kinect. The
young adults stated that both devices were very easy to use (50%
of the responses regarding each device).
On the other hand, in terms of ease of learning and improvement,
Figure 4. Groups’ Index of Performance concerning both both children and young adults thought both devices were
devices relatively easy, but the older-adults argued that the Leap Motion
was easier to learn than the Kinect. Regarding fatigue, all of the
groups thought both devices were relatively demanding except the
An independent-samples Kruskal-Wallis 1-way ANOVA test was older-adults’ group that considered the Leap Motion less
also run to determine if there was a statistically significant challenging than the Kinect. Finally, all groups considered the
difference in terms of performance between the groups. Kinect as a device that requires further concentration to complete
Considering the Kinect, the performance of the children versus the trials, as opposed to the Leap Motion that was simpler in terms
young adults (p = .031) as well as versus older-adults (p = .013) of attention to the task.
was significantly different. Contrarily, the group of young adults
and older-adults did not register significant differences (p = 4. CONCLUSIONS AND FUTURE WORK
1.000). On the other hand, considering the Leap Motion sensor, This investigation was intended to understand (1) if there are age-
the only significant difference presented was between the groups related differences regarding gestural selection tasks when using
of children versus older-adults (p = .034), and not between one device to the detriment of the other; (2) if two different
children and young adults (p = .824) or young adults versus older- gestural sensors, the Kinect and the Leap Motion, could
adults (p = .453). differently influence the users’ interaction performance and target
acquisition times.
We might reach the conclusion that, in terms of performance,
neither the Kinect nor the Leap Motion sensor directly influences We are aware that conducting further tests may help ascertain
gestural interaction. However, they appear to have different whether the devices do truly influence the participant’s movement
results regarding users of different age brackets, which implies times and respective index of performance. Also, it is possible that
that these sensors could affect accuracy of interaction depending by continuing the experiment the groups’ index of performance
on the user’s age. The group that revealed the best performance would increase, and we therefore need to understand if practice
results was the older-adults’ one, which has already shown to could be considered as a way to attenuate the learning curve, and
have worse results when it came to reaction times during selection consider running more tests to set aside potential hardware
tasks. Here, we may infer that since other pointing devices like the problems on detection that could be negatively affecting accuracy.
mouse or even touch inputs may request more accuracy than
gestural commands, these allow older-adults to overcome some Nevertheless, we acknowledge it appears to be evidence that for
problems of dexterity that may exist in this age bracket, and thus selection and target acquisition tasks a gestural interface may not
demonstrate improved performances over other groups of users. be the best approach, as it shows low indices of performance
However, we will not further explore this assumption on the throughout the three groups. We may presume that for selection
course of this paper. In contrast, children may need more time tasks that require a higher level of precision neither device
practicing to overcome the learning curve to better understand the displayed proof that it influenced the user to complete the trials
relation between the physical environment of the gesture itself and with better performance. In fact, the devices’ accuracy did not
its visual feedback. appear to have a direct relation to improved performance.
As stated earlier, mean movement times should not be considered
the only variable when exploring the users’ performance, and here

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 74 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso
we can relate to why: for example, when comparing the finger, stylus and mouse. International Journal of Human-
movement times and respective index of performance, children Computer Studies. 70, 3 (Mar. 2012), 218–233.
exhibited mean times similar to the other groups, but not nearly as [5] Dam, A. van 1997. Post-WIMP user interfaces. Commun.
good performance as the others. Children have as good response ACM. 40, 2 (1997), 63–67.
times as other groups, but they do not present a consistent
interaction, which may reveal that they are faster but not as [6] Findlater, L. et al. 2013. Age-related differences in
precise when interacting with gesture-based interfaces. On the performance with touchscreens compared to traditional
other hand, the group of older-adults was indeed the one that mouse input. Proceedings of the SIGCHI Conference on
exhibited the best results concerning performance with both Human Factors in Computing Systems - CHI ’13 (New York,
devices. However, it was also the one group to present higher New York, USA, Apr. 2013), 343.
movement times, which may indicate that they have a consistent [7] Fitts, P.M. 1954. The information capacity of the human
interaction. Regardless of their movement times being not as fast motor system in controlling the amplitude of movement.
as the other groups, they tend to be more precise throughout the Journal of Experimental Psychology. 47, 6 (1954), 381–391.
several levels of difficulty of each set of trials.
[8] Goldberg, K. et al. 2013. A New Derivation and Dataset for
This is a work in progress, and more tests will be conducted to Fitts’ Law of Human Motion.
further attest these findings, but we may suggest that: (1) in terms [9] Inai, Y. 2013. Age-related differences in pointing movements
of index of performance the groups displayed statistically in restricted visual tasks and their design implication. 2013
different results: the group of older-adults held the best gestural ICME International Conference on Complex Medical
performance results, and the children the worst, when compared Engineering (May 2013), 439–443.
between each other; (2) both devices behave in a similar manner
for selection tasks and there are no statistically significant [10] ISO 2007. ISO 9241-400:2007 - Ergonomics of human--
differences concerning their accuracy when comparing each of the system interaction -- Part 400: Principles and requirements
three groups individually. However, in terms of user preference, for physical input devices: International Organization for
some participants felt that the Leap Motion sensor was easier to Standardization.
use and less demanding in terms of concentration than the Kinect. [11] ISO 2000. ISO 9241-9 International standard: Ergonomic
requirements for office work with visual display terminals
Aside from more tests to be conducted, in the future we also
(VDTs) -- Part 9: Requirements for non-keyboard input
intend to broaden our research to other elemental tasks (as
devices: International Organization for Standardization.
insertion and manipulation) using gestural interfaces and observe
performance considering our targeted age groups. Also, we intend [12] Lemos, G.C.E.M.P. de 2006. Habilidades cognitivas e
to follow other evaluation methods and test the 7 measurements of rendimento escolar entre o 5.o e 12.o anos de escolaridade.
McKenzie [14] and Steering Law [1] for tracking performance Universidade do Minho.
evaluation for these pointing devices. [13] Likert, R. 1932. A technique for the measurement of
attitudes. Archives of Psychology. 22 140, (1932), 55.
5. ACKNOWLEDGMENTS
We would like to acknowledge the support and contribution of [14] MacKenzie, I.S. et al. 2001. Accuracy measures for
“Universidade de Trás-os-Montes e Alto Douro” and the schools evaluating computer pointing devices. Proceedings of the
that took part in our study: “Monsenhor Jerónimo do Amaral”, SIGCHI conference on Human factors in computing systems
“Escola Secundária Morgado de Mateus”, and the studies center - CHI ’01 (New York, New York, USA, Mar. 2001), 9–16.
“Super-Heróis”, all in Vila Real, Portugal. [15] MacKenzie, I.S. 1992. Fitts’ Law as a Research and Design
Diana Carvalho has a PhD fellowship granted by FCT – Fundação Tool in Human-Computer Interaction. Human-Computer
para a Ciência e a Tecnologia (SFRH/BD/81541/2011). Interaction. 7, 1 (Mar. 1992), 91–139.
[16] Mayer, R.E. and Moreno, R. 2003. Nine Ways to Reduce
6. REFERENCES Cognitive Load in Multimedia Learning. Educational
[1] Accot, J. and Zhai, S. 1999. Performance evaluation of input Psychologist. 38, 1 (Mar. 2003), 43–52.
devices in trajectory-based tasks. Proceedings of the SIGCHI
[17] Nielsen, J. 1993. Noncommand user interfaces.
conference on Human factors in computing systems the CHI Communications of the ACM. 36, 4 (Apr. 1993), 83–99.
is the limit - CHI ’99 (New York, New York, USA, May
1999), 466–472. [18] Osborne, J.W. and Overbay, A. 2004. The power of outliers
(and why researchers should always check for them).
[2] Bobeth, J. et al. 2014. Tablet, gestures, remote control? Practical Assessment, Research & Evaluation. 9, 6 (2004).
Proceedings of the 2014 ACM international conference on
Interactive experiences for TV and online video - TVX ’14 [19] Oviatt, S.L. 2007. Multimodal Interfaces. The Human-
(New York, New York, USA, Jun. 2014), 139–146. Computer Interaction Handbook: Fundamentals, Evolving
Technologies and Emerging Applications, Second Edition. A.
[3] Carvalho, D. et al. 2014. Different interaction paradigms for Sears and J.A. Jacko, eds. CRC Press. 413–432.
different user groups: an evaluation regarding content
selection. Proceedings of the XV International Conference on [20] Ren, Z. et al. 2011. Robust hand gesture recognition with
Human Computer Interaction - Interaccion’14 (Puerto de la kinect sensor. Proceedings of the 19th ACM international
Cruz, Tenerife, Spain, Sep. 2014), 40. conference on Multimedia - MM ’11 (New York, New York,
USA, Nov. 2011), 759.
[4] Cockburn, A. et al. 2012. Understanding performance in
touch selections: Tap, drag and radial pointing drag with [21] Sambrooks, L. and Wilkinson, B. 2013. Comparison of
gestural, touch, and mouse interaction with Fitts’ law.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 75 |
 Performance evaluation of gesture-based interaction between different age groups using Fitts’ Law
D. Carvalho | L. Magalhães | M. Bessa | E. Carrapatoso
Proceedings of the 25th Australian Computer-Human years of Fitts’ law research in HCI. International Journal of
Interaction Conference on Augmentation, Application, Human-Computer Studies. 61, 6 (Dec. 2004), 751–789.
Innovation, Collaboration - OzCHI ’13 (New York, New [24] Weichert, F. et al. 2013. Analysis of the accuracy and
York, USA, Nov. 2013), 119–122. robustness of the leap motion controller. Sensors (Basel,
[22] Schapira, E. and Sharma, R. 2001. Experimental evaluation Switzerland). 13, 5 (Jan. 2013), 6380–93.
of vision and speech based multimodal interfaces. [25] Zhai, S. et al. 2004. Speed–accuracy tradeoff in Fitts’ law
Proceedings of the 2001 workshop on Percetive user tasks—on the equivalency of actual and nominal pointing
interfaces - PUI ’01 (New York, New York, USA, Nov. precision. International Journal of Human-Computer
2001), 1. Studies. 61, 6 (Dec. 2004), 823–856..
[23] Soukoreff, R.W. and MacKenzie, I.S. 2004. Towards a
standard for pointing device evaluation, perspectives on 27

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 76 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor

Including multi-stroke gesture-based interaction in user

interfaces using a model-driven method
Otto Parra González 1 Sergio España Oscar Pastor
1
Universidad de Cuenca Universitat Politècnica de València Universitat Politècnica de València
Av. 12 de Abril s/n y A. Cueva Camino de Vera s/n Camino de Vera s/n
01.01.168, Cuenca, Ecuador 46022, Valencia, Spain 46022, Valencia, Spain
+593 07 405 1000 +34 96 387 7000 +34 96 387 7000
[email protected] [email protected] [email protected]

of gesture-based capabilities has to be done entirely in the source

ABSTRACT code by programmers with specific technical skills [3].
Technological advances in touch-based devices now allow users
to interact with information systems in new ways, being gesture- With the aim of overcoming this situation, this paper introduces a
based interaction a popular new kid on the block. Many daily Model-Driven Development (MDD) approach to gesture-based
tasks can be performed on mobile devices and desktop computers information systems interface development. The method is
by applying multi-stroke gestures. Scaling up this type of intended to allow software engineers to focus on the key aspects
interaction to bigger information systems and software tools of gesture-based information system interfaces; namely, defining
entails difficulties, such as the fact that gesture definitions are gestures and specifying gesture-based interaction. Coding and
platform-specific and this interaction is often hard-coded in the portability efforts are alleviated by means of model-to-text (M2T)
source code and hinders their analysis, validation and reuse. In an transformations. We focus on multi-stroke gestures as they are
attempt to solve this problem, we here propose gestUI, a model- expressive and currently wide-spread.
driven approach to the multi-stroke gesture-based user interface Our proposal has the following benefits: (a) the solution is
development. This system allows modelling gestures, integrated with the existing source code of user interfaces; (b) the
automatically generating gesture catalogues for different gesture- solution can be used on any target platform (platform-
recognition platforms, and user-testing the gestures. A model independence).
transformation automatically generates the user interface
components that support this type of interaction for desktop We also implemented a tool prototype to support our approach
applications (further transformations are under development). We that defines multi-strokes gestures and includes them in a user
applied our proposal to two cases: a form-based information interface. This tool prototype is coded in Java and Eclipse
system and a CASE tool. We include details of the underlying Modelling Framework.
software technology in order to pave the way for other research The contributions of this paper are the following:
endeavours in this area.
• We present gestUI, an MDD method for gesture-based
Categories and Subject Descriptors IS interface development consisting of: (i) a modelling
H.5.2 [Information Interfaces and Presentation]: User language to represent multi-stroke gestures, and (ii) a
interfaces. – Graphical User Interfaces, Interaction Styles, Input set of multi-platform model transformations.
devices and strategies.
• We provide a tool support for the method that captures
General Terms multi-stroke gestures sketched by the users, transforms
Design, Experimentation, Human Factors, Verification. them into a model, and automatically generates (i) the
gesture catalogue and (ii) the source code of the gesture-
Keywords based IS interface.
Customised gesture, model-driven engineering, user interface,
gesture-based interaction. • We validate our approach by applying it to two cases in
different types of information systems for desktop-
1. INTRODUCTION computing.
Natural user interface (NUI) considers the use of many different This paper is organized as follows: Section 2 includes the
interaction modalities, including multi-touch, motion tracking, definitions of items contained in the proposal. Section 3 reviews
voice, and stylus [1]. Users interact with computers employing related work. Section 4 describes our proposal. Section 5
intuitive action such as touching, gesturing and speaking. introduces the tool prototype designed to apply this approach.
Several issues may hinder the wide adoption of gesture-based Section 6 describes its application in two cases and Section 70
interaction in complex information systems engineering. contains our conclusions and projected future work.
Currently, gesture-based interaction is limited to its specification
in the implementation stage in the software development lifecycle
2. BACKGROUND
(SDLC) using tools to write source code. Gesture-based interfaces 2.1 Stroke gestures
have been reported to be more difficult to implement and test than
Nacenta et al. [4] demonstrate that users prefer user-defined
traditional mouse and pointer interfaces [2], yet the development
gestures rather than stock and pre-defined gestures. Although
user-defined gestures offer better memorability, efficiency and

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 77 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
accessibility than pre-defined gestures, they have received little use of model-driven engineering in HCI to design user interfaces
attention in the literature [5]. According to the taxonomy of with this type of interaction.
gestures in [6], semaphoric gestures refer to strokes or marks
Aquino et al. [13] emphasize the importance of interaction
made with a mouse, pen or finger. This type of gesture is further
classified as single-stroke and multi-stroke, according to the modelling on the same level of expressiveness as any other model
number of strokes required to sketch them (see Fig. 1). involved in the development life cycle of an interactive
application. They define the presentation model of the OO-
Method as an abstract model from which the model compiler can
automatically generate a user interface for different interaction
modalities and platforms, although they do not include an explicit
reference to a type of interaction modality (e. g., graphical, vocal,
tactile, haptic, and multimodal).
Deshayes et al. [14] propose the use of MDE and model execution
in the context of human-computer interaction (HCI) by means of
Figure 1. Types of semaphoric gestures heterogeneous models obtained with the ModHel’X modelling
environment for developing a simple HCI application for gesture-
based interaction. Their application makes it possible to browse a
According to [7] a stroke gesture is commonly encoded as a time- virtual book using gestures (e.g., swiping, moving) in Microsoft
ordered sequence of two-dimensional points with coordinates (x, Kinect.
y). Optionally, stroke gestures can also have time stamps as the
third dimension so the sampling points are encoded as (x, y, t) if Vanderdonckt [15] describes a set of variables related to the
the temporal aspect of a gesture is to be preserved and used. In development of user interfaces, one of which considers interaction
this work, stroke gestures are used to issue commands, which are devices and styles. Interaction styles include the gesture
the names of the executable computing functions issued by the recognition. However, he point out that an abstract user interface
user. is independent of any interaction modality [16] so that an explicit
reference to a specific type of interaction style is not considered.
2.2 Model related definitions
A model is a description of a system or part of a system written in [17] includes a report regarding user interface plasticity and MDE,
a well-defined language. A well-defined language is a language in which three information spaces are defined: the user model,
with well-defined form (syntax), and meaning (semantics), which environment model, and platform model. The platform model
is suitable for automated interpretation by a computer [8]. A considers the possible interactions that can be included in a user
model, both source and target, is expressed in a language [9], for interface. This report also includes a description of models that
example, XML. have been defined with the aim of creating user interfaces. It also
mentions the variety of interaction modalities currently available
Model-Driven Architecture® (MDA®) is an architectural thanks to the diversity of technological spaces that can be
framework for software development. One of its fundamental included in the definition of concrete user interfaces.
aspects is its ability to address the complete development
lifecycle, covering analysis and design, programming, testing, Calvary et al. in [18] describe the relation between MDE and HCI
component assembly as well as deployment and maintenance. in implementing user interfaces. In this context, they introduce the
MDA specifies three default viewpoints on a system: computation models contained in a number of frameworks (e g., UsiXML,
independent, platform independent and platform specific. MDA CTTe), one being the interaction model considered in the process
specifies three default models of a system corresponding to these of defining user interfaces. However, the interaction modality is
three viewpoints: computation independent model (CIM), not considered.
platform independent model (PIM) and a platform specific model In [19], the authors propose the Abstract Interaction Model that is
(PSM) [10]. added to the Presentation Model in the OO-Method. Two sub-
Model-Driven Engineering (MDE) describes software models are considered to define the Interaction Model: the user
development approaches in which abstract models of software are model and abstract interface model. A set of interaction
created and systematically transformed into concrete components is defined in the abstract interface model that define
implementations [11]. Model-driven development (MDD) its interface with the software system. These components are
automates the transformation of models from one form to another. conceptual modelling elements that describe the interaction
behaviour expected by the user but not how it is implemented, so
Model-driven describes an approach to software development that this system does not include the interaction modality in the
whereby models are used as the primary source for documenting, process of user interface generation.
analysing, designing, constructing, deploying and maintaining a
system [10]. All the works cited above mention the importance of using MDE
and HCI to obtain user interfaces in a short time at a low cost.
Model-to-Text (M2T) transformation is a model management Although they also point out the need for a specification of an
operation that involves generating text (e.g., source code, interaction modality, they do not include gestures in their
documentation, configuration files) from models [12]. proposals. We considered gesture-based interaction in this
proposal in order to obtain a complete definition of user interfaces
3. RELATED WORK using MDE and HCI.
This section briefly reviews the role of model-driven engineering
(MDE) in HCI in which models are used to create a user interface
that includes user interaction. Several studies have reported on the
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)
| 78 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor

4. OUR PROPOSAL precedence order. Each posture in a gesture is related to a figure

This section describes gestUI [20], a user-driven iterative method (line, circle, etc.) with an orientation (up, down, left, right), and is
based on the MDD paradigm. The system defines multi-strokes qualified by a state (initial, executing, final).
gestures, creates a gesture catalogue model and using model In the platform-specific layer there are two activities: (i) Activity
transformations to obtain the interface source code, including A2, “Design gesture-based interaction” which consists of a
gesture-based interaction. gestUI is expected to be integrated into process to define the relationship between a gesture and an action
a full interface development method (shown in Fig. 2 by generic (command) to be executed in an information system. This activity
activities and artefacts in grey). The method can either be model- produces the gesture-based interaction model. If the stakeholders
driven or code-centric. gestUI is user-driven because the users consider that the definition of gestures is incomplete or inadequate
participate in all non-automated activities and is iterative because the process can return to Activity 1 to be redefined. (ii) Activity
it aims to discover the necessary gestures incrementally and A3, “Generate gesture specification”, employs an M2T
provides several loopbacks. transformation to generate a platform-specific gestures catalogue
Our proposal is based on Model-driven Architecture (MDA) specification.
considering three layers: the platform-independent layer, There are two activities in the code layer: (i) Activity A4,
platform-specific layer and code layer. gestUI consists of five “Generate gesture-based interface”, which employs the gesture-
activities and their products which are distributed in the three based interaction model and the platform-specific gestures
layers, as shown in Fig. 2. catalogue specification to obtain the source code of the gesture-
In the platform-independent layer, during the A1 activity, “Define based user interface by applying a model transformation. (ii)
gestures”, the stakeholders and developers have meetings to Activity 5, “Test gestures”, tests the gestures defined by the
discuss gesture definition for the user interface. The stakeholders stakeholders and included in the information system. This is an
try the gestures in the interface implemented in the tool to verify iterative process because it is possible to redefine the gestures if
that these gestures are suitable for the tasks in their information the users are not satisfied with their definition.
system. Finally, a gesture catalogue model is obtained conforms If the gesture catalogue is complete and the users agree with the
to the gesture catalogue metamodel (Fig. 3). In the gesture definition of the correspondence gesture-action in the information
catalogue each gesture is formed by one or more strokes defined system (IS), then the process is finished. The IS then contains
by postures, described by means of coordinates (X, Y). The gesture-based interaction as a complementary interaction modality
sequence of strokes in the gesture is specified by means of to the traditional interaction.

Figure 2. gestUI method overview

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 79 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor

Figure 3. Gesture catalogue metamodel

Figure 4. gestUI tool overview

5. TOOL The third module “Model transformation” contains the definition

This section describes the tool that implements our proposal. This of the M2T transformations for some platforms as targets for
tool is structured in three modules as shown in Fig. 4. The gesture-based interaction in the user interface. Activity A3 is
number in brackets indicates the method activity each component included in this module.
supports. The method’s internal products are not shown. The The tool prototype is developed in Java and Eclipse and permits
relationship with the external gesture recogniser is represented. the proposal to be applied by means of its modules: (i) to define
The first module “Gesture catalogue definition” contains a process multi-strokes gestures, (ii) to obtain a gesture catalogue model
that obtains the gesture catalogue model from the users’ gesture using model transformations, (iii) to define gesture-action
definition made by the users. Activity A1 is included in this correspondence and, (iv) to generate source code to include
module. gesture-based interaction in user interfaces. At present, the tool
generates Java source code embedded into the IS interface source
The second module “Gesture-Action correspondence definition” code.
includes a process to specify the correspondence between gestures
and actions (commands) available in the IS. Activities A2 and A4 6. DEMONSTRATING THE PROPOSAL
are included in this module. This section describes an application of the method with the
implemented tool to include gesture-based interaction in existing

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 80 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
information systems (IS). gestUI is integrated into a code-centric read the information of a record. Using these gestures, we obtain
interface development method considering two legacy desktop- the gesture catalogue model by means of a model transformation.
computing IS: forms-based IS and case tools.
The process of obtaining a user interface with gesture-based
interaction involves the following steps:
a) Define the gestures according to the users’
requirements, and then obtain the gesture catalogue
model.
Figure 6. Gestures defined for form-based IS
b) Obtain the platform-specific gesture catalogue by means
of model transformation using the gesture catalogue Therefore, in order to define the gesture catalogue model the user
model and the target platform as source for the selects gestures from a gesture repository containing all the
transformation. gestures that the user has defined. This catalogue contains the
description of each gesture, but does not contain the action
c) Specify the user interface to which we will add the (command) definition to execute it (in Fig. 7 the action is
gesture-based interaction by definition of gesture-action assigned to “null”) because this feature (to assign an action) is
correspondence. The user interface source code which platform-specific must be specified in the next step.
includes the gesture-based interaction is generated.
d) Add the generated source code to the existing user
interface source code and recompile the system.

6.1 Form-based Information System

In the first case, we consider a form-based IS to manage
information for the computing department of a business company,
formed by departmental offices that have offices. In each
department there are employees who use computers with one or
more operating systems and printers.
The IS manages the information of: (i) the company, (ii)
departments, (iii) devices (computers, printers), (iv) the operating
systems installed in the computers, (v) the operation status of the
devices, (vi) the employee that uses the devices. For the sake of
brevity, we will only consider the option to manage information of
operating systems in the process of inclusion of gesture-based
interaction. Fig. 5 shows the domain class diagram of this IS.

Figure 7. Gesture catalogue model

Figure 5. Domain class diagram of the form-based IS

This software, written in Java using Swing in Eclipse, employs a
three-tier architecture: data access logic, business logic, and
presentation logic [20]. The set of operations that can be executed
in each component are called CRUD operations (create, read,
update, and delete). In this case we need to define gestures in Figure 8. Obtaining platform-specific gesture specification
order to execute these operations in each option (component) of
the IS.
The first step is the creation of new gestures using the interface The next step is the generation of platform-specific gesture
implemented for this task, (see in Fig. 6). The users define specification by choosing the gestures that are used in the IS (Fig.
gestures according to their preferences in order to execute actions 8). Using this specification and entering the target platform ($N)
in an information system. For instance, in a database (a) C, to and the target language (XML) as input data we apply a model
create a new record to enter in the IS, (b) U, to update the transformation to obtain the gestures to be used in the gesture
information of a record, (c) D, to delete a record, and (d) R, to recognition process using $N [21].

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 81 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
Fig. 9 shows an excerpt from a gesture definition generated in
XML to be used with $N. In this case, the gesture which
represents the letter “D” is formed by two strokes (shown in Fig.
6), each of which contains points with coordinates (X, Y) and a
timestamp (T).

Figure 11. Gesture-Action correspondence definition

When generating the user interface Java source code, many
references are included: (i) references to libraries to manage
gestures (e.g. listeners to manage mouse events such as move,
down, up, release, drag), (ii) references to the libraries of the
gesture-recognition technology (e.g. $N), (iii) references to the
method of executing the gesture-action correspondence (i.e.
executingActions in Fig. 12), (iv) references to the methods of
capturing gestures (i.e. mouseDragged, mouseReleased), and (v)
references to the method of loading the multi-stroke gesture
catalogue.
Figure. 9. An excerpt from a gesture written in XML
The next step concerns with the definition of the gesture-action
correspondence in order to complete the gesture definition. In this
case, the process consists of the selection of a user interface
source code to apply a parsing process and determine the actions
to be included in this source code. We consider code which
contains methods in Java containing “action perform” structures
(Fig. 10), which typically define the actions to be executed by
selecting buttons or options in the menu in a form-based IS.
Figure 12. An excerpt of source code to execute actions
Additionally, the definition of some classes have been changed in
order to implement mouseListener and mouseMotionListener to
detect mouse events.
Finally, when we execute the form-based IS (Fig. 13), we use the
gestures included in the source code to execute the actions
specified in the process. In this case, we can use traditional
interaction and gesture-based interaction.

Figure 10. An excerpt of source code of user interface

The developer assign the gesture-action correspondence in
collaboration with the user, supported by the Gesture-action
correspondence definition module. The process consist in select
one gesture (Fig. 11, left) for each action specified in the user Figure 13. Form-based IS with gesture-based interaction
interface (Fig. 11, centre). The correspondence gesture-action is
defined (Fig. 11, right). The next step is the source code
generation by selecting button “Generate” in the interface shown In the form-based IS shown in Fig. 13, the user draws the gesture
in Fig. 11. In this case, we obtain Java source code to include “C” with the aim of creating a new operating system record.
gesture-based interaction in the user interface source code When the gesture recognizer analyses the gesture sketched by the
user as correct, then a new window for the user to fill in the form
related to the operating system details.

6.2 Case tool

Now, we describe the application to the second type of IS: in this
case, we consider Graphical Editing Framework (GEF), an
Eclipse project which provides a development framework for
graphical representation of information. The GEF project is

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 82 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
developed in JFace and Standard Widget Toolkit (SWT) with different actions: (i) in the form-based IS to create a record, and
Eclipse Modelling Framework. GEF is designed with a Model- (ii) in the case tool to draw a rectangle.
View-Controller (MVC) architecture. Briefly, the function of each
Using the process described in this paper, we obtain the gesture
component is [22]:
catalogue model, then in the next step we get the platform-specific
• Model: holds the information (underlying objects being gesture specification and finally, the definition of gestures using
represented graphically) to be displayed and is persisted XML to use in the $N gesture recognizer.
across sessions.
The gesture-action correspondence definition is executed in the
• View: renders information on the screen and provides source code of Shapes. Fig. 14 includes an excerpt from the
basic user interaction, in this case, a canvas containing a source code that contains the reference to the aforementioned
collection of figures. actions: draw rectangles and ellipses.
• Controller: coordinates the activities of the model and
the view, passing information between them as
necessary. The controller has a collection of GEF edit
parts.
According to [22], the GEF project is formed by three
subsections:
Figure 14. An excerpt of source code of Shapes
• Draw2D, a lightweight framework layered on top of We apply the process of parsing in this source code to determine
SWT for rendering graphical information. the actions defined to draw the elements contained in the palette.
• GEF Framework, an MVC framework layered on top of The user selects the gestures in the catalogue (Fig. 15, left), and
Draw2D to facilitate user interaction with graphical defines the relation with the actions that were found in the parsing
information. process (Fig. 15, centre). Finally, the gesture-action
correspondence is obtained (Fig. 15, right).
• Zest, a graphing framework layered on top of Draw2D
for graphing.
In this work, we employ the example called Shapes, included in
GEF project, to apply the method. Considering that GEF has a
plug-in structure, then the Shapes source code is comprised by
two components: (i) source code of GEF which provides the
functionality of an editor (to display and draw elements in a
diagram), and, (ii) source code of the example, which define the
canvas and the palette of the tool and employs the methods Figure 15. Gesture-action correspondence definition
defined in the plug-in to use the editor. The palette defined in When generating the user interface Java source code, many
Shapes permits rectangles and ellipses to be drawn. Therefore, we references are included; for instance, in the GEF source code we
need to modify the View component both in the source code of add references to gesture management libraries (e.g. listeners to
GEF and the example Shapes, in order to define the finger manage mouse events such as mouseUp and mouseDrag), to the
gestures for drawing ellipses and rectangles. gesture-recognition technology libraries (e.g. $N) and the method
We describe below the process applied to the GEF framework in of executing the gesture-action correspondence (Fig. 16).
order to include gesture-based interaction in this case tool. Also, in the methods for managing mouse events we add source
Table 2. Gestures and elements related code to capture gestures (i.e. mouseDrag (Fig. 17), mouseUp),
and the method of loading the multi-stroke gesture catalogue is
Gestures Element related added.

Rectangle

Ellipse

Figure 16. An excerpt of source code to execute actions

Table 13 shows the gestures used to draw the elements defined in

the Shapes palette. In the process of defining gestures, in some
cases it is possible to “reuse” gestures that the users have sketched
previously, for instance, we use the gesture “C” to execute

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 83 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
We then automatically generated the final gesture-based interface
components and integrated them into the application interfaces.
The advantages of the gestUI proposal are: its platform
independence enabled by the model-driven development
paradigm, the convenience of including user-defined symbols and
its iterative and user driven approach.
Figure 17. An excerpt of the source code of GEF Future work will be developed along the following lines: (i) a user
We then consider the source code of the example in order to add study to determine user preferences in defining gestures according
the option of including gesture-based interaction in the palette. In to the task to be executed, (ii) applying this method to extending
the source code containing the definition of the palette we add the the Capability Design Tool (CDT) of the “Capability as a Service”
source code shown in Fig. 18. The option added is shown in a red Project (CaaS project) in order to incorporate gesture-based
circle in Fig. 19. The users thus have two styles of interaction in interaction into this framework.
GEF: (i) traditional interaction (using mouse and keyboard) and
(ii) gesture-based interaction (using fingers to sketch gestures). 8. ACKNOWLEDGMENTS
Otto Parra is grateful to his supervisors Sergio España and Óscar
Pastor for their invaluable support and advice. This work has been
supported by Secretaría Nacional de Educación, Ciencia y
Tecnología (SENESCYT) and Universidad de Cuenca of Ecuador,
and received financial support from Generalitat Valenciana under
Project IDEO (PROMETEOII/2014/039).

Figure 18. An excerpt from the source code to modify the 9. REFERENCES
palette [1] D. Wigdor y D. Wixon, Brave NUI world: designing natural
user interfaces for touch and gesture, USA: Morgan
Kaufmann Publishers Inc., 2011.
Obviously, then the source code should be compiled to obtain the [2] M. Hesenius, T. Griebe, S. Gries y V. Gruhn, «Automating
case tool with gesture-based interaction included. UI Tests for Mobile Applications with Formal Gesture
When the user employs the gestures defined in this case tool to Descriptions,» Proc. of 16th Conf. on Human-computer
draw an ellipse/rectangle on the canvas using his/her finger, the interaction with mobile devices & services, pp. 213-222,
gesture recognizer analyses the sketch. If it is correct, then the 2014.
system draws an ellipse/rectangle defined in the GEF palette. [3] K. Gerken, S. Frechenhäuser, R. Dörner y J. Luderschmidt,
«Authoring Support for Post-WIMP Applications,» Human-
Computer Interaction – INTERACT 2013. Lecture Notes in
Computer Science, vol. 8119, pp. 744-761, 2013.
[4] M. Nacenta, Y. Kamber, Y. Qiang y P. Kristensson,
«Memorability of Pre-designed & User-defined Gesture
Sets,» Proc. of the SIGCHI Conf. on Human Factors in
Computing Systems, pp. 1099-1108, 2013.
[5] U. Oh y L. FindLater, «The challenges and potential of end-
user gesture customization,» Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems -
CHI'13, pp. 1129-1138, 2013.
[6] M. Karam y M. C. Schraefel, «A taxonomy of Gestures in
Human-Computer Interaction,» de Retrieved from
http://eprints.soton.ac.uk/261149/, 2005.
[7] S. Zhai, P. O. Kristensson, C. Appert, T. H. Anderson y X.
Figure 19. Case tool with gesture interaction Cao, «Foundational Issues in Touch-Surface Stroke Gesture
Design — An Integrative Review,» Foundations and Trends
7. CONCLUSIONS AND FUTURE WORK in Human–Computer Interaction, vol. 5, nº 2, pp. 97-205,
gestUI is a model-driven method for developing multi-stroke 2012.
gesture-based user interfaces. We validated the method and [8] A. Kleppe, J. Warmer y W. Bast, MDA Explained: The
supporting tools by applying them to two types of applications: a Model Driven Architecture : Practice and Promise, USA:
form-based application and a CASE tool built using GEF (an Addison-Wesley Prof., 2003.
Eclipse framework with the structure of a plug-in) that can be
used to draw simple diagrams. We generated the ‘Platform- [9] S. J. Mellor, T. Clark y T. Futagami, «Model-driven
specific gesture specification’ for the two types of applications to development: guest editors' introduction,» IEEE Software,
illustrate the multiplatform capability of the approach. The vol. 20, nº 5, pp. 14-18, 2003.
gestures were successfully recognised by the corresponding tools.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 84 |
 Including multi-stroke gesture-based interaction in user interfaces using a model-driven method
O. Parra González | S. España | M. Bessa | O. Pastor
[10] F. Truyen, «The Fast Guide to Model-Driven Architecture. [17] J. Coutaz y G. Calvary, «HCI and Software Engineering for
The Basics of Model-Driven Architecture,» Cephas User Interface Plasticity,» de The Human-Computer
Consulting Corp., 2006. Handbook - Fundamentals, Evolving Technologies, and
[11] R. France y B. Rumpe, «Model-Driven Development of Emerging Applications, Julie, A. Jacko ed., CRC Press
Complex Software: A Research Roadmap,» Future of Taylor and Francis Group, 2012, pp. 1195-1220.
Software Engineering - ICSE, pp. 37-54, 2007. [18] G. Calvary, D.-P. A., A. Occello, R.-G. P. y M. Riveill, «At
[12] B. Ogunyomi, L. M. Rose y D. Kolovos, «On the Use of the Cross-Roads between Human-Computer Interaction and
Signatures for Source Incremental Model-to-text Model-Driven Engineering,» ARPN Journal of Systems and
Transformation,» Model-Driven Engineering Languages and Software, vol. 4, nº 3, pp. 64-76, 2014.
Systems. Lecture Notes in Computer Science, vol. 8767, pp. [19] F. Valverde, J. I. Panach y O. Pastor, «An Abstract
84-98, 2014. Interaction Model of a MDA Software Production Method,»
[13] N. Aquino, J. Vanderdonckt, J. I. Panach y O. Pastor, de Twenty-Sixth International Conference on Conceptual
«Conceptual Modelling of Interaction,» de Handbook of Modeling - ER 2007 - Tutorials, Posters, Panels and
Conceptual Modeling. Theory, Practice, and Research Industrial Contributions, Auckland, New Zeland, 2007.
Challenges, Berlin, Springer, 2011, pp. 335-358. [20] O. Parra, S. España y O. Pastor, «A Model-driven Method
[14] R. Deshayes, C. Jacquet, C. Hardebolle, F. Boulanger y T. and a Tool for Developing Gesture-based Information
Mens, «Heterogeneous modeling of gesture-based 3D Systems Interface,» de Proceedings of the CAiSE'15 Forum
applications,» de Proceedings of the 6th International at the 27th International Conference on Advanced
Workshop on Multi-Paradigm Modeling, 2012. Information Systems Engineering, Stockholm, Sweden, 2015.

[15] J. Vanderdonckt, «A MDA-Compliant Environment for [21] M. Fowler, Patterns of Enterprise Application Architecture,
Developing User Interfaces of Information Systems,» Boston, MA: Addison-Wesley Professional, 2003.
Advanced Information Systems Engineering LNCS in [22] L. Anthony y J. O. Wobbrock, «A Lightweight Multistroke
Computer Science, vol. 3520, pp. 16-31, 2005. Recognizer for User Interface Prototypes,» Proceedings of
[16] J. Vanderdonckt, «Model-Driven Engineering of User Graphics Interface (GI '10), pp. pp. 245-252, 2010.
Interfaces: Promises, Successes, Failures, and Challenges,» [23] E. Clayberg y D. Rubel, Eclipse Plug-ins, Massachusetts,
de ROCHI'08, Iasi, Romania, 2008. USA: Addison-Wesley, 2009.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 85 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López

Interfaces de Usuario Tangibles como Mecanismo de

Interacción en Entornos Multi-dispositivos
Elena de la Guía, María D. Lozano, Víctor Penichet, Luis Orozco, Vicente López
Computer Science Research Institute
University of Castilla-La Mancha
Campus Universitario, Albacete - SPAIN
{mariaelena.guia, maria.lozano, victor.penichet, luis.orozco, vicente.lcamacho}@uclm.es

ABSTRACT Sheiderman[1] a la manipulación táctil que permite digitalizar

En los últimos años la tecnología ha crecido exponencialmente nos objetos y entornos reales, mezclando la parte digital con la real.
encontramos escenarios donde se trabaja con múltiples Actualmente se están insertado las interfaces de usuario tangibles
dispositivos, como son móviles, Tablet PC, portátiles, wereables, como mecanismo de interacción. Cuando hablamos de interfaces
etc. interactuando entre sí para ofrecer servicios fácilmente. Hoy de usuario tangibles (TUI o Tangible User Interface en inglés) [2]
en día encontramos mecanismos de interacción tangible y natural nos referimos a objetos físicos utilizados como representaciones y
que permiten fusionar el mundo real y digital simulando entornos controles de la información digital. De esta forma conseguimos
reales y familiares a los usuarios. Sin embargo, la diversidad de combinar dispositivos digitales con objetos reales creando
usuarios finales se ha incrementado, es decir, nos encontramos escenarios más familiares e intuitivos para el usuario.
usuarios con diferentes habilidades cognitivas, conocimientos, Ahora que tenemos entornos físicos reales surgen las siguientes
discapacidades, etc. Actualmente, el primer desafío es como preguntas. ¿Qué directrices son recomendables seguir para
diseñar y distribuir las interfaces/información alrededor del distribuir interfaces de usuario en los diferentes
entorno sin saturar al usuario. El siguiente desafío es como dispositivos/objetos/usuarios de la manera más eficaz posible?
motivar y guiar al usuario alrededor de entornos MDE sin La organización del artículo está definida de la siguiente manera:
distraerlo de su tarea u objetivo principal. Teniendo en cuenta que Primero se presenta un breve repaso sobre las interfaces de usuario
la interacción tangible es más familiar y fácil de aprender para los distribuidas y las técnicas de interacción tangible utilizadas en
usuarios que no tienen experiencia con la tecnología se han entornos multi-dispositivo. A continuación se describen los
desarrollado y evaluado cuatro prototipos. Son juegos MDE prototipos realizados basados en interfaces de usuario distribuidas
basados en interfaces de usuario distribuidas y tangibles. Después e interacción tangible. Por último, se presentan las
de evaluar los prototipos, analizar los datos y basándonos en las recomendaciones y directrices obtenidas gracias a las evaluaciones
lecciones aprendidas se han propuesto 8 directrices iniciales. El y lecciones aprendidas de los prototipos desarrollados.
objetivo principal es servir de guía para diseñar e investigar dichos
entornos. Las 8 directrices se han basado en los siguientes
2. TRABAJOS RELACIONADOS
elementos (espacio físico/ múltiples usuarios/ motivación). La rápida evolución de la tecnología ha cambiado la forma en que
interactuamos con sistemas interactivos. En los últimos años han
Categories and Subject Descriptors surgido entornos dotados de múltiples dispositivos, también
H5.2. Information interfaces and presentation: User Interfaces. – denominados MDE (Multi-Device Environment). Estos escenarios
Graphical user interfaces. soportan interfaces de usuario distribuidas. Esta se define como
una interfaz de usuario donde sus componentes pueden ser
General Terms distribuidos a través de una o más dimensiones como puede ser
Design, Experimentation, Human Factors, entrada, salida, plataforma, espacio y tiempo [3]. Según González
en [4], las interfaces de usuario Distribuidas (DUI) se pueden
Keywords clasificar en función de las características de la interfaz en un
Interfaces de Usuario Distribuidas, Interacción Tangible MDE. Las interfaces pueden ser divididas o distribuidas en el
entorno físico de acuerdo a sus propiedades. El objetivo principal
1. INTRODUCCIÓN de una DUI es facilitar las tareas a los usuarios en el sistema de
Hace años la interacción persona ordenador se basaba en software, poniendo a su disposición una configuración óptima de
interfaces de usuario controladas a través de un ratón y un teclado interfaz de usuario que están disponibles en el entorno de trabajo
conectados al ordenador. de los usuarios. Según Vandervelpen y Coninx en [6] los
Hoy en día nos encontramos con escenarios dotados de múltiples elementos que forman parte de un entorno de distribución de
dispositivos donde las interfaces de usuario tradiciones se quedan interfaces son los siguientes: Recursos de interacción (RIs) son
limitadas y surge la necesidad de estudiar e investigar nuevos canales de I/O disponibles para que el usuario ejecute una tarea.
mecanismos de interacción e interfaces que se adapten al entorno Incluye recursos de I/O como teclados, pantallas, micrófonos, etc.
y al usuario. Durante estos últimos años ha surgido un nuevo Hablamos de canales limitados a una dirección y una simple
concepto denominado DUIs (Distributed User Interfaces o modalidad. Dispositivos de interacción (DIs) son sistemas que
Interfaces de Usuario Distribuidas) su objetivo principal es integran computación y son capaces de controlar la entrada o
distribuir las interfaces en distintos dispositivos. Pero todavía envían la salida a los RIs individuales a los que se encuentran
existen aspectos a abordar, ¿Cómo interactuamos con las conectados. Un DI es una colección de RIs junto con la unidad de
interfaces de usuario?, los estudios en este campo han computo. Los DIs en este escenario serían ordenadores,
evolucionado desde la manipulación directa que proponía dispositivos móviles, etc. Sin embargo, la distribución de las

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 86 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
interfaces en el espacio es complejo, el desafío principal es:
¿Cómo configurar y distribuir las Interfaces de Usuario entre IR,
ID para conseguir un sistema usable y atractivo al usuario? Hay
estudios al respecto [5], sin embargo, no hacen hincapié sobre las
técnicas de interacción que sería más apropiada dependiendo del
sistema. Tampoco tienen en cuenta la psicología cognitiva humana
innata en los usuarios.
Con el fin de proporcionar diferentes técnicas de interacción
tangible nos hemos centrado en el gesto de acercar y alejar
también llamado “Approche&Remove” [7][8][9] en este caso
distinguimos del recurso interactivo que utilicemos como entrada,
este puede ser el dispositivo móvil, un objeto o los dedos. Estos
tipos de interacción permiten al usuario interactuar con interfaces
de usuario distribuidas fácilmente. De esta forma se ofrece un
estilo natural y fácil de interacción que resulta intuitiva y Figura 1. Distribución de una interfaz de usuario común en
motivadora para las personas que no tienen experiencia con la interfaces distribuidas y tangibles
tecnología. Han sido desarrollados con la tecnología NFC (Near
Field Communication).
3.2 Tipos de Interacción Tangible
Con el fin de interactuar con entornos multi-dispositivo donde la
3. CARACTERÍSTICAS DE LOS MDE interfaz de usuario principal está distribuida en un computador,
DESARROLLADOS tablet, smartphone, o proyección en la pared. Se han digitalizado
Se han desarrollado cuatro prototipos funcionales basados en objetos comunes como pueden monedas, tarjetas, juguetes,
juegos de aprendizaje y estimulación cognitiva. Teniendo en paneles, etc. ahora llamadas interfaces de usuario tangibles.
cuenta la diversidad de los usuarios, se han enfocado en usuarios Se describen tres tipos diferentes de técnicas de interacción, donde
con necesidades especiales. Los cuatro son colaborativos, se utilizan diferentes recursos de interacción (RI). La operación
interactivos y distribuyen interfaces de usuario en el entorno. Cada interna del sistema es la misma.
uno de ellos implementa una técnica de interacción tangible Input: Objeto inteligente
diferente. Esta técnica es llamada “Approach&Remove Object”, consiste en
interactuar con el sistema acercando y alejando la interfaz de
3.1 Distribución de interfaces de Usuario usuario tangible (objeto digitalizado) a un dispositivo de
Para diseñar la distribución de las interfaces de usuario se ha interacción (DI) en este caso cualquier dispositivo móvil
tenido en cuenta la cognición distribuida innata en los usuarios. (smartphone o tablet) que incorpore el lector NFC en su interior.
Como dice Hutchins [10] la cognición distribuida propone que la El dispositivo móvil (DI) es el encargado de conectarse con el
cognición y el conocimiento no se limitan a un individuo, sino que sistema y enviarle la información requerida, posteriormente el
se distribuye a través de objetos, personas y herramientas en el sistema mostrará la información de salida solicitad (Ver figura
entorno. Se ha hecho uso del modelo presentado en [15] donde se 2.a).
distribuye el contenido del juego teniendo en cuenta los modelos Input: Dispositivo móvil
mentales. Para el diseño de entornos MDE, hemos seguido los
Esta técnica de interacción también se denomina
siguientes criterios. La memoria a largo plazo se asocia con las
“Approach&Remove Device” debido a que el gesto que realiza el
interfaces de usuario tangibles, es decir, los objetos digitalizados
usuario para interactuar con el Sistema es el mismo, acercar y
que permiten al usuario interactuar con el sistema. De esta manera,
alejar el dispositivo móvil al panel digitalizado (Interfaz de
el proceso que seguiría la mente de los usuarios al utilizar el
usuario tangible) (Ver figura 2.b).
sistema es el siguiente: en primer lugar, la interfaz del juego es
Input: Dispositivo móvil & Objeto digitalizado
visualizada utilizando la memoria sensorial, a continuación,
mediante la memoria a corto plazo el usuario interioriza las Esta técnica de interacción es tangible pero combina dos tipos de
interfaces tangibles, finalmente, cuando se entiende la imagen interacción; tangible, es decir, el usuario interactúa con objetos
almacenada, se guarda en la memoria a largo plazo, resultando comunes y otra técnica basada en tocar una superficie digital con
intuitivo el uso del sistema. (Ver Figura 1). los dedos (touch). El usuario interactúa con la Tablet y puede
utilizar tantos los dedos como los objetos (Ver figura 2.c).
Output: Dispositivo móvil y Pantalla
Para mostrar los resultados, tenemos multi-dispositivos que nos
muestran mensajes e información multi-modal (audio, visual,
textual y gráfica).

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 87 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
usuarios. En los prototipos se incluyen tareas para que exploren el
entorno, recompensas, puntos, niveles para que mejoren las
capacidades cognitivas y sientan un desafío mientras utilizan el
sistema.
Control: Lepper y Malone [16] aconsejan que los juegos ofrezcan
A el control al usuario. En este caso los usuarios tienen sus propias
interfaces de usuarios tangibles y dispositivos móviles privados
con el fin de ofrecerles la confianza de que utilizan sus propios
recursos.
Feedback: La aplicación a través de audio, imágenes y texto
ofrece información relevante para guiar en todo momento al
B usuario.
Inmersión: La inmersión es la característica del juego que
provoca al jugador para que se vea envuelto en el mundo virtual,
volviéndose parte de éste e interactuando con él. En este caso el
mundo es real y combinamos el factor real (interacción y
búsqueda de objetos) con el factor virtual (tablet, Smartphone,
pantallas, etc. que nos ofrecen los juegos).
Interacción social: El factor real nos ofrece un espacio físico que
al mismo tiempo permite a los usuarios interactuar entre ellos con
para conseguir un objetivo común.

4. DESCRIPCIÓN DE LOS PROTOTIPOS

C A continuación se describen los entornos que soportan múltiples
Figura 2. Inputs en entornos MDE a) (IR) es el objeto dispositivos, interfaces de usuario distribuidas e interacción
inteligente; b) (IR) es el dispositivo móvil; c) (IR) puede ser un basadas en interfaces de usuario tangibles. Se han enfocados para
objeto inteligente o los dedos (touch) tareas/juegos colaborativos que soportan múltiples usuarios
interactuando simultáneamente.

3.3 Diseño del contenido

Para diseñar el contenido de los prototipos se ha buscado activar la
motivación por parte del usuario. La motivación es un factor
interno que incita a una acción, activa, dirige y mantiene la
conducta. Es decir, es el factor que va animar al usuario a utilizar e
interactuar con los sistemas propuestos. Existen dos tipos de
motivación: Intrínseca y Extrínseca: Las personas que 1.A 1.B
tienen motivación intrínseca realizan acciones porque encuentran
satisfacción en el mismo hecho de hacerlas, así como la auto
superación, la sensación de éxito, etc. Por otra parte, la motivación
extrínseca aparece cuando lo que atrae al individuo no es la acción
que se realiza en sí, sino lo que se recibe a cambio de la actividad
realizada (por ejemplo, premios, recompensa, puntos, etc.
Teniendo en cuenta que los prototipos son juegos se ha seguido
una serie de criterios basados en la teoría de flujo propuesta por 2.A 2.B
Sweetser y Wyeth [19].
Los puntos más importantes que se han tenido en cuenta para Figura 3.-Sistemas a-) Co-Brain Training , b) TraInAb
diseñar e implementar el contenido y distribución de las interfaces
de usuario han sido los siguientes.
Co-Brain Training (Collaborative Brain Training) [13] (Ver
Concentración: El objetivo de la aplicación es mantener la Figura 3 1.a) es un juego interactivo y colaborativo diseñado para
atención y concentración del usuario enfocada en la aplicación. estimular habilidades cognitivas como son la memoria, atención y
Para conseguir que la interfaz sea limpia y el usuario no se lenguaje en personas que tienen Alzheimer. Su funcionamiento es
distraiga con otro tipo de información se distribuye la interfaz de el siguiente: la interfaz principal del juego se proyecta en una
la aplicación siguiendo la cognición distribuida descrita pared permitiendo que puedan jugar múltiples usuarios. Los
anteriormente. Por un lado el espacio de trabajo principal y por usuarios interactúan con el sistema acercando el dispositivo móvil
otro lado los controles y dispositivos de navegación. que incorpora el lector NFC(como un apuntador) a un panel
Desafío o Reto: Según Malone [17] cualquier juego tiene que interactivo (Ver Figura 3 1.b) donde se muestran las opciones
proporcionar un reto, es decir, debe despertar la curiosidad de los posibles del juego.

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 88 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
TraInAb (Training Intellectual Abilities)[12] (Ver Figura 3 ser interfaces tangibles a través de las que se interactuará con el
2.a).es un juego interactivo y colaborativo diseñado para terapias juego.
de estimulación cognitiva en usuarios que tienen discapacidad Real Game es un juego ejecutado en el entorno real donde los
intelectual. El sistema integra una nueva forma de interacción a usuarios interactúan con el entorno y entre ellos a través de
través de objetos comunes de la vida cotidiana como pueden ser dispositivos móviles/objetos. Los usuarios exploran el entorno, el
cartas, juguetes, monedas. Su funcionamiento es el siguiente: la sistema detecta donde se encuentran y en función de esto les
interfaz principal del juego se proyecta en una pared con el fin de plantea retos a través de los dispositivos móviles que portan. Los
aumentar su visibilidad y permitir que puedan jugar múltiples retos se resuelven colaborativamente interactuando con los objetos
usuarios. Los usuarios interactúan con el sistema acercando los que se encuentran en una determinada zona del entorno, acercan el
objetos (interfaces tangibles), identificados mediante tecnología objeto a otro objeto/dispositivo móvil. Una pantalla principal se
NFC (Near Field Communication), al dispositivo móvil que muestra las puntaciones, recompensas, retos logrados, etc. (Ver
incorpora el lector NFC (Ver Figura 3 2.b). Figura 4.b)

5. LECCIONES APRENDIDAS
La obtención de datos nos permite definir directrices que ayuden a
futuros diseñadores e investigadores a tomar decisiones a la hora
de desarrollar entornos MDE que soporten interfaces de usuario
distribuidas y tangibles. Después de realizar iterativas
evaluaciones de los prototipos se han recogido los tiempos medios
dependiendo de la interacción, el número medio de errores, la
tangibilidad, affordance, número de interacciones verbales entre
los usuarios, el coste y la infraestructura de los sistemas. A
continuación se describen brevemente algunos de los resultados
obtenidos.
Teniendo en cuenta que los sistemas se diseñan para un un espacio
físico real, es necesario que sea usable, es decir, la interacción
debería ser fácil de aprender, sencilla, atractiva al usuario y
familiar, con el fin de romper la brecha digital que existen entre
varios grupos de usuarios. Por ese motivo, se ha decidido diseñar
entornos que simulen el modo real de trabajar que tienen los
usuarios en su vida cotidiana. La distribución de interfaces de
usuario se realiza siguiendo la psicología cognitiva distribuida, es
Figure 4. a) Sistema NFCBook b) Sistema Real Game decir, teniendo en cuenta los procesos mentales del usuario.
Usabilidad y fácil aprendizaje.
La tangibilidad es el atributo que define a los dispositivos/objetos
NFCBook [14] es un libro-juego, es decir, una obra literaria que son fácilmente detectables con los sentidos. En este caso los
donde se involucra al lector en el seno de la historia permitiéndole objetos son más comunes y familiares teniendo un grado de
decidir sobre el curso de la misma. Su contenido se ha basado en tangibillidad más alto. La técnica de interacción tangible basada
la colección de libros-juegos denominada "Elige tu propia en acercar los objetos a otros objetos o dispositivos comunes
aventura". El objetivo principal de la aplicación es motivar a los camuflados en objetos ofrecen una alta tangibilidad
usuarios para que adquieran el hábito de lectura ayudándoles de proporcionando entornos más familiares y fáciles de utilizar para
esta forma a desarrollar habilidades cognitivas tales como agilidad usuarios que no tengan experiencia previa utilizando las nuevas
mental, concentración, activación del sistema visual, etc. Para tecnologías Affordance es la cualidad de un objeto o ambiente
incentivar al usuario se ha tenido en cuenta factores lúdicos que que permite a un individuo realizar una acción, en este caso, los
pretenden convertir una actividad a priori aburrida en una objetos tienen un grado de affordance más alto. Se asemejan a los
actividad que motive al usuario a participar en ella (Figura 3.a). entornos reales.
El libro que antes era físico se ha convertido en formato digital Distribución de Interfaces de Usuario en RI/DI
para ser leído y consultado desde un dispositivo móvil (tablet, El tiempo medio que tardaban los usuarios en buscar la solución
smartphone, etc.). Para interactuar con el libro-juego se utilizan correcta en el juego dependía del orden en el que se habían
objetos físicos que integran NFC en su interior. De esta forma distribuido las interfaces de usuario. Un ejemplo se dio en el juego
antes de comenzar a jugar es necesaria una fase de búsqueda y de deletrear, teníamos 27 opciones correspondientes al abecedario,
selección de tarjetas comunes que integren NFC, después el buscar la letra correcta entre 27 objetos era muy tedioso y los
usuario tendrá el control para configurarlas y posteriormente usuarios tardaban más tiempo, sin embargo cuando las letras
disfrutar del libro-juego a través del dispositivo móvil y las estaban organizadas en el panel interactivo los usuarios accedían
interfaces tangibles o tarjetas. automáticamente a la opción correcta en cada momento,
La aplicación interactiva distribuye interfaces de usuario en necesitaban menos tiempo para buscar la opción, en tareas donde
dispositivos móviles y objetos comunes. El funcionamiento es el las opciones siguen un orden como son los números, letras, etc.. es
siguiente, el usuario a través de tarjetas físicas casuales que puede más intuitivo utilizar paneles interactivos con la técnica de
encontrar en cualquier lugar, como son tickets de transportes interacción consistente en acercar y alejar el dispositivo móvil a la
públicos, entradas a espacios públicos, museos, cines, etc. interfaz de usuario tangible. Es decir, los paneles interactivos son
personaliza sus objetos y su propio juego. La aplicación ha sido más aconsejables cuando podemos agrupar opciones. En el caso
implementada en Android e internamente utiliza tecnología NFC en el que tengamos que distribuir la información o necesitas
para dar un valor a las tarjetas físicas que posteriormente pasaran a tenerlo más organizado, el panel es preferible como interfaz de
XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)
| 89 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
usuario tangible. De esta manera, los paneles pueden distribuir Tabla 1. Comparación de las propiedades de las técnicas de
menús y atajos para utilizar el sistema. interacción tangible
El número de errores debido a la técnica de interacción se Alto, medio y Approach Approach& Approach&
ocasionaban por los siguientes motivos: bajo &Remove Remove el Remove el
el objeto objeto
1º. El usuario acercaba el dispositivo móvil al panel pero no dispositivo combinado
apuntaba de forma precisa la opción que deseaba seleccionar y el móvil con ‘touch’
lector recogía el valor de la opción más cercana.
% Errores Medio Bajo Medio
2º. El usuario acercaba la interfaz de usuario tangible al
dispositivo móvil, pero no en el área donde se encuentra el lector Coste de la Medio(un Bajo Bajo
NFC, por lo que tenían que buscar el área primero y acercar la Infraestructura dispositivo
tarjeta. Este error se dio cuando comenzaron el juego y más en el móvil cada
caso de la Tablet que el móvil, debido al tamaño del dispositivo. usuario)
Cuando el usuario acercaba la interfaz tangible al dispositivo Tangibilidad Bajo Alto Alto
móvil había menos errores, el dispositivo era más pequeño y la
Affordance Medio Alto Alto
lectura de la tarjeta NFC (que se encontraba en el interior del
objeto) era más precisa. Agrupamiento Alto Medio Medio
El número de errores fue mayor al acercar el dispositivo móvil a
la interfaz de usuario tangible. En cambio la técnica de interacción Interacción Medio Alto Medio
consistente en acercar la interfaz de usuario tangible al dispositivo Social
móvil eran más precisas por lo que el número de errores fue muy
reducido.
El coste es mayor cuando se utiliza la técnica de interacción 6. RECOMENDACIONES DE DISEÑO
basada en acercar el dispositivo móvil al panel interactivo, todos Después de analizar los datos se han definido ocho directrices
los usuarios deben tener su propio móvil. Sin embargo, acercar los iniciales para el diseño de sistemas MDE que soportan interfaces
objetos a un único dispositivo conlleva un coste muy bajo. de usuario distribuidas y tangibles.
Motivación- Múltiples Usuarios Directriz 1. Se debe organizar los objetos de una forma ordenada
La interacción social se facilitaba con las técnicas de interacción y consistente. Esta es una de las reglas más comunes que se siguen
basada en acercar los objetos a un dispositivo móvil. Se mejoraba diseñando GUI (Interfaces de usuario gráficas) y siguiendo las
la comunicación, comentaban entre ellos las posibles soluciones, y teorías de la escuela psicológica de la Gestalt [11], donde
se obtuvieron más interacciones verbales entre los usuarios. Les describen que las personas percibimos los objetos como patrones
resultaba más atractivo y novedoso. Cuando interactuaban bien organizados más que como partes discretas. La mente
acercando el dispositivo móvil al panel que compartían entre configura, a través de ciertas leyes, los elementos que llegan a ella
todos, cobraba importancia los turnos para que puedan interactuar a través de los canales sensoriales (percepción) o de la memoria
todos, el número de interacciones verbales entre ellos fue menor. (pensamiento, inteligencia y resolución de problemas).
Teniendo en cuenta la percepción visual del usuario y las
lecciones aprendidas. Se deben agrupar objetos de la misma
naturaleza en un panel, tabla, surface, etc. como por ejemplo
(Abecedario, números, opciones gestionar ficheros: abrir, guardar,
etc.). Los paneles se van a utilizar como atajos para realizar
acciones en el sistema que se llevan a cabo repetidas veces.
Directriz 2. Se deben dividir los espacios de trabajo en dos tipos
diferente. Por un lado el privado donde el usuario tiene el control
de su información, dispositivos, objetos, etc. Por otro lado el
espacio de trabajo colaborativo donde el usuario puede ver los
datos de los demás usuarios, enviar información a todos los demás
componentes del grupo, etc. De esta forma conseguimos que el
usuario tenga el control de su información dándole confianza [18].
Directriz 3. Añadir el componente “Espacio físico” donde el
entorno ofrece información al usuario permitiéndole explorar e
interactuar, reduce la carga cognitiva. En este caso, las interfaces
de usuario visuales deben ser sencillas e intuitivas con el fin de no
despistar al usuario. Además los mensajes auditivos, sensoriales,
etc. Son preferibles para no saturar al usuario con la información
visual.
Directriz 4. Añadir señales, pistas, etc. Cuando el usuario necesite
explorar a través del entorno. De esta forma se consigue que el

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 90 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
usuario no se sienta perdido y sepa en cada momento donde se in computing systems, pages 234–241, New York, NY, USA.
encuentra. ACM Press. 19, 20, 27, 36, 73, 96, 187
Directriz 5. Se debe reducir la distancia de los usuarios respecto a [3] Elmqvist, N. Distributed User Interfaces: State of the Art
los objetos que se utilizan repetidas veces mientras se usa el .Workshop on Distributed User Interfaces2011 (DUI) at the
sistema. Estos objetos deben estar cerca. 29th ACM CHI Conference on Human Factors in
Directriz 6. El Sistema debe avisar al usuario a través de Computing Systems 2011, ISBN: 978-84-693-9829-6,
alertas/notificaciones/pistas donde se está mostrando la Vancouver, Canadá, May 7-12, 2011.
información importante en cada momento de forma que el usuario [4] González,P.,Gallud,J.A.,Tesoriero,R. WallShare: A
preste atención a la información relevante. Collaborative Multi-pointer System for Portable Devices.
Directriz 7. Añadir componentes motivacionales es recomendable PPD10: Workshop on coupled display visual interfaces.
con el objetivo de que el usuario disfrute y se sienta inmerso en la May 25, 2010: Rome, Italy.
tarea. Se pueden añadir de dos tipos diferentes: Motivación [5] Vanderdonckt, J. Distributed User Interfaces: How to
intrínseca: El usuario debe sentir que tienen el control del juego, Distribute User Interface Elements across Users,
añadirle la posibilidad de comunicación con otros usuarios, la Platforms, and Environments. Proc. of XI Interacción, 20-
facilidad de explorar en el entorno y crearle desafíos. Extrínseca. 32, 2010.
Añadir mecanismos lúdicos, es decir puntos, ranking, premios,
refuerzo positivo, etc. [6] Vandervelpen,C. , Coninx, K. Towards model based design
support for distributed user interfaces, Proceedings of the
Directriz 8. Se debe tener en cuenta el awareness, es decir, el third Nordic conference on Human-computer interaction,
usuario debe ser capaz de percibir, sentir, o ser consciente de los p.61-70, October 23-27, 2004, Tampere,
acontecimientos, objetos, acciones, etc tanto del entorno como de Finland [doi>10.1145/1028014.1028023]
los demás usuarios. Es decir, cuando el usuario trabaje con grupos
de usuarios debe ser consciente de los demás usuarios y a la vez [7] Want, R., Fishkin, K.P., Gujar, A., Harrison, B.L.: Bridging
ser capaz de comunicarse con ellos y compartir información physical and virtual worlds with electronic tags. In:
fácilmente. La comunicación entre los usuarios debe evitar que sea Proceedings of the SIGCHI conference on Human factors
a través de mensajes escritos en dispositivos con reducida in computing systems: the CHI is the limit, ACM Press,
pantallas, es preferible que sea a través de botones, gestos, etc. Pittsburgh, Pennsylvania, United States, 1999.
Estas deberían ser mostradas a los demás a través de [8] Romero, S., Tesoriero, R., González, P. , Gallud, J. A.,
notificaciones, sonidos, mensajes sensoriales, imágenes, etc. Penichet, V. M. R.: Interactive System based on NFC to
manage Georeferenced Documents. Interacción 2009, X
7. CONCLUSIONES Congreso Internacional de Interacción Persona-Ordenador.
Actualmente nos encontramos con escenarios que distribuyen Barcelona. Septiembre 2009. ISBN-13:978-84-692-5005-1
interfaces en entornos con múltiples dispositivos. Interactuar con
diferentes interfaces no es una tarea trivial, por ese motivo hemos [9] Tesoriero, R., Tébar, R., Gallud, J. A., Penichet, V. M. R.,
propuestos la interacción tangible como mecanismo de Lozano, M.: Interactive EcoPanels: Ecological Interactive
interacción, esta resulta intuitiva y permite que la mayoría de los Panels based on NFC . Proceedings of the IX Congreso
usuarios se sientan familiarizados con objetos comunes, resultando Internacional de Interacción Persona-Ordenador Interacción
más agradable su uso. Las interfaces de usuario se han distribuido 2008. ISBN: 978-84-691-3871-7; pp 155-165.
siguiendo la psicología distribuida y se han añadido componentes [10] Hutchins, Edwin (1995). Cognition in the Wild.
para motivar a los usuarios a usar los sistemas. Después de Cambridge, MA: MIT Press.
analizar las evaluaciones realizadas de cuatro prototipos se han [11] Arnheim, R. (1985). Arte y Percepción Visual. Alianza
obtenido ocho directrices de diseño iniciales. Estos resultados son Editorial. Madrid
preliminares, como trabajo futuro se pretende realizar una
evaluación más exhaustiva con el fin de obtener criterios y [12] Elena de la Guía, María Lozano, and Victor R. Penichet.
patrones que ayuden a futuros diseñadores e investigadores a 2013. TrainAb: a solution based on tangible and distributed
construir sistemas de esta naturaleza. user interfaces to improve cognitive disabilities (CHI EA
'13). ACM, New York, NY, USA, 3039-3042.
8. AGRADECIMIENTOS DOI=10.1145/2468356.2479605
Esta investigación ha sido parcialmente subvencionada por el http://doi.acm.org/10.1145/2468356.2479605
proyecto regional con referencia POII-2014-008-A. Agradecemos [13] Elena de la Guía, María Dolores Lozano, Victor M. Ruiz
especialmente a Érica González, Yolanda Cotillas, Francisco Penichet: Tangible and Distributed User Interfaces to
Vizcaino, Ruben Nieto, Vicente López y Mª Dolores Monedero su Improve Cognitive Abilities within People Affected by
colaboración en este trabajo de investigación, igualmente Alzheimer's Disease. DUI@EICS 2013: 10-13
queremos agradecer a las asociaciones AFA, ACMIL y
ASPRONA de Albacete por su participación en las evaluaciones. [14] Elena de la Guía, María D. Lozano, Víctor M. R. Penichet,
and Rubén Nieto. 2014. NFCBOOK: GameBook Digital
9. REFERENCIAS Based on Tangible User Interfaces (Interacción '14). ACM,
[1] Schneiderman, B. Direct manipulation: a step beyond New York, NY, USA, Article 32 , 3 pages.
programming languages. IEEE Cornput. (Aug. 1983), 57-69. DOI=10.1145/2662253.2662285
http://doi.acm.org/10.1145/2662253.2662285
[2] Ishii, H. and Ullmer, B. (1997). Tangible bits: towards
seamless interfaces between people, bits and atoms. In CHI [15] Elena de la Guía, María D. Lozano, and Víctor M. R.
’97: Proceedings of the SIGCHI conference on Human factors Penichet. 2012. Interaction and collaboration supported by

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 91 |
 Interfaces de Usuario Tangibles como Mecanismo de Interacción en Entornos Multi-dispositivos
E. de la Guía | M. D. Lozano | V. Penichet | L. Orozco | V. López
distributed user interfaces: from GUIs to September 18-19, 1980, Palo Alto, California, USA
DUIs. (INTERACCION '12). ACM, New York, NY, USA,, [doi>10.1145/800088.802839]
Article 53 , 2 pages. DOI=10.1145/2379636.2379688 [18] Sara Streng , Karsten Stegmann , Sebastian Boring , Sonja
http://doi.acm.org/10.1145/2379636.2379688 Böhm , Frank Fischer , Heinrich Hussmann, Measuring
[16] Lepper, M. R., & Malone, T. W. (1987). Intrinsic effects of private and shared displays in small-group
motivation and instructional effectiveness in computer- knowledge sharing processes, Proceedings of the 6th
based education. In R. E. Snow, & M. J. Farr (Eds.), Nordic Conference on Human-Computer Interaction:
Conative and affective process analyses: Vol. 3. Aptitude, Extending Boundaries, October 16-20, 2010, Reykjavik,
learning and instruction (pp. 255--286). Hillsdale, NJ: Iceland [doi>10.1145/1868914.1869030]
Lawrence Erblaum [19] Penelope Sweetser , Peta Wyeth, GameFlow: a model for
[17] Thomas W. Malone, What makes things fun to learn? evaluating player enjoyment in games, Computers in
heuristics for designing instructional computer games, Entertainment (CIE), v.3 n.3, July 2005
Proceedings of the 3rd ACM SIGSMALL symposium and [doi>10.1145/1077246.1077253]
the first SIGPC symposium on Small systems, p.162-169,

XVI Congreso Internacional de Interacción Persona-Ordenador | Natural User Interfaces (II)

| 92 |
METHODOLOGIES AND
MODELS
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González

Limitaciones del Modelo de Tareas del W3C para

aplicaciones Post-WIMP
Miguel A. Teruel, Arturo C. Rodríguez, Francisco Montero,
Elena Navarro, Víctor López-Jaquero and Pascual González
Grupo de Investigación LoUISE
Instituto de Investigación en Informática de Albacete
02071 – Albacete (España)
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected]

RESUMEN gran cantidad de datos o los juegos altamente interactivos.

La evolución tecnológica y la aparición de nuevos dispositivos También tienen deficiencias en aplicaciones que requieren una
que ésta conlleva está provocando grandes cambios en el modo en monitorización continua de señales de entrada, representación de
que las personas interaccionan con los sistemas software. Cada modelos 3D o simplemente en interacciones para las que no hay
vez son más los sistemas cuyas técnicas de interacción están widgets estándar. Por esta razón, las interfaces de usuario
basados en realidad virtual, reconocimientos de gestos, personalizadas se han convertido en la mejor alternativa a la hora
dispositivos corporales y muchos otros, que se alejan de realizar tareas complejas.
notablemente de la interacción clásica de los sistemas Post-WIMP es un término usado para describir interfaces que van
denominados WIMP (Windows, Icons, Menus, Pointers). Estos más allá de los paradigmas habituales (ventanas, iconos, menús y
nuevos sistemas, a los que se denomina sistemas Post-WIMP, punteros) [1]. Esta nueva generación de interfaces ha sido
necesitan, por lo tanto, que los lenguajes para la especificación de impulsada tanto por los avances en tecnología como por un mejor
sistemas evolucionen de igual forma para poder hacer frente al entendimiento de la psicología humana. Definidas por van Dam
incremento de la complejidad y a los nuevos conceptos inherentes [1] como interfaces “que contienen al menos una técnica de
a sus formas de interacción. En este artículo se presenta un interacción no dependiente de los clásicos widgets 2D, como
lenguaje de modelado de tareas de sistemas Post-WIMP, basado a menús o iconos”, algunos ejemplos de interacciones Post-WIMP
su vez en CSRML, como alternativa al actual meta-modelo de son: (i) realidad virtual, aumentada o mixta, (ii) interacción
tareas propuesto por el W3C. Este nuevo lenguaje tiene como tangible, (iii) computación ubicua, (iv) computación sensible al
finalidad especificar las tareas que serán realizadas por el usuario contexto, (v) interacción con dispositivos móviles, (vi)
a través de la interfaz de sistemas Post-WIMP con la expresividad computación perceptual, e (vii) interacción ligera, tácita o pasiva.
de la que carece el actual meta-modelo de tareas del W3C. Cabe Aunque algunas personas podrían clasificar estos estilos de
destacar principalmente dos ventajas de esta nueva propuesta interacción como innovaciones dispares, futuristas y heterogéneas,
respecto a la del W3C. Por un lado, se han incluido varios comparten importantes características como una considerable
elementos y relaciones para enriquecer el nuevo meta-modelo con demanda de información de contexto que ayude a entender,
la expresividad necesaria para llevar a cabo la especificación de la conectar y analizar el trabajo que se lleva a cabo.
colaboración de forma detallada. Por otro lado, nuestra propuesta
permite representar la información de awareness que los usuarios Esta evolución de las interfaces requiere que los lenguajes para el
Post-WIMP deberían tener durante la interacción con el sistema. análisis de requisitos y la especificación de sistemas sean
Para mostrar la utilización de este nuevo meta-modelo, adaptados de forma adecuada, pudiendo así recoger las
presentamos un ejemplo de modelado basado en un juego “first necesidades de información de estos nuevos paradigmas de
person shooter” Post-WIMP con una interfaz de realidad virtual. interacción. El análisis de tareas no es una excepción y, a pesar de
que los meta-modelos actuales han supuesto una herramienta
Categorías y descriptores de tema eficaz en el desarrollo de sistemas WIMP, las nuevas interfaces
H.5.2 [User Interfaces]: Interaction styles., I.3.6. [Methodology Post-WIMP introducen la necesidad de revisarlos. Además, el
and Techniques]. Interaction techniques. análisis de tareas está estrechamente relacionado con la
interacción, puesto que, como afirman Gea y Gutiérrez [2], cubre
Términos Generales diferentes técnicas orientadas a describir las interacciones entre
Modelado, Interacción, Experimentación, Lenguajes. las personas y los entornos de una manera sistemática y tiene
como objetivo obtener una descripción formal del conjunto de
Palabras clave acciones que debe realizar el usuario para lograr un fin; acciones
Post-WIMP; CSRML; W3C task model; collaboration; awareness. que pueden ser tanto cognitivas como interacciones con la
interfaz.
1. INTRODUCCIÓN
Durante las últimas dos décadas, investigadores en Interacción En este artículo, proponemos una alternativa al actual meta-
Persona-Ordenador (HCI) han desarrollado una amplia gama de modelo de tareas del W3C [3] considerando dos de los principales
nuevas interfaces que se alejan de las tradicionales WIMP aspectos clave de los sistemas Post-WIMP: awareness (habilidad
(Window, Icon, Menu, Pointing). Estas interfaces WIMP del usuario de ser consciente de lo que le rodea) y colaboración
presentan carencias para llevar a cabo tareas complejas como el entre usuarios. La necesidad de incorporar dichos conceptos
diseño asistido por ordenador, la manipulación simultanea de una surge, no sólo del estudio de la literatura relacionada, sino

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 94 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González
también de la amplia experiencia acumulada en nuestro grupo de pueden definirse cuatro tipo de tareas: (i) User, o tareas
investigación a lo largo de los últimos quince años en el desarrollo de usuario: actividad cognitiva que no se realiza
de interfaces post-WIMP, que se inició con sistemas de directamente a través de la interfaz. (ii) System, o tarea
interacción 3D [4]. de sistema: acción realizada por la aplicación. (iii)
Interaction: acciones realizadas por los usuarios que dan
Como base para esta propuesta, se ha elegido el lenguaje CSRML lugar a respuestas instantáneas del sistema y (iv)
(Collaborative Systems Requirements Modelling Language, Abstract: tareas que se descomponen en subtareas de
[5][6]) ya que dispone de capacidad expresiva para especificar las distintas categorías.
necesidades de colaboración y awareness. Además, otra razón
para escoger este lenguaje es que ha sido evaluado para tratar con • Domain Object: un objeto del modelo de dominio
diferentes dominios, como la edición colaborativa de documentos relacionado con una tarea.
[5] o la inteligencia de negocio [7]. No obstante, el modelo de
tareas CSRML ha sido ampliado y modificado para conseguir dos • Condition Group: operación lógica que puede ser usada
objetivos. En primer lugar para hacerlo compatible con el modelo para especificar una precondición y/o una post-
W3C y en segundo lugar, para enriquecerlo con elementos de condición de una tarea.
awareness ligados a sistemas Post-WIMP, ya que CSRML estaba • Logic Operator: operador que conecta los operandos del
diseñado para atender básicamente a las necesidades de los Condition Group y especifica el tipo de operación
sistemas colaborativos . lógica.
Este artículo está organizado de la siguiente manera. La Sección 2 • Condition Literal: valor u objeto del modelo del
presenta el estado de la cuestión relacionada con el actual meta- dominio usado como operando en una operación lógica.
modelo de tareas propuesto por la W3C. En base a las deficiencias
detectadas en la propuesta de la W3C, en la Sección 3 se presenta • N-ary Operator: operador que establece una restricción
el meta-modelo de tareas para interfaces Post-WIMP. Para que temporal entre subtareas.
puedan apreciarse más claramente las diferencias de la propuesta
de la W3C y la nuestra, en la Sección 4 se presenta un ejemplo de • 1-ary Operator: operador que establece si una tarea es
modelado basado en un juego Post-WIMP que se ha modelado opcional y/o iterativa.
usando ambos meta-modelos de tareas. Finalmente, en la Sección
5 se presentarán algunas conclusiones.

2. META-MODELO DE TAREAS DEL W3C

Gea y Gutiérrez [2] definen el análisis de tareas como el estudio
de lo que un usuario tiene que realizar en términos de acciones
y/o procesos cognitivos para conseguir un objetivo. Este análisis
se traduce en modelos de tareas que tradicionalmente, han
proporcionado una descripción eficaz de sistemas interactivos
WIMP. Representan información estática de las tareas de
aplicación y de usuario a dos niveles: estructural (descomposición
de tareas) y temporal (relaciones temporales). Cada tarea
especificada en el modelo de tareas es una actividad o proceso
cuya realización es necesaria para conseguir los objetivos del
usuario. Además, los analistas pueden representar tareas en
diferentes niveles de abstracción, por lo que pueden considerar Figura 1. Meta-modelo actual del W3C [3]
una actividad como una tarea atómica o descomponerla en sub- El meta-modelo mostrado en la Fig. 1 permite al analista describir
tareas, con el objetivo de proporcionar una especificación con modelos de tareas de sistemas WIMP de forma eficaz. Este meta-
mayor detalle, y después, establecer relaciones temporales entre modelo usa las relaciones de composición para especificar
ellas. Sin embargo, esta descripción no tiene en cuenta detalles estructuras jerárquicas de tareas y asocia los operadores
sobre cómo se llevan a cabo los procesos e interacciones, ambos temporales a estas relaciones. Aunque estos operadores son
necesarios para el correcto diseño del diseño del sistema. suficientemente expresivos para describir dichas relaciones, sería
razonable incluir una nueva meta-clase específica para establecer
El modelo de tareas del W3C está basado en la notación
relaciones temporales entre tareas, asociando el operador a dicha
ConcurTaskTrees (CTT), ampliamente aceptada [8] y tiene como
clase, permitiendo así desligar las relaciones estructurales de las
objetivo definir un estándar para el modelado de tareas. El meta-
temporales y estableciendo un único operador para cada relación
modelo de tareas del W3C proporciona una estructura jerárquica
temporal.
entre tareas y ofrece varios operadores, usados también en CTT,
para definir las relaciones temporales entre tareas. Además, Las aplicaciones actuales tienden claramente a ser colaborativas
proporciona una taxonomía de tipos de tareas. Los elementos del (por ejemplo, juegos, editores de texto, redes sociales, IDEs, etc.)
meta-modelo de tareas (¡Error! No se encuentra el origen de la con un considerable número de usuarios interactuando
referencia.) son los siguientes: concurrentemente. Además, las interfaces de usuario de dichas
aplicaciones han evolucionado enormemente hacía el paradigma
• Task: acción que se realiza desde la perspectiva del Post-WIMP. Así, Jacob et al. [9] identificaron los siguientes
usuario. Desde el punto de vista estructural, hay dos cuatro aspectos del mundo real que una aplicación Post-WIMP
tipos de tareas: Tareas atómicas (acciones indivisibles) debería soportar:
y tareas compuestas (divisibles en subtareras). Además,

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 95 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González
• Naïve Physics: las personas tienen conocimiento general Se ha escogido este lenguaje porque soporta la especificación de
del mundo físico. requisitos de sistemas colaborativos, uno de los elementos
principales de los sistemas Post-WIMP. CSRML es un lenguaje
• Body Awareness & Skills: las personas tienen de modelado de requisitos orientado a objetivos para la
conciencia/conocimiento de sus propios cuerpos y especificación de sistemas colaborativos respaldado por una
poseen habilidades para controlar y coordinar sus herramienta empíricamente validada [13]. Los elementos usados
cuerpos. para el modelado de tareas se describen a continuación (ilustrados
• Enviroment Awareness & Skills: las personas son en Fig. 3 y Fig. 4). Además, la representación gráfica de dichos
conscientes de lo que les rodea y poseen habilidades elementos se puede ver en la Fig. 2.
para negociar, manipular y moverse por el entorno. • Task: El concepto de tarea del W3C ha sido extendido
• Social Awareness & Skills: las personas son para incluir la importancia usando un código de colores
generalmente conscientes de otras personas en su (verde, amarillo, naranja y rojo) siendo el color verde el
entorno y tienen habilidades para interactuar con ellos. de menor importancia y el rojo el de mayor importancia.
Además, se han identificado cuatro tipos de tareas: (i)
Como se puede ver, todos estos aspectos están principalmente Abstract Task: Abstracción de un conjunto de tareas
relacionados con el concepto de awareness [10]. Por ejemplo, la concretas, (ii) User Task: Tarea realizada por un usuario
propiocepción del usuario en el entorno físico juega un papel sin necesidad de interactuar con la interfaz e (iii)
fundamental en entornos de realidad virtual en los que puede Interaction Task: Tarea que implica una interacción con
existir una interacción con cualquier parte del cuerpo. Una la interfaz o respuestas del sistema. Hay cuatro tipos de
representación virtual del cuerpo y otras informaciones, como la tareas de interacción: Individual Task es una tarea que
localización virtual pueden mejorar la experiencia de usuario al no conlleva colaboración (puede ser realizada por un
interactuar con sistemas Post-WIMP. El conocimiento de los participante). Estas tareas contemplan también las tareas
grupos o equipos a los que pertenecen los usuarios es un ejemplo de sistema, ya que éste es considerado como un actor
de Social Awareness. Otro ejemplo de awareness lo encontramos más dentro del modelo. Collaboration/
en sistemas en los que varios usuarios están editando un Communication/Coordination tasks son tareas que
documento al mismo tiempo, necesitan saber con quién están requieren al menos dos participantes para poder ser
colaborando y qué cambios están haciendo en el documento. Esta realizadas, con el objetivo de llevar a cabo algún tipo de
necesidad ha sido ampliamente analizada en Computer Supported colaboración / comunicación / coordinación entre ellos.
Collaborative Work (CSCW). Ellis et al. [11] identificaron las
fuertes relaciones entre awareness y colaboración en el modelo de • Legacy Elements: Con el fin de hacer este modelo de
colaboración 3C. Debido a que awareness y colaboración están tareas compatible con el del W3C, varios elementos han
muy extendidos en las aplicaciones contemporáneas, sido añadidos a su meta-modelo.
especialmente en entornos virtuales colaborativos (Collaborative • Role: Establece el comportamiento de un actor para un
Virtual Environments CVE) [12], un meta-modelo de tareas conjunto de tareas relacionadas. Un actor que
debería permitir especificar tales características. Por lo tanto, desempeña un rol puede participar en una tarea
puesto que la especificación actual del W3C fue originalmente individual o colectiva (mediante las relaciones de
diseñada para especificar entornos WIMP, en este artículo participación). Cabe señalar que la relación entre un
presentamos un meta-modelo de tareas extendido para entornos actor y un rol es especificada en el modelo de usuario,
Post-WIMP basado en CSRML con el fin de especificar estos no en el modelo de tareas.
innovadores sistemas.
• Goal: Contesta a la pregunta “¿por qué?”. Describe un
3. UN LENGUAJE DE MODELADO DE estado del mundo que el usuario querría alcanzar. Sin
TAREAS PARA ENTORNOS POST-WIMP embargo, un Goal (objetivo) no describe cómo debería
Una vez se han identificado los problemas y limitaciones del alcanzarse dicho estado.
meta-modelo de tareas del W3C para especificar sistemas Post-
WIMP, presentamos en esta sección el meta-modelo de tareas
CSRML [5] adaptado para la especificación de tareas Post-WIMP.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 96 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González

D
Dependency

Means-end link

Task decomposition link

+/-/?
Contribution link
[Guard]
Playing link

Responsibility link
1..*
Participation link

Figura 2. Representación gráfica de los elementos del meta-modelo de tareas para sistemas Post-WIMP

Figura 3. Meta-modelo de tareas para sistemas Post-WIMP (Elementos)

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 97 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González

Figura 4. Meta-modelo de tareas para sistemas Post-WIMP (Relaciones)

• Softgoal: Es una condición del mundo que el usuario What – Group goal es un ejemplo de Social Awareness.
quiere alcanzar, pero a diferencia del concepto Goal, Además, estos elementos de awareness están
esta condición no está claramente definida. Un Softgoal clasificados en cuatro categorías relacionadas con el
es normalmente un atributo cualitativo que restringe presente, el pasado, el futuro y aspectos sociales. Por
otros elementos, como un objetivo, una tarea o un último, su importancia puede ser establecida como nice
recurso. to have (N), desirable (D), highly desirable (HD) o
• Resource: Es una entidad que el usuario necesita para mandatory (M).
alcanzar un objetivo o realizar una tarea. El principal • Participation Link: denota quién está involucrado en una
interés de un recurso es saber si está disponible y de tarea. Esta relación tiene un atributo para especificar su
quién. cardinalidad, es decir, el número de usuarios que pueden
• Awareness Resource: Este elemento representa involucrarse en la tarea. Opcionalmente puede tener un
información de awareness que un usuario necesita para Awareness Resource ligado que representa que el Role
completar una tarea. Este elemento es representado tiene una necesidad especial de percepción para
como un conjunto de elementos de Awareness participar en la tarea. Sin esta percepción, la realización
(Awareness Elements) ligados a una relación de de la tarea se vería afectada negativamente o incuso el
participación entre un rol y una tarea. Role podría ver impedida su participación en la tarea.
• Awareness Element: Un Awareness Resource muestra • Task Decomposition Link: Describe elementos
todos los Awareness Elements disponibles (si son esenciales de una tarea. Un Task Decomposition Link
necesarios) e indica su importancia en función de la relaciona una tarea con subtareas a realizar, subgoals a
contribución a la realización de la tarea. Estos conseguir, Resources necesarios y softgoals que
elementos de awareness son una extensión de los normalmente definen la calidad de los objetivos para la
identificados por Gutwin y Greenberg [14] para tarea.
entornos Post-WIMP con el propósito de ser • Means-end Link: documenta qué softgoal, task o
compatibles con los diferentes elementos de awareness resource contribuyen a alcanzar un objetivo.
identificados por Jacob et al. [9]. Por ejemplo, el • Dependency: Es una relación ente un depender y un
elemento Presence – Where – Reach está relacionado dependee para un dependum. El depender y el dependee
con el elemento de Jacob Body awareness; el elemento son actores y el dependum puede ser un goal, una tarea
Future – Where – Next Position está relacionado con el (task), un resource o un softgoal. El depender depende
conocimiento del entorno; y Social & group dynamics – del dependee para alcanzar un objetivo, realizar una

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 98 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González
tarea o usar un recurso. Si el dependee no puede equipo), Shoot, Aim, Pull The Trigger, Get Power-ups, Deploy
proporcionar un dependum al depender la realización de Power-up (tarea de sistema), Show End (tarea de sistema).
la tarea, alcanzar el objetivo o usar el recurso se vuelve Además, puesto que el juego a implementar es tanto colaborativo
difícil o imposible. como Post-WIMP, puede proporcionarse a los usuarios
información de awareness sobre el entorno del juego (por
4. MODELADO DE UN SISTEMA POST- ejemplo, estado del jugador, posición de otros personajes, terreno
WIMP de juego, etc.). Concretamente, la información de awareness que
Para analizar el meta-modelo propuesto, presentamos un ejemplo será proporcionada por el sistema se describe en la Tabla 1.
de modelado que consiste en un juego First Person Shooter (FPS) Una vez explicadas las tareas del juego y las necesidades de
multi-dispositivo de realidad virtual. Este sistema hará uso de awareness, todo ello será modelado usando tanto el actual meta-
diferentes dispositivos para facilitar la interacción entre el usuario modelo de tareas de la W3C (en adelante W3CTM) como nuestra
y el sistema. En primer lugar utilizará un dispositivo de propuesta PWTM (Post-WIMP Task Meta-model).
visualización inmersivo para proporcionar al usuario los gráficos
3D así como para capturar los movimientos de cabeza del usuario Tabla 1. Información de awareness del FPS
para controlar la dirección de la visión de la representación virtual Tipo de Awareness Descripción
del usuario (personaje). Para proporcionar al usuario sonido Own status Información sobre el estado de salud del
posicional, permitiéndole saber la dirección de los disparos y los personaje y los power-ups activos
pasos de otros personajes, se dispone de auriculares de sonido Connected players Lista que mostrará tanto los jugadores
envolvente, los cuales incorporan un micrófono para comunicarse conectados como sus equipos.
con otros usuarios. Por otra parte para incorporar sensaciones Allies location Localización de los usuarios miembros del
hápticas se situarán varios actuadores vibrotáctites sobre el mismo equipo es mostrada en un mini-mapa
cuerpo del usuario para permitir que éste sienta en qué lugar del Power-ups position Las nuevas mejoras del arma y del personaje
cuerpo y cuándo el personaje es alcanzado. Se situarán 7 que se han desplegado en el juego se
muestran en el mini-mapa.
dispositivos colocados en cabeza, brazos, pecho y espalda. Para
Received Damage Cuando el personaje de un usuario es
controlar la posición de la mano dominante del usuario que simula alcanzado por una bala, el usuario sentirá una
el arma, se utilizará un tracker que recoja la posición de la mano. vibración en su propio cuerpo, obteniendo así
Por lo tanto, la dirección y el disparador/gatillo del arma del información de awareness de la dirección del
personaje serán controlados con este dispositivo. Finalmente, en disparo (por ejemplo si el personaje es
su mano no dominante, el usuario dispondrá de un mando para alcanzado desde atrás, el usuario sentirá una
controlar los movimientos del personaje (adelante, atrás y a los vibración en su espalda).
lados). Caused Damage Cuando un personaje es alcanzado por un
disparo se proporciona información auditiva.
Por otra parte, las tareas que un usuario realizará durante el juego Steps and Shooting Las armas y los pasos de los personajes
son: DefeatEnemy (tarea principal), Move, Communicate sound producirán sonidos que pueden dar
(dependerá de la distancia entre los jugadores), Revive Player (dos información de awareness sobre su
jugadores pueden devolver la vida a otro jugador del mismo localización.

Figura 5. Especificación de tareas usando el meta-modelo del W3C

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 99 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González

Figura 6. Especificación de tareas usando el meta-modelo Post-WIMP

4.1 USANDO W3CTM PARA relacionada con la propiocepción del usuario y de las dinámicas
sociales y de grupo.
ESPECIFICAR LAS TAREAS DEL JUEGO
La Fig. 5 muestra una instancia del modelo obtenido usando el 4.2 Post-WIMP Task Meta-model PARA
meta-modelo de la W3C para nuestro ejemplo de modelado.
Como puede observarse, la tarea DefeatEnemy es la raíz del árbol
ESPECIFICAR LAS TAREAS DEL JUEGO
de tareas. Esta tarea se descompone en otras subtareas. Move, Tras especificar las tareas del ejemplo de modelado usando el
LookAt, RevivePlayer, Communicate, Listen y DeployPowerUp meta-modelo de la W3C, se procederá al modelado del mismo
que pueden ser realizadas de forma concurrente. DeployPowerUp sistema usando nuestra propuesta. Como puede observarse en la
habilita la tarea GetPowerUp puesto que ésta no puede ser Fig. 6, esta especificación enriquece la mostrada anteriormente
(Fig. 5) añadiendo nuevos elementos y relaciones que aportan
realizada si no hay power-ups en el escenario. GetPowerUp
habilita la tarea Shoot y ésta, ShowEnd. La tarea Move puede ser características de colaboración y awareness al modelado. Puesto
realizada de dos formas diferentes, por lo que se descompone que este nuevo modelo está basado en un lenguaje de requisitos
Walk y Run y el operador N-Ary entre estas tareas será el operador orientado a objetivos [15], se ha añadido el objetivo Achieve the
Choice. Además, el sistema muestra el movimiento sobre el mapa defeat of enemies al modelo de tareas, ya que es el principal
de forma concurrente. Para conseguir power-ups, el usuario debe objetivo de los usuarios que realizan la tarea Play Game. Además,
moverse a través del escenario virtual. Cuando el personaje pasa a tres recursos llamados Speed, Reach y Power, correspondientes a
través de un ítem virtual que representa el power-up, el sistema tres diferentes power-ups han sido incluidos en el modelo de
tareas. Otra diferencia es que las tareas de interacción han sido
informa al usuario mediante la tarea ShowPowerUp. La tarea Shot
se descompone en dos subtareas: Aim y PullTheTrigger. Ambas detalladas definiendo su tipo y prioridad. Como ejemplo, la tarea
tareas pueden ser realizadas de forma concurrente. Cuando un Communicate es considerada una Communication Task, ya que
equipo gana un juego, el sistema informa a los usuarios del final. permite a los usuarios de un mismo equipo comunicarse entre
El modelo, mostrado en la Fig. 5, presenta varias deficiencias de ellos. Las tareas Listen y Pull Trigger son consideradas
cara a posteriores fases del proceso de desarrollo. En primer lugar, Coordination Tasks. Estas tareas de coordinación son un tipo
no aparecen los participantes de las tareas ni el concepto de rol, lo especial de tareas colaborativas en las que distintas informaciones
que impide describir quiénes intervienen en tareas como (ruido y salud en nuestro caso) permiten coordinarse a varios
jugadores. Además, la tarea Revive Player, es una Collaborative
Communicate o RevivePlayer y el papel que juegan en las
mismas. Por otro lado, este modelo no nos permite definir la Task ya que dos diferentes personajes tienen que aproximarse a un
información de awareness necesaria para la realización de algunas tercero que ha muerto para poder revivirlo. Finalmente, otras
tareas. Por ejemplo, la tarea Communicate, según la propia tareas como Look At Game Scenario o Walk son tareas
especificación del ejemplo, sólo puede llevarse a cabo cuando la individuales que pueden ser realizadas por un solo usuario. Estas
posición de los participantes hace que estén lo suficientemente dos tareas han sido asignadas con la prioridad más alta (color
cerca, por lo que esta información es relevante para el desarrollo rojo) ya que deben ser implementadas en primer lugar para
del sistema. Por último, en un entorno virtual de inmersión obtener un prototipo de la aplicación lo antes posible. Por esta
completa, la tarea de dirigir la mirada hacia distintas zonas del misma razón, Aim también ha sido considerada una tarea de alta
prioridad, mientras que Listen y Deploy Power-Up han sido
escenario requiere de información de awareness que está
consideradas con una prioridad normal porque en nuestro primer

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 100 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González
prototipo no es absolutamente necesario tener el sonido o el identificadas por Jacob [9], el modelo de awareness de CSRML,
sistema de power-ups totalmente desarrollado. llamado Gutwin’s Workspace Awareness [14] ha sido enriquecido
Como se muestra en la Fig. 6, la especificación Post-WIMP no con características especiales como Context Awareness o Social
tiene tareas de sistema. Cuando una tarea es realizada por el Awareness. Además, se ha presentado un ejemplo de modelado en
sistema, se indicará con la participación del Role “System” en el que se ha modelado un juego Post-WIMP tanto con la actual
dicha tarea (por ejemplo la tarea Deploy Power-Up es realizada propuesta del W3C como con el lenguaje presentado en este
por el sistema). Además, otros cuatro roles toman parte en esta trabajo. Esto nos ha permitido ilustrar que las características Post-
especificación: Player, Allies, Enemies y Human, que es el WIMP relacionadas con el awareness y la colaboración sólo
“super-role” de los tres primeros. Sin embargo, hay que señalar pueden ser especificadas con éste último.
que la asociación del usuario con los roles que puede jugar se
especifica en el modelo de usuario, no en el modelo de tareas, que A pesar de que este modelo ha mostrado ser más adecuado para
no se presenta aquí por problemas de espacio. sistemas Post-WIMP, es necesario profundizar más en este
Los Participation Links son usados para especificar qué usuarios sentido. Con el objetivo de mejorar la especificación de requisitos
están involucrados en tareas de interacción. Por ejemplo, dos de sistemas Post-WIMP, nuestros siguientes trabajos se centrarán
diferentes Participation Links están relacionados con la tarea en un modelo de contexto que complemente el modelo de tareas.
Communicate, porque cuando se realiza esta tarea un jugador (1) Así, mediante el uso de dicho modelo, será posible especificar
habla con uno o más (1..*) de sus aliados. cómo las aplicaciones Post-WIMP pueden adaptarse al contexto
Finalmente, cinco diferentes recursos de awareness se han de uso (es decir, hardware disponible, capacidades del usuario,
adjuntado a los Participation Links para representar la características del entorno físico como luz y ruido, etc.). Al
información de awarenes requerida para realizar las tareas. mismo tiempo, la implementación del caso de uso presentado en
Primero, el jugador debe ([M]) tener el recurso de Awareness este trabajo ya ha comenzado. Ya está disponible un prototipo
Allies location para proporcionarle la localización de sus aliados a jugable basado en la especificación de tareas creada mediante la
la hora de realizar la tarea Communicate, ya que la comunicación propuesta de meta-modelo de tareas post-WIMP recogida en este
sólo puede llevarse a cabo si los miembros del equipo están cerca. artículo.
Adicionalmente, sería deseable ([N]) tener información sobre 6. AGRADECIMIENTOS
dónde miran otros jugadores para coordinar ataques conjuntos.
Este trabajo está financiado por el Ministerio de Economía y
Además, cuando se realiza la tarea Look At Game Scenario, los
Competencia de España y por la fundación FEDER de la UE bajo
jugadores necesitan información de Awareness sobre su propio
el proyecto insPIre (TIN2012-34003). También ha sido financiado
estatus, algo que es altamente deseable ([HD]), para ser
por el Ministerio de Educación, Cultura y Deporte de España
conscientes de sus propias habilidades (power-ups). Finalmente,
gracias a la becas FPU (AP2010-0259).
es necesario que cada usuario sepa a qué grupo pertenece así
como los miembros que posee ese grupo. Por último, es deseable 7. REFERENCIAS
([D]) saber quiénes son los miembros de los equipos. Como se ha [1] A. van Dam, “Post-WIMP User Interfaces,” Commun. ACM,
mostrado, la especificación Post-WIMP permite especificar la vol. 40, no. 2, pp. 63–67, 1997.
colaboración y el awareness de forma adecuada, siendo estos dos
aspectos primordiales para el desarrollo de sistemas Post-WIMP [2] M. Gea and F. L. Gutiérrez, El Diseño. Capítulo 5 del libro
contemporáneos. Esta especificación de la información de electrónico de la Interacción Persona-Ordenador. Asociación
awareness solventa los problemas especificados en la sub-sección AIPO, 2002.
anterior. [3] “W3C: MBUI - Task Models.” [Online]. Available:
http://www.w3.org/TR/task-models/.
5. CONCLUSIONES Y TRABAJOS
[4] P. González, F. Montero, V. López, A. Fernández-Caballero,
FUTUROS J. Montañés, and T. Sánchez, “A Virtual Learning
A lo largo de los últimos años, no sólo la apariencia visual de las Environment for Short Age Children,” in IEEE International
interfaces de usuario ha cambiado, sino también la forma en la Conference on Advanced Learning Technologies
que las personas interactúan con los ordenadores. De hecho, las (ICALT’01), 2001, pp. 283–284.
clásicas interfaces WIMP están dando paso a nuevos sistemas
Post-WIMP que van más allá de ventanas, iconos, ratón y [5] M. A. Teruel, E. Navarro, V. López-Jaquero, F. Montero,
punteros. La interacción con estos nuevos sistemas está basada en and P. González, “CSRML: A goal-oriented approach to
reconocimiento de gestos, realidad virtual, dispositivos model requirements for collaborative systems,” in Lecture
corporales, etc. Debido a la evolución de los sistemas de Notes in Computer Science (including subseries Lecture
interacción, los mecanismos necesarios para especificarlos deben Notes in Artificial Intelligence and Lecture Notes in
hacerlo también. Este hecho supone la principal motivación para Bioinformatics), 2011, vol. 6998 LNCS, pp. 33–46.
la realización de este trabajo, que propone el desarrollo de un [6] M. A. Teruel, E. Navarro, V. López-Jaquero, F. Montero, J.
modelo de tareas alternativo a la actual revisión del modelo del Jaen, and P. González, “Analyzing the understandability of
W3C, para adecuar el modelado de tareas a la especificación de Requirements Engineering languages for CSCW systems: A
requisitos de sistemas Post-WIMP. La colaboración es uno de los family of experiments,” Inf. Softw. Technol., vol. 54, no. 11,
aspectos clave en dichos sistemas, por lo que nuestra propuesta se pp. 1215–1228, 2012.
basa en el modelo de tareas Collaborative Systems Requirements
[7] M. A. Teruel, R. Tardío, E. Navarro, A. Maté, P. González,
Modelling Language (CSRML) [5], que implementa el
J. Trujillo, and R. Terol Muñoz, “CSRML4BI: A Goal-
ampliamente aceptado modelo de colaboración 3C [10]. Con el
Oriented Requirements Approach for Collaborative Business
objetivo de hacer que este modelo tenga la capacidad de
Intelligence,” in 33rd International Conference on
especificar las necesidades de awareness de sistemas Post-WIMP
Conceptual Modeling (ER’14), 2014, pp. 423–430.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 101 |
 Limitaciones del Modelo de Tareas del W3C para aplica-ciones Post-WIMP
M. A. Teruel | A. C. Rodríguez | F. Montero | E. Navarro | V. López-Jaquero | P. González
[8] F. Paterno, “Model-based design of interactive applications,” [12] A. García and J. Molina, “Enhancing collaborative
Intelligence, vol. 11, no. 4, pp. 26–38, 2000. manipulation through the use of feedback and awareness in
[9] R. Jacob, R. Jacob, A. Girouard, A. Girouard, L. Hirshfield, CVEs,” Proc. 7th ACM SIGGRAPH Int. Conf. Virtual-
L. Hirshfield, MS, MS, 2008, and 2008, “Reality-based Reality Contin. Its Appl. Ind. ACM., p. 32, 2008.
interaction: a framework for post-WIMP interfaces,” in [13] M. A. Teruel, E. Navarro, V. López-Jaquero, F. Montero,
portal.acm.org, 2008, pp. 201–210. and P. González, “A CSCW Requirements Engineering
[10] I. Steinmacher, A. P. Chaves, and M. A. Gerosa, “Awareness CASE Tool: Development and usability evaluation,” Inf.
support in global software development: A systematic review Softw. Technol., vol. 56, no. 8, pp. 922–949, 2014.
based on the 3C collaboration model,” in Lecture Notes in [14] C. Gutwin and S. Greenberg, “A descriptive framework of
Computer Science (including subseries Lecture Notes in workspace awareness for real-time groupware,” Comput.
Artificial Intelligence and Lecture Notes in Bioinformatics), Support. Coop. Work, vol. 11, no. 3–4, pp. 411–446, 2002.
2010, vol. 6257 LNCS, pp. 185–201. [15] M. A. Teruel, E. Navarro, V. López-Jaquero, F. Montero,
[11] C. A. Ellis, S. J. Gibbs, and G. Rein, “Groupware: some and P. González, “Comparing Goal-Oriented Approaches to
issues and experiences,” Communications of the ACM, vol. Model Requirements for CSCW,” in Evaluation of Novel
34, no. 1. pp. 39–58, 1991. Approaches to Software Engineering, L. A. Maciaszek and
K. Zhang, Eds. Springer-Verlag, 2012, pp. 169–184.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 102 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía

An Ontology-Driven Approach to Model & Support Mobility

and GeoLocation Based Campus Interactions
Maha Khemaja Félix Buendía
PRINCE, ISITC, H Sousse Computer Engineering Dept.
University of Sousse, Tunisia Universitat Politècnica Valencia
+216 23501625 +34 963877000 (ext. 75734)
[email protected] [email protected]

ABSTRACT Therefore, there is a need to incorporate this kind of context

information based on the user's profile and preferences, the user's
University campuses are increasingly becoming places where location, the status or the scheduling of an activity that is
different interactions take place including those related to occurring at the present time. This idea was advanced by authors
instructional and learning tasks. Most of these interactions require such as Weiser [[1]] or Crook & Barrowcliff [[2]] who believed in
gathering contextual information from the campus facilities, the potential of ubiquitous computing to offer new tools and
platforms, services or people that are part of this environment. The patterns of student’s interactions. Such potential was analyzed in
current work aims at providing an ontology-based proposal, which 13 institutions that provided "ubiquitous computing" in very
can enable to model most of these context features that can be different ways [[3]]. DeGagne and Wolk [[4]] examined how
useful to support multiple kinds of interactions in Campus college students used technology in their everyday life on a
settings. This proposal permits to reason and make inferences on campus with ubiquitous computing and communication.
contextual information and features in such a way to provide users Currently, many universities have platforms and apps like
with mobile geolocated services such as path finding, small group iStanford 1 or UCC2 that provide generic coarse-grain contextual
meetings planning or personalized information about campus information useful for their students’ lifetime. Nevertheless, the
facilities. The proposed ontology has been applied in a University support of specific context-aware interactions usually requires a
campus with the purpose of guiding users to improve their more fine-grain approach that represents context concepts and
personal experiences. their relationships in a more detailed way. Several initiatives have
been promoted for applying ubiquitous computing technologies
Author Keywords for campus environments in specific cases.
Ontology proposal; Campus interactions; context modeling;
mobile apps; geolocation services. For example, Aware Campus [[5]] or Active Campus [[6]] explore
location-aware applications addressed to guide users and allow
them to annotate or share location information with others.
ACM Classification Keywords MyCampus [[7]] is another application that was conceived to
enhance everyday campus life at Carnegie Mellon University
H.5. Human computer interaction (HCI). Ubiquitous and mobile
introducing context-aware agents, recommender services and case-
computing, mobile devices.
based reasoning. Moreover, Scott & Benlamri [[8]] describe
1. INTRODUCTION several initiatives to move traditional learning spaces into smart
University campuses are today organized as small or medium-size ones by using ubiquitous context-aware services.
towns where different types of interactions take place. They The current work focuses on the modeling of contextual
usually involve common services such as resident information that can be useful to support Campus interactions. It is
accommodation, catering, or social meetings but, obviously, the practically impossible to cope with every potential interaction in
most demanded services deal with learning activities or student this type of settings but a possible subset or interaction patterns
life in the campus. Even in this type of learning services, there is a can be defined. For example, outdoor activities can be performed
wide range of possibilities that are based on indoor/outdoor in several disciplines from architecture; agricultural studies to arts
scenarios, lecture vs. lab or seminar sessions, presential or online either for collaborative or individual tasks. These activities may
activities, individual or collaborative tasks as well as the diversity require a certain support beyond the academic works to be
of disciplines and domains that are taught in higher education. developed within them. Therefore, models have to be provided to
both represent information items about the context surrounding the
Moreover, campuses increasingly provide spaces for exhibiting
target learning activity and reason and infer related knowledge and
artistic masterpieces or historical work and seek to attract and
information. From these models, context aware mobile and
inform new comers about them. Most of these campus facilities
geolocation apps addressed to campus settings can be further
are generally supported by information systems (internal data
developed.
bases, web portals, intranet sites) allowing users to query for
specific information as lecture schedules, labs availability, tutorial
sessions or specific artistic show location. However, these
information systems do not take into account one’s user context 1
https://itservices.stanford.edu/service/mobile
and act only on a reactive manner to user’s explicit requests.
2
http://www.usca.edu/ucc/index2.html

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 103 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía
The rest of the paper is organized as follows. Section II presents axioms and rules and finally, instances are built and described. (3)
some works related to the proposed approach. Section III Ontology Formalization (in a formal or semi-formal) model (4)
describes the main features of the overall approach with the Ontology Implementation and validation using an Ontology
ontology proposal and the development process fundamentals. development framework or a programming environment. In order
Section IV illustrates the application of the presented approach, to define the domain and scope of the ontology, a list of
and, finally, Section V gives some concluding remarks. competency questions that a knowledge base should be able to
answer are gathered [[17]]:
2. RELATED WORKS • What are the elements that compose the user's context in
Along last years, several ways to model context issues have been a campus setting?
proposed. Strang [[9]] established that a well designed model is a • What kind of contextual information could enable a
key element in any context-aware system and identified several proactive decision making about user's interactions as,
approaches from the more simple based on key-value data for example, indoor or outdoor activities?
structures to mark-up scheme, graphical, object oriented, logic- • What kind of activities could engage a given user?
based or ontology models. Ontology notations represent a • What kind of scenarios and activities could be carried
powerful semantic mechanism that enable not only to reason or out by campus actors accordingly to their preferences
infer new information but also to share it in different ways. In this and profiles?
sense, we can differentiate generic ontology proposals that have • Which kind of information will allow users to plan
been applied to campus samples from other specifically addressed meetings and through which campus facilities?
to this environment. CoBrA-ONT [[10]] is an example of ontology How could new campus users locate buildings, sculptures, other
for context-aware pervasive computing environments that was persons or different objects in the campus?
applied in campus context. The CoCA (Collaborative Context-
Aware) service platform [[11]] is based on a hybrid context To answer these competency questions, several taxonomies are
management model and it was used to build a Pervasive Campus- identified. For instance, a context element taxonomy illustrated in
Aware Smart Onlooker (PiCASO) scenario. The SeCOM model Figure 1 constitutes as explained in [[18]] a first layer of elements
[[12]] was used to develop a Geomobile system at the FUV or primary context types for characterizing the situation of a
(Federal University of Viçosa) that contained information about particular entity. These context types act as access points into
the structure and activities carried out in the campus. Its context other sources of contextual information called secondary context
model was based on ontological concepts such as Actor, Activity, (e.g. the email address of a given user) for the entity as well as the
SpatialEvent and Temporal Event. This Geomobile system was primary context for other related entities (e.g. other people in the
developed using the C#NET language and addressed to PocketPC same location).
emulators though the conceptual ideas are similar to those
proposed in the current paper. Moreover, the SeCOM model was
used to develop the iCas system [[13]] based on a Java+XHMTL
framework to support students and teachers in their campus life
using OWL services. Otherwise, Geo-Wiki [[14]] provided a
semantic geographical knowledge based on Google Maps
technologies but it is addressed to specific wiki services.

3. OVERALL APPROACH
The main idea in the current work aims at providing an ontology-
based proposal which can enable, first, to model most of those
context features that can be useful to represent Campus
interactions, and, second, to support a development process, which
helps with the elaboration of mobile geolocated applications in
this context.
3.1 Context ontology Figure 1. Context Ontology model.
A methodology addressed to model the ontological context has
been applied. First, the category of the ontology to develop is Figure 2 shows part of the Context Ontology that models different
identified. Secondly, a development process inspired from several types of activities and their related interactions. The Activity
ontology construction methodologies is applied [[15]], [[16]]. taxonomy illustrates possible activities that could engage users in
Ontology construction methodologies promote the following basic Campus settings where the Learning Activity concept is
phases: (1) Ontology Specification which specifies the ontology distinguished from Recreational Activity Concept while the
purpose, domain and scope and it requires to decide whether or Facility taxonomy represents items and elements provided by the
not to reuse existing ontologies, (2) Ontology Conceptualization in campus infrastructure that link them to the corresponding
which the important terms of the ontology are enumerated, activities. Figure 3 shows part of the ontology model that
taxonomies i.e. classes and classes hierarchies are defined, binary represents the Geographic area entity where a specific Activity is
relationships, class and instance attributes or properties, formal developed.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 104 |
An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía

Figure 2. Campus Activity Ontology model

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 105 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía
The Geographic Area considered in Campus settings deals with Area and may be part of a Path that could be followed by a
physical elements that can be found in a Campus together with Campus user.
their organizational aspects. The concept of POI (Point of
Interest), for example, is described as a GeoLocation point having
a certain textual description. It is located in a certain Location

Figure 3. Campus GeoLocation Ontology Model.

All concepts, their properties and types are defined clearly in a Environment The surroundings or conditions in which a
Glossary as well as in a dictionary. Table 1 contains main person lives or operates.
ontology terms representing classes or entities such as User, The setting or conditions in which
Activity, and POI concepts. This latter is specifically interesting a particular activity is carried on
because it represents a specific location in a campus that could Campus The grounds and buildings of a university
attract new comers or students and which could be useful for them
either for recreational, learning or cultural related activities. Scenario The way in which a situation may develop.

Table 3. Ontology elements. Geographical An area, especially part of a country or

area the world having definable characteristics an
Name Description d boundaries
Location A particular place or position
Context The set of circumstances or facts that surroun
d a particular event, situation, etc Point of A specific point location that someone may
Context A kind of circumstance or fact composing Interest find useful or interesting
element context Path A road, way, or track made for a particular pu
User/End The person or the organization that is part of rpose. The route or course along which a
User the campus environment and to which the person moves
software/applications are intended. Campus Something (such as a building or large piece
Activity A thing that a person (user) or a group does Facility of equipment) that is built for a specific
or has done purpose. It makes an action, operation, or

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 106 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía
activity easier. logic should also be taken into account as illustrated by Figure 4.
This architecture model based on MVC (Model View Controller)
Moreover, constraints on concepts and relations between them are enables a development process which could be carried out by a
identified and expressed as axioms or domain/range constraints. development team with different roles and skills.
The ontology expressivity power is enhanced by a set of logical
rules. These rules are defined and used to enable inferences and
reasoning in the context of the proposed ontology. Table 2 shows
an excerpt of such axioms and logical rules.
Table 2. Ontology Constraints and Rules.
Constraint Example
tDisjoint An object or instance of student for example
classes could not be an instance of faculty. An
individual activity cannot be at the same time
a collaborative activity.
If Student(?S) and Faculty(?F)
→isDifferent(?S,?F)
If Activity(?A) and isCollaborative(?A) →
Existential A Path with POI is a path that fulfils the
restriction constraint at least one POI exists and pertains
to the path
isPath_With_POI(?P) → ∃ POI(?po) and
isLocatedIn(?po,?P)

Rule Example

Add_GUI_ A user U who's GUI interface is a Map m and

Component who have visited a POI p1 at Time t1 and
who's next planned Activity A at Time t2 is to
visit another POI p2 is informed about the
next POI location by adding a Marker to his Figure 4. Application architectural model.
interface:
In order to grant a best user experience, the interaction model as
If IsLocated (U, p1,t1) and well as the user interface should be intuitive, ergonomic and
Next_InPath(p1,p2) and Activity(A,t2)="Visit compliant to the user's habits and preferences. It should also avoid
POI" → her/him having an overloaded view. User inputs are requested
set_Up_GUI_Component(Marker(Map only when they are strictly necessary. The user's interactions are,
m,p2)) therefore, driven by a set of components that are able to detect
her/his context and provide her/him consequently and seamlessly
Draw_Directi A user U who's current Location l is far from with the best fitted services.
on_Path form the POI p2 to be visited is informed
about the next POI location by drawing Taking into account all these considerations, the core application
Direction path on his interface: process structure addresses these actions (1) initialize the first
application's State, (2) sense and analyze Context info, (3) update
If IsLocated(U,l,t1) and IsFarFrom(l,p2) → the application State, (4) display the next State. The process
Draw_Direction_Path(p1,p2) continues at point (2) until the user explicitly stops the application.

Apply_Zoom When the distance between two POIs is small The development process targeting this kind of application is
it is better to ZoomUp the Map: incremental. It also follows a "bottom-up" approach through the
next steps:
If Distance (p1, p2) < 2 Km
→Apply_Zoom(m, 13) • Identify possible users’ needs and interaction rules that
drive the application behavior.
• Identify and develop information sources and data
structures (Ontology sources and Databases).
3.2 Development process • Identify existent components that could be reused and/or
Modern software development is increasingly requiring separation reassembled as for instance, third party services such as
of concerns. Therefore, each component to be developed should those provided by Google Play services, Weather
individually address a different purpose and brings itself Forecast services or also transportation and traffic
contribution to achieve the global application goal. Separation services.
between the user's interface, the interaction mode and the business

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 107 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía
• Identify issues such as User Interface structure, on the action to be realized as, for example, to add one GUI
components, GUI services and attributes. component or to update one another.
• Develop context sensing and Events Listeners
components.
• Develop Events handlers and implement their
interactions with business services.
• Develop business services that firstly, reason and
request ontology sources and databases and, secondly,
act on the user's interface components accordingly to
interaction rules by triggering GUI services.
In the context of the present work, point (1) and (2) are
represented by developed ontologies as well as their associated
rules.
The point (4) considers Widgets and Graphical User Interface
components provided by mobile devices.
The point (5) relies on sensor modules (e.g. GPS) present on the
user's devices, on remote services (e.g. Weather services) as well Figure 5. Services taxonomy.
as on the user's interaction with the app GUI components. Each 4. SAMPLE APPLICATION
detected event (e.g. Location changed) will trigger the relevant As a proof of concept, a sample application is provided that deals
Event handler. Mappings between Event Handlers and capabilities with specific interactions within a University Campus. These
provided by services developed in point (6) are inferred interactions can be addressed to educational purposes for Campus
automatically which constitute the core of the interaction logic students but also to allow other users to take advantage of its
provided to the user. As illustrated by Figures 4 and 5, service services or visit its patrimony facilities. First, the geographical
categories vary from Decision Making to GUI Services. Moreover, campus environment is described. Thereafter, technical details of
these services collaborate to detect the user's context and decide the mobile application used within this environment are provided.

Figure 6. University Campus map.

Figure 7 shows a detailed view of one of the sectors represented in
4.1 Environment description the map. It is located close to the main street and there are two
The context of the sample application is based on an existing
campus entries labeled as L and M. There are also two near bus
University Campus. Figure 6 shows a map of the target campus,
stops and a tram station. The selected sector is composed by
which represents the geographical areas that integrate it. Each area
several buildings (6A-G), sports courts (green areas), two Meeting
is identified by a sector label (e.g. sector 2C) and it includes a set
Points (P6A and P6B), and a set of numbered red spots.
of buildings, objects and other campus facilities such as sport
arenas or yards. Additionally, this map displays some numbered
blue little squares that represent “Meeting points” for each campus
sector.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 108 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía

Figure 7. Campus sector map.

These spots are signaling special artistic points (e.g. sculptures
that are part of the Patrimony campus) which can be interesting for
Art students or an occasional visitor for cultural exhibitions at the
campus. Figure 8 shows a group of students observing a sculpture
called “Cactus” and listening to the tutor explanations. These
students are in a course of Sculpture Conservation and they want
to know those technical issues that can be harmful for sculpture
materials. This situation could be a sample scenario where
interactions can take advantage of geolocation services.

Figure 9. Campus path

The application is structured according to the architecture layers
illustrated in Figure 4, with self contained components and
services within each layer.
The resource layer constitutes information sources as databases
and ontologies for managing campus organizational data as well as
contextual information to be stored and processed. The context
ontology is implemented using the Protégé ontology editor and it
is expressed by means of the Ontology Web Language (OWL).
The business or application model layer contains services for
reasoning and querying the ontology using the SPARQL language
and the Andro Jena API. These services have been implemented as
an external Android service exposing its interface via the Android
Interface Definition Language (AIDL).
The controller layer implements several categories of listeners
Figure 8. Campus Sculpture exhibition. related specifically to context change events as, for example,
location changes or user's interactions with the map. These
4.2 Technical solution listeners are associated to event handlers which capture event
The sample application called OntoMapAndro has been developed information and delegate it to be processed by services in the
through an Android framework and deployed using a mobile application model layer.
device. It tracks the user's location and allows her/him to interact
Decisions made by this layer after considering new contextual
with a main GUI component which is a Map representation of the
information may specifically trigger GUI services to update the
campus space, interesting POIs and paths (see Figure 9) within the
user's Interface. For example, Figure 10 shows an OntoMapAndro
previously introduced campus environment.
screenshot displaying information (text & image) about the object
that is currently visited. This information is displayed over the
MapView (Google Map based) as a consequence of triggering a
NextStep event.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 109 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía

Figure 11. Zoom level adaptation.

5. CONCLUSION
The current work has presented an approach to support the
modelling of contextual information in campus settings using a
systematic development process. The proposed approach has been
Figure 10. Object information display based on an ontology proposal focused on representing concepts
and relationships either for indoor or outdoor campus scenarios.
Finally, the view layer represents mainly the client components or This ontology includes the definition of axioms and rules that are
services that interact with the Google Map API. Client's services useful to reason or infer new information promoting learning
implement specifically functionalities to display maps, to change purposes or sharing knowledge. An implementation of this
map attributes as the zoom level and camera position, to also ontology has been developed using well known semantic
display the user's positions, add specific markers, and draw paths technologies such as Protégé, JENA and SPARQL tools
among two or more POIs. These information items are adapted demonstrating the feasibility of the approach.
according to the ontology rules defined previously. For example,
Figure 11 shows how the zoom level has changed because the Moreover, the developed ontology has been applied in the context
distance to the next object to visit is under a specific threshold. of a university campus showing its potential to model context
information that allows campus users to perform multiple kinds of
queries focused on different types of interactions. It is important to
highlight the possibilities provided by using ontology sources in
order to drive the application execution and improve its
adaptability in a changing context. Further works plan to address
development issues such as the generation of mobile apps in
multiple platforms and the improvement of accessing ontology
sources via web services. Another research line consists in
analyzing the users’ interactions and their behaviour when they are
using geolocated applications based on the current proposal. These
findings could lead to recognize mobile behaviour patterns and
provide better user adaptations.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 110 |
 An Ontology-Driven Approach to Model & Support Mobility and GeoLocation Based Campus Interactions
M. Khemaja | F. Buendía
Ontologies for Distributed Systems, Knowledge Review,
6. REFERENCES 2003, Acapulco MX.
[1] Weiser M. The future of ubiquitous computing on campus.
Communications of the ACM, 1996, 41(1):41–42. [11] Ejigu, D., Scuturici, M., Brunie, L. Hybrid Approach to
Collaborative Context-Aware Service Platform for Pervasive
[2] Crook, C., & Barrowcliff, D. Ubiquitous computer on
Computing. Journal of Computers, 2008,Vol 3, No 1, 40-50.
campus: Patterns of engagement by university students.
International Journal of Human-Computer Interaction,2001, [12] Lamas, A. R., Filho, J. Oliveira, A. P,. Júnior, R. M. A
13,245−256. mobile geographic information system managing
contextaware information based on ontologies. Ubiquitous
[3] Brown D.G. Ubiquitous Computing: The Universal Use of
Computing and Communication Journal, 2009, vol. 4, pp.
Computers on College Campuses. David G. Brown (Ed.)
71–727.
2003. Anker Publishing Company, Incorporated.
[13] Sousa, J.-P., Carrpatoso, E., Fonseca, B., Pimentel, M., Neto,
[4] DeGagne and Wolk. Unwired: Student Use of Technology in
R. A Service-Oriented Middleware for Composing Context
the Ubiquitous Computing World. Information Systems
Aware Mobile Services. Composition of context aware
Education Journal, 2007, 5 (13)..
mobile services using a semantic context model. International
[5] Burrell, J., Guy, G., Kubo, K., Farina, N. Context-aware Journal on Advances in Software, 2009, 2:2-3, p 275-287.
computing: A test case. In Proceedings of UbiComp Springer
[14] Gao, Y., Gao,S., Liu, R. Liu, Y. A semantic geographical
, 2002, 1–15.
knowledge wiki system mashed up with Google Maps.
[6] Griswold, W. G., Shanahan, P., Brown, S. W., Boyer, R., Science China Technological Sciences, 2010, Volume 53,
Ratto, M. , Shapiro, R. B.,. Truong T. M. Activecampus – Issue 1 pp 52-60.
experiments in community-oriented ubiquitous computing.
[15] Fernández-López, M., Gómez-Pérez, A., & Juristo, N.
Paper CS2003-0750, Computer Science and Engineering, UC
(1997). Methontology: from ontological art towards
San Diego (2003).
ontological engineering. Proceedings of the Ontological
[7] Sadeh, N. Gandon, B. and Kwon, O.B. Ambient Intelligence: Engineering AAAI-97 Spring Symposium, 1997.
The MyCampus Experience. Chapter 3 in Ambient
[16] Gómez-Pérez, A., & Suárez-Figueroa, M. C. Scenarios for
Intelligence and Pervasive Computing, Eds. T. Vasilakos and
building ontology networks within the NeOn methodology.
W. Pedrycz, ArTech House.
In ACM Proceedings of the fifth international conference on
[8] Scott, K., Benlamri, R. R. Context-Aware Services for Smart Knowledge Capture, 2009 pp. 183-184.
Learning Spaces, IEEE Transactions on Learning
[17] Grüninger, M., & Fox, M. S. Methodology for the Design
Technologies, 2010, vol.3, no. 3, pp. 214-227.
and Evaluation of Ontologies. Workshop on Basic
[9] Strang, T., Linnhoff-Popien, C. A Context Modeling Survey, Ontological Issues in Knowledge Sharing, 1995.
in: J. Indulska, D.D. Roure (Eds.), Proceedings of the First
[18] Abowd, G. D., Dey, A. K., Brown, P. J., Davies, N., Smith,
International Workshop on Advanced Context Modelling,
M., & Steggles, P. Towards a better understanding of context
Reasoning and Management, in conjunction with UbiComp
and context-awareness. In Handheld and ubiquitous
2004, University of Southampton.
computing. International Symposium on Handheld and
[10] Chen H., Finin T., Joshi A. An ontology for context-aware Ubiquitous Computing, 1999. pp. 304-307.
pervasive computing environments. Special Issue on

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 111 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga

SEGA-ARM: A Metamodel for the Design of Serious

Games to Support Auditory Rehabilitation
David Céspedes-Hernández1, Juan Manuel González- Jaime Muñoz-Arteaga1
3
Jorge Luis Pérez-Medina2, Calleros3, Francisco J. Álvarez Benemérita Universidad Autónoma
1 de Puebla
Universidad Autónoma de Rodríguez1,
Aguascalientes 2
Université catholique de Louvain Av. 14 Sur y San Claudio, Puebla,
Av. Universidad 940, Ags., Mexico Place des Doyens 1 Mexico
{dacespedes, fjalvar, Louvain-la-Neuve, Belgium [email protected]
jmunozar}@correo.uaa.mx [email protected]
ABSTRACT to provide an application that may be used to help on the
Hearing impairments are widely recognized as one of the most performance of assisted therapy sessions, but to propose a
common disability that affects many people. There are several definition of modeling elements along with their relationships and
works addressing the design and development of systems for the rules in order to enable the creation of semantic models, that may
rehabilitation of users with disabilities, as well as for the later result on the implementation of a family of serious games
definition of models to express their capabilities and disabilities. with the purpose of supporting the process of auditory
Unfortunately, when we want to develop interactive systems to rehabilitation, meaning by this, a metamodel [12] for the design of
assist hearing impairment therapy, knowledge of analysis and serious games to support auditory rehabilitation.
design, typical of the methodologies is absent. In this paper, we The rest of the paper is structured into sections. Section two
introduce SEGA-ARM: a metamodel to support the design of presents a state of the art with relevant concepts to auditory
serious games for auditory rehabilitation, considering concepts rehabilitation and serious games, as well as other related works,
related to speech therapy sessions, user capabilities, the context of and the description of useful techniques and tools for the
use, and concepts related to serious games design. The proposed development of this project. For the description of the elaborated
solution is provided using a semi-formal UML notation. Two model, section three firstly presents the packages in which it is
game prototypes are presented to validate the viability of the divided. Later, in order to facilitate the comprehension of the
application of the metamodel for serious games development and model’s classes, a case study is introduced. Finally, instancing the
to point out its reutilization and extension. case study, each one of the packages’ element is described. In the
fourth section, a discussion on the characteristics of the full model
Categories and Subject Descriptors and the relationships that exist between classes of its different
H5.2 [Information interfaces and presentation]: User packages is carried out, introducing a second case study. Last,
Interfaces–Prototyping; user-centered design; K8.0 [Personal conclusions and future work are mentioned in section number
computing]: Games. five.
General Terms 2. STATE OF THE ART
Design, Theory
2.1 Concepts on Auditory Rehabilitation
Keywords In order to better understand the aim of the present work, a set of
Model-driven development, speech therapy, serious games, concepts relative to auditory rehabilitation and serious games has
Human-computer interaction, hearing impairment, cochlear to be defined.
implant, user-centered design.
Rehabilitation is the dynamic process by which physical, sensory,
1. INTRODUCTION and mental capacities are restored or developed in people with
In several contexts such as healthcare, military and educational, disabling conditions [13]. When it comes to rehabilitation for
games have been applied as a way to improve skills and for auditory disabilities, it is covered mainly by speech therapy and
training professionals [1] [2]. This type of games is called serious reinforced by occupational and physical therapy. The success of a
games [3] [4]. Despite the fact that the concept of serious games rehabilitation program depends on various factors: timing, patient
itself, might be considered as an oxymoron since “Games are diagnosis and treatment planning [14]. For the purposes of this
inherently fun and not serious” [5], it is important to clarify that particular work, only speech therapy is addressed. Speech therapy,
many works had been conducted in the field addressing the also known as speech-language pathology, is the rehabilitative
development of these games, always considering fun as a main treatment of physical and/or cognitive disorders resulting in
component of them [6]. Several works addressing the design and difficulty to perform verbal communication. Speech therapists
development of systems for the rehabilitation of users with asses, diagnose, treat and help to prevent disorders related to
disabilities [7] [8], as well as for the definition of models to speech, language and other elements of communication [15]. The
express people’s capabilities and disabilities [9] [10] had been ultimate goal of these specialists is to help patients develop or
carried out. However, even though hearing impairment is widely recover reliable communication and other skills. Most speech-
recognized as one of the most common disabilities [11], there is a language pathologists work with determined age groups, such as
lack of work focusing on providing tools to ease the design and children or elderly and focus on the treatment of certain
development of applications for the domain of auditory communication problems, such as those resulting from strokes or
rehabilitation. In this sense, the objective of this work is not only hearing loss [16].

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 112 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
Hearing impairment may be caused by different factors, such as, 2.3 Related Work
accidents, viral diseases, noise, and genetic causes [17]. Software designers may achieve the goal of expressing the
Depending on the background of the patients with hearing capabilities and disabilities, along with other valuable insights of a
impairment, among other classifications, they can be categorized user to developers via “Personas” models [9]. “Personas” are
into two groups: prelocutive (loss hearing toke place before the hypothetical archetypes of actual users and their description may
acquisition of spoken language) and postlocutive (patients whose consist on different aspects according to what is intended to make
loss of hearing has taken place after the development of the basic the development team aware of. Alternately, in [10] a metamodel
skills of spoken language) [18]. Once a hearing impairment has was proposed by Kaklanis to allow the expression of the whole
been diagnosed, the specialists involved on the patient’s user capabilities and disabilities with the intention of virtual user
treatment, evaluate the best option in order to improve his/her modeling. The main advantage that was encountered while
conditions. In this sense, some of the patients may be candidates comparing the “Personas” representation to this second proposal,
to receive a cochlear implant as the one mentioned in the case is that “Personas” provide only a natural language description of
study (Subsection 3.1) of the present work. the characteristics of the user, while Kaklanis’ disability and
A cochlear implant [19] is an electronic device that stimulates the capability models’ structure allows their representation on a
auditory nerve to allow sound perception. It is composed by an machine-readable format.
external component that receives incoming sound, processes it For the present work, the disability model by Kaklanis, shown in
and transfers the resultant signal across the skin. Another device Figure 1, is considered as it was defined, since it provides a
receives and decodes the transmitted signal and stimulates the generic description of any kind of disabilities with the affected
auditory nerve directly, bypassing the hair cells that implement tasks on it involved, while the capability model was in turn
the first stage of auditory neural processing in normal ears. simplified to extract from it only those parts related to the hearing
According to the National Institute on Deafness and Other and speech skills, and enriched with characteristics that authors
Communication Disorders (NIDCD) on the United States of mention as relevant for the domain of speech therapy in children,
America, by December 2012, approximately 324,200 people such as loudness, speech rate and phrasing [22][23].
worldwide have received cochlear implants [20].
Concerning serious games design, an effort has been realized by
several authors, in order to abstract components that lead to the
definition of a guide on what to be aware of when developing a
game with these characteristics [24][25]. However there is not yet
a consensus on the components or elements that should be
considered when performing design and implementation, Table 1
presents some of the design elements that are usually taken into
account for the development of games [24-28].
Moreover, Longstreet & Cooper [29] proposed a conceptual guide
for development of serious games to support higher education,
their conceptual definition includes most of the design elements
reported on Table 1, but a structural notation was not provided
and the relationship between the involved elements is only
explained in a narrative manner, making its reutilization
Figure 1. The disability model proposed by Kaklanis [10]. complicated.
In this section, a general panorama of the domain concepts was
2.2 Concepts on Serious Games provided in order to facilitate the comprehension of further
To define a serious game, [21] begins by defining a game as a
sections. Also, related works are considered and described,
physical or mental challenge, that is played according to specific
pointing out their usefulness for accomplishing the objectives
rules, with the objective of amusing or entertaining the
presented on section one, and also reporting their limitations.
participant. Then, continues to build the definition of a videogame
as a mental challenge that is played through a computer according Finally, taking into account the concepts addressed in the last two
to certain rules for entertainment, fun or to achieve a goal. And subsections, a decision was made to consider the proposition of a
finally, provides the definition of a serious game as a mental metamodel composed by three main packages: context, theoretical
challenge, played through a computer according to specific rules, therapy, and performed therapy, and to add it the definition of a
which uses entertainment mechanisms to achieve training or serious games design elements extension.
capacitation goals in areas such as education, health and military.
Notice that in the rest of the paper, this is taken as the definition
of a serious game, just pointing out that it may also consist of
physical challenges (e.g., tongue movement, phoneme
pronunciation, and lips movement). The interest on the usage of
serious games on rehabilitation lies on the fact that is has been
demonstrated that they increase motivation towards therapy
sessions, which represents a major problem due to the repetitive
nature of exercises [14].

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 113 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
Table 4. Game design elements in the literature [24-28]. 3.1 Case Study
Game The case study consists on a scenario in which the design of a
Design Description serious game is required in order to support the therapy sessions
Element conducted by a speech therapist for a six year old male patient
The definition of characters allows user diagnosed with deep bilateral hypoacusia who has recently
engagement to the game, and provides the received a cochlear implant. The therapist has already performed
Characters possibility to define actions. Helps improving the tests over the patient to determine his exact condition and counts
player experience. Considers both, player and on a full expedient describing his capabilities towards hearing and
non-player characters speech skills. The intention with the required game is to extend
The rational of the game described in a narrative and complement the therapy sessions that the patient already
Narrative way, promotes the immersion of the player, helps receives with sessions conducted by himself in his home as those
description to define the needed resources, characters, game suggested in [30]. The language therapist wants the game to
mechanics and challenges. reinforce one of the therapy activities that the patient finds to be
The definition of challenges to be faced during repetitive since it has to be performed in almost every single
the gameplay, gives the player the opportunity to session.
Challenges
try his/her skills and to compete either with a
partner or with the game itself.
Determines the level of immersion that is
decided to use in the game. The most used point-
Type of view of-view perspectives for gameplay are: first-
person, third-person, third-person trailing,
overhead, and three-fourths isometric.
Constraints determine characteristics of the
Constraints challenges (e.g., limited time or resources,
competitive and sequential activities).
Establishing a reward system (e.g., points,
badges, levels, and leaderboards) allows
Rewards
improving the player experience and motivating
him/her to achieve a goal.
As with rewards, the establishment of
punishments, such as a decrement of points, and
Punishments
losing a life in the game, motivates the user to Figure 15. The exercise conducted in the therapist’s office.
achieve a determined goal.
The definition of interactive elements supports The exercise consists on that the therapist chooses a series of
Interactive phonemes sharing a consonant (e.g., ma, me, mi, mo, and mu),
the immersion of the player on a virtual world.
elements write them down on a notebook as shown in Figure 2, putting the
Their definition helps to design player tasks.
For each action performed by the player, the consonant on one column of the page and the vowels in another
game provides a reaction. In this way, the user is next to it, then asks the patient to pronounce each one properly
Feedback aware of the consequences of his/her acts on the while connecting the consonant with the vowels with lines, and
game and an appearance of continuous dialog is then repeat this task several times and with different phonemes.
given to him/her. This activity is designed to be performed in approximately ten
minutes and also considers phoneme visual recognition by the
3. SEGA-ARM: A METAMODEL FOR THE patient. It is also known by the therapist that the patient counts on
a 10.1” tablet with Android 4.2 OS. The information that the
DESIGN OF SERIOUS GAMES TO specialist possesses has been structured in terms of the here
SUPPORT AUDITORY REHABILITATION proposed metamodel as a way to facilitate the communication
For achieving the objectives mentioned on the introduction, a between the speech therapist and the game development team.
metamodel (SEGA-ARM) is proposed, considering for its The following section reports as an instantiation of the
definition concepts related to context, auditory rehabilitation, metamodel’s classes and how data was arranged and organized.
planned therapy, performed therapy, and serious games design
elements. The elaborated model is divided for its best 3.2 Description of the Classes Involved on the
comprehension and usage into three packages, presented in Figure Model
3, and an extension shown in Figure 4, each one of them identified The following subsections are for used for providing a detailed
by a different color and name, and designed to contain elements description of the packages and classes that take part on SEGA-
that support the design of serious games for auditory ARM. For every single package that is described, there is an
rehabilitation. instantiation in terms of our case study introduced on section 3.1.
In the rest of this section, SEGA-ARM is described in detail and 3.2.1 The Context Package
explained through a case study of a serious game design for A package designed to understand the Context [31] in which the
supporting the process of rehabilitation of a patient with a deep user interacts with the system, and to provide the developers with
hypoacusia condition, recipient of a cochlear implant. a brief description of relevant characteristics to make decisions
about what interactive modality to implement, and resources

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 114 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
selection among others. The Context is identified by an id and a and a user (Patient).
descriptive name, and conformed by an Environment, a Platform

Figure 3. The Context, Theoretical part of the Therapy, and Dynamical part of the Therapy packages with their relationships to
the serious game extension of the SEGA-ARM metamodel proposed.
The Environment describes the physical place in which the non-noisy interior environment, a 10.1” tablet with Android Jelly
interaction occurs. For the specific domain of this work, it is Bean 4.2 operating system, audio output, voice input and camera
necessary to point out if it is noisy or not, and to give it a name, an usage enabled, with android speech recognition/synthesis software
id and a type (i.e., exterior or interior). as platform, and the patient as user. Note that even though the
The Platform definition is carried out to determine the characteristics of the context must be taken into account by the
characteristics of the device that is going to host the application, serious game’s designer; the implemented solution may work
and is formed by a SoftwarePlatform and a HardwarePlatform. under other similar contexts as well.
The SoftwarePlatform represents those characteristics relative to 3.2.2 The Static Part of the Therapy Package
the software in the hosting platform i.e. operating system name The purpose that is pursued with the design of this package is to
and version, as well as the speech recognition/synthesis software provide the speech therapist with a semiformal notation to express
on it installed, while the HardwarePlatform class defines the characteristics of a therapy plan, keeping that information
characteristics regarding the hardware of the hosting platform and structured in a way that results understandable for serious games
relevant to the domain, such as the possibility of using sound designers. The definition of this part of the model was inspired
outputs, voice inputs or a camera. mainly by the work done by Kaklanis for modeling a Virtual User
In our example, the context in which the therapy sessions take [10], extended to allow the expression of a full auditory therapy
place is defined by a therapist’s office as a non-noisy, interior session program, and enriched with concepts found out to be
environment, has not platform and the user role on it is taken by relevant for patients of auditory rehabilitation on the literature
the patient, meanwhile, the context of use of the application that is [22] [23] and through the attendance to speech therapy sessions.
intended to be developed, is composed by the patient’s home as Abstracted from the full capability model [10] that describes

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 115 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
physical, cognitive and behavioral user characteristics, for the part of this paper, only considers the elements related to speech
specific domain of this work, the CapabilityModel presented as and hearing.

Figure 4. The serious games design elements extension provided for SEGA-ARM
The only parameter in the Hearing container is the Ear that in Corresponding to the identified disability, a rehabilitation
turn acts as one for the values related to each ear of the user: program was designed comprising approximately 300 hours, to
ResonanceFrequency and HearingThreshold. The Speech be provided in half an hour, two times a week sessions. One of
container includes Voice and Prosody elements. Voice elements the activities to be performed by the patient consists on
include Pitch and Loudness, each one with its measure units and repeating a series of phonemes in a determined order. The
values; while Prosody is formed by SpeechRate, VoiceStress therapist mentions a phoneme and expects the patient to repeat
and Phrasing. it. If a phoneme is not correctly pronounced, the therapist
A DisabilityModel [10], allows the description of all the reinforces its pronunciation and motivates the user to try again.
disabilities of the user as well as the affected by them tasks. For The objective of the task is to improve the patient’s
each disability, a name and a type (e.g. motor, auditory, and pronunciation and to get him/her used to repeat the sounds that
vocal) must be provided, and it is related to affected tasks. For a he/she listens to. For this particular activity, only a notebook and
determined DisabilityModel, a TheoreticalTherapy may be a color pencils are used.
proposed. A TheoreticalTherapy is a full rehabilitation plan and 3.2.3 The Dynamic Part of the Therapy Package
consists of a certain number of sessions and a series of In order to represent the current progress and performance of the
theoretical activities. A TheoreticalActivity is in turn, a series of Patient towards the therapy program, and to provide the
actions with a common objective to be performed by the patient development team with a structure to understand how to
either using or not PhysicalResources (e.g., a mirror, sticks or evaluate patient’s activities, the dynamic part of the therapy
cards). The planned duration, objective and description are package was created. This package contains then the Patient
attributes of a TheoreticalActivity. class, with an age attribute, related to a PerformedTherapy
For our case study, the disability and capability models which reports the number of completed sessions and groups
correspond to the diagnosis that was realized by the specialist in objects of type PerformedActivity.
speech therapy. The disability that was encountered is deep A PerformedActivity is based on a TheoreticalActivity and its
hypoacusia, and the affected tasks by it, are hearing and main goal is to keep a record of the advance presented by the
speaking. For the capability model, in the hearing branch, the patient while performing a given activity. In order to allow the
patient presents hearing thresholds of more than 90 dB nHL for evaluation of a PerformedActivity, a Metric class was also
frequencies of 1000, 2,000 and 4,000Hz in both ears; and a defined. Metrics allow the use of a formula in order to compute
resonance frequency between 800 and 1,200Hz. For the speech a value and to express it on a determined measure unit (e.g.,
branch, the patient’s voice was measured, getting as values for centimeters, seconds and decibels). For the evaluation of a
the pitch 250-300Hz, and 73dB for the maximum loudness. The formula, a series of Parameters i.e., criteria to be used while
prosody was also analyzed, obtaining for it typical results for a evaluating, may be defined. There could be nested metrics since
patient of the age and condition as the previously described. some formulas may need some others to be previously computed

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 116 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
in order to take values from them. Metrics in this domain may for this package, a series of challenges has to be designed to
be useful for measuring among others, tongue movement (i.e., represent the goals that are pursued while playing the game. A
tongue tip position, tongue vibration, and tongue position), lips Challenge consists of a description and is composed by at least
movement (upper lip movement and lower lip movement), one Constraint. Constrains may include one or several
pronounced phonemes and blow action. conditions (viz. time, limited resources, rewards, punishments,
In the context of the case study, the performed therapy and and mandatory collaboration).
performed activity classes keep track on how the patient is For the given case study, taking into account the description
evolving and advancing through the therapy program and their provided by the speech therapist for the context, theoretical and
involved values are constantly updated. The metric to be used dynamic packages, the serious game designer proposes to
for the specific theoretical activity described in section 3.2.2, elaborate a game with the characteristics that are presented in
consists on evaluating how a phoneme is pronounced by the Table 2 and which resultant user interface is shown in Figure 6.
patient, recognize it and compare it to an expected phoneme It is important to point out that the definition of some of the
passed as parameter according to a specific tolerance, using for game design elements was achieved through the establishment
this purpose the formula represented on Figure 5 where the of relationships between the therapy packages and the serious
pronounced by the patient phoneme (pp) is compared to the games extension (e.g. TheoreticalActivity and the Action). Next
expected phoneme (ep) and then, if the distance between them is section emphasizes this characteristic of SEGA-ARM along
smaller than the tolerance value (t), the utterance from the with others that had not been yet addressed.
patient is accepted as correct.
Table 2. Characteristics of the designed serious game.
Game name Froneme the frog.
Froneme the frog is in the edge of a river
looking for food to eat. In order to go find it, he
Figure 5. Formula for recognition considering expected has to jump between leaves avoiding to fall on
phoneme (ep), utterance (pp), and a tolerance value (t). the water. To perform a jump, the player has to
Narrative
identify the phoneme associated to the
3.2.4 The Serious Games Extension description
destination leaf and pronounce it. If the player
In attention to the main objective of this paper, an extension to is unable to provide a correct utterance,
the therapy metamodel for considering serious games design Froneme falls down into the water and
elements was elaborated. For the proposal of this extension, therefore the game is over.
mainly Longstreet´s approach is considered [29], enriched with
concepts obtained through a deeper survey on serious games Player The patient represented by a frog.
design elements [26-28]. The intention on the definition of this Mosquitoes representing Froneme’s food;
extension is to provide developers with a tool for passing from a Interactive leaves to allow Froneme jump from one to
narrative description of a game to a semiformal structure that objects another identified by a phoneme; water that has
may be related to the one of the core metamodel. A to be avoided by Froneme.
SeriousGame is composed by Mechanics to record the player Eat mosquitoes by landing on the leaves that
progress and to manage the punctuations, one or many instances they are on, jump from one leaf to another by
Player
of the Character class to define players as well as non-player pronouncing phonemes, and fall down on the
actions
characters (NPC), and SimulationGameElements such water if not correct utterance is provided on the
challenges, rules, scenarios and interactive objects. given time interval.
A Character object represents an entity that realizes actions. The game is designed with an isometric type of
These instances can be either a Player or a Non- view, and the following game situations:
PlayerCharacter, and need Resources in order to be properly Simulation Froneme in the border of the river (initial state
displayed. For the definition of NPCs, it is necessary to give environment of the game), Froneme on a free leaf, Froneme
them a name, decide the relationship that they maintain towards on a leaf occupied by a mosquito (eating), and
the Player, and when possible, provide their gender and cultural Froneme falling on the water.
information e.g. language, nationality, degree among others The challenges involved in the game are the
[29]. In order to handle events on the game and describe their following: jump from a leaf to another
consequences, it was necessary to define the Action class. An pronouncing a correct phoneme, jump from a
Action is represented by an event (e.g. key press, click, tap, leaf to another within a determined time
recognized word or movement, or scripts) that triggers it, along interval (time limit) to earn points (reward), or
Challenges
with a description, duration, and a script to execute as let Froneme to fall into the water (punishment);
consequence. Actions are performed by Players or executed by choose a leaf to jump on from a series of
NPCs and may whether use or not an InteractiveObject that are provided options (limited resources) and if
in turn, objects that belong to the SimulationEnvironment and there is a mosquito on the selected one, earn a
are created to support the game interaction, for instance, the bigger amount of points (reward).
scenario, the floor, and colliding objects. The For each time the player takes Froneme to a
SimulationEnvironment description contemplates the definition valid leaf, 10 points are going to be added to
of a type of view (e.g., first person, third person, or isometric) Mechanics the budget, If there is a mosquito on the
[26], and helps to hold the game situation i.e. level, state, and occupied leaf, ten extra points should be added.
maximum amount of errors [27] in a narrative manner. Finally The time available for producing the phoneme

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 117 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
is five seconds. Time and punctuations are
recorded and reported.

4. DISCUSSION
As it may be seen in Figure 3, there are relationships between
the therapy packages and some of the serious games design
elements extension classes. These connections allow the
developers to know where to extract information that is
supposed to be represented in the game, and how to present it.
Punctually, there are relationships between: the Patient
(Dynamic part of the therapy) and the Player (Serious games) to
remark the fact that the patient has to be considered as the player
and to get some information from his/her profile; the
PerformedTherapy (Dynamic part of the therapy) and the
Mechanics (Serious games), to track the advance of the player
through the game and give a feedback for the therapist; and the
TheoreticalActivity (Theoretical part of the therapy) and the
Action (Serious games) in order to understand the tasks that the
patient is supposed to perform while playing the serious game. It
is also important to point out that three levels were considered
while modeling the therapy: (1) the patient (capabilities and
disabilities), (2) the therapy program (theoretical therapy), and
(3) the involved activities to meet the plan (theoretical activity).
Each one of those three levels has equivalence on the dynamic
part of the therapy in order to keep track of the advance and
performance of the patient through the therapy. Figure 6. Screenshot of the serious game Froneme the frog.
When compared to the proposal in [29], our metamodel has the One of the aimed advantages of our proposal is reutilization, and
advantage of providing a semiformal notation which may be in order to demonstrate it, a second instantiation of serious
used in a methodological process as a conceptual guide for game, called Roberto the robot, was realized in which the very
achieving the goal of expressing and therefore implementing same definitions of the case study described in section 3 for the
serious games for a specific set of therapy activities. Along with context and for the static and dynamic parts of the therapy
its description, it also allows identifying the actor that is packages are taken into account, varying only the serious game
supposed to fulfill the different classes and attributes, and extension elements and therefore creating a different serious
provides a definition of the relationships that exist among them. game but for the same rehabilitation purpose. For this second
However, in this paper the use of knowledge bases and game, a character representing the player called Roberto the
taxonomies is not considered resulting on the lack of a robot is set on a scenario representing the moon surface looking
repository of activities to provide to the developers. for pieces to assemble a rocket to return to the Earth. In order to
find them, he has to run and jump over rocks avoiding hitting
them. To perform a jump, the player has to pronounce a required
phoneme. If the player is unable to provide a correct utterance,
Roberto hits a rock and finally after 5 impacts falls down. The
interactive objects are: small rockets representing rocket pieces,
and rocks that have to be avoided by the player. The player
actions are: to take rocket pieces by reaching them, jump over
rocks, hit rocks if no correct utterance is provided on a given
interval. The game was designed with a third person type of
view, and the following game situations: Roberto on the surface
of the moon, Roberto jumping over a rock, and Roberto reaching
a rocket piece. The challenges involved in the game are the
following: jump over rocks pronouncing a correct phoneme
(limited resources) within a determined time interval (time limit)
to earn points (reward), or hit rocks and fall (punishment); and if
a rocket piece is reached earn a bigger amount of points
(reward). As mechanics of the game, for each time Roberto
jumps a rock, 10 points are added to the budget, if a rocket piece
is reached, ten extra points are added. The time available for
producing the phoneme is five seconds. On the development of
this game, shown in Figure 7, also interface elements from the
Froneme the Frog game were reused. There are also works on
the definition of rules for defining different user interfaces as
output [32] but those objectives are out of the aim of this paper.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 118 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga

7. REFERENCES
[1] Arnab, S., Dunwell, I., & Debattista, K. (2013). Serious
games for healthcare: Applications and implications.
Medical Information Science Reference.
[2] McDowell, P., Darken, R., Sullivan, J., & Johnson, E.
(2006). Delta3D: a complete open source game and
simulation engine for building military training systems.
The Journal of Defense Modeling and Simulation:
Applications, Methodology, Technology, 3(3), 143-154.
[3] Michael, D. R., & Chen, S. L. (2005). Serious games:
Games that educate, train, and inform. Muska &
Lipman/Premier-Trade.
[4] Breuer, J. S., & Bente, G. (2010). Why so serious? On the
relation of serious games and learning. Eludamos. Journal
for Computer Game Culture, 4(1), 7-24.
[5] Kuipers, D. A., et al. (2013). Design for Transfer. In
Serious Games Development and Applications (pp. 239-
Figure 7. Screenshot of the serious game Roberto the robot. 246). Springer Berlin Heidelberg.
It is important to note as another advantage of the here proposed [6] Ritterfeld, U., Cody, M., & Vorderer, P. (Eds.). (2010).
metamodel that since the packages included on it keep a certain Serious games: Mechanisms and effects. Routledge.
level of independence towards the serious games design [7] Kikuchi, T., et al. (2007, June). Quasi-3-DOF rehabilitation
extension, it is possible to use them, and to define new system for upper limbs: its force-feedback mechanism and
extensions in order to design and develop other types of software for rehabilitation. In Rehabilitation Robotics,
software solutions for auditory rehabilitation (e.g., diagnosis, 2007. ICORR 2007. IEEE 10th International Conference on
tracking, and advising systems). Likewise, by developing (pp. 24-27). IEEE.
models for other domains than auditory rehabilitation such as [8] Burdea, G., Popescu, V., Hentz, V., & Colbert, K. (2000).
educational and military, it may be possible to relate them to the Virtual reality-based orthopedic telerehabilitation.
extension here provided to enable the development of serious Rehabilitation Engineering, IEEE Transactions on, 8(3),
games for those ambits. 430-432.
[9] Casas, Roberto, et al. User modelling in ambient
5. CONCLUSIONS AND FUTURE WORK intelligence for elderly and disabled people. Springer Berlin
In this paper we proposed SEGA-ARM, a metamodel for the Heidelberg, 2008.
design of serious games to support auditory rehabilitation. The [10] Kaklanis, N., A Coupled User and Task Modelling
mentioned model consists of three packages, regarding the Methodology for Accessible Product Design. PhD
context of use, the theoretical part of the therapy, and the dissertation, University of Surrey. March 2013.
dynamic part of the therapy; and an extension for its [11] Ventry, I. M., & Weinstein, B. E. (1982). The hearing
implementation through serious games. As the description of handicap inventory for the elderly: a new tool. Ear and
every single package and class was provided, the actors on it hearing, 3(3), 128-134.
involved (i.e., designer, development team, and speech therapist) [12] Koch, N., & Kraus, A. (2003). Towards a common
were mentioned along with tasks to be performed by them, and a metamodel for the development of web applications. In
case study was in it instanced to allow a better understanding. Web Engineering (pp. 497-506). Springer Berlin
Two prototypes of serious game to support a therapy activity Heidelberg.
were developed. In the near future the developed serious games [13] Brandt Jr, E. N., & Pope, A. M. (Eds.). (1997). Enabling
are going to be evaluated by a speech therapist in order to America: Assessing the Role of Rehabilitation Science and
determine if it is possible to include them at first as support tools Engineering. National Academies Press.
for in-consulting-room therapy sessions, and then as a way to [14] Rego, P., Moreira, P. M., & Reis, L. P. (2010, June).
complement and extend the received therapy sessions with Serious games for rehabilitation: A survey and a
sessions conducted by the patient himself. Also, more extensions classification towards a taxonomy. In Information Systems
are going to be developed in order to verify the completeness of and Technologies (CISTI), 2010 (pp. 1-6). IEEE.
the metamodel for different scopes. [15] McDonnell, A., & Akruvala, S. D. (2012). Rehabilitative
Therapists in Geriatric Health Services. Managing Geriatric
6. ACKNOWLEDGEMENTS Health Services, 45.
We acknowledge the support that was given by CONACYT, [16] Bureau of Labor Statistics, U.S. Department of Labor,
The Université catholique de Louvain and the Autonomous Occupational Outlook Handbook, 2014-15 Edition,
University of Aguascalientes for the development of this work Speech-Language Pathologists, on the Internet at
through a research stay program. We also acknowledge http://www.bls.gov/ooh/healthcare/speech-language-
Professor Jean Vanderdonckt from Université catholique de pathologists.htm (visited March, 2015).
Louvain for his valuable support on the development of this [17] World Health Organization, Fact sheet No.300 Updated
project. March 2015, on the Internet at
http://www.who.int/mediacentre/factsheets/fs300/en/
(visited March, 2015).

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 119 |
 SEGA-ARM: A Metamodel for the Design of Serious Games to Support Auditory Rehabilitation
D. Céspedes-Hernández | J. L. Pérez-Medina | J. M. González-Calleros | F. J. Álvarez Rodríguez | J. Muñoz-Arteaga
[18] Pereira, A. (2010). Criteria for elaborating subtitles for deaf Proceedings of the 15th International Academic MindTrek
and hard of hearing adults in Spain: Description of a case Conference: Envisioning Future Media Environments (pp.
study. Listening to Subtitles. Subtitles for the Deaf and 9-15). ACM.
Hard of Hearing, 87-102. [26] Taylor, L. N. (2002). Video games: Perspective, point-of-
[19] Svirsky, M. A., et al. (2000). Language development in view, and immersion (Doctoral dissertation, University of
profoundly deaf children with cochlear implants. Florida).
Psychological science, 11(2), 153-158. [27] Salen, K., & Zimmerman, E. (2004). Rules of play: Game
[20] National Institute on Deafness and Other Communication design fundamentals. MIT press.
Disorders, U.S. Department of Health & Human Services. [28] Sánchez, J. L. G., Zea, N. P., & Gutiérrez, F. L. (2009).
Quick Statistics. On the Internet From usability to playability: Introduction to player-centred
http://www.nidcd.nih.gov/health/statistics/pages/quick.aspx video game development process. In Human Centered
(visited February, 2015). Design (pp. 65-74). Springer Berlin Heidelberg.
[21] Zyda, M. (2005). From visual simulation to virtual reality [29] Longstreet, C. S., & Cooper, K. (2012, July). A meta-
to games. Computer, 38(9), 25-32. model for developing simulation games in higher education
[22] Chin, S. B., Bergeson, T. R., & Phan, J. (2012). Speech and professional development training. In Computer Games
intelligibility and prosody production in children with (CGAMES), 2012 17th International Conference on (pp.
cochlear implants. Journal of communication disorders, 39-44). IEEE.
45(5), 355-366. [30] Ittleman, M. A. (2012). The Teaching of Talking: Learn to
[23] Lenden, J. M., & Flipsen, P. (2007). Prosody and voice Do Expert Speech Therapy at Home with Children and
characteristics of children with cochlear implants. Journal Adults. Morgan James Publishing.
of Communication Disorders, 40(1), 66-81. [31] Calvary, G., et al. (2003). A unifying reference framework
[24] Aldrich, C. (2009). The complete guide to simulations and for multi-target user interfaces. Interacting with Computers,
serious games: how the most valuable content will be 15(3), 289-308.
created in the age beyond Gutenberg to Google. John Wiley [32] Florins, M., et al. (2006). Splitting rules for graceful
& Sons. degradation of user interfaces. In Proceedings of the
[25] Deterding, S., et al. (2011, September). From game design working conference on Advanced visual interfaces (pp. 59-
elements to gamefulness: defining gamification. In 66). ACM.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 120 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez

Towards the Definition of a Framework for the

Management of Interactive Collaborative Learning
Applications for Preschoolers
Liliana Rodríguez-Vizzuett1, Jaime Muñoz-Arteaga1, Francisco J. Álvarez-
Jorge Luis Pérez-Medina2, Josefina Guerrero-García3, Rodríguez1
1 2 3
Universidad Autónoma de Université catholique de Louvain Benemérita Universidad Autónoma
Aguascalientes Place des Doyens 1 de Puebla
Av. Universidad 940, Ags., Mexico Louvain-la-Neuve, Belgium Av. 14 Sur y San Claudio, Puebla,
[email protected], {fjalvar, jorge.perezmedina Mexico
jmunozar}@correo.uaa.mx @uclouvain.be [email protected]
ABSTRACT acquiring knowledge. In most countries, preschool education
The adoption of technologies in education has been changing the begins at the age of three years, but there are cases in which
traditional teaching system considerably. New educational children between 2 and 6 years are enrolled into it. However,
paradigms make use of technological tools to enhance the our proposal focuses on children aged between 4 and 6 years.
knowledge given in a classroom. In the literature review there Preschool educational institutions of several countries
aren't reports of Frameworks that are based on models and established official documents which are provided to educators,
methods for developing Interactive Collaborative Learning with skills that must be met by the children, so that at the end of
Applications for Preschoolers (ICLAP). This paper proposes a this stage students have begun a process of formal contact with
meta-model for a Framework supporting the development of the language writing, through the exploration of texts with
ICLAP taking into account the technology, the educational different characteristics (e.g., books, newspapers and
aspect and collaborative techniques that were observed during instructions) [2-6].
the attendance to sessions in schools. The ICLAP meta-model Augmented Reality (AR) can be used in many application
includes useful concepts for the use of Augmented Reality since domains. In recent years, a number of prototypes and
we found that it emerges as a technology of interest in diverse applications have been created in areas such as Computer
areas and especially in education. Our contribution could be graphics, Human Computer Interaction (HCI), vision, and
used for teachers, designers and developers as a conceptual collaborative work including the following sectors: architecture,
guide. In addition, a study case is presented to illustrate the main design, engineering and production planning, games, education
concepts included in the meta-model. and learning [7-9]. AR can be designed not only to support
Categories and Subject Descriptors learning related disciplinary contents but also to provide other
skills such as critical thinking, collaboration and information
D.3.3 [Programming Languages]: Language Constructs and
sharing [9]. Applications that make use of AR achieve to
Features – frameworks; H.5.2 [User Interfaces]: Prototyping,
generate the basic principles of Collaborative Learning: (1)
User- centered design; K.3.1 [Computer Uses in Education]:
Positive Interdependence; (2) Promotion of Interaction; (3)
Collaborative Learning
Individual Accountability; (4) Interpersonal and Small Group
General Terms Skills; and (5) Group Processing. Positive Interdependence
Design, Human Factors, Collaboration. ensures that team members perceive that they are linked with
each other so that one cannot succeed unless everyone succeeds.
Keywords In Promotion of Interaction the students promote each other's
Collaborative Learning, Augmented Reality, Framework, success by assisting, supporting, encouraging, and praising each
Collaborative Application. other's efforts to learn. The Individual Accountability means that
each individual student's performance is assessed and the results
1. INTRODUCTION are given back to the group and the individual. The Interpersonal
The inclusion of technology in education should be considered and Small Groups Skills suggests that the student is required to
only a tool to support since it is not to coming to replace the work as part of a team (teamwork), and finally for the Group
professor, but to help the student to have many auditory and Processing the team members discuss how well they are
visual elements to enrich the process at teaching–learning. An achieving their goals and maintaining effective working
important factor to be considered is the help that relationships.
Communication and Information Technologies (TIC) give Our proposal, for the specific case study that is here presented,
because it provides the students with many kinds of tools to be makes use of Augmented Reality. The role of the AR is to
used within in the classroom and for enhancing the way in facilitate interaction among preschoolers using QR codes. These
which knowledge is acquired [1]. In traditional education, the codes are the markers that will be recognized by the application.
teacher is responsible for student learning by defining learning
objectives or content area learning tasks, and designing and There are many interactive applications to support the learning
making an assessment of what has been learned by the students. of reading and writing, however, these do not report to use a
In the traditional model of learning, the concept of collaboration methodology of development involving educational content,
is included to allow individuals to enrich the experience of making the applications not only to entertain children but also to

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 121 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
help them to obtain knowledge [10]. Moreover, the literature communication [16]. Collaborative learning has as goals to
survey does not report models for Interactive Applications promote the interaction among students, valuing the individual
supporting Collaborative Learning in Children, despite the fact contribution of each student that assumes tasks and
that there are collaborative applications, those do not report to responsibilities, development of group and personal skills such
be developed under a user-centered paradigm, besides, even as listening, participating, coordinating activities and leading.
though they count with collaborative characteristics, the absence For last the self-assessment is promoted since the students need
of a model describing their design makes their components to continuously know from their performance and their
reutilization limited. In order to develop literacy skills in equipment. To carry out collaborative learning, four important
children and considering the benefits that TICs offer, in this components are needed: students, the professor or facilitator,
paper a set of rules and constructs for the ellaboration of technological resources, and the environment for the
semantic models for ICLAPs is proposed, meaning by this, a collaboration. In collaborative learning there are different roles
meta-model for development ICLAP involving several actors to carry out a task, however, each member of team is responsible
(i.e., student, professor, software engineer, designer and for a task and when all the team members finish their activities,
programmer) within each development phase, this meta-model they present to the rest of the team what they did, so it is
pursues the purpose of including a collaborative component that important that there is good communication between all team
allows its reuse in any type of ICLAP, and is based on the members to do a better work. Collaborative activities enhance
available literature, the observation of the techniques used by learning by allowing individuals to exercise, verify, solidify, and
preschool professors, as well as technological and educational improve their mental models through discussions and
aspects. information sharing during the problem-solving process [17].
The rest of this paper is structured as follows: Section 2 Collaborative learning helps to generate a common language, by
introduces our approach by making an analysis of the literature establishing rules for the group functioning and in order to
related to the concepts of framework, collaborative learning, reduce the fear of criticism and to promote feedback. By taking
learning technologies, augmented reality, and a subsection with advantage of the knowledge and experience of the team,
related work. Section 3 introduces an ICLAP Meta-model. The according to their area of expertise and different viewpoints, the
meta-model is later discussed using a case study in section 4. quality of the decisions and of the obtained products is
Finally, section 5 presents our conclusions and future work. improved.

2. REVIEW 2.3 Learning Technologies

Technology is a tool that can change the nature of learning [19].
2.1 Framework The multimedia resources have the potential to create high
A framework provides an abstraction of a solution to different quality learning environments because they provide to the
kinds of problems that have some similarities. It also provides students a better way to learn. In the literature, some authors say
the steps or phases that must be followed to implement a that for any design to be successful, it must be based on the
solution without getting into the details of what activities are needs and interests of users as this ensures that it will fulfill the
done in each phase [11]. The using of a framework for functions for which it was elaborated [20]. In a classroom
developing ICLAP aims to generate a conceptual and supported by technology, students don't "learn" technology
technological support structure, in which other software because it only provides the tools to be used for authentic
development can make reuse of required components. For this learning, so the technology is a mean, not an end. It is well
purpose, firstly the construction of a meta-model that possesses known how difficult it is to engage children in learning,
basic elements such as collaborative learning, learning contents however, it has been reported that children enjoy the process
and the technology is required. when their tutor uses a touch screen and other devices in order to
make classes more interactive and interesting [21]. This leads to
2.2 Collaborative Learning the assumption that by involving technology, it’s easier to attract
A skill is defined as the ability to coordinate knowledge and the kid’s attention and to make the educational process more
know-how to respond to a given situation or to solve problems enjoyable for both, the tutor and the pupils. As classroom
in everyday life [12]. The skills in preschool that children should computer technology is being used for different types of
develop are: (1) the ability to assume different roles, both in the communication i.e. for presentation, for class interaction, and for
game and in various activities, (2) rely on their peers to achieve collaboration, students are required to assume different roles and
goals, (3) resolve conflicts through dialogue, and (4) respect and must be willing to collaborate with others. Another advantage of
recognize the rules of coexistence in the classroom, school and using technology in the classroom is its flexibility and
beyond [2][13][14]. Within the strategies that have to favor the adaptability to different type of learning [22]. A collaborative
learning, collaborative learning is the most important activity application is a software application that allows multiple users
that takes place in small groups mainly in a classroom in order interaction to achieve a common goal, receives input from
to share different viewpoints to form a more complete multiple users and displays outputs to multiple users. Also
knowledge [15]. The collaborative learning supported by couple those users, ie, allows the entrance of one of them to
technology consists on that two or more persons share the influence the output shown to the other [23].
responsibility of building knowledge, based on the interaction
and decision making using the technological resources as 2.4 Augmented Reality
mediators for this process. The collaborative learning process Augmented Reality (AR) is a paradigm of interaction that stems
consists of several tasks that must be performed by a facilitator from the desire to merge the processing capabilities of a
or teacher and students, also, has as key elements, the computer in the user's real environment. The goal of this
cooperation between group members, active learning, positive technology is overcoming the boundary between the real world,
interdependence, responsibility, coordination and including the user, his/her environment and the computing

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 122 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
world. A key feature of AR systems is that real-world elements model (apprentice and facilitator), the content model which
are involved in the interaction. The existence of real and virtual mentions that the contents are characterized as learning objects,
worlds implies that the purpose of the task may match to one of the communication model where the process is defined by which
the two. The term Mixed Reality corresponds to the interval is conducted the interaction between users and fosters
between the real and the virtual. Likewise, Dubois [7] proposes collaboration and finally the assessment model which takes into
a taxonomy based on the objective of where the crucial point in account the evaluation of service quality. The points that are
their approach was identifying the task supported by the system evaluated are: number of subscriptions that are the recognized
to be analyzed and designed. The taxonomy considers two users by the application, the number of connections of each
continuous: (1) Augmented Reality and (2) Virtual Reality subscription, the statistics of users who sent a message, the
(VR). In AR systems the user’s interaction with the physical response time of the facilitator to respond a message sent by an
world is enriched by the aggregation of data and services offered apprentice user, the evaluation of the contributions made on
by the numerical world (computer), in the natural environment collaboration and lastly the relevance of the message to both the
of the user. Comercial applications such as Dokéo+ learner and facilitator.
(http://www.dokeo-realiteaugmentee.com/), and L’Aphablet
(http://www.clementoni.com/) make use of AR and help to 3. AN INTERACTIVE COLLABORATIVE
reinforce learning in children, however, they do not report being LEARNING APPLICATIONS FOR
designed to support collaborative learning. Dokéo+ allows
children to learn different topics (science, animals, history,
PRESCHOOLERS META-MODEL
mathematics), but does not allow the interaction of more than This section shows the meta-model for ICLAP. The purpose of
this meta-model is to present a conceptual guide that allows the
one user at a time. L'Alphabet enables learning letters as well as
objects which names contain them. This tool uses audio and development of ICLAP considering collaborative techniques
animation resources, and has a bilingual mode, however, it that were observed while attending sessions in schools in
doesn't report either to enable collaboration among children. It is Mexico. The ICLAP meta-model follows a User-Centered
important to point out that both applications use AR markers. approach. The authors have had the opportunity of proposing
Figure 1 allows distinguishing the two streams corresponding to activities for the children, carry them out and thus obtain the
the both extremes of a continuum between real and virtual definition of collaborative strategies that are used by teachers to
accomplish the acquisition of collaboration skills.
worlds.
3.1 General Description
The main concepts of the ICLAP meta-model are presented on
Figure 2 that is divided into six packages. The first one refers
the Evaluation package of the whole application and its usage.
The second package concerns the Technology and is formed by
Interface and GameElement. The third is for description of
Collaborative Learning, conformed by the class
InteractiveCollaborativeLearning and by the one referent to
InteractiveCollaborativeApplication. The fourth package
concerning the LearningContent and formed by
LearningContent and LearningStyle, the fifth package
concerning the Context model formed by the User, the Platform
and the Environment. Finally, the Interaction Resources package
consisting of the elements that will be used to develop an
application.

Figure 1. Classification of Mixed Systems [7][24]

3.2 The Evaluation Package
An important part to be considered, is the evaluation of
2.5 Related Work interactive applications concerning usability, skills and
In [25] the theory of the activity is defined as a strategy for the collaborative learning using instruments and also using quality
development of collaborative learning systems in which takes metrics [2] [29-33]. The evaluation is the measurement of
into account the activity, goal, result, community, rules, works’ knowledge, skills and performance of students when using a
division and tools. Likewise, the modeling methodology collaborative interactive application. It uses instrument that are
AMENITIES [26] define four views for cooperative systems. documents which purpose is to justify that interactive
They’re: group view (groups, roles and actors), cognitive view collaborative applications working for learning and
(tasks, sub-activities and actions), interaction view (protocols, collaboration. An Instrument class is composed of an
devices, media) and information view (resources, documents, IntegratedMeasure element that compute the Measure of each
messages). Penichet [27] highlights the importance of user Metric. Each instrument has one or several sections that relates
interaction with collaborative systems, and makes a proposal to to the topic to be evaluated (usability, learning skills,
collaborative environments from obtaining requirements to final collaborative skills, perception of technology, collaborative
system maintenance, through the phases of analysis, design, work, and quality assessment). In turn, the section has a name
implementation, and evaluation. Finally [28] makes a literature and one order. Moreover the dimension refers to each of the
review which mentions that it is important to consider the user characteristics to be evaluated in each instrument (e.g., Easy to
interaction with the system to be developed, with special remember, language and communication, cordination). The
emphasis on user interfaces, without leaving aside the user dimension is composed of a description and order.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 123 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez

Figure 2. An Interactive Collaborative Learning Applications Meta-Model

Every dimension must carry at least one item; the attributes of an and the errorSolution; GameWorld, considering the scene in
item are the question to be answered, and the order representing which the game takes place, the view (e.g. first person, third
the value that was given to it. A Metric is a mean to understand, person, isometric and cabin view), and the interactiveElements,
monitoring, controlling, predicting and testing the software which are those elements that the User may use to interact with
development and maintenance projects [34]. Metrics has a set of the system (e.g., images, audio and animations); and
attributes: name, description, and also a lowValue and upperValue InteractiveSystem, to describe the modality of interaction. Table 1
that are the measured values for each metric. Metrics produce shows an OCL Rule defined to ensure that instances of
measures that throw values that are the results for each InteractiveCollaborativeApplication class have three different
instrument. elements of type GameElements at the same time
3.3 The Technology Package
For the part of the model concerning the Interface, the model
provided by [35] is taken. Its definition is achieved by using the
theoretical framework CAMELEON [36] that comprises four
levels of abstraction: (1) Tasks & Concepts: description of the
users’ activities. (2) Abstract User Interfaces (AUI): Definition of
abstract containers and individual interaction components. (3)
Concrete User Interfaces (CUI): Definition of the interaction
modality and usage of concrete interaction objects. (4) Final User
Interfaces (FUI): Operational elements, this is, the implemented
UI for a determined platform. Elements for the design of
videogames according to [37] are considered. Those elements
being: ErrorManagement, for the presentation of errors to the
User, including an icon, a message, a trigger called detectedError,

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 124 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
Table 1. OCL rule for InteractiveCollaborationApplication the acquisition of theoretical knowledge, but comprehension, and
instances discovery of the environment.
Table 2. The Structure of Learning Content
Name
Description
Objective
3.4 The Collaborative Learning Package Learn to know Learn to be Learn to do
Evaluation of learn to Evaluation of learn to Evaluation of learn
For the part of the model concerning the CollaborativeLearning,
know be to do
an InteractiveCollaborativeActivity [38] is an activity that Learning Type
involves the Feedback between two entities (viz. tutor-pupil and Learning Style
pupil-pupil) [39], and a series of attributes: rules to follow during The LearningStyle describes the way in which Users learn as well
the activity, collaborativeSkills, rewards as incentives for the as the actions to be performed in order to promote a more
User (e.g., badges and levels), executionTime in seconds, effective learning(e.g., Felder and Silverman models, Neuro
monitoring performed by the Tutor, teamSize that is the number of linguistic programming, Kolb model, and Multiple Intelligences
members in each group between 2 and 6 [40], and the model). However, due to the generality of the model proposed,
numberOfTeams participating on the collaborative activity. The more styles can be adapted to this class. The attributes that are
collaborativeSkills are those to be achieved during the considered for the definition of the LearningStyle are the
collaborative learning process, and are determined by a set of voicePitch, the posture of the User while conducting activities,
strategies to allow the development of mixed skills (concerning the preferred perceptionChannel for the User to get knowledge
learning and personal/social development), in which every team (e.g. images, audio and physical activities), and voiceSpeed.
member is responsible for both, his/her and the other members’
learning. The InteractiveCollaborativeActivity is related to a Stakeholders may invoke Table 2 to elaborate the Learning
CollaborativeComponent. The CollaborativeComponent has as Contents. The table is formed based on the elements of the
attributes a name, a description, an objective and a package. The elements are: Name, Description of the learning
collaborativePattern. Papageorgiu [41] defined a collaboration content, Objective, Learnings, EvaluationOfLearning,
pattern as a means to capture best practices about recurring LearningType and finally the LearningStyle.
collaborative problems and solutions. As part of this component, 3.6 The Context Package
there are Roles defined for each team member. For the Roles
The Context [45] represents the conditions in which interaction is
definition, it is necessary to specify a roleObjective and roleTasks
carried out and is formed by the User, this class contains as
that are the purpose of the role and the actions to be performed by
attributes an id, that is a key for the identification of each User;
each one. For their relevance to this work, the Roles that are
the preferred language for communication; a stereotypeName, the
considered are: Resource Manager, Time Keeper, and Leader. The
role assigned for an activity (viz. Tutor or pupil); the
Leader is in charge of asking the teacher for assistance whenever
systemExperience, representing how experienced is the User on
it is needed. This role also must be able to organize other team
the usage of interactive applications; the taskExperience,
members in order to properly carry out the activity. The Resource
containing the explanation of how familiar the activity is o the
Manager is supposed to ensure that the given application is used
User; the taskMotivation, motivation or stimuli that cause interest
in a proper manner, and finally the Time Keeper must notify each
on the User for completing the task in a correct manner; the
member of the group about the available time for successfully
deviceExperience, the experience of the User on the usage of
performing the assignment [42].
electronic devices such as tablets, smartphones, computers and
3.5 The Learning Content Package consoles; and the age of the User.
The LearningContent is the information referent to a subject of
study and is formed by a LearningObject [43] that is a basic
reusable structure that contains detailed information about certain
topics and allows the relation to other similar objects. A
LearningObject has as attributes an id, a name, a date of creation,
a description, and an objective. And it’s contained into a category
(e.g., Reading and Writing), subset of subjects that are related to
an area (e.g., Language). The LearningObject contains in turn,
Content, which is the information to be presented and has as
attributes an id, a name, and an associated LearningType (e.g.
casesBased, problemBased, projectBased and gameBased).
Content may be presented in several formats such as videos, texts,
images, audios and animations. Related to the LearningObject,
there are the Author class that represents its creator, the
EvaluationOfLearning and the Learnings class that conform it.
Learnings [44] are that knowledge acquired during the education
process and are specified as learn to be (Being), referring to the
skills developed to be practiced on the daily life; learn to do Figure 3. The Interaction Resource package
(Doing), that is the integration of knowledge to be useful on the
professional life; and learn to know (Knowing), meaning not only

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 125 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
The User can be either a Tutor (professor, parent or supervisor) or
a Pupil (student or trainee). The Platform entity represents a series
of elements that allows the execution of interactive applications.
The attributes are id, name and description and is composed by
softwarePlatform and hardwarePlatform. The only attribute for
the softwarePlatform is the operatingSystem, while for the
description of the hardwarePlatform, it is necessary to specify a
deviceType (e.g. tablet, smartphone and computer), the screenSize
in inches, the availability of a touch screen on it (isTouchScreen)
and the availability of an internetConnection. The Environment
class describes properties of interest about the place in which the
users perform a collaborative activity. Their attributes are an id, a
name, the statement of whether the place is noisy or not (isNoisy),
and the statement of whether the place is stressing or not
(isStressing), this means, if there are or not characteristics that Figure 5. The Task tree model of the case study application
propitiate a change on the Users’ behavior during the activity. The The task model shows the activities that must be carried out in the
definition and consideration of the last two attributes of this class application. The application is aimed at learning the letters and
is important since they may help to determine characteristics that objects that children see in their daily lives and must do so
directly impact on the learning process. collaboratively. The first task that user need to do is select the
language in which they will play, for this, the users have French
3.7 The Interaction Resources Package and Spanish as options. Next, the user must select the number of
The interaction resources package presented in Figure 3 considers players, the application allows to play for 2-5 players, followed by
all interaction elements. Its elements are important within any
this, the user must choose the difficulty level, these levels are:
application that is implemented because they are the basis with
easy, medium and hard. After this selection the game begins. To
which the user interacts. These elements may be images, audio,
play, each child will have a pack of cards with QR codes and must
numbers, words or letters regardless of the amount used of these. be sequentially showing to webcam the letters to complete the
AR applications should require markers associated to these name of the object that appears on the screen. The game consists
resources. For example, QR codes as those shown in Figure 4. on presenting pictures (animals, household items, food, human
Moreover in Figure 4, we show the interaction resources that can body, etc.) and boxes where users must place the letters of their
be used in any type of interactive application.
names. For the first level of difficulty the application will show
the boxes with the letters in watermark to point out where the user
should place each one. For the second level, the system only will
show the first letter of the word and the users should be able to
complete it, and finally at the third level there will be only boxes,
without any aid, in which users should be able to complete the
word. Figure 6 presents screenshots for each level of difficulty.
When the letter shown by the user is correct the application will
fill the corresponding box with it and also, the application will
present an animation with balloons, a trophy and audio. When a
user makes a mistake, the application shows an image to the user
asking him to try again.

Figure 4. The interaction resources

4. CASE STUDY A COLLABORATIVE

LEARNING ALPHABET APPLICATION
In this section we consider a demonstrative case study. The aim is
to carry out benefits of the proposed meta-model. In this case
study the following packages are being instanced: Learning
Content, Collaborative Learning and Technology since they are
most important for the design phase. In order to take decisions for
the development phase, the Context package should be considered
and on the evaluation stage, the Evaluation package has to be
taken into account. The case study consists of a set of sequential
tasks accomplished for children in order to indicate the letters of
the word that is presented by the application. The interaction
scenario is defined in a task tree model depicted by Figure 5.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 126 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
correspond to each one. To identify each child within the system,
distinct markers must be used.

Figure 8. Cards with QR codes given to each child

The application displays the word to be completed and among all
children should complete the word based on each turn generated
by the application. When the letters are insufficient for every child
to participate, turn assignation in the next word should start with
those who did not. Table 3 shows the design of Learning Content
Figure 6. The levels of difficulty on the developed application required by the interactive application. The table is constructed by
Collaborative skills to be achieved in children using this the professor based on skills for learning and collaboration that
interactive application are: (1) Show facility to interacting with he/she wants to develop in children by using the application. Once
others; (2) Accept to perform different roles and assume their this table is constructed, with help of a designer, prototypes of
responsibility with the team; (3) Respect the turns when carrying interfaces are created and the type of elements to be used for
out an activity; and (4) Respect their peers. To achieve these displaying learning content is defined. The most important
collaborative skills, it is required to have a collaborative strategy. elements that the developer is supposed to know in order to
One is the use of turns. A turn consists of assigning a task to a implement the interactive application are: a name, a description,
particular child. The collaborative component implemented in this an objective, the learnings and their associated evaluations, as
application will be the Turn Assignation pattern as shown in well as a learning type and a learning style.
Figure 7. Table 3. The Learning Content for “Learning the letters”
Learning the letters
Use the knowledge that they have of their names and other words to write
something they want to express
Recognizes characteristics of the writing system use resources (marks,
graphs, letters) to express in writing their ideas
The letters and its Show some images The words in
representation are shown with daily objects that watermark are
to the student. Also, contain letters shown to the
their utility in daily life students and they
is explained are required to put
letters in place
Students should Questions are asked The words without
Figure 7. Collaboration Pattern in Spanish recognize letters and about the letters in watermark are
their order in the objects’ names. shown to
The objective of this pattern is to allow each team member to alphabet Students are asked to the students and
fulfill a role and carry out assigned activities. The pattern begins complete those words they are required
with an activity to do, then the number of participants who will to check if they to put letters in
perform the activity should be select, later it is possible make a correctly identify place
turn assignation in a sequential or random manner as requested by them
the teacher. To ensure the participation of all team members in our Based on Games
interactive application turn assignation must be performed Visual, Auditive and Kinesthetic
sequentially. We must also ensure that all children participate.
This will be done as follows: Each child will be given a pack of 5. CONCLUSIONS AND FUTURE WORK
letters with QR codes as those shown in Figure 8; each package This paper presents a meta-model for ICLAP to support the
will have different colors to allow children to identify those that education process. It will serve as base to our framework and
permits a conceptual guide for teachers, designers and developers
for the implementation of this type of applications. The proposed
XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models
| 127 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
model considers technological and educational aspects, as well as [9] Klopfer, E., Perry, J., Squire, K., & Jan, M. F. (2005, May).
collaborative techniques that were observed during the attendence Collaborative learning through augmented reality role
to sessions in schools in Mexico through the performance of playing. In Proceedings of the 2005 conference on Computer
activities that teachers carried out with children using support for collaborative learning: learning 2005: the next 10
collaborative strategies for teaching letters and their use on daily years! (pp. 311-315). International Society of the Learning
life. One of the advantages that the meta-model provides is the Sciences.
capability of its packages to be reused in other type of [10] Proyecto H@z TIC. Guía práctica de aprendizaje digital de
collaborative applications, another advantage is that it can be used lectoescritura mediante tablet para alumnos con síndrome de
in other areas where there is collaborative work. The Technology Down, Ministerio de Educación, Cultura y Deportes, 2012.
Package allows to add other game design elements according to [11] Johnson, R., Foote, B. « Designing Reusable Classes”.
the requirements of the application to be developed. The learning Journal of Object-Oriented Programming, vol. 1, no. 2, pp
content package allows adaptation to learning styles and types as 22-35 (1988)
required. The proposed meta-model presents concepts that can be [12] Nova, A. P. (2011). Formación docente: hacia una definición
easily understood by the different actors involved the del concepto de competencia profesional docente. Revista
development process of interactive collaborative learning electrónica interuniversitaria de formación del profesorado,
applications i.e., professor, software engineer, designer, 14(1), 67-80.
programmer and tester. Actually, the presented case study is in [13] Delors, J. (2013). Los cuatro pilares de la educación. Galileo,
development phase and once a first prototype is produced, we will (23).
be able to easily evaluate the interest and the feasibility of our [14] Brophy, J. (1998). Un ambiente propicio para el aprendizaje
approach by conducing user experiments supported by the en el aula. “Oportunidades para aprender”,“Construcción de
instruments that were mentioned as part of the evaluation un soporte para lograr el trabajo comprometido de los
package. Besides, in parallel a methodology is being designed in alumnos”,“Aprendizaje colaborativo” y “Expectativas de
order to guide the development process, by establishing different logros”, en La enseñanza, México, UNESCO/SEP (Serie
actors and the tasks that each one must fulfill for producing Cuadernos. Biblioteca para la actualización del maestro).
collaborative interactive applications, taking into account all [15] Ordóñez, C. A. C., & Mendoza, J. (2006). Cómo aprovechar
concepts of ICLAP meta-model. As well, we will perform el "aprendizaje colaborativo" en el aula. Educación y
different case studies with children. educadores, 9(2), 61-76.
[16] Panitz, T. (2008). 44 Benefits of Reflective Learning.
6. ACKNOWLEDGMENTS Retrieved May 30, 2008, from Global Development
We acknowledge the support that was given by CONACYT, the Research Centre: http://www.gdrc.org/kmgmt /c-
Université catholique de Louvain and the Autonomous University learn/44.html
of Aguascalientes for the development of this work through a [17] Alavi, M. (1994). Computer-mediated collaborative learning:
research stay program. We also acknowledge Professor Jean An empirical evaluation. MIS quarterly, 159-174.
Vanderdonckt for his valuable support for the development of this [18] Glassett, K., & Schrum, L. (2009). Teacher beliefs and
project. student achievement in technology-rich classroom
environments. International Journal of Technology in
7. REFERENCES Teaching and Learning, 5(2), 138-153.
[1] Graells, P. M. (2000). Impacto de las TIC en educación: [19] Spiliotopoulos, T. (Ed.). (2010). Integrating Usability
funciones y limitaciones. Departamento de Pedagogía Engineering for Designing the Web Experience:
Aplicada, Facultad de Educación, UAB. Methodologies and Principles: Methodologies and Principles.
[2] Secretaría de Educación Pública de México SEP. Enfoque IGI Global.
Centrado en Competencias. Available online at http://www. [20] Kosakowski, J. (1998). The Benefits of Information
dgespe.sep.gob.mx/reforma_curricular/planes/lepri/plan_de_ Technology. ERIC Digest.
estudios/enfoque_centrado_competencias. Consulted on [21] Schmid, E. C. (2008). Potential pedagogical benefits and
February 2015. drawbacks of multimedia use in the English language
[3] Szinger, V. Pre-school literacy in the light of European classroom equipped with interactive whiteboard technology.
tradition. Computers & Education, 51(4), 1553-1568.
[4] Sharp, C. (2002, November). School starting age: European [22] Dewan, P., Choudhary, R., & Shen, H. (1994). An editing
policy and recent research. In LGA Seminar 'When Should based characterization of the design space of collaborative
Our Children Start School’. applications. Journal of Organizational Computing and
[5] Lisle, J. (1998). Proyecto principal de educación en América Electronic Commerce, 4(3), 219-239
Latina y el Caribe. El informe Delors dentro del contexto [23] Milgram P., Kishino, F. A Taxonomy of Mixed Reality
Americano. Visual Displays, Transactions on Information Systems, pages
[6] Rose, P., et al. (2014). Informe de Seguimiento de la EPT en 1321-1329, 1994.
el mundo. Enseñanza y Aprendizaje: Lograr la calidad para [24] Barros, B., Vélez, J., & Verdejo, F. (2004). Aplicaciones de
todos. Informe 2014. la Teoría de la Actividad en el desarrollo de Sistemas
[7] Dubois, E.: Chirurgie Augmentée: un Cas de Réalité Colaborativos de Enseñanza y Aprendizaje. Experiencias y
Augmentée; Conception et Réalisation Centrées sur Resultados. Inteligencia artificial, 8(24), 67-76.
l'Utilisateur. PhD Thesis University of Grenoble I, France [25] Gea, M., Gutiérrez, F., Garrido, J., & Cañas, J. (2002).
pages 275, 2001. AMENITIES: Metodología de Modelado de Sistemas
[8] Hannes Kaufmann, Dieter Schmalstieg. Mathematics and Cooperativos. In Workshop de Investigación sobre nuevos
geometry education with collaborative augmented reality.
Computers and Graphics, 2003.

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 128 |
 Towards the Definition of a Framework for the Manage-ment of Interactive Collaborative Learning Applications for Preschoolers
L. Rodríguez-Vizzuett | J. L. Pérez-Medina| J. Muñoz-Arteaga | J. Guerrero-García | F. J. Álvarez-Rodríguez
paradigmas de interacción en entornos colaborativos [35] Calvary, G., Coutaz, J., et al., A Unifying Reference
aplicados a la gestión y difusión del Patrimonio cultural. Framework for Multi-Target User Interfaces, Interacting with
[26] Penichet, V. M., Lozano, M. D., & Gallud, J. A. (2005). Computers, 15(3), June 2003, pp. 289–308.
Hacia una propuesta metodológica para entornos [36] Aldrich, C., The complete guide to simulations & serious
colaborativos. III Taller de Sistemas Hipermedia games, Pfeiffer, 2009
Colaborativos y Adaptativos. [37] Álvarez, I., & Guasch, T. (2006). Diseño de estrategias
[27] Aular, Y. J. M., & Pereira, R. T. (2007). Metodologías para interactivas para la construcción de conocimiento profesional
el desarrollo de ambientes de aprendizaje en entornos en entornos virtuales de enseñanza y aprendizaje. Revista de
colaborativos: Una reflexión teórica. Multiciencias, 7(1). Educación a Distancia, (14).
[28] Jung, H. W., Kim, S. G., & Chung, C. S. (2004). Measuring [38] Calzadilla, M. E. (2002). Aprendizaje colaborativo y
software product quality: A survey of ISO/IEC 9126. IEEE tecnologías de la información y la comunicación. Revista
software, 21(5), 88-92. Iberoamericana de Educación, 1(10).
[29] Nielsen, J. (1994). Usability engineering. Elsevier. [39] Adams, D., & Hamm, M., Cooperative Learning, Critical
[30] Oviedo, A., Instrumento de Evaluación del Grado de Thinking and Collaboration Across The Curriculum. 2nd ed.
Apropiación Asistido por TIC de Competencias en Springfield III: C.C. Thomas, 1996
Preescolar. 2014 [40] Papageorgiou, N., et al. (2009). A collaboration pattern
[31] Quezada, P., Método de evaluación para TIC's aplicadas en model for virtual organisations. In Leveraging Knowledge
la educación preescolar, caso: Sistema del Estado de for Innovation in Collaborative Networks (pp. 61-68).
Aguascalientes. Tecnologías y Aprendizaje: Avances en Springer Berlin Heidelberg.
Iberoamérica Vol2. 2013 [41] Olguín, S. C., et al. (2006). Ciencias. Antología. Primer
[32] García, M., Santana, P. (2015). Introducción a la usabilidad Taller de Actualización sobre los Programas de Estudio
de los videojuegos. La Interacción Humano-Computadora en 2006. Reforma de la Educación Secundaria. Coordinador
México. Muñoz, J., González, J., Sánchez, A., Eds. Pearson, editorial. SEP
México. [42] Rodríguez-Vizzuett, L., et al. Metodología para el Desarrollo
[33] Basili, V. R., Briand, L. C., & Melo, W. L. (1996). A de Sistemas Interactivos para Gestión y Visualización de
validation of object-oriented design metrics as quality Objetos de Aprendizaje.
indicators. Software Engineering, IEEE Transactions on, [43] Zhou, N. Z. (1996). Four ‘Pillars of Learning’ for the
22(10). Reorientation and Reorganization of Curriculum: Reflections
[34] Stanciulescu, A. (2008). A methodology for developing and Discussions.
multimodal user interfaces of information systems (Vol. [44] Limbourg, Q. (2004). Multi-path development of user
556). Presses univ. de Louvain. interfaces (Doctoral dissertation, UCL).

XVI Congreso Internacional de Interacción Persona-Ordenador | Methodologies and Models

| 129 |
 DOCTORAL
COLLOQUIUM
 Intelligent Playful Environments for Animals
P. Pons | J. Jaen | A. Catala

Intelligent Playful Environments for Animals

Patricia Pons, Javier Jaen, Alejandro Catala
Grupo ISSI, Departamento de Sistemas Informáticos y Computación
Universitat Politècnica de València
Camí de Vera s/n, 46022 Valencia (Spain)
[email protected], [email protected], [email protected]

ABSTRACT [12] initiative are studies where the animals interact with
We are evolving towards an interconnected and ubiquitous world, specialized keyboards and/or touch screens to communicate with
where digital devices and interfaces progressively adapt the trainers. Technology has also been used to improve critical
themselves to fit our needs and ease our daily activities. Although tasks of some animals: ACI principles have been applied to
we coexist with plenty of animal species, such as our pets, we are develop suitable dog-oriented interfaces for Diabetes Alert Dogs
approaching the evolution of technology in a strictly human- (DAD), allowing them to alert emergency services when their
centric manner. A new field in Computer Science, called Animal- owners suffer a hypoglycemic attack [8].
Computer Interaction (ACI), aims at filling this technological gap In recent years, several studies have focused on enhancing animal
by developing systems and interfaces specifically designed for play using technology for different purposes. LonelyDog@Home
animals. Supporting animals’ natural behavior and habits with [1] proposes a web interface where the human can connect to a web
suitable technology could improve both humans and animals’ camera located at her home and check how their pets are, feed them or
wellbeing. As a consequence, this doctoral research aims to play with them remotely. Canine Amusement and Training [11] is a
explore, design and develop animal-centered intelligent systems training system where, using a projector and cameras, the training
that focus on enhancing one of the most natural animal behaviors: activity becomes a playful experience. Felino [10] is a tablet-
play. Therefore, the main goal of this research is to expand ACI based game where a human and a cat can play together capturing
with the ability of automatically manage and adapt animals play different creatures on the screen. The human controls several
activity in order to improve their wellbeing. aspects of the game, adapting it to the cat’s reactions and
preferences.
Categories and Subject Descriptors
Although there are already games for animals which make use of
H.5.3 [Information Interfaces and Presentation]: User
technology to improve their experience, these games have some
interfaces – interaction styles, user-centered design, input devices
limitations which will be outlined in the next section.
and strategies.

General Terms 3. INTELLIGENT PLAYFUL

Design, Experimentation. ENVIRONMENTS FOR ANIMALS
3.1 Motivation
Keywords The focus of this research activity on animal play is motivated by
Animal Computer Interaction; Playful Interaction; Intelligent two reasons. On the one hand, play is one of the most natural
Environment; Animals; Interfaces. behaviors among all animal species. Animals do not need to be
taught on how to play [2], as it stands as a natural and voluntary
1. INTRODUCTION activity, which has also its repercussion in elder stages of their
Human Computer Interaction (HCI) has brought countless lives. As an example, felines’ play is always based on hunting or
benefits to human wellbeing. By easing the interaction with chasing behaviors, as in the wild, these animals traditionally used
computer systems, technology has become more user-friendly and to hunt for food. On the other hand, verbal or written
it is being applied to our everyday activities in a seamless manner. communication is unfeasible when considering animals as target
However, other species are not taking advantage of all these users, and thus, other methods should emerge in order to
progresses. Due to different physical features or mental determine the perceived usability and usefulness of the system.
perceptions of the world, animals are not able to understand our Therefore, playful interactions could be an effective way of
technology as we do. Therefore, a new research field called communicating with animals and gathering information about the
Animal Computer Interaction (ACI) [3,4] has emerged. It focuses usefulness of a system. As summarized in Section 2, animal play
on providing animals with similar technological improvements to has already started to be the object of a digital revolution.
the ones humans already have. ACI considers animals as the However, existing digital games for animals are focused on a
target users of digital systems, and proposes the development of single activity or purpose and only use one digital device.
interfaces specifically designed for them. The doctoral research Moreover, they require human mediation for the animal to play
presented in this paper proposes expanding ACI with the ability of and with time the animal might lose interest when it gets used to
automatically managing and adapting animals play activity in play the same game.
order to improve their wellbeing, focusing on animal pets and
specifically in cats and dogs. The main contribution of this doctoral research is two-fold.
Firstly, we will consider digital games for animals as playful
2. RELATED WORKS environments composed of several interconnected digital devices.
There have been several research studies involving animals which Those devices have to be appealing and suitable for the animal
use technology as a bridge to communicate or interact with them. and for the purpose of the game. Secondly, a playful environment
The LANA Project [9], the Ai Project [5] or the SpeakDolphin of this characteristics will be an intelligent system which should

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 131 |
 Intelligent Playful Environments for Animals
P. Pons | J. Jaen | A. Catala
intelligently make use of its composing devices in order to create digital systems. We have already defined these environments,
suitable playful activities for different purposes. The intelligent their features and potential [6,7]. Our immediate future work will
environment should gather reliable information about the animal, be conducting a set of experiments in order to answer question Q1
its preferences, physical features and its surrounding context. in Section 3.3. For this purpose, we will study how different
With this knowledge, the intelligent environment should create an devices and stimuli affect the attention and motivation of the
engaging playful activity for a specific situation, deciding which animal for starting a playful activity. Based on the results of these
devices and interaction mechanisms would be more suitable for experiments, we will design suitable activities in order to conduct
the given context. There are plenty of scenarios where intelligent future studies to answer the remaining research questions.
playful environments could help to improve animals’ wellbeing,
e.g., create playful activities to alleviate the stress and anxiety of 5. REFERENCES
animals alone at home, zoos or shelters. It could also encourage [1] Hu, F., Silver, D., and Trude, A.LonelyDog@Home.
these animals to perform physical exercise by means of play. The 2007 IEEE/WIC/ACM International Conferences on Web
system should adapt the game to the animal’s attributes such as Intelligence and Intelligent Agent Technology - Workshops, IEEE
species, weight or age - old animals should not perform as much (2007), 333–337.
physical exercise as young ones. An intelligent playful [2] Huizinga, J.Homo ludens. Wolters-Noordhoff,
environment could also assist in animal training, making the Groningen, The Nederlands, 1985.
activity more amusing and less repetitive, and without requiring
human supervision. The environment could chose the optimal [3] Mancini, C.Animal-computer interaction: a manifesto.
time of the day to perform training, i.e., when the animal’s Magazine interactions 18, 4 (2011), 69–73.
attention and motivation seems to be higher. [4] Mancini, C.Animal-computer interaction (ACI):
changing perspective on HCI, participation and sustainability.
3.2 Definition CHI ’13 Extended Abstracts on Human Factors in Computing
As part of the planning stage of the research, we have already Systems, ACM Press (2013), 2227–2236.
defined the scope of the systems being developed [6,7]:
[5] Matsuzawa, T.The Ai project: historical and ecological
An intelligent playful environment for animals, or IPE4A, is an contexts. Animal cognition 6, 4 (2003), 199–211.
animal-centered ecosystem with intelligent capabilities which is [6] Pons, P., Jaen, J., and Catala, A.Animal Ludens :
able to learn from the animals’ behaviors and interactions, using Building Intelligent Playful Environments for Animals. 11th
the acquired knowledge to adapt itself to the context, creating Conference on Advances in Computer Entertainment - Workshop
engaging playful activities which do not necessarily need human on Animal Human Computer Interaction, (2014).
mediation to evolve.
[7] Pons, P., Jaen, J., and Catala, A.Envisioning Future
We have also outlined the different features and dimensions of Playful Interactive Environments for Animals. In A. Nijholt, ed.,
intelligent playful environments which can affect the design and More Playful User Interfaces. Springer, 2015.
development of these systems [6,7]: number of participants,
[8] Robinson, C., Mancini, C., van der Linden, J., Guest,
species of the participants, human participation, human presence,
C., and Harris, R.Empowering assistance dogs : an alarm interface
control, information acquisition, learning inputs, types of stimuli,
for canine use. Intelligent Systems for Animal Welfare, (2014).
purpose of the activity.
[9] Rumbaugh, D.M., Gill, T. V., Brown, J. V., et al.A
3.3 Research questions computer-controlled language training system for investigating
This thesis will give answer to several fundamental questions on the language skills of young apes. Behavior Research Methods &
how to provide successful interactions and communication Instrumentation 5, 5 (1973), 385–392.
between the animal and the intelligent system. Some of these [10] Westerlaken, M. and Gualeni, S.Felino: The
research questions are: Philosophical Practice of Making an Interspecies Videogame. The
• Q1: Which are the most appropriate interactive Philosophy of Computer Games Conference, (2014), 1–12.
mechanisms for the animal, e.g. a cat or a dog? [11] Wingrave, C.A., Rose, J., Langston, T., and LaViola,
J.J.J.Early explorations of CAT: canine amusement and training.
• Q2: How can the system gather information
CHI ’10 Extended Abstracts on Human Factors in Computing
unobtrusively about the animals’ preferences, actions
Systems, (2010), 2661–2669.
and context?
[12] SpeakDolphin. http://www.speakdolphin.com.
• Q3: How can the system use all these information in
order to adapt the playful activity to the context and the
animals’ preferences and wills?
• Q4: How can we objectively measure the success of the
system?

4. CONCLUSIONS AND FUTURE WORK

This thesis proposes the definition, design and development of
intelligent playful environments for animals, focusing on pet
animals. These systems aim to automatically create and adapt
playful digital experiences for animals, improving their wellbeing
and providing a better understanding of their interactions with

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 132 |
 Multi-Display Environments to Foster Emotional Intelligence in Hospitalized Children
F. Garcia-Sanjuan | J. Jaen | A. Catala

Multi-Display Environments to Foster Emotional

Intelligence in Hospitalized Children
Fernando Garcia-Sanjuan Javier Jaen Alejandro Catala
[email protected] [email protected] [email protected]
DSIC/ISSI, Universitat Politècnica de València
Camí de Vera, S/N, 46022 Valencia (Spain)

ABSTRACT better workspace awareness, since face-to-face settings allow

Long-term and frequent hospitalized children are under high loads users to know what the others are doing.
of emotional stress, which affects their well-being in addition to Despite research showing these advantages, it is rare to see
the illness they are suffering. This thesis proposes and will focus tabletops embedded in real settings. This is due to a number of
on an approach to use Multi-Display Environments (MDE) in disadvantages: their high cost; their limited workspace
pediatric hospitalization contexts to improve patients’ emotional dimensions, which can only accommodate a certain number of
intelligence so they can deal with the negative emotions produced participants; and the fact that their form factor complicates
by their situation. mobility. Therefore, this work will address the development of a
collaborative play environment based on multiple interactive
Categories and Subject Descriptors surfaces to support the learning of emotional intelligence in
H.5.2, H.5.3 [Information Interfaces and Presentation (e.g., pediatric hospitalization contexts aimed at reducing the negative
HCI)]: User Interfaces – Input devices and strategies, interaction emotional impact caused by long-term or frequent hospital stays.
styles. Group and Organization Interfaces – Computer-supported
cooperative work. 2. RELATED WORK
K.3.1 [Computers and Education]: Computer Users in 2.1 Emotional Intelligence and Tabletops
Education – Collaborative learning, Computer-assisted Several previous works have considered technology as a way of
instruction (CAI). acquiring emotional intelligence abilities. Morris et al. [6], for
example, involve several users collectively around big interactive
General Terms surfaces in an emotional identification task, through the tagging of
Design, Experimentation, Human Factors, Standardization different images according to the emotions they cause. During the
course of the activity, the researchers observe the users speaking
Keywords with one another about their feelings towards the images, and,
Multi-Display Environments (MDE), Hospitalization, Emotional before tagging an image, they self-reflect and ask themselves
Intelligence, Child-Computer Interaction, Socialization questions like: “Am I happy?”, “Am I sad?” According to the
authors, their explorations show people’s desire of having
1. INTRODUCTION technologies that allow the complex expression of emotions
According to hospital morbidity statistics from INE for 2013, the through a game experience. However, most of previous works in
number of pediatric hospital stays in 2012 in Spain amounted to this area target adults or adolescents, and the activities performed
1,813,009. Besides the fears associated to their illness, frequent aim mainly at identification of emotions, whereas the practical use
and long periods of hospitalization provoke on children high loads of those emotions to solve problems has not yet been fully
of emotional stress and insecurity for being away from their home explored.
and their comfort zone with parents and friends. To tackle these
problems, there are previous studies that demonstrate that play Others target children with autistic spectrum disorders (ASD) to
activities allow children to face the anxiety, conflicts, and fear increase socialization through technology. As an example, Gal et
caused by hospital experiences [4]. However, the application of al. [1] make use of an interactive tabletop to perform storytelling
new technologies in the context of hospital schools has been activities to foster collaboration among kids with ASD. They
limited mostly to scenarios of individual use in which emotional observe an increment of the number of approaches between the
stress is addressed in a simplistic way by using technologies to children, a higher load of positive affect as well as a higher
enable communication channels between children and their homes tendency to express emotions. These type of works, however, are
or external schools. However, we not only aim at deviating the not aimed at making these children reason about their emotions in
children’s attention from the clinical processes they are subject to, a way they can overcome an unfavorable personal situation and
but also at facilitating the emotions that help them accept their they rely on interactive tabletops, which may be complex to
new medical situation and adapt their mental state to face the implant in real hospital environments, as stated previously.
issues that could arise by means of collaborative game activities. 2.2 Multi-Display Environments (MDE)
Since emotional intelligence is mainly of social nature, traditional MDE have been object of much previous research. Traditionally,
desktop computers are not the best option to support its training their main purpose was enlarging the visualization space via the
because these devices are mostly designed for individual use. On physical union of several smaller screens (e.g. [8]), and the
the other hand, digital tabletops have been shown to be very interaction played a more secondary role, therefore leading to
suitable tools for use in collaborative environments (e.g., [3]). mono-user environments with difficulties to favor collaboration,
Their multi-touch capabilities allow simultaneous interaction, which is fundamental in emotional learning. Besides, the
which contributes to a more democratized participation [9] and a configurations tended to be static (i.e., the number of screens was

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 133 |
 Multi-Display Environments to Foster Emotional Intelligence in Hospitalized Children
F. Garcia-Sanjuan | J. Jaen | A. Catala
fixed a priori), and to not allow the mobility of the participants. interact with many devices at once. In order to achieve more
The posterior popularization of mobile devices, such as phones dynamic scenarios users will be able to modify the resulting
and PDAs, introduced a higher degree of mobility and multiuser surface by moving the displays through the environment during
(parallel) interaction since each user could interact with their own the course of the collaboration. And, in order to obtain richer
device, which they carried with them. Mandryk et al. [5] study the ways of interaction, our proposal will support interacting with
impressions of teenagers when using PDAs in a collaborative alternative media such as tangibles, which can result more
game. Because of the reduced dimensions of these devices, the intuitive for children as they can grab and manipulate actual
researchers encouraged them to enlarge the visualization space by physical objects. Moreover, it will also accept aerial interactions
joining the screens. The participants, however, reported low sense using gestures with fingers or dedicated objects with the purpose
of liberty of movements because they had to maintain the devices of avoiding interference with another user’s actions.
together during the course of the activity. Therefore, the mobility In sum, our goal is to design activities to foster emotional
was achieved in detriment of the visualization space. This intelligence in hospitalized children. Since this type of
problematic is due to the fact that the approaches commented so intelligence is mainly social, we propose to build collaborative
far rely basically on joining smaller screens to form a bigger one environments using several affordable surfaces such as tablets and
(and, therefore, rectangular). In this respect, some existing works smartphones. The dimensions identified previously will then be
have proposed alternatives. For instance, Pinch [7] is a technique explored beyond their technical aspects regarding how they
to couple smartphones and tablets via a manual gesture, and it impact collaboration among the children and how they foster
supports free-form topologies and dynamic modification of the emotional intelligence that might help them overcome the
device group to the extent of not needing the displays to be stressful situations they experience during their hospitalization.
physically joint once coupled. A limitation of this work, however,
is that the resulting topology is still flat and the only interaction 4. ACKNOWLEDGEMENTS
technique supported is touch. As this analysis suggests, previous This work is supported by Spanish MINECO (TIN2010-20488
research on MDE has mainly focused on technical aspects, and and TIN2014-60077-R), from Universitat Politècnica de València
their capabilities have not yet been proven in real settings but (UPV-FE-2014-24), and from GVA (APOSTD/2013/013 and
rather in experimental contexts in order to show the performance ACIF/2014/214).
of a given approach. This implicates that, in their design, the
ultimate activity to be performed by the users with these MDE has 5. REFERENCES
not been considered deeply enough. In our context, the software [1] Gal, E., Bauminger, N., Goren-Bar, D., et al. Enhancing
system and underlying interaction techniques will be considered Social Communication of Children with High-functioning
effective not only if the children are able to perform quickly the Autism Through a Co-located Interface. AI & Society 24, 1
manipulations and they understand how to make them, but also if (2009), 75–84.
the system supports effectively the construction of experiences [2] Hornecker, E., Marshall, P., Dalton, N.S., and Rogers, Y.
that develop emotional intelligence through games. Collaboration and Interference: Awareness with Mice or
Touch Input. CSCW '08, ACM (2008), 167–176.
3. RESEARCH OBJECTIVES
From previous works’ limitations, we have identified several [3] Kaminski, M., Pellino, T., and Wish, J. Play and Pets: The
technical dimensions that will be addressed in our work in order Physical and Emotional Impact of Child-Life and Pet
to successfully build collaborative MDE for children in Therapy on Hospitalized Children. Children’s Health Care
hospitalized environments. These are classified into dimensions 31, 4 (2002), 321–335.
regarding the construction of the environment and interaction [4] Mandryk, R.L., Inkpen, K.M., Bilezikjian, M., Klemmer,
requirements. S.R., and Landay, J.A. Supporting children’s collaboration
With respect to the former, our approach will allow several across handheld computers. CHI EA '01, ACM (2001), 255–
displays to form a common environment of any topology, not 256.
necessarily the traditional rectangular one, and possibly [5] Morris, M.E., Marshall, C.S., Calix, M., Al Haj, M.,
discontinuous to remove the need for the devices to be physically MacDougall, J.S., and Carmean, D.M. PIXEE: Pictures,
joint. This way, the users will be able to choose more freely their Interaction and Emotional Expression. CHI EA ’13, ACM
location. The number of devices will be unbounded, and their (2013), 2277–2286.
annexation to one another will be done in a transparent way in
order to avoid complex registration methods. Concretely, our [6] Ohta, T. and Tanaka, J. Pinch: an interface that relates
approach to address these requirements will rely on using the applications on multiple touch-screen by ‘pinching’ gesture.
built-in camera of tablets and smartphones to track their own ACE '12, Springer-Verlag (2012), 320–335.
position with respect to a wallpaper situated on the ceiling using [7] Ohta, T.Dynamically reconfigurable multi-display
computer vision algorithms, allowing the system to build a logical environment for CG contents. ACE '08, ACM (2008), 416.
map of all the tablets in the group. Since the only requirement is [8] Rick, J., Marshall, P., and Yuill, N.Beyond One-size-fits-all:
having the wallpaper/marker partially in sight, this approach can How Interactive Tabletops Support Collaborative Learning.
also allow some 3D topologies by simply placing the displays in IDC '11, ACM (2011), 109–117.
different planes.
Regarding the interactions that our MDE will allow, the most
basic one, in order to benefit from the advantages of tabletops in
terms of collaboration, is to consider all displays as part of a
whole, public surface, and therefore to permit many users interact
with the same display at the same time, and also a given user to

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 134 |
 KINDERTIVITY: Using Interactive Surfaces to Foster Creativity in Pre-kindergarten Children
V. Nacher | J. Jaen

KINDERTIVITY: Using Interactive Surfaces to Foster

Creativity in Pre-kindergarten Children
Vicente Nacher, Javier Jaen
ISSI Group, Dpto. Sistemas Informáticos y Computación, Universitat Politècnica de València.
{vnacher, fjaen}@dsic.upv.es

ABSTRACT individual development [6]. Moreover, the technological support

Taking into account the existent educative and pedagogical to be used when developing creative characteristics in pre-
techniques, which have proved its effectiveness to foster the kindergarten children is critical since the choice of the underlying
innovation and creativity, this thesis poses to develop, experiment supporting technology has a great impact on the nature of the
and evaluate a new technological framework based on interactive pedagogical activities to be performed in a creative learning
surfaces to be applied in the context of preschool education. The context.
goal is to facilitate the three factors required for creative learning: The initial hypothesis for this thesis is that it is possible to trigger
knowledge, creative thinking and motivation but taking into in a more effective way creative collective constructivist
account the cognitive and interaction limitations of these very processes in pre-kindergarten children through the participation of
young users. multiple users in physical spaces of shared games based on
interactive surfaces. Therefore, we pretend to study the use of
Categories and Subject Descriptors these surfaces in the early phase of the cognitive development and
H.5.2. [Information interfaces and presentation]: User its suitability for the creation of educational tools.
Interfaces - Evaluation/methodology, Interaction Styles
2. RELATED WORK
General Terms Multi-touch technology has evolved quickly in recent years, from
Design, Experimentation, Human Factors, Standardization the adoption of graphical user interfaces to its wide acceptance
nowadays [1]. This technology offers new sophisticated input and
Keywords processing mechanisms that enable users to interact in a more
Pre-kindergarten, Touch interaction, Education. natural and intuitive way [19]. These characteristics have
1. INTRODUCTION triggered a new approach to developing applications for even very
young children. Supporting this evidence, the Horizon report [10]
The European Union recognizes creativity as a key factor for the
identifies mobile devices (smartphones and tablets) as one of the
economic development, hence, “Increase the creativity and the
two emerging technologies suitable for children under two years
innovation, including the entrepreneurship spirit, in all levels of
old and Rideout pointed out that children between zero and eight
the education and training” is the 4th goal of the strategic
years old are frequent users of digital media and they meet with
framework for the European cooperation in the field of education
touch technology often before they can even speak [4].
and training and it is one of the constituent elements of the new
R&D strategy in the 2020 Horizon [3]. This is a necessary This inherent ability of touch systems to engage children’s
strategy because, as Cropley points out [5], the traditional attention is being widely exploited to promote learning activities
education systems tend to frequently assume the existence of one from pre-kindergarteners to adolescents [12, 18, 11]. Moreover,
valid answer (or, at least, a preferred one) for any type of some studies have demonstrated that the technology can also be
problem. This fact prevents the generation of new ideas and used to promote collaboration between peers [16, 17, 7] and to
innovation processes. According to this, a creative student, or with foster creativity [8, 2].
a different cultural basis, can be considered as a distortion source However, the increasing interest in multi-touch technology has
or distraction in the knowledge acquisition process imparted by not as yet given rise to studies on the design of multi-touch
the teacher. However, while the individual ability to obey exactly systems for the youngest age range (2-3 years old) [9].
the given orders with discipline were in line with necessities of an
industrial society and massive production systems, the ability to Taking into account the previous works, we believe our proposal
be part of collaborative processes and direct the divergent is a step forward because of the following reasons: (1) the
thinking as the motor of creativity and innovation are essentials suitability of the interactive surfaces to support social learning
nowadays in the information society. since several subjects share the same physical space and, as it
happens in traditional technology-free games, the communication
Fostering creativity must be addressed from a very early age, even during the creation process, experimentation and reflection is
in the preschool phase, since the main cognitive processes direct and no mediated by a computer. (2) The collaborative
associated to creativity have their sources in this phase of the nature of the technologic infrastructure, in which users can carry
Permission to make digital or hard copies of all or part of this work for out different tasks in parallel and on the same table; and (3) the
personal or classroom use is granted without fee provided that copies are creative nature of the infrastructure in which users select the game
not made or distributed for profit or commercial advantage and that elements and the reactive behavior that its offer. This allows
copies bear this notice and the full citation on the first page. To copy
educators to have a direct feedback about the evolution of
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee. children’s creative mental models. These models are internalized
Interaccion’15, September 7–9, 2015, Vilanova i la Geltrú, Spain. in a collective way since the activity itself is based on reflection,
Copyright 2010 ACM 1-58113-000-0/00/0010 …$15.00. creation and experimentation processes. The goal is that also

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 135 |
 KINDERTIVITY: Using Interactive Surfaces to Foster Creativity in Pre-kindergarten Children
V. Nacher | J. Jaen
educators have a tool on which they can measure in an effective [2] Catala, A., Jaen, J., van Dijk, B., and Jordà, S. Exploring
way the level of knowledge development, in depth and breadth, of tabletops as an effective tool to foster creativity traits. In
the divergent/convergent thinking processes and the motivation of Proc. of TEI’12, pp. 143–150.
users in the activity. [3] Comisión Europea. Conclusiones del Consejo de 12 de mayo
3. CONTRIBUTION de 2009 sobre un marco estratégico para la cooperación
In order to assess the actual skills of pre-kindergarten children europea en el ámbito de la educación y la formación («ET
with interactive surfaces, we have performed an evaluation of a 2020»). 2009.
set of basic multi-touch gestures [14]. The results showed that [4] Common Sense Media. Zero to Eight: Childrens Media Use
although only the tap and drag gestures are used in commercial in America 2013. 2013.
applications targeted to pre-kindergarten children, there are [5] Cropley, A.J. Creativity in Education and Learning: A Guide
additional multi-touch gestures that can be performed by them for Teachers and Educators. Kogan Page, (2001).
(one finger rotation, scale up and scale down). In addition, this
study provides a set of design guidelines to define and improve [6] Damon, W., Lerner, R.M., Kuhn, D., and Siegler, R.S., eds.
the interactions of these particular users. The application of Handbook of Child Psychology, Volume 2, Cognition,
several of these design guidelines [15] showed that more Perception, and Language. Wiley, 2006.
problematic gestures, such as double tap and long press, can be [7] Fleck, R., Rogers, Y., Yuill, N., et al. Actions speak loudly
suitable for pre-kindergarten children too. with words. Proc. of ITS’09, pp. 189-196.
On the other hand, we have carried out another study addressing [8] Helmes, J., Cao, X., Lindley, S.E., and Sellen, A. Developing
communicability of multi-touch gestures [13] and the results the story. Proc. of ITS’09, pp. 49–52.
proved that the use of animated languages to communicate [9] Hourcade, J.P. Interaction Design and Children. Foundations
gestures to pre-kindergarten children is possible. This opens a new and Trends® in Human-Computer Interaction 1, 4 (2007),
opportunity to new studies with training sessions in order to 277–392.
evaluate the acquisition ability of this type of languages with these
young users. [10] Johnson, L., Adams, S., and Cummins, M. The NMC Horizon
Report: 2012 K-12. The New Media Consortium, Austin,
These preliminary results have been published in international Texas, 2012.
forums and allow tackling with a lot of motivation the important
milestones that this thesis poses. To sum up, the main milestones [11] Khandelwal, M. and Mazalek, A. Teaching table: a tangible
mentor for pre-k math education. Proc. of TEI’07, 191–194.
of the thesis are the following:
[12] Mansor, E.I., De Angeli, A., and De Bruijn, O. Little fingers
-Contextualization: the evaluation of the types of interaction that on the tabletop: A usability evaluation in the kindergarten.
can be performed by pre-kindergarten children in multi-touch Proc. of TABLETOP'08, 93–96.
surfaces.
[13] Nacher, V., Jaen, J., & Catala, A. (2014). Exploring Visual
-Assistance: the definition of assistive techniques of interaction Cues for Intuitive Communicability of Touch Gestures to
that allow the increase of effectiveness of pre-kindergarten actions Pre-kindergarten Children. Proc. of ITS'14, 159–162.
with interactive surfaces.
[14] Nacher, V., Jaen, J., Navarro, E., Catala, A., and González, P.
-Communicability: the definition of effective mechanisms to Multi-touch gestures for pre-kindergarten children.
communicate which actions are expected from the user through International Journal of Human-Computer Studies 73
animated languages that allow children to be autonomous when (2015), 37-51.
using the multi-touch technology without the continuous [15] Nacher, V., Jaen, J., Catala, A., Navarro, E., and Gonzalez, P.
supervision of adults. Improving Pre-Kindergarten Touch Performance. Proc. of
-Adaptability: the definition of strategies that allow the ITS ’14, 163–166..
adaptation of the Assistance and Communicability mechanisms to [16] Rick, J., Francois, P., Fields, B., Fleck, R., Yuill, N., and
the specific skills of each user. Carr, A. Lo-fi prototyping to design interactive-tabletop
-Creativity: the construction and validation of environments that applications for children. Proc. of IDC’10, pp. 138–146.
foster creativity and allow collaboration between users (pre- [17] Rick, J. and Rogers, Y. From DigiQuilt to DigiTile: Adapting
kindergarten children) to obtain in the future more adequate educational technology to a multi-touch table. Proc. of
educational tools for these users. TABLETOP'08, pp. 73–80.
[18] Sluis, R.J.W., Weevers, I., van Schijndel, C.H.G.J., Kolos-
4. ACKNOWLEDGMENTS Mazuryk, L., Fitrianie, S., and Martens, J.B.O.S. Read-It:
Work supported by the MINECO (grants TIN2010-20488 and
Five-to-seven-year-old children learn to read in a tabletop
TIN2014-60077-R) and from GVA (ACIF/2015/075).
environment. Proc. of IDC’04, pp. 73–80.
5. REFERENCES [19] Smith, S.P., Burd, E., and Rick, J. Developing, evaluating
[1] Buxton, B. Multi-touch systems that I have known and loved. and deploying multi-touch systems. International Journal of
2013. http://billbuxton.com/multitouchOverview.html. Human-Computer Studies 70, 10 (2012), 653–656.

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 136 |
 Facial Emotion Analysis in Down’s syndrome children in classroom
P. Torres-Carrión | C. González-González

Facial Emotion Analysis in Down’s syndrome children in

classroom
Pablo Torres-Carrión Carina González-González
Universidad Técnica Particular de Loja Universidad de La Laguna
Ecuador Spain
[email protected] [email protected]
ABSTRACT have expanded considerably in the last three decades [14]
Sentiment analysis has allowed a degree of machines "intelligent" improving your quality of life standards.
interaction, although its development is still at an early stage. This People with DS usually have well-developed emotional
document presents the progress of the research project in this line, intelligence, maintain good social interaction skills. They are
with a particular focus on children with Down Syndrome (DS). communicative and respond well to the demands of the
Their singularities are considered through personalized learning environment [2]. The emotions of the people seldom are
resources, and interaction with Kinect HCI, and the Tango:H expressed in words, so it is necessary to interpret them from non-
platform. In this first moment we made an initial interaction, verbal channels such as: the gestures, tone of voice and facial
resulting in video images of the interaction, and a subjective expression [7]. People with SD shows emotional sensitivity that
assessment of emotions through one extension of EMODIANA. allows them to capture emotions that others go unnoticed [2].
The next step will be the comparison of the patterns of facial Each person's emotional state can be expressed in different ways;
recognition applied in people without SD, to identify whether the facial expression is one of the most common ways to show
there are significant differences. others our feeling with respect to any act or thing [4]. In the case
Categories and Subject Descriptors of persons with SD, we have to consider that certain facial
characteristics distinguish them from the rest of population [3,9],
K.4.2. [Computers and Society]: Computing Milieux – Social
therefore, studies and identification of emotions patterns based on
Issues: Assistive technologies for persons with disabilities.
face, may not extend and apply to this population in general.
General Terms 2. AVANCE DE INVESTIGACION
Human Factors, Theory. Research in the field of the SEN, and specifically in people with
DS have increased over the last three decades [14]. However they
Keywords have not been significant in the area of human-computer
Down Syndrome, Emotion, Affective Computing, AI, Digital interaction (HCI) with affective computing (AC), and the
Learning. consideration of emotions during interaction.
1. INTRODUCTION 2.1 Metodology
The human person as a social entity, learn from inside the The methodology that guides this research, is a continuous work
environment that develops and lives; it is necessary therefore to qualitatively and quantitatively (see Table1). In the first time we
develop the skill of reading behaviors and emotions to infer the plan doing the emotional assessment in environments of
possible intentions [15]. In this context, the emotional intelligence interaction in the classroom. This work was done with a sample of
as part of the multiple intelligences of Gardner [6], joins two types six children (3 male and 3 female) of the Down Tenerife
of intelligences: intrapersonal and interpersonal; that refer to how Association, located in the autonomous community of Canary,
we know ourselves, the way in which we managed to control and Spain.
regulate our feelings; the ability to understand others, what you
feel and the skills to relate to others[7], respectively. The pre-test phase included meetings with teachers to identify the
population, for the experimental group and the control sample,
With the development of these intelligences, teachers and keeping the equity in terms of motor and cognitive profiles. We
psychologists comprehenden (understand from inside) to each organized an initial intervention, composed of three working
other and their environment (social field). Explicitly in the sessions. The experimental group used the platform Tango:H and
learning process, the student and teacher through educational MS Kinect; the control group used conventional resources.
activities can know, appreciate and share, generating one compre-
h-ensión integral (comprehensive understanding), which is The first data were obtained from a subjective assessment by
reflected in a significant learning [15]. teachers, about the emotions displayed by the children during the
interaction; it was carried out according to the methodology of the
The above applies to the entire student population, from initial EMODIANA [8] and their assessment scales. This instrument was
training to top, without making a cognitive or mental age adapted in a proprietary format, which allowed to measure
discrimination. Against this background, in this paper we focus on continuously the 10 basic emotions, and their possible causes:
the people with DS, that within the population with Special subject, activity and external.
Educational Needs (SEN), represent one of the largest and most
vulnerable around the world [10]. The process of educational and
social adaptation of this population requires an individual
treatment, due to psychological, cognitive and kinesthetic abilities
[1]. Scientific research about their characteristics and possibilities
of performance in different areas of life: social, work and school,

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 137 |
 Facial Emotion Analysis in Down’s syndrome children in classroom
P. Torres-Carrión | C. González-González
Table 1. Methodology for the emotional assessment[13] Additionally, there is the need to relate these algorithms in
Fase Tool Actors classrooms and in our case with children with SD.
Pre-test - Competency Test - Testers 4. REFERENCES
- DS’s people [1] Amadó, A., Benejam, B., Mezuca, J., Serrat, E., and Vallès-
Majoral, E.Socio-cognitive abilities in children with Down’s
Test - EMODIANA - Testers
syndrome: results of a preliminary study. International
- Rehabilitation platform (cognitive - Teachers Medical Review on Down Syndrome 16, 3 (2012), 34–39.
and physical) TANGO:H. - Physiotherapist [2] Angulo A. Luna, M., M.G. and Prieto, I.Manual de Atención
- Record of videos - DS’s children al Alumnado con Necesidades Específicas de Apoyo
Post-test - Competency Test - Testers Educativo Derivadas de Sindrome Down. Junta de
Andalucía, Sevilla, 2010.
- EMODIANA - DS’s people
- Observation Method (videos)
[3] Carvajal, F., Fernández-Alcaraz, C., Rueda, M., and Sarrión,
L.Processing of facial expressions of emotions by adults with
- HER Tool (videos) Down syndrome and moderate intellectual disability.
For a further analysis all sessions were recorded on video. These Research in developmental disabilities 33, 3 (2012), 783–
videos will undergo evaluation patterns, to contrast with results 790.
obtained by these tools in environments of interaction with non- [4] Cerezo, E. and Hupont, I.Emotional Facial Expression
SD. This contrast, in addition to the subjective assessment Classification for Multimodal User Interfaces. In Articulated
performed with the EMODIANA, will allow establish scientific Motion and Deformable Objects. Springer Berlin Heidelberg,
findings about the possible differences between people with and Berlin, Heidelberg, 2006, 405–413.
without SD, in terms of patterns and AI algorithms on recognition
of emotions previously validated. [5] Dibeklioğlu, H., Salah, A.A., and Gevers, T.Recognition of
Genuine Smiles. IEEE Transactions on Multimedia 17, 3
The objective pursued is to customize in real time, both teaching (2015), 279–294.
resources such as teaching strategies, based on the cognitive,
motor and emotional profile of the student with SD. We believe [6] Gardner, H.Estructuras de la mente: La teoría de las
that learning will be more significant. inteligencias múltiples. Fondo de Cultura Económica,
México, 2001.
3. BRIEF REVIEW OF LITERATURE [7] Goleman, D.Inteligencia emocional. Kairo´s, Barcelona,
3.1 HCI and Affective Computing 2011.
Every human relationship contains many and varied doses of [8] González-González, C.S., Cairós-González, M., and
affection, knowledge and interaction [15]. The literature about Navarro-Adelantado, V.EMODIANA: Un instrumento para
"affective" includes research and study areas, where relates: la evaluación subjetiva de emociones en niños y niñas. Actas
psychology, science of consciousness, neuroscience, engineering, del XIV Congreso Internacional de Interacción
computer science, sociology, philosophy, and medicine; from this PersonaOrdenador, (2013).
epistemological context seeks to understand the feelings, such as: [9] Hippolyte, L., Barisnikov, K., Van der Linden, M., and
emotion, motivation, attention, memory, among others [11]. Detraux, J.-J.From facial emotional recognition abilities to
3.2 Emotions through the face and gestures emotional attribution: A study in Down syndrome. Research
Research about emotion recognition are held mainly in facial in developmental disabilities 30, 5 (2009), 1007–1022.
expressions, intonation of voice and gestural traits [11]. In the [10] Hodapp, R.M. and Freeman, S.F.N.Advances in educational
corresponding face, muscle movements associated with strategies for children with Down syndrome. CURRENT
sentimental or emotional situation have been related to patterns OPINION IN PSYCHIATRY 16, 5 (2003), 511–516.
that relate basic emotions like raised them by Ekman: joy, [11] Picard, R.W., Papert, S., Bender, W., et al.Affective learning
surprise, disgust, anger, sadness and fear, along with a Neutral - a manifesto. BT Technology Journal 22, 4 (2004), 253–268.
option [4]. These emotions can be extended up to levels of
valuation of complex emotions, product of its combination and [12] Tapia, S.A.A., Gomez, A.H.F., Corbacho, J.B., et al.A
semantic reading of the sequence in which certain gestures are contribution to the method of automatic identification of
shown [12]. human emotions by using semantic structures. 2014
International Conference on Interactive Collaborative
The recognition of genuine emotions [5] against those that occur Learning (ICL), IEEE (2014), 60–70.
in environments where the user poses specifically for an
evaluation of the emotion, varies significantly. Therefore, it is [13] Torres-Carrion, P., González-González, C.S., and Mora
necessary that these studies are carried out in the natural Carreño, A.Methodology of emotional evaluation in
environment of interaction, avoiding as far as possible the education and rehabilitation activities for people with Down
variation of expressions by external agents [11]. syndrome. XV International Conference on Human
Computer Interaction, (2014), 12–15.
In the TED-Woman 2015 event, the PhD. Rana el Kaliouby of the
Affectiva company, presented the Affdex SDK, that allows the [14] Troncoso, M.V. and Flórez, J.Comprensión en la lectura de
recognition of emotions in real time, supported in a study of las personas con síndrome de Down. Revista Síndrome de
approximately two million videos, coming from more than 75 Down: Revista española de investigación e información
countries. This is one of the reference works in the area. sobre el Síndrome de Down, 2011, 50–59.

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 138 |
 Facial Emotion Analysis in Down’s syndrome children in classroom
P. Torres-Carrión | C. González-González
De Zubiría, M.Inteligencia Emocional. Fundación Alberto
Meraní para el Desarrollo de la Inteligencia, Bogotá, 1999.

XVI Congreso Internacional de Interacción Persona-Ordenador | Doctoral Colloquium

| 139 |
CHILD COMPUTER INTERACTION
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado

Métodos y técnicas para la evaluación de la experiencia

emocional de niños y niñas con videojuegos activos
Carina S. González-González Vicente Navarro-Adelantado
Departamento de Ingeniería Departamento de Didácticas
Informática y de Sistemas Específicas
Universidad de La Laguna Universidad de La Laguna
[email protected] [email protected]

ABSTRACT La definición, el modelado y la representación de las emociones

En este trabajo se presenta una propuesta metodológica para la sigue siendo objeto de estudio en el campo de la psicología y de la
evaluación de la experiencia emocional de niños y niñas con computación afectiva [6]. Existen dos enfoques
videojuegos activos. Esta propuesta pretende dar respuesta a fundamentalmente [7]: el enfoque categórico o discreto y el
algunos de los problemas más frecuentes encontrados en la enfoque dimensional. El enfoque categórico se basa en la
evaluación de productos interactivos con menores. Para ello, se ha clasificación de emociones en categorías [8], por lo que supone
realizado un análisis de los principales inconvenientes de las una representación discreta de las emociones incapaz de reflejar el
técnicas tradicionales de evaluación de la usabilidad, calidad de amplio rango de emociones complejas que un ser humano puede
uso y de la experiencia de usuario y emocional en niños y niñas, expresar. El segundo enfoque es dimensional [9], y establece que
así como de las relaciones entre las emociones, los videojuegos y las emociones conforman un espacio continuo multidimensional
los juegos activos. En base a este análisis, se ha elaborado una en el que unas y otras no son independientes sino que se
propuesta de evaluación que contiene: un conjunto de emociones relacionan de forma sistemática, siendo el modelo tridimensional
más representativas para los videojuegos activos y diferentes (valencia, activación y control) [10] uno de los más extendidos.
instrumentos y técnicas creados y/o adaptados para el caso de los Existen distintos instrumentos que realizan este tipo de
juegos motores con mediciones de tipo subjetiva (verbal, no evaluaciones, entre los que encontramos SAM [11] o Premo©
verbal), objetiva (biométrica) y de interacción social (Product Emotion Measurement Instrument) [12]. Este último se
(comportamiento), organizados en diferentes momentos de una basa a su vez en SAM, pero está especialmente diseñado para
sesión de evaluación. Esta propuesta permite conocer si las medir la experiencia emocional de los usuarios con productos.
respuestas emocionales son debidas a razones internas a la Aunque este método ha sido utilizado con éxito en niños, la
persona, a la actividad realizada o externas (contexto). cantidad de emociones que tiene (14) es potencialmente
problemático para su uso como un método de auto-reporte durante
Categories and Descriptors una actividad [13]. Esto se debe a que la carga cognitiva que
H.5.2 [User Interfaces]: Evaluation/Methodology produce es significativa para los niños y niñas, ya que deben
identificar, distinguir y seleccionar sus respuestas emocionales
Terms generals entre 14 emociones. Igualmente, existen dificultades de
Measurement, Human Factors asociación entre el lenguaje utilizado por los niños y su asociación
con la representación gráfica de la emoción [13]. Por este motivo
Palabras clave nuestra metodología utiliza un instrumento denominado
EMODIANA [14], específicamente diseñado y validado para
Evaluación de emociones en niños y niñas, metodología de tener en cuenta las dificultades cognitivas de los niños y niñas, y
evaluación emocional, evaluación de la experiencia de usuario que se basa en las fortalezas de los métodos anteriores. Además,
(UX) en niños y niñas, videojuegos activos, juegos motores. como videojuego activo utilizaremos una plataforma de juegos de
rehabilitación cognitiva y física basada en interacción gestual
1. INTRODUCCIÓN creada por el equipo de investigación, llamada TANGO:H
La evaluación emocional intenta recopilar y medir información
(http://tangoh.iter.es).
sobre aspectos cualitativos y cuantitativos de la experiencia de un
usuario [1]. El instrumento de medición dependerá en gran Para poder crear una metodología adecuada al caso de
medida de lo se busca obtener, pudiendo distinguir entre videojuegos activos y niños, primero se deben identificar los
instrumentos no verbales (objetivos) y verbales (subjetivos). problemas más frecuentes encontrados en la evaluación de la
Actualmente existen distintas técnicas de medición emocional, experiencia del usuario (UX) con niños y cuáles son las técnicas
tales como técnicas que permiten analizar las expresiones faciales, más idóneas para la evaluación con menores. Luego se deben
técnicas que miden reacciones fisiológicas (por ejemplo, latido del identificar las emociones relacionadas con los juegos y
corazón, sudor, dilatación de la pupila o las ondas cerebrales) o videojuegos, específicamente activos, para ver las técnicas e
las técnicas de medición subjetiva de sentimientos a través de instrumentos que permitan medir de forma fiable una intervención
cuestionarios, entrevistas y auto-informes [2-5]. Nuestra lúdica-educativa utilizando juegos activos con niños. Por ello, a
metodología de evaluación emocional utiliza técnicas objetivas y continuación, se describen los problemas más frecuentes de la
también subjetivas. evaluación emocional en niños y las técnicas de UX más
recomendadas. Posteriormente, analizaremos el conjunto de
XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction
| 141 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
emociones relacionadas al juego motor y videojuego activo. Método Ventajas Inconvenientes
Luego, presentaremos la metodología creada específicamente para
Observación -La interacción con -La estructura de la
este caso y las conclusiones obtenidas.
pasiva el sistema es más observación puede
2. TRABAJOS RELACIONADOS (estructurada) natural ya que el influir en los resultados
niño/a no se obtenidos.
2.1 Evaluación de la experiencia interactiva y encuentra
emocional en niños y niñas Observación presionado por la -Con alguna frecuencia
pasiva (no evaluación y los aparecen cuestiones
La diversión es un atributo de la experiencia de usuario que es
estructurada) evaluadores. fuera del ámbito de la
importante medir, ya que es una de las principales motivaciones
guía de observación que
que tienen los niños/as para interactuar con la tecnología [15] y -Permite observar la pueden pasar
además, es uno de los factores más relevantes asociados con los experiencia del desapercibidas
juegos. Por tanto, podemos decir que la diversión es una variable niño/a en su entorno
que se requiere medir en un estudio de la experiencia del usuario (familiar). -Los datos obtenidos
con los niños/as. Existen varios métodos de evaluación de la suelen ser demasiado
-Adecuado para difíciles de analizar.
experiencia del usuario con niños/as, entre los que podemos
descubrir problemas
encontrar: tarjetas gráficas para la identificación de problemas
inesperados en la
(Problem Identification Picture Cards), el kit de herramientas de
interacción como
diversión (Fun Toolkit) y la escala (Laddering) [16]. Muchos de
son los patrones de
estos nuevos métodos de evaluación de la experiencia del usuario
uso.
se basan en el uso de instrumentos como encuestas u otras
técnicas relacionadas. El problema es que existen grandes -Resultados fiables.
diferencias en las habilidades cognitivas y de desarrollo entre los
niños de la misma edad [17], lo cual en las técnicas subjetivas, se
pone en juicio la validez y fiabilidad de las respuestas de los niños Pensamiento -Los niños/as - Puede ser exigente
[18]. Al evaluar a niños/as encontramos problemas tales como en voz alta pueden explicar por cognitivamente e
complaciencia (satisficing), sugestionabilidad (suggestibility) e qué encuentran el incómodo para los
incomprensión (misunderstanding) [19]. Por ello, es necesario producto fácil, niños/as.
Método divertido, aburrido,
trabajar en la línea de maximizar la fiabilidad de las respuestas de
conductor difícil, etc.
los niños/as con el fin de asegurar la validez e integridad de los
- Las respuestas suelen
resultados. -Se puede adecuar la ir ligadas a cuestiones
sesión de evaluación específicas planteadas y
La investigación ha evaluado el Fun Toolkit con distintos métodos
para hacerla cómoda no sobre el proceso
de evaluación de juegos [16]. Los resultados de este estudio
y agradable, interactivo real.
mostraron que los dos métodos produjeron resultados muy
adecuada al niño/a.
similares y fueron comparables para identificar las preferencias en
el juego. También se han evaluado técnicas de evaluación de -El evaluador puede
usabilidad usadas en adultos con los niños, tales como las técnicas influenciar al niño/a -Normalmente se
de pensar en voz alta, entrevistas y cuestionarios [20]. Se ha y guiar el proceso realizan en laboratorio,
demostrado que los niños pueden identificar y reportar los hacia un fuera del contexto
problemas de usabilidad. Por ejemplo, los niños fueron capaces de determinado familiar al niño/a.
detectar problemas de usabilidad que ayudarían en el diseño de resultado o proceso.
juegos para niños de 4-9 años [21]. Sin embargo, en la realización Mago de Oz -Permite evaluar un -Se requiere la
de una investigación de usabilidad con los niños hay una serie de producto no participación de un
retos que deben tenerse en cuenta. Así, en un estudio [22] de 70 terminado evaluador que conozca
niños sólo 28 de ellos hicieron observaciones verbales durante la todo el sistema que
prueba de usuario. Esto puede atribuirse a su personalidad, ya que -Permite descubrir
aspectos de la cumpla dicho rol y
un estudio mostró que las características de personalidad influyen brinde las respuestas
en el número de problemas identificados [23], por consiguiente, se interacción no
contemplados interactivas simuladas.
hace necesario investigar para comprender las limitaciones y
aplicaciones en niños/as con relación a diferentes métodos. En anteriormente
Padilla Zea et al (2012) [24], podemos encontrar una clasificación -Para los niños/as
de diferentes métodos de evaluación de productos interactivos puede resultar
procedentes de la evaluación de usabilidad y calidad de uso, y los divertido y fácil
pros y contras de su utilización en niños y niñas. Asimismo, los realizar un juego de
mismos autores proponen una metodología para la evaluación de rol, por lo que se
la jugabilidad en videojuegos educativos en niños y niñas de 3 a 7 aumentan las
años. emociones positivas
hacia el producto
Tabla 1. Métodos de evaluación de la experiencia interactiva
con productos para con niños (Adaptado de Padilla-Zea et al.
[24]). Cuestionarios - Se puede preguntar - Poca fiabilidad en la
Método Ventajas Inconvenientes a los niños a la vez y respuesta de los

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 142 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
Método Ventajas Inconvenientes con videojuegos activos incluyendo elementos de interacción
social y colaborativa. Por ello, en este trabajo se propone una
obtener gran niños/as.
metodología específicamente diseñada para la evaluación de este
cantidad de datos.
tipo de productos con niños y niñas.
Entrevistas -Permite conocer en -Dificultad de los
profundidad la Las respuestas emocionales pueden darse a través de diversos
niños/as en comprender canales. El canal tradicional de detección emocional han sido las
experiencia las preguntas.
individual de cada imágenes faciales, pero en los últimos años se han ido
participante. -Análisis de datos incorporado otros canales como el sonoro (voz) y, muy
laborioso, con mucha especialmente, los fisiológicos [26]. Asimismo, existen diversos
-Se pueden obtener información de tipo tipos de instrumentos para medir las respuestas emocionales. Por
datos cualitativos cualitativa. ejemplo para las respuestas de tipo verbal, algunos de los
más ricos que los instrumentos más comunes son la escala Likert, escala de
obtenidos en otros diferenciador semántico, el perfil emocional estandarizado (SEP,
métodos. el perfil de reacción (Reaction Profile) o la escala sentimiento
Grupos - Permite debatir hacia anuncios (Feelings Toward Ad Scale). Para las respuestas de
focales sobre diversos temas tipo no verbal, los instrumentos más comunes son PrEmo,
en profundidad EmoCards, SAM,, LemTool o GEW. Y para medir las respuestas
fisiológicas, para las expresiones faciales algunos instrumentos
más comunes son: Affdex Facial Coding, FaceAPI, Facesense,
- El evaluador guía Facereader, FaceSDK; para señales cerebrales el Emotiv-EPOC y
la discusión hacia para otras variables biométricas los Affective wearables [27].
los temas que
interesan evaluar, En Méndez et al (2015) [27] se presenta un marco de referencia
permitiendo ahondar como complemento a la evaluación de sistemas interactivos con
en variables tres componentes principales de evaluación: comportamiento,
cualitativas de la fisiológico y subjetivo. El componente de comportamiento se
experiencia del centra en evaluar el comportamiento del usuario, considerando
usuario específicamente las características relacionadas con la expresión
(satisfacción, facial, dirección de la mirada, postura, movimientos de la cabeza
impacto emocional, y gestos. El componente fisiológico se centra en los aspectos
etc.) fisiológicos de los usuarios, considerando específicamente la
actividad cerebral del usuario durante la evaluación. Por último, el
Métodos -Permite la -Las heurísticas se componente subjetivo se centra en recoger la propia opinión del
basados en evaluación cuando centran tradicionalmente usuario, al respecto de las emociones que el usuario considera que
heurísticas la participación de en aspectos pragmáticos evocó mientras realizaba las diferentes tareas durante la
los niños/as es (usabilidad) y se puede evaluación del sistema interactivo. Este enfoque metodológico no
difícil. perder información contempla el componente social y cooperativo de los usuarios,
Métodos hedónica (diversión, cuestión que sí es abordada en nuestra propuesta.
basados en - Ayuda a obtener
información con un entretenimiento, etc.)
recorridos 2.2 Emociones, videojuegos y juegos activos
número reducido de - Pueden mostrar
expertos (5) problemas que no Aunque existe una gran variedad de interpretaciones y
existen y se puede definiciones del concepto de emoción [28], podemos decir que se
-Son válidos en trata de un estado complejo del organismo caracterizado por una
evaluaciones perder los problemas
que realmente si están. excitación o perturbación que predispone a una respuesta
tempranas de organizada [29], de acuerdo con la evaluación subjetiva que
prototipos - Los expertos pueden realiza cada persona del significado del evento que la ha originado
desconocer lo que [30].
realmente siente al
interaccionar un niño/a Los videojuegos son poderosos generadores de emociones. Varios
con el producto autores, se han centrado en analizar qué emociones pueden
interactivo. generarse, y de cómo surgen [31]. Es posible distinguir dos
niveles en la vivencia de emociones [32]:
-Dificultad de encontrar
expertos en los diversos • El jugador como observador-participante: Se refiere a
tipos de videojuegos las emociones producidas por la interacción con las
educativos/activos. imágenes y el sonido del videojuego, por aquellos
La observación pasiva es uno de los métodos clave para nuestra elementos que el participante no puede cambiar, y los
metodología no invasiva. mecanismos que las inducen son similares a los
En nuestro caso, el rango de edad para el cual hemos diseñado la utilizados por las películas de cine. Por ejemplo, un
metodología es de 8 a 12 años. En estas edades, aumenta la jugador podría asombrarse al percibir la belleza de un
discriminación emocional, la interacción social y el juego grupal escenario de juego.
[25]. Sin embargo, no existen metodologías o un conjunto de • El jugador como actor participante: Se refieren a las
métodos y técnicas que permitan evaluar la experiencia de usuario emociones generadas por las propias acciones del

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 143 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
jugador, o de las interacciones del jugador con otros. Rechazo No admitir o aceptar a otra persona que participa
Por ejemplo, un jugador podría sentir alegría como en el mismo juego.
resultado de haber conseguido superar una fase de un
videojuego. En los videojuegos, a diferencia de otros Vergüenza Timidez que se siente ante determinadas
formatos multimedia, y particularmente los videojuegos situaciones de juego que impide o inhibe la
multijugador, se posibilitan estas emociones. realización de una acción motriz. Fracasar en vivir
de acuerdo con el yo ideal.
Los juegos motores generan una vivencia intensa de reacciones
emocionales asociadas a estados activos e interactivos [33]. Sorpresa Reacción a alguna situación de juego imprevista.
Según Lavega et al. (2015) [28] “las emociones se relacionan al Esperanza Confiar en qué ocurrirá o se logrará el objetivo de
estado activo, asociado a diferentes tendencias de acciones, a juego que se desea alcanzar.
códigos de comunicación no verbales o disposiciones corporales,
y a la intervención procedimental; es decir, vivencial e Compasión Sentirse afectado por el sufrimiento de otro
interactiva”. También, hay que distinguir entre el juego motor real participante en el juego y sentir el deseo de querer
y el juego interactivo simulado, así como tener en cuenta los ayudarle.
fundamentos teóricos procedentes de la praxiología motriz de
Parlebas [34,35, 36]. A partir del criterio de ‘interacción motriz’, En el campo de los videojuegos en particular, varios autores han
se establecen cuatro grupos de juegos o ‘dominios de acción analizado cómo los videojuegos favorecen la motivación por ser
motriz’ [37]. Juegos psicomotores (sin interacción motriz), en los intrínsecamente satisfactorios [38]. Esta satisfacción provendría
cuales el protagonista interviene sin tener ningún oponente que lo esencialmente del caudal de emociones generado al obtener
perjudique o compañero que le ayude en la realización de logros, al disfrutar de libertad de acción y al interactuar con otros
acciones motrices; b) Juegos de cooperación, en los que diferentes jugadores.
protagonistas han de ayudarse para superar un objetivo común; c) Otros autores han investigado cuáles son las emociones más
Juegos de oposición, en los cuales los protagonistas desafían a frecuentes en videojuegos. González Tardón [39] presenta una
uno o más rivales para conseguir su objetivo y d) Juegos de taxonomía de emociones frecuentes en videojuegos, las cuales
cooperación y oposición a la vez, en los que varios jugadores que son: interés, humor, felicidad, sorpresa, ansiedad, amor,
forman parte de un equipo deben enfrentarse y superar a otros hostilidad, tristeza, repulsión e ira. Moorcock [40] presenta los
adversarios, generalmente también organizados en equipo. La resultados basados en 1,040 respuestas sobre las emociones que
metodología que proponemos está especialmente diseñada para más frecuentemente se daban al jugar a videojuegos, siendo las
los tipos de juegos motores de cooperación, aunque puede ser diez emociones más frecuentes: felicidad, alivio, satisfacción,
adaptada para otros tipos de juegos con acción motriz. sorpresa, triunfo, curiosidad, excitación, admiración, alegría,
Las emociones se pueden clasificar en tres categorías en función diversión. Perron [41] también intentó caracterizar algunas
de si se cumplen o no las expectativas buscadas: emociones emociones en videojuegos tipos, teniendo en cuenta la teoría del
positivas (alegría, humor, amor y felicidad), emociones negativas cine. Este autor identificó las siguientes siete emociones: interés,
(miedo, ansiedad, ira, tristeza, rechazo, vergüenza) y emociones disfrute, preocupación, miedo, sorpresa, ira y frustración.
ambiguas: sorpresa, esperanza y compasión [29]. A continuación Asimismo, en el estudio de Ravaja et al. [42] se analizan los
se describen las 13 emociones seleccionadas por Bisquerra [29] y patrones de las respuestas emocionales de los usuarios ante varios
relacionadas con el juego motor (Tabla 2): videojuegos. Los resultados sugieren la utilidad del análisis de los
patrones de las respuestas emocionales en el diseño de
Tabla 2. Emociones relacionadas a juegos motores [28] videojuegos.
Emoción Descripción relacionada con el juego 3. PROPUESTA METODOLÓGICA
Alegría Disfrutar de un suceso durante el juego Como se ha mencionado en la Sección 2, la evaluación de la
satisfactorio. experiencia de usuario con niños y niñas conlleva una serie de
inconvenientes para los métodos e instrumentos tradicionales,
Humor Inspira alegría y mueve a la risa durante el juego. sobre todo en los métodos de tipo test, ya que son especialmente
Amor Desear o participar con afecto respecto a otra complejos para los niños por la cantidad de ítems, adjetivos,
persona del juego, habitualmente, pero no puntuaciones, abstracciones, entre otros métodos [24]. Por otra
necesariamente recíproco. parte, para no afectar la evaluación emocional debe mantenerse un
clima de familiaridad y confianza durante la realización de los
Felicidad Hacer progresos razonables hacia el logro del tests, tratando de mantener los contextos conocidos y personas de
objetivo que plantea el juego. su entorno. También se debe utilizar un lenguaje y elementos
gráficos adecuados a las edades de los menores. La metodología
Miedo Sentir peligro físico real e inminente, concreto y que aquí se propone utiliza instrumentos y técnicas que permiten
arrollador durante el juego. una evaluación emocional mínimamente invasiva y adaptada a los
Ansiedad Enfrentarse a una amenaza incierta durante el menores de 8 a 12 años. Estos instrumentos y técnicas se aplican
juego. en diferentes momentos de una sesión de evaluación: pre-test, test
y pos-test. La duración de la sesión es de una hora. En la Tabla 3
Ira Sentir una ofensa contra mí o mi equipo durante el se observan los momentos e instrumentos utilizados.
juego.
Tabla 3. Momentos e instrumentos para la evaluación
Tristeza Experimentar una pérdida irreparable durante el emocional en niños y niñas con videojuegos activos
transcurso del juego. colaborativos.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 144 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
Tipo
Momento Instrumento Enfoque
Subjetivo – Categórico –
Pre-test EMODIANA
No verbal Dimensional
Pulsómetro –
Objetivo -
Acelerómetro
Biométricas
Categórico
Observacional
Test No verbal
pasiva-
Verbal Categórico
estructurada
Interacción
(grabación en
Social
vídeo)
EMODIANA
Subjetivo –
Categórico –
No verbal
REP (Escala de Dimensional
Pos-Test
esfuerzo
Subjetivo-
percibido de Categórico
Verbal
Borg)
A continuación describiremos los instrumentos utilizados en los
diferentes momentos de la evaluación.

3.1 EMODIANA
En un trabajo anterior se presentó la elaboración y validación del
instrumento EMODIANA [14] (Figura 1). En el mismo, se
analizó, por un lado, el diseño gráfico del instrumento de
evaluación emocional en referencia a la comprensión por parte de Figura 1. EMODIANA: Instrumento de evaluación emocional
los niños y las niñas de las representaciones gráficas emocionales subjetiva para niños y niñas.
(expresiones faciales o caras), y, por otra parte, la intensidad El procedimiento de utilización de la EMODIANA se divide en
emocional asociada a cada emoción. Además, en el mismo trabajo dos fases, según los momentos de la sesión en donde se aplicará el
se presentó el estudio sobre la coherencia entre el lenguaje instrumento: fase pre-sesión y fase post-sesión. De esta forma, al
utilizado al definir las emociones en Premo© y el lenguaje de los entrar en la sesión, el evaluador/a presenta al niño la EMODIANA
niños y niñas utilizado en la identificación de las emociones. (en formato de cartel gráfico), le pregunta al niño/a cómo se siente
La EMODIANA es un instrumento de evaluación emocional y le pide que ubique un imán gráfico en la diana, diciéndole que
subjetiva para niños y niñas, con edades comprendidas entre 7 y en el centro de la diana es la mayor intensidad (mucho) y el
12 años. Este instrumento fue creado para evaluar emociones en exterior la menor intensidad (poco). Luego, le preguntará que
situaciones educativas con juegos motores y videojuegos. La explique las causas o razones de por qué ha dicho que se siente de
validación del instrumento se basó en el análisis del diseño gráfico esa forma. Este procedimiento se registra en una hoja de registro
y el significado emocional atribuido a las diferentes expresiones emocional de la sesión, incluyendo valores cuantitativos (emoción
gestuales presentadas a 168 niños y niñas de entre 7 y 12 años. e intensidad) y cualitativos (explicaciones dadas por los niños y
Asimismo, se validó el lenguaje infantil utilizado para definir las niñas). Al terminar la sesión, el evaluador/a repetirá el
emociones y el grado de discriminación emocional de los 168 procedimiento de registro, pidiéndole al niño/a que diga qué
menores participantes en el estudio. Al conjunto de emociones de siente, en qué intensidad y por qué se siente así, y lo registrará en
Premo© se añaden tres nuevas emociones no recogidas en la la hoja de registro de la sesión. El registro se realiza tal y como el
herramienta original (ansiedad/nerviosismo y sorpresa), y que son contenido es aludido literalmente por los niños y niñas. La
relevantes para la evaluación emocional en niños y niñas en información recogida en las sesiones de intervención se analiza
juegos y videojuegos. También se evaluó la comprensión del siguiendo procedimientos estadísticos para el análisis cuantitativo
significado de la intensidad emocional, y por ello se diseñó una (medias, frecuencias, varianzas), y para el análisis cualitativo
diana, con mayor intensidad de color en el centro y menor en el (codes y categorías, fiabilidad, y concordancia). La EMODIANA
exterior, y se les preguntó a los menores sobre su significado. El nos permite además conocer si las emociones son internas o
instrumento está diseñado estrictamente para valorar las externas a la intervención que se está realizando, ya que los ítems
emociones referidas al sujeto. Las emociones son consideradas de valoración subjetiva o justificaciones dadas por los niños y
mutuamente excluyentes para el análisis, aunque se recogen las niñas, se clasifican como internos o externos a la actividad
explicaciones de los niños y niñas, de forma que puede extraerse realizada. La dimensión interna incluye las justificaciones
cualitativamente si hay alguna otra emoción asociada a la referidas al sujeto o persona (P) y las relacionadas a la estructura
emoción elegida (considerada predominante). de la actividad (E). La dimensión externa comprende a las
justificaciones o explicaciones referidas al contexto (C). Este
sistema de categorías, permite al observador identificar las
categorías que representan con más precisión la afirmación
subjetiva de la justificación de la emoción declarada por cada
niño/a. Así, para el análisis de las justificaciones subjetivas se
definen 7 categorías posibles (Tabla 4).

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 145 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
Tabla 4. Siete categorías excluyentes para el análisis de la referidas. La fiabilidad del instrumento se ha calculado utilizando
naturaleza interna o externa de las emociones manifestadas la Kappa de Fleiss [43] que permite analizar los registros
por los niños y niñas. observados por más de dos observadores. En la validación del
instrumento realizada sobre 89 casos válidos, el índice de
Categoría Definición
fiabilidad k ha sido 0,903 (donde 0 es la mínima concordancia
Cuando la justificación se realiza en primera posible y el 1 la máxima). Por consiguiente, podemos asegurar la
persona, vinculándose a la justificación de la fiabilidad de este instrumento de evaluación emocional.
emoción declarada.
1. Persona (P) 3.2. Sensores biométricos
Por ejemplo: “porque me reí y me divertí”,
“porque tengo miedo de portarme mal como En nuestra propuesta metodológica incluimos sensores
la última vez” biométricos de bajo coste que puedan ser utilizados en la
realización de actividades motoras con niños y niñas y que
Cuando la justificación se realiza sobre permitan medir frecuencia cardíaca, pulso y distancia recorrida,
elementos o agentes de la intervención etc. Para ello, los sensores más comunes que podremos utilizar
2. Estructura específica programada. serán los pulsómetros y los acelerómetros. Los pulsómetros son
de la
Por ejemplo: “porque hoy voy a hacer aparatos diseñados para poder llevarlos durante el ejercicio con el
actividad
ejercicios nuevos y voy a aprender a hacer propósito de medir y registrar la frecuencia cardiaca en
(E) pulsaciones por minuto. Estos dispositivos proveen un registro
juegos”, “porque los juegos eran muy
divertidos”. completo de las pulsaciones por minuto (bpm, del inglés beats per
minute) durante una sesión de actividad física, con mayor
Cuando la justificación se realiza sobre exactitud que la toma manual de pulsaciones. Miden la frecuencia
aspectos del entorno del niño y de la actividad cardíaca, permitiendo ajustar la frecuencia límite, máxima y
3. Contexto programada. mínima. Existen diferentes tipos de pulsómetros, los más comunes
(C) se componen de dos piezas principales: la cinta emisora digital
Por ejemplo: “Porque hoy es el cumpleaños
que se coloca a la altura del esternón y contiene un sensor y un
de mi padre”, “porque tengo puente y ahora
transmisor que envía las señales al reloj-pulsera. Estos aparatos
vienen las navidades”. tienen diferentes accesorios y su precio puede variar entre los 30
Cuando se dan de forma conjunta las a los 300 euros dependiendo de las características y ventajas que
4. Persona y justificaciones relacionadas con las categorías ofrecen, entre ellas el poder conectarse directamente a un
estructura P y E. ordenador para descargar y analizar la información a través de una
de la memoria USB/SD. Los acelerómetros incorporan sensores que
actividad Por ejemplo: “porque me siento bien al hacer miden la zancada, la cadencia y el ángulo del pie, entre otras
(PE) el ejercicio que he hecho”, “porque me gustan medidas posibles y con esta información generan datos sobre la
mucho los juegos pero me he portado mal”. distancia y velocidad. Los sensores vienen incorporados en el
Cuando se dan de forma conjunta las propio reloj, en los relojes de última generación.
justificaciones relacionadas con las categorías
5. Persona y P y C.
contexto
(PC) Por ejemplo: “porque hoy tuve un buen en
clase, en mi casa y aquí”, “porque estoy
satisfecho porque ya hice un examen”.
6. Estructura Cuando se dan de forma conjunta las
de la justificaciones relacionadas con las categorías
actividad y E y C. En este caso no se han encontrado en el
contexto estudio justificaciones que se correspondan
(EC) con esta categoría.
Cuando se dan de forma conjunta las
7. Persona, justificaciones relacionadas con las tres Figura 2. Solución empleada en la propuesta para la medición
estructura categorías P, E y C. de variables biométricas.
de la
actividad y Por ejemplo: “porque ya no vamos a venir
contexto más, solo una semana más”, “porque me
(PEC) siento satisfecho cada vez que vengo aquí, y
además hoy tuve excursión”.
La dificultad del fenómeno de justificación de emociones en niños
y niñas requiere de una validación que contenga opiniones de
expertos relacionadas con distintos campos disciplinares
(psicología, juego motor y videojuegos). Por ello, el análisis
cualitativo en nuestra metodología se realiza por evaluadores
expertos de cada campo, que analizan los codes y categorías de las
justificaciones dadas por los niños y niñas a las emociones

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 146 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
que se hayan producido durante la interacción con el videojuego.
En la Tabla 5 podemos apreciar las dimensiones, los criterios y su
relación con las categorías emocionales.
Tabla 5. Guía para la observación.
Dimensiones Criterios Relación con
las
categorías
emocionales
Lenguaje no Gestos ilustrativos Se
Figura 3. Colocación de sensores antes de la sesión para la
medición de variables biométricas. verbal (vinculados al discurso: por categorizan
ejemplo, V de victoria etc.). en positivas,
En nuestro caso, hemos optado por una solución de bajo coste negativas y
(menor a 100 euros) compuesta por un reloj, un acelerómetro, un ambiguas
cinturón cardio-frecuenciómetro (Figura 2 y Figura 3). Para Expresiones faciales Atenderemos
facilitar la utilización de los diferentes sensores, acelerómetro y a las
cinturón cardio-frecuenciómetro, se utiliza el protocolo de expresiones
comunicación ANT+. Además, se pueden transferir y analizar las positivas,
sesiones en el ordenador a través de una memoria USB SD negativas y
Connect. Las medidas que obtenemos con estos sensores son las ambiguas -
siguientes: FC: frecuencia cardíaca (salvo W 500 SD), Además se
VELOCIDAD: velocidad en km/h, RITMO: ritmo en min/km y indicará su
CAL: calorías por hora. intensidad:
alta, media o
3.3.Observación pasiva estructurada baja
Las sesiones se graban en vídeo y luego se realiza el proceso de
observación estructurada. Se graban dos vídeos: uno de frente a Miradas, por ejemplo si fija la Se
los niños/as (se observan los rostros) y otro sobre la interacción de mirada en el suelo si está categorizan
los niños/as con el sistema. Luego se combinan ambos vídeos para distraído o si hay conflicto. en positivas,
realizar el procedimiento de registro en un software específico de negativas y
análisis de vídeos en deportes, ya que tiene elementos específicos ambiguas
para el análisis de actividades físicas grupales (etiquetas, Gestos emocionales: Se
categorías, subcategorías, equipos, jugadores, etc.), las cuales
• Reducir tensión: categorizan
facilitarán nuestra tarea de registro estructurado observacional en en positivas,
acicalarse el pelo,
videojuegos activos realizados en grupo. En nuestro caso el negativas y
rascarse, etc.
software elegido fue LongoMatch (http://www.longomatch.org).
• Estados de conflicto o ambiguas
Los registros son realizados por 3 evaluadores y luego se analiza
emociones negativas:
la concordancia de las decisiones de categorización y codificación
chuparse el dedo
emitidas.
• Cansancio: tocarse el
pelo, orejas, bostezar
• Agresividad: se
inclinan hacia adelante
• Empatía: imita a la otra
persona
• Otros
Tacto: contactos táctiles Se
Figura 4. Vídeos combinados para el análisis estructurado en entre los niños/as, sienten categorizan
LongoMatch. nerviosismo o están en positivas,
Las emociones observadas se agrupan en tres categorías indicando molestos negativas y
además la intensidad (alta, media o baja). De esta forma, tenemos ambiguas
los siguientes conjuntos de emociones: positivas (alegría,
satisfacción, cariño), ambiguas (vergüenza, sorpresa, seriedad, Lenguaje Positivas: ¡Jajaja!, ¡Sí!, Se
aburrimiento) y negativas (tristeza, miedo, nerviosismo, enfado). Verbal ¡Toma!, Yupi!, ¿Cuándo categorizan
Si bien la literatura habla de una categoría emocional “neutra” empezamos?, ¡Yo también en positivas,
[27], en nuestro caso el equipo de psicología ha decidido cambiar quiero participar!, ¡Muy negativas y
esta conceptualización a “ambigua”, ya que no se entiende que la bien!, ¡Qué bien lo hiciste!, ambiguas
vergüenza, sorpresa o aburrimiento sean estados emocionales ¡Ánimo!, ¡Sigue así!, etc.
neutros. Además se
- Negativas: ¡Nooo!, ¡Jolín!, indicará su
Al observar los vídeos se registran medidas atendiendo a tres ¡Aarrr!, ¡Psk!, ¡Diablos!, ¡Jo!, intensidad:
dimensiones: el lenguaje no verbal, el lenguaje verbal y las ¡Yo paso!, ¡Yo no alta, media o
interacciones sociales (con otros compañeros o con el profesor) puedo!, ¡Yo no sé!, ¡No me baja

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 147 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
Dimensiones Criterios Relación con actividad y otros factores. Este instrumento se muestra últil para
las evaluar tareas específicas dentro de una sesión de entrenamiento,
categorías así como para evaluar la sesión completa. En la elaboración de la
emocionales escala del instrumento, se observó que añadiéndole un cero al
atrevo!, ¡Me siento mal!, valor de esfuerzo percibido, se podría obtener una estimación
¡Qué nervios!. - aproximada de la frecuencia cardíaca (por ejemplo, una
puntuación de 8 equivalía a 80 pulsaciones por minuto, 15 a 150,
Ambiguas: Silencio etc.). Aunque no se ha podido comprobar que esta equivalencia
¡Pobrecito! ¡Lo siento! ¡Te sea real, sí se halló una relación lineal entre el esfuerzo percibido
ayudo! ¡Qué pena! ¿Estás y la frecuencia cardíaca, que aconseja la utilización de esta
bien? ¡Ojalá! ¡A ver si medida subjetiva para evaluar y controlar la intensidad del
puedo!. esfuerzo en el entrenamiento [45]. La Tabla 6 muestra una
equivalencia aproximada entre cuatro medidas de intensidad: la
- Duración o tiempo escala propuesta por Borg; las pulsaciones por minuto; una escala
empleado de 0-10 puntos; y el porcentaje de intensidad máxima posible
Interacción -Tipos de interacción: Se [46]. Este instrumento puede ser de gran ayuda en el proceso de
social categorizan evaluación del entrenamiento, ya que son pocos costosos y fáciles
1. Interactúa con los con en positivas, de incorporar al entrenamiento, y al mismo tiempo permiten
observadores (IO): los negativas y obtener un indicador de la intensidad del esfuerzo como feedback
participantes preguntan, ambiguas de la actividad realizada. Este indicador fortalece la percepción de
dudan, etc. autocontrol y el progreso, aportando información muy valiosa al
diseñador de los ejercicios.
2. Comportamientos
Tabla 6. Escala de esfuerzo percibido [46]
desencadenados por los
observadores (OTB): si los Escala de Equivalencia Grado de Equivalencia
comentarios de los esfuerzo aproximada de intensidad de una escala
percibido de pulsaciones por del de esfuerzo
observadores activan tanto Borg minuto esfuerzo percibido de
las conductas verbales y no (% de la 0-10 puntos.
verbales en los jugadores. capacidad
máxima
3. Compartir experiencias posible)
(SE): los jugadores comparten 6 60-80 0
experiencias con los demás,
reflejando el comportamiento 10
expresivo (por ejemplo, 7 Muy, 70-90 1
riendo juntos) muy
suave
d) Cooperación (COO):
algunos jugadores 8 80-100
compartieron consejos y se 20 2
ayudaban unos a para superar
9 Muy 90-110
las dificultades.
suave
e) Competencia (COM): los 10 100-120
jugadores competían; por 30 3
ejemplo, por la puntuación.
11 Bastante 110-140
f) Intentos de interactuar suave
fallidos (AI): Los jugadores 12 120-140 40 4
intentan interactuar pero el
otro jugador hace caso omiso
de ellos. 13 Algo 130-150 50 5
duro
14 140-160 60 6
3.4. REP (Escala de Esfuerzo percibido)
En la literatura científica especializada [44] se conoce por las
siglas Rango de Esfuerzo Percibido (REP) o en inglés Rating of 15 Duro 150-170
Perceived Effort (RPE) o escala de Borg. El instrumento consiste
en una tabla con números entre 20 y 6, colocados verticalmente y 16 160-180 70 7
acompañados de valoraciones cualitativas entre muy, muy fuerte y 17 Muy 170-190 80 8
muy, muy ligero. Las valoraciones subjetivas de una persona que duro
practica un deporte estarán influenciadas por el nivel deportivo, el
grado de preparación, el estado de salud, la motivación por la
XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction
| 148 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
Escala de Equivalencia Grado de Equivalencia permitirá simplificar y automatizar el análisis de las expresiones
esfuerzo aproximada de intensidad de una escala faciales de los vídeos.
percibido de pulsaciones por del de esfuerzo
Borg minuto esfuerzo percibido de 5. AGRADECIMIENTOS
(% de la 0-10 puntos.
capacidad Este trabajo fue financiado por el Proyecto de Investigación
máxima "Desarrollo de hábitos saludables y la educación física a través de
posible) Videojuegos educativos y juegos motores para la infancia
hospitalizada y la adolescencia", Ref. EDU 2010- 20010 el
18 180-200 90 9
Ministerio de Educación y Ciencia de España. Los autores desean
19 Muy, 190-210 100 10 agradecer especialmente el grupo de profesionales que desarrolló
muy la intervención en la escuela.
duro
REFERENCIAS
20 200-220 [1] ISO 9241-210:2010. Ergonomics of human-system
interaction – Part 210: Human-centred design for interactive
4. CONCLUSIONES systems.
En este artículo se presenta el problema de la evaluación de la [2] Ekman P., Friesen W.V., Ellsworth P. (1972). Emotion in the
experiencia interactiva y emocional en niños y niñas con human face: Guideline for research and an intergration of
videojuegos activos. Para ello, se ha realizado una revisión de findings. Pergamon Press, New York, 1972.
técnicas y de instrumentos de evaluación de la UX, con sus
ventajas e inconvenientes en la evaluación con niños y niñas. [3] Kaiser, S. & Wehrle, T. (2008). Facial expressions in social
Además, también se ha realizado un análisis del conjunto de interactions: Beyond basic emotions. In D. Cañamero & R.
emociones que se presentan frecuentemente en la interacción con Aylett (Eds.), Advances in Consciousness Research (Vol.74):
videojuegos y con los juegos motores o activos. En base a este Animating Expressive Characters for Social Interactions (pp.
53-69) . Amsterdam : John Benjamins Publishing Company.
análisis, se ha diseñado y presentado una propuesta metodológica
de evaluación de la experiencia emocional de niños y niñas con [4] Kaiser, S., Wehrle, T., & Schenkel, K. (2009). Expression
videojuegos activos. Esta propuesta está especialmente adaptada faciale. In D. Sander & K. R. Scherer (Eds.) 77-108). Paris:
para la evaluación emocional en edades comprendidas entre 8 y Dunod.
12 años. Los métodos e instrumentos que componen esta [5] Mauss, I. B., & Robinson, M. D. (2009). Measures of
metodología fueron diseñados para resolver algunos de los emotion: A review.Cognition & Emotion, 23(2), 209–237.
problemas frecuentes que se encuentran en la evaluación doi:10.1080/02699930802204677
emocional con niños y niñas, siendo mínimamente invasivos y
maximizando la fiabilidad de los resultados obtenidos. [6] Picard, R.W. (1997). Affective Computing. MIT Press
[7] Loscertales, Antonio, Cerezo, Eva, Creatura, Marco,
La metodología se organiza en diferentes momentos (pretest, test
González, Jose Luis, Minguez, Javier, Gil, Rosa, Baldassarri,
y postest) y contempla medidas subjetivas del sujeto y del
Sandra and López, Yolanda (2012): NeuroGame project:
observador, así como medidas objetivas (biométricas). Se propone
multimodal videogame neuro-evaluation. In: Proceedings of
un procedimiento de análisis de las medidas subjetivas que
the 13th International Conference on Interacción Persona-
permitan garantizar la fiabilidad, tanto para los cuestionarios
Ordenador 2012. p. 18.
subjetivo de la EMODIANA y de REP, así como para el
procedimiento de observación estructurada. Por ello, en el análisis [8] Gunes, H. and Pantic, M. (2010). Automatic, dimensional
cualitativo, la metodología propone una inter-observación and continuous emotion recognition. Int. Journal of Synthetic
realizada por dos o más observadores buscando así más rigor en la Emotions (IJSE), 1(1), 68–99.
observación. Asimismo, a través de los análisis cruzados de datos [9] Ekman, P., Dalgleish, T. and Power. M. 1999. Handbook of
se pueden proponer y validar nuevas hipótesis, entre ellas, Cognition and Emotion. Wiley Online Library.
comprobar si el esfuerzo percibido no coincide con las
pulsaciones registradas. [10] Russell, J.A. and Mehrabian, A. 1977. Evidence for a
threefactor theory of emotions. Journal of Research in
La metodología ha sido aplicada en la validación de un programa Personality 11(3), 273-294.
formativo sobre hábitos saludables llevado a cabo en un centro
[11] Bradley M.M. and Lang P.J. (1994). “Measuring emotion:
escolar. En este programa formativo se ha utilizado un videojuego
the self-assessment manikin and the semantic differential,”
activo con interacción gestual, denominado TANGO:H [47], tal y
Journal of behavior therapy and experimental psychiatry, vol.
como se ha mencionado en la sección introductoria. Los
25, 1994, pp. 49-59.
resultados de la evaluación de dicha experiencia han sido
altamente positivos. Sin embargo, debido a la limitación de [12] Desmet P. (2005). “Measuring emotion: development and
extensión del artículo no pueden incluirse los resultados. Como application of an instrument to measure emotional responses
resultado de lo obtenido, actualmente estamos desarrollando una to products,” Funology, Kluwer Academic Publishers, 2005,
librería de integración sensorial para poder conectar distintos tipos p. 123.
de sensores biométricos, así como realizar el análisis de los [13] Lewis M., Haviland-Jones J.M., and Barrett L.F. (2008),
registros en un software especialmente diseñado para satisfacer Handbook of emotions, The Guilford Press, 2008.
nuestros requisitos y conseguir de este modo automatizar los
registros y análisis para nuestro caso. También estamos adaptando [14] González C.S., Cairós M., Navarro V. (2013). EMODIANA:
el software TANGO:H para trabajar con Kinect 2, y registrar Un instrumento para la evaluación subjetiva de emociones en
expresiones faciales de forma automática, ya que esto nos

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 149 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
niños y niñas. En Actas del Congreso INTERACCIÓN 2013. [28] Lavega P., Filella G., Lagarderaa F., Mateub M. & Ochoa J.
Madrid. (2015). Juegos motores y emociones. Cultura y Educación:
[15] González Sánchez, J.L., Gil Iranzo, R.M. and Gutiérrez Vela, Culture and Education, 25:3, 347-360, DOI:
F. L. 2011. Enriqueciendo la Evaluación en Videojuegos. 10.1174/113564013807749731
Actas del XII Congreso Internacional de Interacción [29] Bisquerra R. (2000). Educación emocional y bienestar.
Persona-Ordenador. Interacción 2011 (Lisboa, Portugal, 2-6 Barcelona: Praxis.
de Septiembre). Néstor Garay y Julio Abascal (Eds.), 273- [30] Scherer K.R. & Brosch T. (2009). Culture-Specific appraisal
282. biases contribute to emotion dispositions. European Journal
[16] Sim G., Cassidy B. & Read J. (2013). Understanding the of Personality, 23, 265-288. doi: 10.1002/per.714
Fidelity Effect when Evaluating Games with Children. IDC [31] Frome J. (2007). Eight Ways Videogames Generate
2013, New York, NY, USA. Emotion." Conference of the Digital Games Research
[17] Borgers, N., Leeuw, E. D. and Hox, J. Children as Association. Tokyo, Japan. Sep 27, 2007.
Respondents in Survey Research: Cognitive Development [32] González, C.S. y Blanco, F. (2008). Emociones con
and Response Quality. Buletin de Methodologie videojuegos: incrementando la motivación para el
Sociologique, 66 (2000), 60-75. aprendizaje. "Teoría de la Educación: educación y cultura en
[18] Horton, M. and Read, J. C. Interactive Whiteboards in the la sociedad de la información", 9 (3), 69-92
Living Room? - Asking Children about their Technologies. [33] González C.S. & Navarro (2015). A Structural Theoretical
In Proceedings of the 22nd British HCI Conference Framework Based on Motor Play to Categorize and Analyze
(Liverpool, 2008). Active Video Games. Games and Culture.
[19] Read, J. C. and Fine, K. Using Survey Methods for Design 1555412015576613, first published on March 26, 2015 as
and Evaluation in Child Computer Interaction. In doi:10.1177/1555412015576613
Proceedings of the Interact 2005 (Rome, 2005) [34] Parlebas, P. (1981). Contribution á un lexique commenté en
[20] Markopoulos, P. and Bekker, M. On assessment of usability science de l’action motrice. París: INSEP.
testing methods for children. Interacting with Computers, 15 [35] Parlebas, P. (1988). Elementos de sociología del deporte.
(2003), 227-243. Málaga: Unisport.
[21] Hoysniemi, J., Hamalainen, P. and Turkki, L. Using peer [36] Parlebas, P. (2001). Juegos, deporte y sociedad. Léxico
tutoring in evaluating the usability of a physically interactive comentado en praxeología motriz. Barcelona: Paidotribo.
computer game with children. Interacting with Computers,
15, 2 (2003), 203-225. [37] Lagardera F. y Lavega P. (2003). Introducción a la
praxiología motriz. Barcelona: Paidotribo.
[22] Donker, A. and Reitsma, P. Usability Testing with Children.
In Proceedings of the Interacton Design and Children [38] Ryan, R. M., Scott Rigby, C. & Przybylski, A. (2006). The
(Maryland, 2004). ACM Press. Motivational Pull of Video Games: A Self-Determination
Theory Approach.
[23] Barendregt, W., Bekker, M., Bouwhuis, D. G. and Baauw, E.
Predicting effectiveness of children participants in user [39] González Tardón C. (2006). Emociones y Videojuegos,
testing based on personality charactersitics. Behaviour & Actas del III Congreso del Observatorio de la Cibersociedad.
Information Technology, 26, 2 (2007), 133-147. [40] Moorcock M. (2008). Top Ten Videogame Emotions.
[24] Padilla Zea N., González-Sánchez J.L., Gutierrez Vela F., Available online:
Abad-Arranz A., López Arcos R. (2012). Evaluación de http://onlyagame.typepad.com/only_a_game/2008/04/top-
Emociones en Videojuegos Educativos. El caso particular de ten-videoga.html
los Niños. En Actas del XIII Congreso Internacional de [41] Perron, B. (2005). “A cognitive psychological approach to
Interacción Persona Ordenador (Interacción 2012), 3-5 gameplay emotions.” Conference Proceedings of DiGRA’05,
octubre 2012, Elche, España. Changing Views: Worlds in Play, Digital Games Research
[25] Navarro V. (2015). El juego en el desarrollo: entre el juego Association.
motor y el videojuego activo En Libro Uso de las TIC para la [42] Ravaja, N., Salminen, M., Holopainen, J., Saari, T., Laarni, J.
Atención Educativa Hospitalaria y Domiciliaria. González C. and Järvinen, A. (2004). Emotional response patterns and
y Violant V. (Coord.). Editorial Mc Graw Hill. Madrid. sense of presence during video games: potential criterion
[26] Loscertales A., Cerezo E., Creatura M., González J.L., variables for game design. In Proceedings of the third Nordic
Minguez J., Gil R., Baldassarri S. and López Y (2012). conference on Human-Computer Interaction (Tampere,
Proyecto NeuroGame: Neuro-Evaluación Multimodal de Finland), 339-347.
Videojuegos. En Actas del XIII Congreso Internacional de [43] Fleiss, J. L. & Cohen, J. (1973) "The equivalence of
Interacción Persona Ordenador (Interacción 2012), 3-5 weighted kappa and the intraclass correlation coefficient as
octubre 2012, Elche, España measures of reliability" in Educational and Psychological
[27] Méndez Y. (2015). Marco de Referencia para la Captura Measurement, Vol. 33 pp. 613–619
Semi-Invasiva y Análisis de las Emociones en el Proceso de [44] Moya Morales J.M. (2004). La percepción subjetiva del
Evaluación con Usuarios de Sistemas Interactivos. Tesis esfuerzo como parte de la evaluación de la intensidad del
doctoral. Facultad de Ingeniería Electrónica y entrenamiento. http://www.efdeportes.com/ Revista Digital -
Telecomunicaciones de la Universidad del Cauca. Colombia. Buenos Aires - Año 10 - N° 73 - Junio de 2004.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 150 |
 Métodos y técnicas para la evaluación de la experiencia emocional de niños y niñas con videojuegos activos
C. S. González-González | V. Navarro-Adelantado
[45] Arruza, J, et al. (1996) Esfuerzo percibido y frecuencia [47] Carina González, Pedro Toledo, Miguel Padrón, Elena
cardiaca: el control de la intensidad de los esfuerzos en el Santos, Mariana Cairós.Including Gamification Techniques
entrenamiento de Judo. Revista de Psicología del Deporte, 9 in the Design of TANGO: H Platform, Jurnal Teknologi, v.
- 10: 29 - 40. 63. 2013
[46] Buceta, J.M. (1998) Psicología del entrenamiento deportivo.
Dykinson,. Madrid.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 151 |
 The SARA Project: An Interactive Sandbox for Research on Autism
D. Arellano | V. Helzle | U. M. Schaller | R. Rauh | M. Spicker | O. Deussen

The SARA Project: An Interactive Sandbox for Research

on Autism
Diana Arellano Volker Helzle Ulrich Max Schaller
Filmakademie Filmakademie University Medical Center
Baden-Württemberg Baden-Württemberg Freiburg, Germany
Ludwigsburg, Germany Ludwigsburg, Germany
diana.arellano@filmakade
mie.de

Reinhold Rauh Marc Spicker Oliver Deussen

University of Konstanz Konstanz, University of Konstanz Konstanz,
University Medical Center Germany Germany
Freiburg, Germany

ABSTRACT Motivated by these numbers and the amount of research done in

this area, we propose SARA (Stylized Animations for Research
SARA is an ongoing research project that investigates in a in Autism), an ongoing project that investigates the causes
novel and artistic way the causes for social communication behind communication and emotion perception deficits in
and emotion recognition deficits in children and adolescents children and adolescents with high-functioning ASD. To achieve
with high-functioning autism spectrum disorders (ASD). The this, SARA combines psychology, real-time non-photorealistic
novelty of our work resides in the real-time generation and rendering (NPR) and 3D computer animation, with the main
parameterization of emotional facial expressions of virtual goal of studying how abstracted faces, with different levels of
characters by means of speed, intensity and abstraction, the details are categorized by children and adolescents with ASD.
latter achieved by non-photorealistic rendering (NPR)
techniques. Although the project is currently in an ongoing The interest in using NPR is that it permits a variation in the
phase, it shows the potential of using virtual characters and real- level of abstraction and visuo-spatial information, adapting
time techniques for interactive experiments, which otherwise images to “focus the viewer’s attention” [6]. Thus the
would be impossible using “linear stimuli” (e.g. pre-rendered information load in the characters’ facial expressions can be
animations). reduced, conveying the emotional information more efficiently.
In the end, a set of virtual characters are used in an interactive
Categories and Subject Descriptors computer-based psychological test, where each character
H.5 [Information Interfaces and Presentation]: Mul- timedia displays emotional facial animations generated in real-time. In
Information Systems; I.3 [Computer Graphics]: Applications; the following, we will shortly refer to previous research in this
J.4 [Social and Behavioral Sciences]: [Psy- chology] area, we will explain the current status and first results of
SARA, and finally, we will conclude with the ongoing and
General Terms future work.
Autism, Social Skills, Animation, Emotions 2. RELATED WORK
Keywords Many interactive applications created to develop or enhance the
social skills of individuals with autism make use of virtual
Autism Spectrum Disorder, Non-Photorealistic Rendering, Real- characters. Among the tools used for research in a lab
time Animation, Facial Expressions, Interaction environment we found the work of Whyte et al. [14], who used
game components (e.g., storyline, long-term goals, re- wards) to
1. INTRODUCTION create engaging learning experiences, especially in computer-
Autism spectrum disorder (ASD) is a developmental disability based interventions. Milne et al. [9] employed autonomous
that tosses significant communication, behavioral and social agents as social skills tutors for teaching children with ASD
challenges. conversation skills and how to deal with bullying. Grawemeyer
According to the Center for Disease Control and Prevention et al. [7] developed an embodied pedagogical agent together
(CDC), research on ASD has increased a great deal in recent with, and for young people with ASD.
years [1], as well as the number of children and adults with this ECHOES VE [2] presents a virtual environment where children
disorder. In 2014 the CDC estimated that about one percent of with ASD need to assist a virtual character in selecting objects
the world population has ASD. In the United States the by following the character’s gaze and/or pointing at the object.
prevalence of ASD was in the same year one in 68 births [5]. In In JeStiMulE [12] participants are taught to recognize emotions
Spain, the “Confederaci´on Autismo Espan˜a” communicated on the faces and gestures of virtual characters, while considering
that approximately 350.000 people in this country has a form of the context. LIFEisGAME [3] deploys a low cost real-time
ASD [8]. animation system embedded in a game engine to create a
game that helps individuals with ASD to recognize emotions
in an interactive way. FaceSayTM [11] aids children with ASD

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 152 |
 The SARA Project: An Interactive Sandbox for Research on Autism
D. Arellano | V. Helzle | U. M. Schaller | R. Rauh | M. Spicker | O. Deussen
to recognize faces, facial expressions and emotions by offering the experimenter has more control over it. It is the experimenter
students simulated practice with eye gaze, joint attention, and who explains the child or adolescent what it needs to be done
facial recognition skills. Let’s face it! [13] is a program and even aids him in the selection of the emotional choice, as
comprised of seven interactive computer games that target the seen in Figure 1.
specific face impairments associated with autism.
One thing all these applications have in common is the use of
virtual characters to enhance or develop skills in subjects with
ASD. In all the previous cases, the characters present a defined
visual style, which can be either cartoony or realistic. In this
sense, one of the assets of our project is the possibility to change
the visual representation of the characters (from more realistic to
more abstract) in real-time, opening up new possibilities for
more personalized applications.

3. SARA
Similarly to previous research, SARA (Stylized Animation for
Research in Autism) combines clinical psychology and
computer animation to create a tool for assessing the
categorization of dynamic emotional facial expressions by
children and adolescents with high-functioning ASD. Moreover,
our project’s innovation includes real-time non-photorealistic Figure 1: Participant during a DECT session.
rendering (NPR) algorithms to abstract the faces of the virtual The test deliberately does not contain any GUI-centric terms, so
characters used in the test. This will allow us to explore how a participants with ASD do not focus on other elements than the
reduction in the level of details of facial expressions affects their facial expressions of the characters and the emotional answers.
categorization by individuals with ASD. A session with the DECT consists of several trials. Figure 2
shows the basic screens that are sequentially displayed in each
The core of SARA is the DECT (Dynamic Emotion trial. It begins with a pink-colored screen (Fig. 2(0)), which
Categorization Test) [10], an interactive computer-based tool serves as separator in between trials. By pressing the Enter key
created to assess the feasibility of using real-time animations by the trial and the interaction is initiated. It begins with a fixation
comparing virtual characters to video clips of human actors. A cross (Fig. 2(1)) that appears for 0.5 seconds and indicates
previous version of the test contained material of two human where the participant should fixate his gaze. After this time, one
actors and two virtual characters displaying dynamic facial of two characters (Hank, an old male; or Nikita, a young
expressions of the basic emotions: anger, disgust, fear, female) appear displaying a real-time generated animation of a
happiness, sadness, and surprise, on three intensity levels: weak, facial expression with certain intensity (weak, medium, strong)
medium, and strong. The results of this very first version of
and speed (very slow, slow, moderately slow, normal,
DECT showed that the three levels of intensity were equally moderately fast, fast, very fast) (Fig. 2(2)). Then, a screen
categorized in the virtual characters and in the human actors. with white noise ((Fig. 2(3)) appears for 0,5 seconds, loading
This motivated us to continue exploring the use of virtual participants’ iconic memory with task-irrelevant information.
characters in autism research. Finally, a screen with the answers options represented by
The design and development of the DECT, as well as the emotional labels is shown (Fig. 2(4)). Here the user needs to
implementation of the NPR algorithms and real-time animations select the one corresponding to the expression that he just saw.
is being done in the software development plat- form In order to select an emotion, each of the basic emotions were
Frapper 3 , in particular using the Agent Framework [4]. mapped to a number between 1-3 and 7-9, which was then
Frapper is a C++, Ogre3D and Qt based development selected using the numeric pad of the keyboard. The reason for
environment consisting of a node-based scene model, a model- not using the row 4-6 was to allow space between the fingers
view-controller architecture, a panel-oriented user interface and avoid experimental errors by inadvertently pressing the
similar to commercial 3D packages. The Agent Framework is wrong key. The pairing emotion-number is done randomly each
the set of functionalities (nodes and plug- ins) that allow users time the test is carried out.
the rapid prototyping of applications that make use of virtual
For the SARA project, three versions of the DECT have been
characters. Both Frapper and the Agent Framework are provided
planned. The first version, R-DECT was implemented and
with two human-like characters distributed under the Creative
executed in the context of a pilot study to assess “Rapid Social
Commons Attribution-NonCommercial-ShareAlike 3.0
Cognition” of children and adolescents with ASD. The
Unported License.
innovation
4. INTERACTING WITH DECT
DECT has been conceived as an interactive research tool, rather
than as an intervention tool, where both experimenters and
participants can work in a real-time environment. Currently, the
interaction with the software has been designed in a way where

3
http://sourceforge.net/projects/frapper/

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 153 |
 The SARA Project: An Interactive Sandbox for Research on Autism
D. Arellano | V. Helzle | U. M. Schaller | R. Rauh | M. Spicker | O. Deussen

5.1 R-DECT
Participants of the experiment were 39 adolescents with ages
between 14.0 and 17.9 years and IQ ≥ 70. The group of
neurotypically developed adolescents (NTD group: n=22)
consisted of 18 males and 4 females. The group of individuals
with high-functioning ASD (ASD group: n=17) consisted of 12
males and 5 females. The R-DECT consisted of 2 (characters)
x 6 (basic emotions) x 3 (intensity levels), resulting in 36
animations. Regarding the speed variable, it was assigned
according to a certain scheme to each of the 36 animations,
ranging from 1 (normal speed) up to 2.25 times of normal speed.
In total, six levels were used (1.00, 1.25, 1.50, 1.75, 2.00, and
2.25).
Figure 2: Screens of the DECT: (0) Initial screen, (1) In total, 62.2% of the animations were categorized correctly.
Fixation Cross, (2) Character with an angry expression, (3) Accuracy rate for the NTD group was 65.7 % whereas for the
White noise, (4) Forced choices with emotion names, (5) ASD group was 57.7%. A 2x6 MANOVA with repeated
Initial screen for next trial. measurements showed no significant interaction between group
of this test was the random presentation of real- time animations and basic emotion (F < 1). However, the two main effects were
of facial expressions with different speeds (going from normal significant (basic emotion: F(5,33) = 77.63, p < .0001; group:
to very fast) and intensities (weak, medium, strong). The R- F(1,37) = 5.36, p = .026), indicating that the ASD group
DECT served also to validate the improved facial animation performed significantly worse than the NTD group (Figure 4).
from the first DECT, as well as to validate the test itself as a tool The order of accuracy for basic emotions was the same for both
for the interactive categorization of emotional expressions. groups, being “happiness” the one recognized with the highest
accuracy and “fear” with the least. However, all post-hoc
The second version called NPR-DECT comprises one of the
comparisons did not reach statistical significance. As for the
novelties of our project: the use of NPR algorithms to abstract intensity of facial emotions, we only considered the NTD group
and manipulate visuo-spatial information in the faces of our where typical facial emotion recognition is expected. The results
virtual characters. It not only constitutes a way to reduce showed that in general varying intensities from weak over
information load in the characters’ facial expressions, but also a medium to strong affected accuracy rates correspondingly:
way to include more artistic approaches to investigate how Weak: 59.5%, Medium: 65.5% and Strong: 72.0%.
these abstractions affect the recognition of the facial
expressions of emotions, in comparison to their more realistic
representations. Figure 3 shows different facial abstractions to
be tested in the NPR-DECT.

Figure 3: NPR styles. Left to right: original, pencil

drawing, watercolors, line drawing, loose & sketchy.
The third version called i-DECT is the one with the highest
interactivity, stimulating a visual interaction between the
participant and the virtual character. This test will study the
differences in eye contact and mutual gaze between neurotypical
subjects and subjects with ASD. It is worth mentioning that the
interactivity to be achieved in this test is possible thanks to the
real-time characteristic of our frame- work, providing flexibility
and opportunity for more elaborated interactive experiments.
Another level of interaction that plays a main role is the one of
the experimenter with the test. Having a tool that generates
animations and visual representations in real-time allows the
psychologists and experimenters to fine tune and parameterize Figure 4: Mean of relative frequencies of correct
the tests themselves according to their requirements, or the categorizations of six basic emotions in ASD and NTD
participant’s needs. This flexibility makes them independent groups. Error bars represent 95% CI.
from the animator, an important aspect to consider when using a
computer-based research tool.
5.2 NPR-DECT
5. EVALUATIONS Participants of the experiment were 31 (9 male, 22 female)
Until now we have evaluated the R-DECT, and partially the neurotypically developed psychology students with an age
NPR-DECT with neurotypical participants. range from 20 to 35 years. The NPR-DECT comprised 2

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 154 |
 The SARA Project: An Interactive Sandbox for Research on Autism
D. Arellano | V. Helzle | U. M. Schaller | R. Rauh | M. Spicker | O. Deussen
(characters) x 6 (basic emotions) x 13 (1 + 12 NPR style x 8. REFERENCES
abstraction combinations) = 156 trials, which were presented in [1] Facts About ASD, 2014. Retrived April 06, 2015, from
a pseudorandomized order. The NPR styles: coherent line Center for Disease Control and Prevention:
drawing (CLD), pencil drawing (PD), image abstraction (IA) http://www.cdc.gov/ncbddd/autism/facts.html.
and watercolors (W) were instantiated in one of three levels of
abstraction (low, medium, high). During the whole NPR-DECT [2] A. Alcorn, H. Pain, G. Rajendran, T. Smith,O. Lemon, K.
session participants’ gaze was tracked by an RED-250 eye Porayska-Pomsta, M. E. Foster,K. Avramides, C.
tracker (SMI), which API was integrated in Frapper to allow the Frauenberger, and S. Bernardini. Social communication
communication between each other. between virtual characters and children with autism. In
AIED, 2011.
In total, 71.4% of the emotional expressions were correctly
[3] S. Alves, A. Marques, C. Queir´os, and V. Orvalho.
categorized. However, we found a considerable difference of
LIFEisGAME prototype: A serious game about emotions
accuracy between both characters (Hank: 66.5% vs. Nikita
for children with autism spectrum disorders. PsychNology
76.3%). Differences in levels of accuracy seem to be dependent
Journal, 11(3):191–211, 2013.
on the character in a certain style with a certain level of
abstraction. The lowest percentage for Hank was obtained with [4] D. Arellano, V. Helzle, U. Schaller, and R. Rauh.Animated
the CLD - high abstraction (55.9%), whereas the high- est faces, abstractions and autism. In Intelligent Virtual Agents
percentage was with W - medium abstraction (73.7%). The 2014, volume 8637 of LNCS, pages 22–25, 2014.
lowest percentage for Nikita was obtained with the PD - high [5] J. Baio. Prevalence of autism spectrum disorder among
abstraction (71.5%), whereas the highest percentage was noted children aged 8 years – autism and developmental
for both IA - low abstraction (81.7%). disabilities monitoring network, 11 sites, united states,
As for the recognizability and likeability of the abstracted faces, 2010. In Surveillance Summaries, volume 63, pages 1–21.
they were measured through a computer-based questionnaire 2014.
with 26 images of the two characters, stylized by the four NPR [6] B. Gooch and A. Gooch. Non-Photorealistic Rendering.
styles in three levels of abstraction, plus the original A.K. Peters, 2001.
photorealistic representation. For each image, two questions
[7] B. Grawemeyer, H. Johnson, M. Brosnan, E. Ashwin, and
were posed: (1) How good were you able to recognize the
L. Benton. Developing an embodied pedagogical agent
emotions from this representation? (2) How good did you like
with and for young people with autism spectrum disorder.
this representation? For each question, the answers were
In ITS 2012, volume 7315 of LNCS, pages 262–267.
presented in a 7-point Likert scale ranging from very good (“1”)
Springer, Heidelberg, 2012.
to very bad (“7”). On average, recognition ratings were best for
the original representation (mean = 2.02), followed by IA (mean [8] J. C. Hidalgo. Ciudades en azul en el d´ıa del autismo,
= 2.44), CLD (mean = 3.69), W (mean = 3.73), and finally PD April 2015. El Pa´ıs. Retrieved from http://goo.gl/Yzqdw6.
(mean = 4.17). [9] M. Milne, M. Luerssen, T. Lewis, R. Leibbrandt, and D.
Powers. Designing and evaluating interactive agents as
6. CONCLUSIONS social skills tutors for children with autism spectrum
We have presented the ongoing research project SARA and the disorder. Conversational Agents and Natural Language
tests that have been implemented so far to assess the Interaction: Techniques and Effective Practices, pages 23–
categorization of emotional facial expressions: R-DECT and 48, 2011.
NPR-DECT. In their current status, both are more a re- search
tool than an intervention tool. Therefore, the interaction with the [10] R. Rauh and U. Schaller. Categorical perception of
tool has not been exploited to its maximum. Results have shown emotional facial expressions in video clips with natural and
that there is a significant difference in the categorization of artificial actors: A pilot study. Technical report, University
emotions between the NTD and ASD groups, being the latter the of Freiburg, 2009.
one that performed the worst. However, it was not possible to [11] L. Rice, C. Wall, A. Fogel, and F. Shic. Computer-assisted
say which emotion(s) contributed to this general difference. As face processing instruction improves emotion recognition,
for the NPR styles, no decisive conclusions were achieved. At mentalizing, and social skills in students with asd. Journal
the end of this project it is our goal to distribute all DECTs as a of Autism and Developmental Disorders, pages 1–11, 2015.
open-source tool. Moreover, based on the results obtained with
[12] S. Serret, S. Hun, G. Iakimova, J. Lozada, M. Anastassova,
SARA we will create new interactive applications or tests
A. Santos, S. Vesperini, and F. Askenazy. Facing the
considering HCI elements, taking advantage of the NPR
challenge of teaching emotions to individuals with low- and
elements and artistic abstraction techniques.
high-functioning autism using a new serious game: a pilot
7. ACKNOWLEDGMENTS study. Molecular Autism, 5(37), 2014.
The SARA project (officially Impact of Non-Photorealistic http://www.jestimule.com.
Rendering for the understanding of emotional facial expres- [13] J. W. Tanaka, J. M. Wolf, C. Klaiman, K. Koenig, J.
sions by children and adolescents with high-functioning Autism Cockburn, L. Herlihy, C. Brown, S. Stahl, M. D. Kaiser,
Spectrum Disorders is funded by DFG - German Research and R. T. Schultz. Using computerized games to teach face
Foundation (AR 892/1-1, DE 620/18-1, RA 764/4-1). Thanks to recognition skills to children with autism spectrum
Kai Goetz for the graphics. disorder: the let’s face it! program. J Child Psychol
Psychiatry, 51(8):944–952, 2010.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 155 |
 The SARA Project: An Interactive Sandbox for Research on Autism
D. Arellano | V. Helzle | U. M. Schaller | R. Rauh | M. Spicker | O. Deussen
[14] E. Whyte, J. Smyth, and S. Scherf. Designing serious game
interventions for individuals with autism. Journal of Autism
and Developmental Disorders, 2014.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 156 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador

Model for Analysis of Serious Games for Literacy in Deaf

Children from a User Experience Approach

Sandra Cano Jaime Muñoz Arteaga César A. Collazos

Universidad del Cauca Univ. Autónoma de Aguascalientes, Universidad del Cauca
Grupo IDIS México Grupo IDIS
[email protected] [email protected] [email protected]

Viviana Bustos Amador

Univ. Autónoma de Aguascalientes,
México
[email protected]

ABSTRACT Visualization is a valuable communications tool that involves a

Learning to read and write, or acquiring skills in literacy, is a variety of graphical aspects that can be used to represent
basic educational need. Deaf children traditionally face greater knowledge in the teaching of pronunciation [46]. Various research
challenges and encounter difficulties particular to their abilities. works [23] [24] [28] [37] consider video games as an important
Their inclusion today in regular schooling places much educational tool able to foster a range of skills and development
responsibility for their success on the support teachers designated capabilities in learning in children. Other studies meanwhile [25]
to provide them with the assistance they need. More and more [36] have applied serious games to literacy as a form of support
however, ICT is being used in the field of education. Video that enables children to improve their reading and writing skills,
Games offer a first class option for generating meaningful while motivating their learning. Literacy difficulties arise as a
learning experiences, and serious games in particular are common special educational need in children most commonly by
employed in class to motivate learning. The design of serious means of dyslexia 4 and dysgraphia5. In deaf children this
games to meet the needs of teachers where deaf children are difficulty represents a real challenge, since through their hearing
struggling to master literacy would therefore seem to offer a loss they have great problems in the acquisition of written
hugely important contribution. Based on the User Experience language and reading comprehension, resulting in limitations in
approach, a model is therefore put forward for the design of such language proficiency at the lexical, syntactic and semantic levels
a serious game. The model assesses the quality of the game and [27].
the user experience from others proposed games and examines the Recent studies have shown that games have positive effects on
aspects that are important for creating a game in this case that can visual skills and working memory skills [34] [35]. Games are
help teachers as support for the learning of both deaf and hearing considered serious when they strikes a balance between the
children in the acquisition of reading and writing skills. educational and simply entertainment [20], so the challenge for a
new serious game is to meet its objectives based on these two
Categories and Descriptors scenarios.
H5.m. [Information interfaces and presentation] (e.g HCI):
User Interfaces Before defining the scenario of a serious game the different
components that can influence user experience must be analyzed
General terms to understand how to create a positive experience for the user
Human Factors, Design, User Experience. when interacting with a serious game in a specific context, and in
turn, which factors allow selection of the qualities of serious
Keywords games to match user aspects. User eXperience (UX) can be
Serious games, Tablet App, Deaf Children, User Experience. User understood as the set of sensations and emotions that occur in a
Profile. user on interacting with the serious game. Arthippainen & Tähti
[10 define it as the experience that a person lives when he or she
1. INTRODUCTION interacts with a product in particular circumstances. Hassenzah &
Information Technologies are transforming education. Serious Tractinsky [13] meanwhile define it as a consequence of the
games are being used as an alternative tool for generating internal state of the user (biases, emotions, motivation, needs and
meaningful learning experiences. The video game now plays a mood), some characteristics of the system (complexity,
fundamental role in psychological development, learning, and in usefulness, usability, functionality, among others) and of the
the emotional and social aspects in children. Video games help
stimulate cognitive skills such as visual attention span, decision-
4
making, visio-motor coordination and reasoning, so that these Language difficulties that render proper comprehension unattainable
5
games, could quite well support the development of children’s This is used to design the disorder of writing that affects the shape or content and is
learning in many different contexts. seen in children who do not show intellectual, neurological, sensorial, motor,
affective or social problems.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 157 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador
context in which the interaction takes place. Elsewhere, Dillon communication channel to convey the extraction of meaning of a
[26] proposed a simple model to define user experience as the concept. As such, teachers require play-based tools that can
sum of three levels: action, that which the user carries out; result, motivate children in their learning and that can integrate them
what the user gets; and emotion (what the user feels). Using these within their educational planning.
definitions it is important to consider such aspects as graphical
interface, challenges, goals, rules, etc., which are able to convey a Besides, the serious games currently available do not meet all the
positive user experience, but which in turn may vary according to needs of the deaf child with learning difficulties in literacy, since
the aspects of the user and the context of use. they are designed for hearing children with learning difficulties in
literacy and have not taken into account that children with
This paper presents a model for designing a serious game from a cognitive problems, such as ADHD 6, cognitive deficit, among
user experience perspective for a case study applied to deaf others. As such, these cognitive problems that may arise in deaf
children experiencing difficulty in literacy learning. The proposal children ought to be considered in assessing a set of aspects in the
of the model is that it can provide support for identifying the serious game. However, the deaf child by default has problems in
needs of the child with auditory discapacity and the teacher cognitive development and to have some other deficiency can
starting with the analysis of a set of existing serious games that result in a greater challenge to learning literacy, which indicates
are adapted for the use context in reading and writing. This that he or she may become frustrated and demotivated more
analysis includes identifying aspects to be taken into account with quickly.
the aim of creating a positive experience in literacy learning, to
measure the quality of the game as educational material. The 3. SERIOUS GAMES IN LITERACY
needs identified will be useful in extracting the requirements for LEARNING
the production of a serious game applied to deaf children in Games promote the overall development of the child, as they
learning literacy. Section 2 provides a brief description of the allow the testing of rules, capabilities or limitations that can then
problem faced by teachers in literacy learning for deaf children. In be explored in real situations. A number of researchers in the field
section 3, reference is made to the importance of the inclusion of of pedagogy and psychology [1] [2] have proposed serious games
serious games as educational material. Section 4 describes the that exist in the market and have studied how these games can be
user experience model in the context of literacy learning. In seriously used in classroom to support the educational process.
section 5, the user experience model in the context of literacy One of the factors contributing to the success of these games for
learning for deaf children in a school in Mexico. In Section 6, children is the motivation component, which is a crucial condition
attention is drawn to research related to lines of user experience for the success of the children in their tasks, where they face
research and literacy using serious games. Finally, in Section 7 a challenges of learning while they play.
number of conclusions and future work areas are presented.
A hearing child develops language skills through sounds (sound-
2. PROBLEM OUTLINE letter-word-meaning), which relates letter to sound to find the
lexical component. A deaf child cannot depend on the same
The program run in Mexico by the Regular Education Support educational strategy, learning instead only sound-word. Schools
Services Unit (USAER from the Spanish acronym) is a system of are choosing pedagogical models that fit the characteristics of the
support services for regular schools, intended to encourage access child as well as aids that promote literacy learning in deaf children
and improve the performance of students with disabilities and [27].
outstanding skills integrated into regular school. In USAER [45],
Serious games allow their participants to experiment, learn from
the role of the support teachers is to educationally integrate
their mistakes and gain experience safely. Serious games today
children with Special Educational Needs (SEN), with or without
are actively applied in the area of education, with the aim of
disabilities. Support of teachers use different strategies in literacy
learning, understanding and communicating a specific activity. It
learning, and interest has grown in combining their activities with
is therefore possible to make the most of the situation and enhance
ICT as another way for children to improve and strengthen their
the use of serious games in order to help in encouraging the
skills in reading and writing. Management of ICT therefore
development of reading and writing skills in deaf children, and
requires new skills from support teachers to link the various tools
support teachers can in turn integrate them as educational material
within the learning strategies.
in their planning.
Today there still are many complications for deaf children as their
A number of researchers have conducted studies related to user
main communication channel is sign language and there is a need
experience that have suggested ways to make the game more fun,
to teach them how to communicate through reading and writing to
increasing the motivation of the player. In 2008, [8] proposed four
be able to play their part in the wider society. They have
keys for user experience. These are: analyze what the player likes
difficulties learning spoken languages, and this becomes a
to play, heighten emotions without making use of the story,
challenge for reading comprehension and written language.
analyze the most successful games, and apply psychological
Written language following after sign language is the most
techniques for developing the personality of the characters in the
accessible way to get the information to enable them to be aware
games. Using the four keys, four types of user experience based
of what is happening around them.
on emotions are proposed. These are: Hard fun, Easy Fun, Altered
Deaf children further, fail to develop skills at the same pace as a States, and People Factor. In [9] a set of aspects were formulated
hearing child, making it difficult to identify problems in the that could help to improve motivation in the user experience.
development of their basic cognitive skills and this may affect
progress in the acquisition of learning. However, they have a 6
better developed visual attention capacity [34], [35], so teachers Attention Deficit Hyperactivity Disorder is a developmental neuropsychiatric
disorder in which there are significant problems with executive functions (e.g.,
are using tools accompanied by images and texts as a attention control and inhibitory control).

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 158 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador
These are: challenges, curiosity, control, and fantasy. In 2009,
[43] using existing frameworks for designing educational games
proposed a framework for designing educational games, taking
account of the pedagogical side and the content of the model. In
2010, [44] proposed a user-centered design method for serious
games that involves three aspects: interactive design, evaluation,
and validation that ought to be taken into account and are
necessary in the different stages of design of the serious game.
The model proposed has the following phases: research,
validation, design, and implementation.
Elsewhere, [5] made a Tablet app for children with dyslexia,
where the exercises are tailored to the specific difficulties of the
player. Tests were also performed with children where they
confirmed that the game is fun and more attractive compared to
traditional activities. Further, in [27] the authors outline problems
that teachers face in teaching literacy to deaf children, which
involve pedagogical models that fit the characteristics of the child
and in turn display all the aids that favor literacy learning in deaf
children.

4. CONCEPTUAL MODEL
A model for the design a serious game is proposed, which
involves user experience in evaluating a set of serious games for
learning literacy. In the evaluation, aspects are identified that meet Figure 1. Model for the analysis of serious games to support
the needs required for the design of a serious game for literacy literacy learning for deaf children, from a UX approach.
learning with deaf children are identified. The model (Figure 1) is Using research techniques (questionnaires and observation)
composed of three elements. These are User, Learning strategies, aspects are identified that can influence the experience with the
serious games. The proposed model considers user responses serious game in its context of use, such as literacy. These aspects
obtained from hedonic attributes [14], pragmatic attributes [14] on are: profile of deaf child, serious game-child interaction, quality
interacting with the serious games, game responses on managing attributes of serious game, usefulness, hedonic qualities
to communicate/transmit literacy learning, utility of serious games (emotions, needs and motivation), and learning.
as educational material and adaptation of learning strategies
A description of elements that include the model:
incorporated in serious games according to the characteristics of
user. Research methods (interviews, questionnaire and 4.1 User
observation) are also applied with each specific game for each The user is the person who will be using the game as a support for
child according to needs and level of literacy learning found. The learning, so it is important to understand the needs of the student
model identifies user characteristics to determine which game can as well as those of the teacher. At this stage, therefore, it is
adapt according to the user attributes in: learning level, age, genre advisable to apply research techniques in order to identify
among others. Learning strategies are then established, and in a characteristics particular to each including for example
further stage a set of serious games aimed at learning of literacy is concentration span, ability to express an experience, gender
selected and different aspects of user experience are evaluated, by differences, experience of technology, cognitive abilities, and
which means information is obtained and problems identified to emotions. At this stage it is important to know the user profile to
be transformed into new pedagogical objectives/entertainment to identify issues that could affect their experience of using the
be accomplished which are reflected in a final stage responsible game. Aspects defined in the user profile are name, age, gender,
for designing a serious game. The methods are applied in order to academic year, skills/abilities, physical/cognitive disabilities,
obtain information about the user, their needs and so translate learning level, emotion and motivation. Estas características se
them into indispensable requirements for the use of the game. seleccionaron a partir de un estudio previo de investigaciones
When speaking about a game, it is not just how useful or usable it realizadas[47][48][49] donde han propuesto un conjunto de
can be [14]. Other factors can be included, such as the capacity for características para la construcción de un perfil del usuario en un
recreation of a virtual world, story, character design, how the contexto académico.
player feels, the rules of the game, and so on. The experience of a Table 1. Aspects for defining user profile.
user can be widened compared to a traditional interactive system,
which obliges greater reflection on properties that identify and Aspects Description
measure these experiences.
Involves important data that can act as a
support to define the needs and learning
Personal data
level. These are: Name, Age, Gender, and
Academic year.

Discover skills that can be taken into

Skills / Abilities
account in establishing learning strategies.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 159 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador

Involves physical disability, which is 4.3 Serious Games

hearing-related, such as hearing loss In this stage, it is proposed to use a set of serious games aimed at
Disability (Mild, Moderate, Moderate-Severe, learning to read and write, according to the needs of each child.
(Physical/Cognitive) Severe, and Profound), but in turn is To select whether a game has a serious approach and can be
related to a cognitive impairment that can applied to learning literacy the following attributes should be
occur in children. considered: objectives to be achieved, literacy level, and previous
knowledge. In this stage it is proposed to conduct for each game a
This is related to literacy levels, these are: cognitive walkthrough test [29] with the help of the teacher to
Pre-syllabic, Pre-syllabic-Syllabic, select one or more serious games that can support the process of
Learning level
Syllabic, Syllabic-Alphabet, and literacy learning and that the teacher can integrate in the learning
Alphabet. activities for deaf children.
A set of attributes (Table 2) is proposed in order to analyze the
These are the reactions that can be quality of the serious games to determine if they meet both
detected in the children on interacting educational and entertainment purposes, i.e. if there is a balance
Emotion
with the technology to carry out their between motivation and learning. Therefore, if it is desired to
educational activities. keep the child motivated, he or she should be informed of the
progress of the activity (Feedback), and there must be a balance
These comprise determined actions the between the levels of difficulty (Challenges) with respect to the
Motivation child carries out and persists with them level of ability of the child to achieve the objectives. Attributes
until complete. that may affect the ease of use of a game are also identified,
The user profile is therefore a representation of a set of attributes because for a deaf child the result may be different from that of a
that describe a person, in this case the deaf child in his or her role hearing child.
as user on interacting with the games, so that they are adapted to
the characteristics of the child and can be associated with the Table 2. Attributes for measuring the quality of serious games
learning strategies. in learning literacy.
Attribute Description
In turn, the user is an input into the model since he or she interacts
with the product and a number of results are obtained, such as the Challenges Levels of difficulty in reaching targets.
emotions the game might inspire. These emotions may be Challenges can add to the fun and the
negative (frustration, stress, anger or boredom) or positive (joy, competition [19]. This attribute is related to the
interest or fun). Using an analysis of the processes defined by
difficulty level, and in turn will lead to
[28], certain factors are included to be taken into account, such as achieving the goal.
anticipation, expectations that the user may afford the product, the
connection as the first experience of using the product, the first Rules/goals The rules are the objectives of the game
reactions to the interaction and the interpretation that includes the established in order to win, which is very
sense that the user gives to his or her interaction with the product. important in the learning part. There are three
types of rules. System rules (inherent in the
4.2 Learning strategies game), procedural rules (actions in the game to
Support teachers use strategies so that the child acquires
regulate behavior), imported rules (rules
knowledge and skills related to literacy. Within the strategies, the oriented to real world situations).
support teachers establish a set of support resources and
educational techniques, included among these ITC mediated by Feedback Display each of the actions carried out. The
games. The type of strategies used by the support teacher are indications can be visual, audible or tactile. [19]
support strategies, whose interest is to establish motivation, focus
attention and concentration, and manage the time for interacting Assessment This measure compares the performance
with the game. between players. This can be identified as a
The teacher performs activities that depend on the level the child measure of achievement. [20]
is at in literacy learning. These activities depend on the fulfillment
of educational goals to be achieved with the child. For this reason, Surprise Random elements within the game [21]
the support teacher develops educational strategies that may
involve use of a Tablet in activities for each child. Using different Interaction Adaptability and manipulation of the game,
educational games, the experience that influences the teacher and where the game changes in response to player
the child are analyzed. Educational strategies where serious games actions [22].
are incorporated like support material for the literacy learning are
developed. The support teacher carries out these strategies, he is Fantasy The environment through which the game,
who decides which game is best suited to form part of the activity. setting, and characters are developed, which
Such a selection process takes into account the level of literacy of involves the player [19].
the deaf child, expected knowledge, and the child’s profile. It can
also be said that this pedagogical scenario should be adapted to Usefulness and usability influence the experience of the product.
the profile of each child and the serious game must in turn For this reason, it is sought to evaluate each proposed game using
establish a communication with the pedagogical scenario in such a a usability test, where such attributes are taken into account as
way that it enables the child to acquire learning. effectiveness, efficiency, satisfaction, emotions, and learning.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 160 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador

4.4 Requirements
On analyzing all the aspects that influence user experience,
information was obtained on different factors such as the profile
of the deaf child, educational objectives, learning scenarios,
entertainment strategies, etc., that will be useful for proposing the
design of a serious game to help improve literacy learning in deaf
children.

5. CASE STUDY
The case study was carried out in Mexico in the USAER school in
Aguascalientes, a school that enables children with hearing
impairment to be included in regular secondary schooling. The Figure 2. Results of the inquiry evaluation to determine the
deaf children here are aged from 12 to 15 and have problems in level of skills/abilities, hearing loss, and literacy, learning
learning literacy. Using research techniques, including activities, literacy for each child.
such as observations, interviews with the teachers, and
questionnaires, information is obtained on every child in order to Of the children assessed, four are in the alphabetic literacy level
discover the needs and level of literacy learning. Using the and the other two at syllabic level. These deaf children are at a
proposed model comprising four elements - user, learning low academic literacy level compared to a hearing child and
strategies, serious games and requirements - a brief description is according to their academic grade. Therefore, they make less use
made of each element applied to the case study. of the phonological code because of the difficulty of uniting
sounds with letters. They are able to use this code when graphic
5.1 User information is presented, though never reaching the levels of a
hearing child due to the fact that they are faced with the difficult
An evaluation to discover the profile of the deaf child was and complex task of memorizing vocabulary and if they come
performed on the Likert scale with a score of 1-5. It was decided across a new word they will not understand its meaning until it is
to evaluate the following aspects: attention span, visual displayed graphically.
perception, visual memory, comprehension, spatial orientation,
visual discrimination, social environment, learning level and From the information obtained, a user profile is built for each
degree of hearing loss. Figure 2 shows the results of an inquiry child, paying attention to the attributes proposed in Table 1. The
evaluation conducted with six children with hearing impairment. attributes, obtained from the inquiry test include personal data,
Five are boys in high school and one is a girl in elementary skills/abilities, learning level and physical/cognitive disability.
school. Of the children assessed, 4 of them (Child 2, Child 3, The attributes of motivation and emotion meanwhile are obtained
Child 4 and Child 6) have a profound level of hearing loss (91- by observing each child as they interact with the serious games.
119db), so that their main channel of communication is sign The emotions shown by the children were positive, as can be seen
language; Child 1 has a severe level of hearing loss (71-90db) and in Figure 4, however Child 1 with his ADHD suffered some
uses a hearing aid; while Child5 has a moderate level of hearing frustrating moments when he did not receive an immediate
loss (41-55db). Both Child 1 and Child 5 use speech as their main response to his actions or when he could not complete actions
means of communication. carried out with a game because it had a high level of difficulty.
The support teacher must involve the game in their planning as
The aspects evaluated indicate that the children have skills for educational material, so it is necessary to adjust their educational
acquiring learning but that their hearing impairment makes strategies in order to integrate the game in their activities as well
obtaining any level of learning difficult. Furthermore, it is as to assess the learning impact of the incorporation of the
observed that Child1 has low skill levels since he is unable to technology. It is further important to take into account that the
distinguish very much and this is because, in addition to having a learning material must be supported by a visual communication
hearing impairment, he has a disability known as ADHD able to capture the attention, in order to motivate learning.
(Attention Deficit Hyperactivity Disorder), which makes learning
slower compared with the other children. 5.2 Learning strategies
The girl meanwhile, Child 6, has low skill levels in such aspects The support teacher carries out a planning strategy around the
as spatial orientation, comprehension and visual memory. She is process and techniques to be used with each child, where every
unaware of many semantic concepts, leaving her in a low level of strategy has educational goals to be reached. This component
literacy compared to other children in the school. In the takes place in the context of their use for teaching literacy, so that
observations obtained, it is noted that the support teachers use what is sought here is that, for each child, one or more serious
visual aids in their educational strategies to convey the different games proposed is suitable to meet the level of literacy and skills.
concepts that are associated in the meaning of a word or sentence. Support teacher must involve the game in their planning as
visual. educational material, so it is necessary to adjust their educational
strategies in order to integrate it in their activities as well as to
assess the learning impact of incorporating of the technology. It is
further important to take into account that the learning material
must be supported by a visual communication able to capture the
attention, in order to motivate learning.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 161 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador

5.3 Serious Games defined in Table 2 are applied, shown in Figure 3. It can thus be
seen that the fantasy attribute was not achievable in any of the five
A search performed for games for Tablet applied to literacy games, since there is no character that the children can identify
learning produced 20 games able to be applied to such learning. themselves with, nor a story that integrates the activities. Also the
The games proposed are for use only with Tablet, since children activities available in the games are not random, i.e. after playing
showed a better response, being more motivated to work with a the game several times it can become very repetitive and can
tablet than with a computer. cause the child to become demotivated. It can also be noted that
some of the games have no difficulty levels, so that when faced a
Using the technique of cognitive walkthrough for each proposed
very high level challenge the child wants to stop playing and
game, the game’s capacity to support learning was evaluated with
when it is too low, it results in boredom.
the help of the teacher. The evaluation enabled five games to be
selected for incorporating into the educational activities for each
child. The games proposed are described in Table 3. These
operate on Tablet Android and are applied to the deaf children in
order to assess pragmatic attributes such as utility and usability
and hedonic attributes such as emotions and motivation. For each
child a usability test was applied to assess such aspects as
effectiveness, efficiency, satisfaction, emotions and learning.
Table 3. Games selected for teaching reading and writing.
Game Description

ABC Español Educational app for learning the meaning of

different concepts, where an image is
associated with the text [38]. Figure 3. Results of assessing each attribute defined in Table
2, applied to each game.
To assess the experience of children on interacting with the
games, a usability test was conducted using the observation
Learning to read technique. The game proposed for carrying out the analysis is
and write Educational app that helps in learning early
ABC Spanish [38], since it is used in all the literacy levels. The
words, as well as in understanding the basis
objective of the game is to convey the meaning of words, and that
for intuitively deciphering the meaning of
is what the children need to improve their reading comprehension.
new words [39].
Figure 4 shows the results produced using an instrument to
determine the degree of acceptance of the game for the child, as
well as its success as a learning tool. The test consists of a total of
50 questions, structured based on the QUIS [30], USE [31], GEQ
Easy Fun Spanish Educational app for learning Spanish [32] and UEQ [33] questionnaires, taking into account the
language concepts. This is contained in a effectiveness, efficiency, satisfaction, emotions and learning
range of activities as progress is made, with attributes.
the aim of learning and memorizing the
concepts [40]. The results obtained with the ABC Spanish game (Figure 4) show
that the game is helpful in learning to read and write, but offers no
feedback when the user fails to type the word correctly or does not
Communication App offering an alternative or augmentative know the word. In both cases it is not possible to proceed with the
book communication system based on next word. Neither does it include an assessment within the
pictograms. Works with Fitzgerald keys, activities, which robs the teacher of any quantitative support for
where grammatical categories are the learning level. It was also noted that the vocabulary provided
associated with colors [41]. by the games contains words that are different to Mexican Spanish
words for example for peach, tennis shoes, jacket, t-shirts among
others meaning that some activities contained in the game are left
Pictogram Augmentative and alternative out.
communication app that comes
accompanied by images and text [42].

The five games to be integrated into the learning strategies the

support teacher comprise: three games used in the Syllabic and
Alphabet levels of literacy (ABC Spanish, Easy Fun Spanish, and
Pictogram) where the goal aims at teaching concepts in extracting
meaning, and two games oriented to the Syllabic (Learn to read)
and Alphabet (Communication Book) levels, beginning with an
analysis of each of the proposed games where the attributes

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 162 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador
their learning. Support teachers currently use FitzGerald keys [6]
in literacy learning for deaf children. This is a visual tool for
organizing oral and written language. It is used to teach, practice
and correct syntactic structures of the oral and written language.
When the review of games for Tablet was conducted, few games
were found that applied the keys. Of those selected,
Communication Book does so (Table 2). However it also brings
problems in the vocabulary, containing many concepts that are not
the same in Mexican Spanish, while the structuring of categories
is not very understandable for a child who is learning literacy. It
also fails to keep track of the activities and achievements.
The proposed model therefore leads to identifying the needs and
requirements of support teachers in learning literacy for deaf
Figure 4. Usability test applied to the ABC Spanish game. children. Already defined at this point are the following aspects to
The learning attribute is related to the motivation attribute, consider when designing a serious game (Figure 5) that meets the
implying that if the child experiences a low level of learning, the needs of teachers for a specified use context. Figure 5 shows the
game should convey the concepts (objectives/rules) clearly proposal for a non-functional prototype of the game, based on the
enough to learn without effort so that the child feels motivated previous stages and integrating FitzGerald keys, where some
and has the confidence to face the various challenges posed by the features that integrate both the educational and entertainment
game. In Figure 4, a value of 43% achievement is obtained, with scenarios will be met, such as challenges (difficulty levels),
respect to questions relating to assessment of the learning surprise, evaluation, fantasy, learning, feedback and
attribute. This is because when carrying out the activity of rules/objectives.
completing a word through writing, the game does not allow for
steady progress if the activity is not performed correctly, nor offer
any help to progress. This means that if the user cannot come up
with the correct spelling of the word, they cannot pass to the next
activity, resulting in a frustration in the child that may diminish
the motivation to continue playing.
Easy Fun Spanish, on the other hand, has the same pedagogical
goal as ABC Spanish, but contains more vocabulary and
activities, leading support teachers to use it when they need to
further extend concepts of meaning and reinforce more activities
that enable the child to memorize the spelling of words, i.e. when
they want to increase the level of difficulty in terms of linguistic
competence. There is also Communication Book, which applies a
methodology called FitzGerald keys. It is used for the teaching of
language and grammar rules with deaf children. Through a set of
categories, the child has to learn to string together words to form a
sentence, beginning with whom, how, where, or when. The results Figure 5. Proposal for a graphical interface for a serious game
in Figure 3 show that in the challenges, surprise, and in literacy learning for deaf children according the
rules/objectives attributes, the game has low values compared to requirements.
the other games. This is because it does not assess the child or
offer challenges to solve the activities. It functions more as a
6. CONCLUSIONS AND FUTURE WORK
communication board, but no responses are obtained as to whether The process of identifying words by deaf children is slow and
the action was right or wrong. laborious, so they often make mistakes, such as substitution,
omission and distortion. With the inclusion of ICT through video
5.4 Requirements games in literacy activities, the desire is to motivate these children
After obtaining the results of the usability testing for each serious in their learning and in turn enable them to develop other skills.
game and carrying out an analysis of it, it can be said that needs The video games proposed were shown to be an alternative for
were identified and using these needs a set of requirements has literacy learning for deaf children, where teachers can incorporate
been generated that will be used as a starting point for developing information technology as an educational material.
a serious game in literacy learning. The analyzed games tend to be Furthermore, it can be seen that the games included in activities
more educational than entertaining. The need therefore arises to do not each perform a single function, and that in some cases they
design a serious game taking into account the characteristics of fail to meet all the needs required by the support teacher. The
deaf children, as well as aspects to be included in a serious game analysis model for serious games for learning literacy is useful for
(Table 2), which in turn enables the generation of challenges in identifying aspects of user experience, content and product,
the children, with the aim of motivating them to achieve the through which it is possible to analyze in detail different factors
educational objectives. It also requires the game to assess the that can be considered for the proposal, as future work, of a game
child in each one of the activities. This can help the support that can be adaptable to the child profile and which in turn enables
teachers keep track of the activities and progress of the children in

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 163 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador
the assessment of the activity and benefit from a constant [11] Gerken Jens, Jetter Hans-Christian. 2006. A Simplified
feedback of the actions carried out on interacting with the game. Model of User Experience for Practical Application.
Proceedings 2nd COST294-MAUSE International Open
The majority of the serious games assessed aim to enrich the Workshop User eXperience- Towards a unified vie. 106-111.
meaning of concepts for the child, which is very helpful for them
to have a better understanding of reading. However, they need [12] Laugwitz B., Schrepp M., Held T. 2008. Construction and
more games that can serve as support to complement the evaluation of a user experience questionnaire. In Holzinger,
methodology followed for the child to expand their repertoire of A (Ed): USAB. 63-76.
syntactic structures and learn the logical order of a sentence. It is [13] Hassenzahl M. and Tractinsky N. 2006. User Experience - a
also important to note that support teachers use only one or two Research Agenda. Behaviour and Information Technology,
games as educational material, so that it is not ideal to offer a 2582. 91-97.
game for each specific objective to be fulfilled. They therefore
[14] Hassenzahl M. 2007. The Hedonic/Pragmatic Model of User
need a game that integrates the activities they undertake following
the FitzGerald keys and which in turn allows them to assess the Experience. Communication in International Conference HCI
UK.10-14.
learning through different activities that help increase the
semantic repertoire, since a single activity does not guarantee [15] Mahlke S. 2007. User experience: Usability aesthetics and
dominion of a subject, just as the number of activities required is emotions in human-technology interaction. On International
not the same for each child. Conference HCI, Lancaster, UK. 26-30.
The analysis model can deemed to have been useful for [16] Roto V. 2007. User experience from product creation
identifying the relevant needs both of support teachers and deaf perspective. On International Conference HCI, Lancaster,
children. The participation of children and their support teachers UK. 31-34.
has enabled pedagogical aspects to be identified: learning curve, [17] Kort J. Vermeeren A., Fokker J. E. 2007. Conceptualizing
clear objectives, a clear progression, assessment, and support, and Measuring UX. On International Conference HCI,
which are important aspects to consider in designing a game. Lancaster, UK. 57-64.
7. REFERENCES [18] Robert J-M., Lesage A. 2011. From Usability to user
[1] González Carina, Blanco Francisco. 2008. Emociones con experience with user interfaces (chp 14). In G.A. Boy (Ed).
Videojuegos : incrementando la motivación para el Handbook of Human-Computer Interaction. A human-
aprendizaje, Revista Electrónica teoría de la educación. 9,3, centered design approach. 303-320.
69-92. [19] Garris Rosemary, Ahlers Robert, Driskell James E. 2002.
[2] Mejía Zuluaga Cesar Augusto. 2012. "Videojuegos y Games, motivation and learning: A research and practice
Evaluación Cognitiva" Las Tecnologías De La Información model. Simulation & Gaming. 441-467.
Y La Comunicación En Contextos Educativos: Nuevos [20] Chen S. Michael D. 2008. Proof of learning: Assessment in
Escenarios De Aprendizaje. 223-240. Serious Games Gamasutra: CMP Media LLC.
[3] Troyer de Olga, Janssens Erik. 2014. Supporting the [21] Owen M. 2004. An anatomy of games: A discussion paper.
requirement analysis phase for the development of serious Futurelab.
games for children. International Journal of Child-Computer [22] Prensky, M. 2001. Fun, play and games: What makes games
Interaction. 2, 76-84. engagins. In M. Prensky (Ed), Digital game-based learning,
[4] Hauge J.B, Duin H., Oliveira M, Thoben K.D. 2006. User 16-47.
Requirements Analysis for Educational Games in [23] Hakulinen Lasse. 2011. Using Serious Games in Computer
Manufacturing. Proceedings 12 th International Conference Science Education. Proceedings of the 11th Koli Calling
on Concurrent Enterprising. 1-8. International Conference on Computing Education Research,
[5] Rello Luz, Bayarri Clara, Górriz Azuki. 2013. Dyslexia 83-88.
exercises on my tablet are more fun. Proceedings of the 10th [24] Hussan Aarij Mahmood, Sehaba Karim, Mille Alain. 2011.
International Cross-Disciplinary Conference on Web Helping children with cognitive disabilities through serious
Accesibility. 1-2. games: project CLES. Proceedings of the 13th International
[6] Fitzgeral E. 1954. Straight Language for the Deaf. ACM SIGACCESS conference on Computers and
Washington D.C. Volta Bureau. accessibility. 251-252.
[7] José Luis González. 2010. Jugabilidad: caracterización de la [25] Wu Jianhua, Hao Yang Harrison, He Xiuling. 2012. A
experiencia del jugador en videojuegos. Tesis doctoral, preliminary study on developing computer games for
Universidad de Granada. 1- 454. information literacy education. Proceedings of the 4th
[8] Lazzaro M. 2008. The Four Fun Key. G.U Experience. International Conference on Internet Multimedia Computing
Morgan Kaufmann. and Service. 223-226.
[9] Lepper M, Malone T. 1987. Intrinsic motivation and [26] Dillon A. 2001. Beyond Usability: Process, Outcome and
instructional effectives in computer based education. Snow. Affect in human computer interactions. Lazerow Lecture
2001, at the Faculty of Information Studies, University of
[10] Arhippainen L, Tähti M. 2003. Empirical Evaluation of User Toronto.
Experience in Two Adaptive Mobile Application Prototypes.
2nd International Conference on Mobile and Ubiquitous
Multimedia, Norrköping. 27-34.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 164 |
 Model for Analysis of Serious Games for Literacy in Deaf Children from a User Experience Approach
S. Cano | J. Muñoz Arteaga | C. A. Collazos | V. B. Amador
[27] Ruiz Linares Esther. 2009. El aprendizaje de la lectoesctirua [38] https://play.google.com/store/apps/details?id=com.gonliapps.
en los niños y niñas sordos. Revista digital de contenidos learnspanish&hl=es, visitado el 24 de marzo del 2015.
educativos. 2, 1-8. [39] https://play.google.com/store/apps/details?id=air.APRENDE
[28] McCarthy John, Wright Peter. 2004. Tecnology as ALEER&hl=en, visitado el 24 de marzo del 2015.
Experience. Interactions.11, 5, 42-43. [40] https://play.google.com/store/apps/details?id=com.funeasyle
[29] John B. E, Packer H. 1995. Learning and using the cognitive arn.spanish&hl=en, visitado el 24 de marzo del 2015.
walkthrough method: A case study approach. In Proceedings [41] https://play.google.com/store/apps/details?id=es.geeknekodr
of CHI'95. 429-436. oid.librodecomunicacion&hl=es, vistiado el 24 de marzo del
[30] Chin John, Virginia Diehl, Kent Norman. 1988. 2015.
Development of an Instrument Measuring User Satisfaction [42] https://play.google.com/store/apps/details?id=es.pictogramas.
of the Human-Computer Interface. Proceedings of ACM pictogramaslite, visitado el 24 de marzo del 2015.
CHI'88 Conference on Human Factors in Computing
Systems. 213-218. [43] Ibrahim Roslina, Jaafar Azizah. 2009. Educational Games
(EG) Design Framework: Combination of Game Design,
[31] Lund Arnold M. 2001. Measuring Usability with the USE Pedagogy and Content Modeling. International Conference
Questionnaire. STC Usability SIG Newsletter. on Electrical Engineering and Informatics. 293- 298.
[32] W. A Ijsselsteijn, Y.A.W de Kort, K Poels. 2008. The Game [44] Holloway A. & Kurniawan S. 2010. Human-Centered Design
Experience Questionnaire: Development of a self-report Method for Serious Games: A Bridge Across Disciplines.
measure to assess the psychological impact of digital games. University of California. 1-10.
Manuscript in preparation.
[45] Esther Adriana Maldonado Cisneros. 2009. La práctica
[33] Rauschenberger M., Schrepp M, Olschner S., docente del maestro de apoyo. X Congreso Nacional de
Thomaschewski J., Cota M.P. 2012. Measurement of User Investigación Educativa. 1-10.
Experience. A Spanish Language version of the User
Experience Questionnaire (UEQ). In: Rocha A., Calvo- [46] Cano Sandra, Álvarez Gloria Inés, Collazos César. 2014.
Manzano J.A., Reis L.P & Cota M.P (Eds), Sistemas y Visualization Model for Learning of Pronunciation with an
Tecnologías de Información- Actas de la 7a conferencia Approach form Human Computer Interaction. Proceedings of
ibérica de Sistemas y Tecnologías de la Información. the XV International Conference on Human Computer
Interaction. 1-2.
[34] Grigonis A. Narkevicienè V. 2010. Deaf Children's Visual
Recall and Its Development in School Age. Vytauro Didziojo [47] Alex Frazer, Alejandra Recio-Saucedo, Lester Gilbert, Gary
Universitetas K, 52. Wills. 2013. Profiling the Educational Value of Computer
Games. Interaction Design and Architecture(s) Journal –
[35] Marschark M, Everhart V.S. 1999. Problem-solving by deaf IxD&A, 19. 9-27.
and hearing students: twenty questions. Deafness Educ int.
65-82. [48] Cristina Carmona Márquez, Gladys Castillo Jordán, Eva
Millán Valldeperas. 2009. Modelo Bayesiano del Alumno
[36] Cagatay M., Ege P, Tpkdemir G, Cagiltay N.E. 2012. A basado en el Estilo de Aprendizaje y las Preferencias. Revista
serious game for speech disorder children therapy. Health IEEE-RITA 4(2). 139-146.
Informatics and Bioinformatics (HIBIT) 7th International
Symposium on. 19-22. [49] Héctor M. González G, Nestor D. Duque M. Demetrio A.
Ovalle. 2008. Modelo del Estudiante para Sistemas
[37] Lissi M, Grau V, Raglianti M, Salinas M, Torres M. Adaptativos de Educación Virtual. Revista Avances en
2001.Adquisición de la Lectoescritura en Niños Sordos: Una Sistemas e Informática. 5(1). 199-206.
Visión desde los profesores en Chile. Pyskhe. 10, 1, 35-48.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 165 |
 Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje
J. Muñoz Arteaga | D. Morales Hndz | R. Mendoza | A. Torres Soto

Uso de Aplicaciones Interactivas para Apoyo a la Escritura

en Niños con Problemas de Aprendizaje
Jaime Muñoz Arteaga Dulce Morales Hndz Ricardo Mendoza Carina S. Gonzalez
Univ. Autónoma de Univ. Autónoma de Ins. Tecnológico de Univ. de la Laguna
Aguascalientes Aguascalientes Aguascalientes Avda. Astrofísico Fco.
Av. Universidad. #904 Av. Universidad. #904 Av. Adolfo López #1801 Sánchez s/n.
Aguascalientes, México Aguascalientes, México Aguascalientes, México La laguna,Tenerife, España
[email protected] [email protected] mendozagric@yahoo. [email protected]
com.mx
RESUMEN. Para sacar provecho de esta diversificación de funciones y
En la actualidad, los problemas de aprendizaje en la población aprovechando sus componentes físicos y lógicos, se propone el
latinoamericana se han incrementado a un ritmo considerable, tal uso dispositivos móviles para apoyar la educación en niños que
es el caso de México, donde se han realizado diversas estrategias presenten problemas de aprendizaje.
para mitigar la diferencia que exista entre los alumnos regulares y Los problemas de aprendizaje se presentan cuando un alumno
aquellos con problemas de aprendizaje. Debido a que los muestra una demora general en todo el proceso de aprendizaje,
problemas de aprendizaje se han diversificado, es imposible observándose lentitud, desinterés, deficiencia en la atención y
atacarlos desde un solo punto. Una de esas variaciones, son los concentración, afectando el rendimiento global [2]. Incluso la
trastornos de aprendizaje en la escritura. Como una solución afectación puede resultar en un área más específica como:
alternativa se propone el uso de las aplicaciones interactivas para
dispositivos móviles como recurso de apoyo en la estrategia • La comprensión y expresión de la comunicación escrita
docente para el área de escritura en alumnos con problemas de llamada dislexia.
aprendizaje. • La afectación de la forma y el significado de la escritura
conocida como disgrafía.
PALABRAS CLAVE. • Las dificultades en el desarrollo de las habilidades
Aplicaciones Interactivas, Dispositivos Móviles, Problemas de relacionadas con las matemáticas.
Aprendizaje, Escritura.
Por las necesidades existentes en el ámbito de la educación,
1. INTRODUCCIÓN. resulta vital diseñar nuevas estrategias educativas con los recursos
En general, en una sociedad se presentan constantes cambios que disponibles. Con el propósito de aprovechar las ventajas que
requieren de una respuesta por parte de la tecnología. brindan los dispositivos móviles, se propone el uso de
Actualmente, la necesidad de constante movilidad por parte de las aplicaciones interactivas como apoyo a la metodología utilizada
personas ha motivado el desarrollo de una herramienta por docente, para el desarrollo del aprendizaje de la escritura en
tecnológica muy útil como lo es el dispositivo móvil. El los niños con problemas de aprendizaje.
dispositivo móvil por definición, es aquel dispositivo que se usa Las aplicaciones interactivas proveen un alto grado de
fuera de una posición fija y que ha sido manufacturado retroalimentación con el alumno, basándose en patrones de
específicamente para ser portátil y útil mientras se está en respuesta – interacción, se ofrece una instrucción que permite la
movimiento [1]. experimentación con temas acordes al contexto de aprendizaje
En la actualidad, los dispositivos móviles gozan de un gran requerido [8], que en este caso es la escritura.
número de usuarios, conforme un estudio realizado en 2014 por El objetivo principal de este artículo es proponer el uso de
iabmexico.com tan solo en México el 84% de la población aplicaciones interactivas para dispositivos móviles, dentro de la
contaba con un dispositivo móvil. Un dispositivo móvil puede ser metodología de aprendizaje para la adquisición de la escritura que
un teléfono celular, tablet’s, PDA´s, smarthphones, entre otros. se aplica en los niños con problemas de aprendizaje. En la
Cabe mencionar que el uso de los dispositivos móviles es muy siguiente sección se describe a detalle la problemática que existe
variado, pueden realizar diversas tareas relacionadas con el en torno a la enseñanza de la escritura y el uso de dispositivos
entretenimiento, conectividad, actividades especializadas y móviles como apoyo en las aulas.
continuar con las actividades elementales de sus funciones; de
esta manera, los dispositivos móviles vienen a configurar un 2. PROBLEMÁTICA.
nuevo paradigma económico, social, cultural y educativo. Aunado El problema relacionado con la adquisición de la escritura,
a eso, los componentes de hardware y software del dispositivo provoca retrasos en el niño, en el desarrollo y el aprendizaje;
móvil hacen posible el uso de nuevas formas de interacción, concretamente en la recuperación de la forma de las letras y las
específicamente, pantallas táctiles y la interpretación de gestos. palabras. Las características que presentan estos niños son las
Esto permite que grupos de usuarios no especializados en el uso siguientes [7]:
de componentes tradicionales (teclado, mouse, pantalla no táctil) • Dificultades para escribir palabras (trazo)
[9], particularmente los niños, manejen con facilidad aplicaciones • Dificultades en la composición escrita (concepto)
en estos dispositivos, debido a que la interacción resulta muy • Raramente se presenta aislado, suele aparecer asociada con
intuitiva. dificultades de lectura (trazo-concepto-comprensión)

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 166 |
 Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje
J. Muñoz Arteaga | D. Morales Hndz | R. Mendoza | A. Torres Soto
El aprendizaje de la escritura demanda un doble proceso: trabajo que apoyan en el fortalecimiento de las habilidades básicas
cognoscitivo y perceptivo – motriz. El aprendizaje de la escritura en niños con problemas de aprendizaje. Inicialmente, el docente
inicia con el ejercicio repetitivo para lograr el dominio de trazos hace una evaluación al alumno para diagnosticarlo con respecto a
con diferente dirección y figura (hacia arriba, abajo, derecha, los niveles de escritura y generar su perfil. Con el perfil del
izquierda, en line recta o curva), para que posteriormente el trazo alumno y un listado de aplicaciones interactivas, el maestro de
se combine para ajustarse a la forma de las letras del alfabeto. apoyo genera un plan de actividades para trabajar con el alumno.
Esto solo indica que el alumno ha aprendido a dibujar las letras Finalmente implementa el plan de actividades con el apoyo de
del abecedario con un mayor dominio de sus movimientos [10]. aplicaciones interactivas.
Pero ¿cómo hacer para que el alumno adquiera la
conceptualización o el significado de lo que escribe?
Como se mencionó la escritura demanda dos tipos de proceso [6]:
Proceso cognoscitivo: Que enfocado al aprendizaje específico de
escritura, consiste en el proceso de conceptualización o
alfabetización de lo escrito.
Proceso perceptivo – motriz: Que consiste en la coordinación de
los músculos para poder reproducir la información requerida.
La maduración de este doble proceso permite el desarrollo de las
habilidades que se requieren para una reproducción gráfica
apropiada de los sonidos del lenguaje [6].
En busca de mejorar diversos procesos de aprendizaje, en México
se ha desarrollado material con diversas metodologías para el
aprendizaje, dentro del marco de este material se encuentra el
Manual para favorecer las competencias de lectura y escritura [5]:
primer ciclo, que apoya al entendimiento del proceso de la
escritura. En dicho manual se menciona que los niños pasan por
varias etapas relacionadas con la hipótesis y la conceptualización Figura 1. Proceso de asignación de aplicaciones de escritura
que adquieren acerca de cómo funciona la lengua escrita. En cada en base al perfil del alumno.
nivel se explican las características específicas de la escritura y se A continuación se presenta una descripción detallada del modelo
proponen algunas actividades que permiten desarrollar este nivel. planteado en la figura 1 para el proceso de asignación de
Un problema es que la forma de tratar al proceso de escritura, solo aplicaciones interactivas de escritura con respecto al perfil de
se ha realizado mediante actividades didácticas en papel. usuario obtenido:
Además de eso, se menciona que para esta metodología el proceso 1. Diagnóstico. Inicialmente, el maestro de apoyo debe contar con
principal es la alfabetización del alumno [5], para después evaluaciones de habilidades básicas ya desarrolladas, él interviene
ayudarlos a formular hipótesis nuevas como: para aplicar dichas evaluaciones al alumno y observar cómo
• En qué dirección se escribe. responde a cada una de ellas. Esto le permite realizar un perfil
• Existe un espacio entre palabras. para identificar a cada alumno, es decir, realizar un diagnóstico.
• Existen artículos para formar ideas. Algunos de los elementos que contiene la evaluación son:
• En ciertas palabras hay letras que se repiten separación de elementos ya sea por color, tamaño o forma,
clasificación de elementos y dictados.
Lamentablemente no existen los recursos suficientes dentro de las
escuelas para que los maestros puedan implementar en sus 2. Perfil del alumno. Con base en los resultados obtenidos de las
metodologías de aprendizaje el uso de dispositivos móviles, evaluaciones y las observaciones realizadas por el maestro de
además de que no cuentan con el conocimiento tecnológico apoyo, se define un perfil del alumno, identificando las
suficiente para hacerlo. Lo cual lleva a que el maestro en general debilidades en las habilidades requeridas para la correcta
no considere el uso de herramientas tecnológicas para llevar a adquisición de la escritura. Dichos niveles se clasifican en 3:
cabo sus actividades educativas.
• Pre – silábico.
Por lo anterior, en la siguiente sección se propone un modelo para • Silábico.
incluir dentro de la metodología de enseñanza el uso de • Alfabético.
aplicaciones interactivas para apoyar al alumno al desarrollo de
sus habilidades de escritura. 3. Plan de actividades. Con respecto a cada perfil del alumno,
además de apoyarse en una tabla de relación de habilidades
3. MODELO DE SELECCCIÓN DE escritas y aplicaciones interactivas sugeridas, se genera un plan de
APLICACIONES INTERACTIVAS. actividades específico. Se busca seleccionar aplicaciones que
Con el objetivo de brindar una herramienta que apoye al sirvan de apoyo para trabajar, con el objetivo de apoyar al alumno
desarrollo de las habilidades de escritura de los alumnos, se a adquirir un nivel satisfactorio en cuanto a las habilidades
propone un modelo que representa un proceso que se pueda escritas acorde a su edad.
utilizar para desarrollar aplicaciones interactivas que apoyen a la
4. Aplicar las actividades propuestas al alumno. La
evaluación y recuperación de los niveles de en los alumnos con
implementación del plan de actividades, se realiza en conjunto
problemas de escritura (ver figura 1). Estas actividades se
con las actividades planeadas por el maestro de apoyo. Es decir,
obtuvieron durante la primera observación realizada a un grupo de

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 167 |
 Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje
J. Muñoz Arteaga | D. Morales Hndz | R. Mendoza | A. Torres Soto
aquellas actividades independientes de la tecnología que permiten
la regularización del alumno.
5. Evaluación de resultados observados. Como último paso del
modelo viene una nueva evaluación donde se observará el avance
del niño al hacer uso de las aplicaciones interactivas. Con los
resultados observados el maestro determina si el alumno avanza al
siguiente nivel o se genera un nuevo plan de actividades con el
uso de otras aplicaciones interactivas de apoyo.
6. Nivel requerido de adquisición de escritura. Tomando en
cuenta los resultados del paso anterior el maestro de apoyo ya
definió que el alumno ha alcanzado el nivel requerido de escritura
acorde a su edad y diagnóstico, lo cual indica que al alumno se le
puede continuar reforzando el aprendizaje o terminar las
actividades de apoyo.
Para concretar el modelo propuesto en la Figura 1. Se presenta
una tabla de correspondencia sobre los niveles de escritura y las
actividades que permiten reforzar estos niveles y finalmente se
presenta, por cada nivel, una lista de aplicaciones sugeridas que
apoyan el desarrollo de estas habilidades.
La siguiente tabla se formó a partir de la consideración de que la
escritura se adquiere por medio de un doble proceso que requiere
habilidades de alfabetización y habilidades motrices. Las
habilidades de alfabetización se tomaron de los niveles de
adquisición de la escritura propuestos por Emilia Ferreiro y Ana
Teberosky [4], mientras que para las habilidades motrices se
consideró el trabajo de Swearingen [6] y de Lica [3].
Tomando en cuenta estos referentes, se propone una metodología
para implementar aplicaciones interactivas (Tabla 1) para apoyar
el desarrollo de los niveles de escritura. Inicialmente se describen
los niveles de escritura, tomando en cuenta que el desarrollo de las
habilidades perceptivo motrices difieren independientemente del
nivel de alfabetización en algunos alumnos [5]. Además, se
agrega una descripción de los niveles y una lista de aplicaciones
interactivas encontradas en Play Store de Google para desarrollar
cada nivel de escritura.
7

7
https://play.google.com/store Ultimo acceso:14/06/2015

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 168 |
 Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje
J. Muñoz Arteaga | D. Morales Hndz | R. Mendoza | A. Torres Soto
Tabla 1. Uso de aplicaciones interactivas en la planeación docente para la adquisición de escritura.

Niveles de Nivel de Aplicaciones

adquisición desarrollo Descripción de niveles. sugeridas en los
de escritura motriz niveles.1
Se presenta cuando el niño solo realiza dibujos o simplemente raya la hoja, sin *Tracing line &
Motricidad
existir un orden claro en los trazos, conocido lo anterior como garabato. Algunas shapes
visualactividades propuestas *Escribo en letras
- Trazo de líneas rectas, horizontales y verticales, curvas, diagonales, con una guía. de imprenta.
Pre – silábico

Comienzan a aparecer símbolos, aunque siempre son los mismos, y al escribir se

*Aprender escribir.
Memoria mantiene constante el número de símbolos que escribe.
*ABC Escritura.
visual - Trazo de letras tanto con guía como sin ella.
*Redacción ABC
- Trazo de silabas con guía y sonido.
Tanto la cantidad como la variedad de símbolos en cada palabra es diferente; sin
*Aprender escribir.
embargo, se puede observar la presencia de un símbolo inicial igual cuando existen
Percepción *ABC Escritura.
palabras que inician con la misma sílaba, sobre todo si se trata de vocales iniciales,
visual *Silabario Lite.
aunque el símbolo no necesariamente coincide con las letras correspondientes.
* Palabras domino.
- Solicitar el trazo de silabas sin guía y solicitar el fonema de la sílaba.
El niño no encuentra una idea acerca de la relación existente entre el símbolo y el
sonido, por lo que en su escritura sigue sin existir una correspondencia sonoro- *Silabario Lite.
Motricidad
gráfica. *Palabras domino.
visual
- Solicitar el trazo de palabras y que al mismo tiempo se le indique el fonema de la *Kids doodle.
letra de cada sílaba en la palabra.
Silábico

El niño asigna una letra por cada sílaba que escucha, algunas de estas letras tienen *Palabras domino.
Memoria
correspondencia con la sílaba. *Leer y Escribir
visual
- Formar palabras por medio de sílabas ya sea mediante imágenes o trazos. Jugando
Comienzan a aparecer vocales y consonantes correspondientes a la silaba que tratan
*Tarjetas
Percepción de representar en una palabra.
visual - Palabra generadora con palabras de dos silabas. educativas.
- Dictado de palabras con sonidos similares. *Kids doodle.
Aunque todas las vocales escritas sean correspondientes, el fallo se encuentra en las *Primeras palabras
consonantes, cuando existen múltiples errores donde no coinciden con la letra 1 y 2.
Motricidad
correspondiente; sin embargo, se respeta la relación entre símbolo y fonema. *Ahorcado
visual
- Pedir que completen palabras a partir de un número de letras que contiene la educativo
palabra (Ahorcado básico). *Kids doodle.
Alfabético

Los errores grafo fonéticos de su escritura son menos frecuentes, generalmente se

Memoria presentan en sílabas trabadas o mixtas, a razón de un error en cada palabra o por
*Kids doodle.
visual cada dos palabras, aunque puede ser menos frecuente.
- Tomar un dictado o elaborar un dictado.
En este nivel sólo se presentan errores polivalentes, siendo los errores grafo
Percepción fonéticos muy poco comunes, o en el mejor de los casos, inexistentes. Cada letra
*Kids doodle.
visual escrita coincide correctamente con el sonido correspondiente.
- Relatar las experiencias de un día anterior
Con el objetivo de ejemplificar el uso del modelo de la Figura 1 y trabajar con el alumno, con el objetivo de avanzar al siguiente
la Tabla 1 en la siguiente sección se describe un caso de estudio. nivel.
4. CASO DE ESTUDIO Con respecto a este caso que se encuentra en el nivel silábico,
Utilizando el modelo propuesto en la Figura 1 y las aplicaciones como se menciona en la Tabla 1, las actividades para trabajar con
propuestas en la tabla 1, se presenta el siguiente caso de estudio, él son la palabra generadora y el dictado de palabras con sonidos
que en particular permite el desarrollo de percepción visual en el similares. Se trabajará con el dictado de palabras ya que para la
nivel silábico del niño. El caso de estudio se realizó durante una segunda actividad aún no se cuenta con una aplicación acorde a
visita a una escuela de educación básica en la ciudad de ella.
Aguascalientes, México. Donde se nos permitió trabajar con Dictado de palabras con sonidos similares. Para esta actividad se
algunos niños con problemas de aprendizaje, el presente caso es solicitó al alumno que escribiera una serie de palabras que se le
de un niño de 10 años que cursa actualmente su 4to grado de dictaron, las palabras dictadas son sencillas de dos silabas y con
primaria y se encuentra en el nivel silábico de escritura. un repertorio muy similar. En esta ocasión se utilizó la aplicación
Como primer paso del modelo, se aplica una evaluación de Kids doodle que es una aplicación que permite la escritura libre
diagnóstico al alumno, una vez realizada, el maestro de apoyo con elementos llamativos para realizar trazos y fondos de diversos
tiene los resultados para determinar el nivel en el que se encuentra colores. Las imágenes de la actividad se muestran a continuación.
actualmente el alumno. Con respecto a este nivel, el maestro de
apoyo generó una planeación de actividades regulares para

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 169 |
 Uso de Aplicaciones Interactivas para Apoyo a la Escritura en Niños con Problemas de Aprendizaje
J. Muñoz Arteaga | D. Morales Hndz | R. Mendoza | A. Torres Soto
de escritura, ya que la mayoría de las aplicaciones existentes no
tienen un sustento pedagógico fuerte.
Como trabajo futuro se pretende desarrollar aplicaciones
sistematizadas en las que el contenido provenga de los
requerimientos pedagógicos para la escritura, además se planea
que exista una capacitación para los docentes sobre el uso de
1) 2) aplicaciones interactivas dentro de la metodología de trabajo con
niños con problemas de aprendizaje.

6. REFERENCIAS
[1] J. Muñoz A, F. J. Martinez, J. Guerrero, J. A. Hernández,
Una clasificación de patrones de interacción para
dispositivos móviles, MexIHC 2006, Mexico D.F., 2006.
3) 4) [2] E. Lopéz F., Problemas generales y trastornos especificos del
Figura 2. Kids Doodle aplicación que permite al usuario el aprendizaje en edad escolar., Aldanis.net., La revista de la
trazo libre de palabras. educación., N° 11., 2007.
En esta actividad se muestran 4 pantallas por orden: en la pantalla [3] M. Lica A., D. Barriga, A. González., Relación entre
1, se le solicitó al alumno que escribiera su nombre y él lo escribió ejecuciones deficientes de motricidad fina con dificultades de
completo. En la pantalla 2, se muestran dos palabras que se le escritura: Análisi de un caso., Revista Educación y
dictaron al niño, oso y yoyo. En la pantalla 3, se muestra la Desarrollo., 2010.
representación del niño de las palabras sapo y ojo. Finalmente, en
la pantalla 4, las palabras que se le dictaron al alumno son sol, [4] E. Ferreiro, A. Teberosky, Los sistemas de escritura en el
nube y lluvia. Lo que se muestra en la figura 2 es la forma en la desarrollo del niño., Siglo Veintiuno Editores, segunda
que el alumno representa de forma escrita las palabras. El uso de edición, 2005.
esta aplicación tuvo dos beneficios: el primero, es que el alumno [5] SEP, Manual para favorecer el desarrollo de competencias de
pudo escribir en la Tablet palabras sin líneas que le sirvieran de lectura y escritura. Primer ciclo, 2011.
guía; y el segundo beneficio es que permitió observar y guardar lo
[6] A. Searingen, MOT, OTR/L, T. Calder., La escritura necesita
realizado por el alumno, para mantenerlo como evidencia de su
habilidades motrices visuales y de percepción., Super Duper
avance.
Handy Handouts!, N° 138., 2009.
5. CONCLUSIONES. [7] M. Castro, A. J., Dificultades de aprendizaje, Revista Digital
En este trabajo se propone un modelo que apoye la adquisición de Enfoques educativos, No 42, 2009.
la escritura en alumnos con problemas de aprendizaje, de esta
[8] R. Cruz F., G. López M., Framework para Aplicaciones
forma se muestra que las aplicaciones interactivas pueden
Educativas Móviles (M-Learning): un Enfoque Tecnológico
convertirse en un apoyo para los niños con problemas de
– Educativo para Escenarios de Aprendizaje Basados en
aprendizaje, ya que el modelo puede ser tomado como guía para
Dispositivos Móviles, Virtual Educa Brasil, 2007.
facilitar al maestro su metodología de trabajo, se agrega la
intervención de aplicaciones interactivas en dispositivos móviles y [9] R. Vargas G., Desarrollo de aplicación interactiva para
la tabla generada a partir de una búsqueda de aplicaciones para enseñar vocabulario en ingles a niños desde edad preescolar.,
cada nivel de la escritura. Sobre las aplicaciones interactivas Memoria para obtener el título de Ingeniero civil en
encontradas, son útiles y pueden usarse en más de un nivel con computación., Santiago Chile, 2012.
diferentes enfoques, pero se necesitan aplicaciones más [10] J. A. Varela, G. Nava, B. Lara, Enseñar a escribir: ¿Cuál es
sistematizadas que permitan el avance del alumno en los niveles el método?, Revista de Educación y Desarrollo, No 11, 2009.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 170 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea

Enseñando Emociones a Niños Mediante Videojuegos

Noemí Marta Fuentes García Francisco Luis Gutiérrez Vela Patricia Paderewski Rodríguez
GEDES, Universidad de Granada GEDES, Universidad de Granada GEDES, Universidad de Granada
[email protected] [email protected] [email protected]

Rafael López Arcos Natalia Padilla Zea

GEDES, Universidad de Granada GEDES, Universidad de Granada
[email protected] [email protected]

ABSTRACT tarjetas para enseñarle a identificar los rasgos característicos de la

Los videojuegos tienen el poder de hacer sentir a quien los juega expresión correspondiente a cada emoción [3][6].
un amplio abanico de emociones de la misma forma en que lo De hecho, existen prácticas docentes especialmente diseñadas
haría si realmente estuviera en el mundo virtual en el que se está para hacerlo que especifican ciertas actividades, métodos y guías
desenvolviendo el juego. La Inteligencia Emocional está cada vez de actuación que se suelen aplicar para mostrar los diferentes tipos
más presente en los centros de enseñanza y se está utilizando con de emociones (especialmente las básicas, como son la ira, la
alumnos en edades cada vez más tempranas. Por ello, el propósito alegría, la tristeza o la sorpresa) y cómo tratar con ellas de manera
de este trabajo es proporcionar una introducción para encontrar la apropiada [4][6][7]. Se utilizan cuentos e historias [8][9][10] que
forma de enlazar un programa para la Educación Emocional de se suelen apoyar en otros materiales como pueden ser tarjetas,
niños en edad de preescolar con mecánicas de juego. Para lograr cartas, fotos o vídeos que muestran rostros que ilustran lo que les
este objetivo, hemos empezado por estudiar la inteligencia están enseñando. Otras veces juegan a juegos adaptados y
emocional y así como algunos programas y herramientas representan obras de teatro o marionetas que les ayudan a
utilizadas para enseñarla. Hemos analizado también cómo los comprender estos conceptos complejos [11][12][13]. También
diferentes medios de entretenimiento producen emociones. Como existen algunas aplicaciones informáticas que son más o menos
resultado, hemos propuesto un conjunto de requisitos, basados en parecidas a los recursos descritos anteriormente, pero utilizando
el estudio anterior,con el fin de poder fijar una base para el diseño un apoyo digital [9][10].
de juegos cuyo objetivo sea doble: que los niños se diviertan y que
los adultos puedan evaluar cómo estos jóvenes aprenden a Por otro lado, los videojuegos son una de las formas más
identificar las emociones y a hacer frente a los sentimientos que conocidas de entretenimiento tanto en niños como en adultos
producen. [16][18]. Este uso generalizado de los videojuegos se debe a
diferentes aspectos. Uno de ellos es que los jugadores son capaces
Categories and Subject Descriptors de desempeñar un rol en una historia y ser transportados a otros
H.1.2 [Information Systems]: User/Machine Systems – Human mundos, donde sienten que pueden dirigir lo que sucede,
factors. sumergiéndose en la historia hasta sentirse parte de la misma. Los
videojuegos tienen el poder de hacer que la gente sienta un amplio
General Terms rango de emociones de la misma forma que si realmente
Human Factors, Design, Experimentation, Languages estuvieran enfrentándose a los problemas que el protagonista tiene
y viviendo las situaciones que el juego proporciona. Distintos
Keywords estudios [16][17] han demostrado que estos juegos tienen como
Videojuegos Educativos, Inteligencia Emocional, Infancia, efecto colateral a la diversión el desarrollo de ciertas capacidades
Emociones, Interacción Persona-Ordenador. como la agudeza visual, los reflejos, la concentración, e incluso la
socialización (especialmente en los juegos cooperativos y online).
1. INTRODUCCIÓN Es por ello que cada vez son más los profesionales de la
La Inteligencia Emocional es cada vez más importante y tiene un educación que deciden incluir como parte de su metodología el
gran impacto en diversos aspectos de nuestras vidas, tanto en el uso de videojuegos, con el objetivo de hacer más eficiente y
ámbito escolar como profesional [1-4]. El aprendizaje, las amena su labor.
relaciones interpersonales, el trabajo en equipo o incluso la
felicidad están relacionados con este tipo de inteligencia [5]. Por El propósito de este trabajo es analizar las características de esta
todo ello, los profesores tratan de introducir a los niños cada vez herramienta y comprobar cómo puede ayudar a enseñar a los
antes en el mundo de las emociones. En la actualidad, se intenta niños cuáles son las principales emociones y cómo identificarlas y
educar a los niños desde preescolar para que aprendan a tratarlas, siempre de forma amena y motivadora. Para hacerlo se
identificar las emociones, ya que esto es importante no sólo de ha seguido un proceso de estudio y análisis de las principales
cara a ser capaces de transmitir sus propios sentimientos, sino de áreas relacionadas: psico-pedagogía y educación de las emociones
empatizar con los de los demás y facilitar su desarrollo y la forma y diseño de videojuegos educativos, fundamentalmente; sin
en que se enfrentan a las distintas situaciones de la vida (tanto olvidar la premisa de que en cualquier videojuego de carácter
infantil como adulta) [1-4]. Para ello se suelen realizar ejercicios educativo, siempre debe existir un equilibrio entre diversión y
como nombrar los tipos de emociones cuando se habla con el niño educación [26]. A modo de ejemplo, y como resultado del análisis
de los acontecimientos de la vida diaria o realizar actividades con realizado, se muestran los requisitos que debe tener un videojuego
diseñado específicamente para enseñar emociones.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 171 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
El resto del artículo muestra parte de la información obtenida en Objetivo López’0 Bisquerra’ López’1 Ribes’0
dicho proceso como sigue: en la Sección 2 se muestra una 5 [2] 12 [4] 0 [6] 5 [7]
recopilación del estudio realizado acerca de la Educación Basada    
Autoconocimie
en Emociones; la Sección 3 identifica Elementos que Inducen
nto
Emociones; en la Sección 4 se realiza una Propuesta de Requisitos
para un Videojuego que enseñe emociones; finalmente, en la Identificación    
Sección 5 se encuentran las Conclusiones y Trabajo Futuro. de las
Emociones
2. EDUCACIÓN BASADA EN EMOCIONES Ajenas
Existen varias definiciones del término Emoción, todas ellas con 
Vocabulario
un punto en común: es una alteración del estado de ánimo que Emocional
puede ser agradable o no, y en la que normalmente influyen las
relaciones con los demás y la propia experiencia. Las dos Regulación   
siguientes definiciones parecen dar un sentido más preciso y Emocional
completo del concepto de emoción: Tolerancia a la  
“Es un estado afectivo que experimentamos, una reacción frustración
subjetiva al ambiente que viene acompañada de cambios Prevención de  
orgánicos (fisiológicos y endocrinos) de origen innato, influidos los efectos
por la experiencia. Las emociones tienen una función adaptativa nocivos de las
de nuestro organismo a lo que nos rodea” [19]. emociones
“Es un complejo estado psicológico que implica tres componentes negativas
distintas: una experiencia subjetiva, una respuesta psicológica, y Generación de  
una respuesta de comportamiento o expresiva” [20].
emociones
Por otro lado, son muchas las interpretaciones que se han positivas
realizado sobre el concepto de Inteligencia Emocional (entre las   
Automotivació
que destacan las de Goleman [21] y Gardner [22]). Sin embargo, n
todas ellas parecen coincidir en definirla como la capacidad de
reconocer las emociones, tanto propias como ajenas y saber Actitud positiva  
tratarlas de manera adecuada. Se considera un complemento de la frente a la vida
inteligencia cognitiva (concepto tradicional de inteligencia) y es Desarrollar las  
posible desarrollarla mediante la aplicación de ciertas técnicas, Competencias
actividades y ejercicios [1]. Esto es especialmente interesante y Emocionales
útil en edades tempranas, ya que es cuando el ser humano está
desarrollándose y creciendo en todos los sentidos y aspectos. Esta
capacidad se consigue mediante la Educación Emocional, que Uno de los términos más utilizados en la bibliografía relacionada
según Bisquerra [4] se define como “El proceso educativo, con la alfabetización de las emociones, y de hecho se considera
continuo y permanente, que pretende potenciar el desarrollo de como uno de los objetivos a cumplir, es el de la Competencia
las competencias emocionales como elemento esencial del Emocional. Ésta se define como el conjunto de conocimientos,
desarrollo humano, con objeto de capacitarle para la vida y con capacidades, habilidades y actitudes básicas para la vida,
la finalidad de aumentar el bienestar personal y social” y consiste esenciales para el desarrollo integral de la personalidad y
en “Optimizar el desarrollo humano, el desarrollo integral de la necesarias para comprender, expresar y regular de forma
persona (desarrollo físico, intelectual, moral, social, apropiada los fenómenos emocionales [4][7]. En este sentido, el
emocional…)”. desarrollo de competencias emocionales a través de la Educación
Emocional puede, por tanto, representar una mejora que afecta
2.1 Objetivos de la Educación Emocional positivamente a múltiples aspectos de la vida.
Considerando las definiciones anteriores, y en concreto el
concepto de Educación Emocional, es lógico plantear una serie de 2.2 Educación Emocional en la Educación
objetivos o metas a incluir en los programas de estudio de las Infantil
emociones. Dichos objetivos deben permitir que los educadores
Como ya se ha mencionado, la Educación Emocional es muy
tengan claro a qué metas deberían aspirar cuando de inteligencia
importante a la hora del desarrollo y el éxito del individuo. Del
emocional se trata.
mismo modo, una mala Educación Emocional, o lo que otros
A continuación se muestra una lista de objetivos obtenida a partir denominan analfabetismo emocional es la principal causa de
de los trabajos de varios autores, relacionados tanto con la fracaso escolar, así como de otros problemas que se generan a lo
definición de inteligencia emocional como con la propuesta de largo de la vida y que, en ocasiones, pueden llegar a ser graves si
programas especializados en la introducción de la misma en las no se tratan a tiempo y se dan ciertas circunstancias que empeoran
aulas de educación infantil. Cada programa se centra más en unos la situación [1][4].
u otros, dependiendo del enfoque del mismo.
Si a ello se le añade que el mundo emocional de los niños es
Tabla 1. Objetivos de la Educación Emocional complejo, se pone de manifiesto la necesidad de ofrecerles las
Programa Emocional herramientas que le faciliten la identificación de lo que sienten y
cómo les afecta, así como la forma de enfrentarse a ello y a las
situaciones de la vida diaria que les puedan perjudicar o dificultar
XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction
| 172 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
sus relaciones. Esto se debe realizar desde la familia pero también sentimientos [28], Educación emocional. Programa para 3-6 años
en la escuela, optando por un modelo donde haya cabida para los [6], Emotional Intelligence Activities for children ages 8-10 [23] o
conocimientos y las emociones y fomentando el desarrollo de la Programas de reforzamiento en las habilidades sociales,
inteligencia emocional [1-4]. autoestima y solución de problemas [27]. Estos programas se
Es por ello que se hace imprescindible plantear un buen programa centran más en unos u otros aspectos dependiendo de la edad de
de enseñanza que permita identificar las capacidades a desarrollar los alumnos a los que se dirigen y los objetivos concretos que
y, de forma igual o más importante, la manera de llevarlo a cabo. pretenden lograr. Sin embargo, todas las propuestas de una forma
u otra tienen como meta ampliar el vocabulario relacionado con
2.3 Estrategias y Propuestas las emociones y facilitar la exteriorización de sentimientos de
Existen varios programas propuestos por distintos autores que manera adecuada y promueven la conciencia de las emociones,
pretenden promover la Educación Emocional y el desarrollo de las tanto propias como ajenas. Las actividades propuestas ayudan en
Competencias Emocionales mediante su integración en la vida del la canalización de dichas emociones mediante ejercicios que
niño, tanto a nivel familiar como en la escuela. En este apartado se suelen consistir en la relajación a través de la respiración u otras
reúnen algunos de los más destacados, aunque enfatizando más la técnicas que la favorezcan.
parte que se refiere al colegio y a la enseñanza de emociones
organizada en planes de estudio. 2.4 Herramientas para Enseñar Emociones
Parte del trabajo realizado previamente a la propuesta que se
Cuando se trata de elaborar un currículum escolar para enseñar plantea en este artículo ha consistido en la localización de material
emociones, al igual que para cualquier otra materia, es necesario que tanto especialistas como padres pueden utilizar y utilizan para
establecer la planificación del programa y de cada una de sus facilitar su labor educadora.
actividades, no sólo en cuanto al número de horas sino también en
cuanto a la franja horaria más adecuada, así como la época del A continuación se muestra una clasificación que pretende agrupar
curso más idónea [1]. Es por eso que en la mayoría de los trabajos los recursos encontrados atendiendo a sus características. En cada
estudiados aparece una estimación del tiempo a dedicar en cada uno de ellos se citan algunos de los ejemplos más relevantes.
ejercicio. Además, se indican las edades y tipo de alumno 2.4.1 Cuentos
objetivo, pues en ocasiones puede haber tareas más propicias para Colección Emociones [9]. Es una aplicación que incluye la
un niño con determinado perfil o, por el contrario algunas que no colección de cuentos de Mireia Canals y Sandra Aguilar
deba realizar, bien por no tener la edad adecuada, bien por sus “Emociones” [8] para hacerlos más interactivos y enseñar a niños
circunstancias personales [1][4]. entre 2 y 8 años las emociones. Además de los cuentos en sí
Según propone la autora María del Mar Vera en [3], la incluye una serie de actividades para favorecer la comprensión y
Inteligencia Emocional se puede desarrollar desde los primeros asimilación de los conceptos.
años de la vida, concretamente en la etapa que dura hasta los 6
años, donde se refuerzan las emociones enfatizando el elogio y la
aprobación. Además, incluye las siguientes estrategias de
estímulo:
- Dar nombre a los sentimientos: Consiste en ser capaces de
nombrar emociones. Esto les ayuda a reconocer esas
emociones cuando las sienten. Se puede realizar a través de
cuentos, películas o tarjetas con dibujos.
- Relacionar gestos con sentimientos: Por ejemplo, conversar
con el niño acerca de las emociones que podrían estar
sintiendo los personajes de un cuento o los actores de
televisión son buenas prácticas para enseñarles a identificar
las emociones mediante el lenguaje corporal.
- Orientar: Una vez que el niño sabe reconocer sus
emociones, hay que darle normas básicas para enfrentarse a
ellas. Es necesario explicarle lo que sí puede hacer. Expresar
cómo se siente es saludable siempre que se haga de manera
aceptable. Es conveniente enseñarle a relajarse para afrontar Figura 1. Colección Emociones: "La Muerte. Mi Amiga
esos momentos. Invisible"
- Actuar con empatía: Imitar los gestos ayuda a entender lo The Allen Adventure [10]. Es una historia interactiva creada para
que siente la otra persona. fomentar el conocimiento de las emociones y, sobre todo, para
- Alabar lo positivo. Felicitarle cuando se enfrente bien a sus aprender a interpretar las de los demás y saber qué les molesta.
emociones o muestre preocupación por los demás. Pertenece al gobierno australiano y fue desarrollada en una
- Enseñar con un ejemplo. Consiste en mostrarle estrategias iniciativa contra el bullying. Esta aplicación es para menores de 8
años pero existen otras para adolescentes. Narra la historia de un
para calmar el estrés: respirar hondo, darse un baño de agua
caliente, llamar a un amigo o escribir en su diario. extraterrestre que llega a la tierra y tiene que integrarse y
adaptarse a las costumbres propias.
Muchos de los trabajos analizados incluyen en su contenido parte
de las estrategias y consejos anteriormente citados. Algunos
ejemplos son Buenos días, ¿cómo te sientes?[4], La rueda de los
XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction
| 173 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea

2.4.2 Videojuegos grupos según las emociones (todas ellas negativas). El juego está
Gomins [14]. Es un juego que pretende ayudar a los padres a basado en el libro El utopífono.
educar a los hijos emocionalmente. Esto se logra mediante la
superación de pruebas que los propios padres van configurando 3. ELEMENTOS QUE INDUCEN
con las indicaciones recibidas en la aplicación para padres EMOCIONES
(planes). Cada nivel superado supondrá un paso más en la Una emoción es algo muy subjetivo que, a pesar de estar presente
evolución emocional del niño. El juego consiste en ayudar al en nuestro día a día, puede ser muy variable incluso en la misma
protagonista “Gomin” a volver a su planeta, ya que se ha perdido. persona. Depende de diferentes factores que van desde el
Para ello tendrán que cuidarlo y superar los juegos propuestos. individuo en sí mismo hasta el lugar en el que se encuentre y de
Estos juegos pertenecen a los géneros puzle y plataformas. sus experiencias vividas, tanto en soledad y como en su relación
IF… The Emotional IQ Game [15]. Este juego es una aventura con los demás.
gráfica en la que el protagonista avanza guiado por distintos seres Por eso generar emociones no es algo trivial, sino que por el
que aparecen para explorar el mundo en el que está. Como parte contrario es algo muy complejo. Por una parte se puede identificar
de las pruebas a superar, se encuentra el reconocer los el factor relacionado con los sentidos, aunque por otra se
sentimientos que plantea algún personaje de los que interactúan encuentra la asociación de ideas y conceptos. En ambos casos se
con el protagonista. suele relacionar estrechamente con situaciones y momentos ya
vividos, así como con la propia cultura.
Partiendo del enfoque anterior, vamos a comenzar por analizar los
sentidos que posee el ser humano [29], pues parte del entorno que
nos rodea es percibido por ellos. Posteriormente veremos cómo se
pueden combinar para inducir emociones con dos ejemplos de
medios de entretenimiento: el cine y los libros.
- Vista. Por lo general, es el sentido más desarrollado en los
seres humanos. Se dice que el 80% de la información que
recibimos a lo largo del día se percibe mediante el canal
visual [30]. Las principales características que se pueden
apreciar son el color, la forma, la textura y el volumen de
seres vivos y elementos artificiales, como pueden ser
animales, personas y objetos.
- Oído. Es el segundo sentido más desarrollado. Con él se
perciben vibraciones y ondas sonoras de orígenes diversos
siempre que se produzcan en una frecuencia que seamos
capaces de percibir. Lo más destacado probablemente sea la
Figura 2. IF... The Emotional IQ Game música, sin embargo, continuamente estamos expuestos a
sonidos de diferentes tipos, como pueden ser los producidos
2.4.3 Sitios Web por animales, personas (tanto en lo que se refiere a la voz
El Perruco [13] es una web para promocionar un proyecto que
como por ejemplo una palmada, caminar…) y objetos, tanto
fomenta la educación de la Inteligencia Emocional (en niños)
al moverse o caer, como los específicos que se utilizan para
mediante el personaje que da nombre a la misma. Incluye varios
producir o emitir sonido (instrumentos musicales, radio,
recursos educativos, tanto informativos, para los padres: técnicas, televisión…).
definiciones y actividades (incluye un blog); como para los niños:
el cuento de El Perruco, la presentación y el juego de mesa. - Gusto. No existen muchos elementos con los que
estimulemos este sentido, por lo general se reducen a
2.4.4 Juegos de Tablero comidas y bebidas.
El Perruco [13] es un juego de mesa, similar al juego de la oca, - Olfato. Percibido mediante la nariz, a él se asocian más
para enseñar las emociones avanzando por las distintas casillas del
elementos que al gusto, por ejemplo el perfume, la comida,
mismo y superando pruebas.
incluso lugares como el mar o el campo pueden tener un
El Bosque Encantado [12]. Es un juego de mesa online olor característico. De igual forma el humo, las hierbas o los
desarrollado por la AECC (Asociación Española Contra el excrementos se relacionan normalmente con el sentido del
Cáncer) ambientado en un mundo ficticio de ogros, hadas y olfato.
duendes en el que se plantean pruebas relacionadas con las
- Tacto. Aunque no nos demos cuenta, es uno de los más
emociones en cada casilla y se avanza o retrocede según el
utilizados, pues podemos percibirlo en todo el. Podemos
comportamiento demostrado.
relacionarlo con objetos, personas o animales, y nos permite
2.4.5 Juegos de Cartas describirlos con adjetivos como rugoso, suave, áspero,
El Taller de Reparancio [11] es un juego de cartas en el que hay pegajoso, líquido, sólido, viscoso, caliente o frío.
que elegir una en cada turno. Cada carta muestra la prueba a
realizar (existen cinco distintas: adivinanza, prueba, todos pintan,
mímica y todos cuentan); por ejemplo: “pensar en algo que…” (te
asuste, te irrite… según la carta). Los demás tienen que averiguar
lo que es con preguntas de tipo sí o no. Las cartas se dividen en

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 174 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
película para que el espectador intensifique las
sensaciones que percibe mientras la está viendo. Jugar
con el tipo y el volumen de la música, incluso
suprimirla puede ser fundamental para crear suspense y
añadir emoción.
• Efectos Especiales que profundizan en la trasmisión de
la historia y en la consecución de que el espectador
forme parte de la misma. Pueden ser tanto visuales
como sonoros.
Una de sus mayores ventajas es que consta de varias dimensiones
para expresar lo que necesita y captar la atención de quien la está
viendo (audio, vídeo, guión, y en ocasiones efectos 3D que
Figura 3. Porcentaje de uso de los sentidos humanos [30]
favorecen aún más el sentir que se es parte de ella).
3.1 Sentidos, Medios de Entretenimiento y Su principal desventaja es que hay muy poco tiempo disponible
Emociones para mostrar todo el contenido, por lo que hay que ser cautelosos
Vamos ahora a mostrar la relación entre los medios de para aprovechar al máximo los recursos disponibles y explotar las
entretenimiento y los sentidos. Analizaremos cómo influyen estos ventajas de este medio.
últimos en la generación de emociones mediante dos casos Las características anteriores podrían extenderse al teatro, aunque
concretos: el cine y los libros. Este análisis que proponemos se en este caso cobra mucha más importancia el escenario y la
basa en la observación de los medios mencionados, realizada calidad de los actores, ya que todo trascurre en directo. Ganaría
como parte del proceso de estudio para la identificación de por tanto el poder de trasmitir de manera visual y a través del
elementos que ayuden a inducir emociones. diálogo de los personajes.
Ambos suelen utilizar de forma directa el sentido de la vista. Libro. Una buena novela debería tener:
Aunque en el cine, además se usa el oído como complemento al
mismo, si bien en el caso del cine el énfasis es mayor. • Una historia que capte al lector.
A pesar de sólo aprovechar uno o dos de los sentidos humanos, • Personajes carismáticos, con los que el lector se
estos medios son capaces de generar emociones tan profundas que identifica y siente lo que ellos sienten.
llegan a hacer que el lector o el espectador sienta que forma parte • Descripción y narrativa que incrementen las
de la historia, que llegue a creer que realmente está en el lugar sensaciones y sentimientos y que hacen que el lector se
donde se desarrolla la misma. En definitiva, que es el identifique con la historia narrada. En ausencia de
protagonista. Se sabe que los sentidos pueden ser el estímulo de elementos audiovisuales (como mucho alguna escena
muchas emociones, pero ¿qué relación existe y cómo consiguen estática, probablemente en blanco y negro), se centran
entonces los autores de novelas y películas emocionar al público? en la descripción y su mayor poder reside en la
La respuesta corta es “describiendo”. Pero esto no parece ser narrativa.
suficiente para encontrar la clave de cómo utilizando tan sólo uno
Su mayor ventaja es que puede recrearse en los detalles, ya que es
o dos de los sentidos se puede conseguir el efecto de inmersión
parte de su atractivo; además, no hay límite de tiempo, y el lector
deseado. Para lograr esto se hace uso de recursos como son la
es el que decide cuánto tiempo seguido lo lee. También es el
historia en sí, buenos personajes, un escenario convincente y una
narrativa que consiga mantener la atención. En concreto: lector el que aporta su propio enfoque, ya que interpreta lo que
sucede según su propio criterio. La fuerza de las palabras y la
Cine. Una buena película debería tener: habilidad del escritor para hacer uso de ellas pueden conseguir
• Una historia que capte al espectador (guión). que un libro proporcione una experiencia más satisfactoria que la
proporcionada por una película a pesar de sólo contar con la vista
• Los diálogos entre los personajes son fundamentales, como sentido activo.
tanto lo que se dice como lo que no marca el desarrollo
del guión y por tanto la evolución de la historia. El inconveniente es que sólo cuenta con una dimensión para
mostrar lo que necesita, y es por eso que debe poner todo su
• Actores que encarnan los personajes, y con los que el esfuerzo en ella. Necesita aprovechar los elementos descriptivos y
espectador se identifica y siente lo que ellos sienten. el vocabulario para que el lector se sienta parte de la historia.
• Escenario/Fotografía. Es lo que ambienta la escena, es Tras esta pequeña revisión por dos medios de entretenimiento,
decir, lo que da vida y soporte al argumento de la parece claro que para obtener buenos resultados al generar
película y que además cambia en función del momento emociones se utiliza lo que se puede mostrar explícitamente a
concreto de la misma. través de la percepción sensorial. Pero esto no es suficiente, se
• Vestuario. Este aspecto también es muy importante pues deben combinar de manera adecuada para evocar recuerdos
todo debe concordar para que sea creíble, incluso incluso relacionados con los sentidos no activos. No hay que
aunque se trate de ciencia ficción debe existir olvidar que al fin y al cabo los sentidos no son más que
coherencia entre la trama, el vestuario, el escenario y los dispositivos de entrada que nuestro cerebro, a modo de ordenador,
personajes. interpreta en base al conocimiento adquirido con anterioridad. Por
tanto, es posible que aunque no estemos oliendo o saboreando
• Banda Sonora que incremente las sensaciones y algo seamos capaces de recordar la sensación (positiva o negativa)
sentimientos, reforzando los momentos críticos de la debido a que en el pasado tuvimos en una situación similar.

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 175 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
Para mostrar una situación no basta con describirla sólo con mueve. Incluso aunque la historia esté pre-definida, es decir, que
palabras. Es necesario hacer uso de la memoria asociativa que los no sea un juego de final abierto o sandbox, el jugador tendrá la
humanos tenemos. Así, mediante los sentidos de los que se percepción de que lo que hace tiene impacto en el curso de la
disponga en el medio con el que se quiere trasmitir, se podrá historia, y por tanto su sensación de inmersión y sus emociones
estimular la memoria y evocar recuerdos, que al fin y al cabo son serán más fuertes. Para ello se hace uso de mecánicas como
los que generan las emociones. plantear retos y sub-retos, que deben ser acordes con el tipo de
juego y con el argumento del mismo. También se utiliza, como ya
3.2 El caso particular de los Videojuegos se ha mencionado en este mismo párrafo, la interacción con los
Los videojuegos se pueden ver, de manera muy simplificada, elementos del juego, ya sean personajes u objetos, lo que permite
como una película interactiva en la que el jugador avanza de sentir que se puede controlar y modificar la situación.
manera más o menos libre hasta finalizar el juego. Sin embargo,
esto sería una definición muy pobre, especialmente teniendo en Cabe recordar que estas características, aunque comunes por lo
cuenta que existen distintos géneros en los cuales puede tener más general a todos los videojuegos, podrían variar e incluso
o menos peso la historia, la acción, el escenario o los puzles. suprimirse en algunos. Tal es el caso de los juegos de tipo puzle,
Aunque es cierto que la mayoría de los juegos se podrían que en ocasiones carecen de historia y personaje, aunque siempre
caracterizar por tener una historia, un protagonista y unos existe un contexto narrativo. En ellos la emoción es el propio reto
objetivos, que variarán y tendrán mayor o menor impacto en el de conseguir avanzar a través de los niveles para lograr mayor
desarrollo del juego dependiendo del tipo del mismo. dificultad y puntuación e incluso competir con otros jugadores
para aumentar el valor de la misma. En estos juegos sigue
Por tanto, cuando se trata de analizar qué elementos son utilizados presente el hecho de que es necesario conseguir uno o varios
para generar emociones en un videojuego se podría partir de la objetivos para poder superar el nivel y desbloquear el siguiente, y
suposición de que el juego es una película, para posteriormente también la interacción, pues es la única forma de poder avanzar en
añadir matices que son los que lo distinguen de la misma. la partida.
Por una parte, como ya se ha explicado, la propia historia es un
mecanismo de generación de emociones, debido en concreto a 4. REQUISITOS PARA UN VIDEOJUEGO
componentes como los personajes [24], los sucesos o imprevistos EDUCATIVO QUE ENSEÑE EMOCIONES
que surgen a lo largo de ésta y la narrativa [25]. A NIÑOS
El nivel de inmersión que puede provocar un videojuego en un A modo de ejemplo, en este punto vamos a presentar y analizar
jugador hace que las emociones sean vividas por el jugador de una algunos aspectos que son relevantes a la hora de diseñar un
forma mucho más efectiva. Además, los videojuegos tienen la videojuego que facilite la práctica de la inteligencia emocional en
particularidad de que pueden adaptar la historia al estado niños entre 3 y 5 años. El videojuego ayuda a identificar una serie
emocional del jugador. Para ello sería necesario hacer uso de de emociones y sentimientos, así como a lidiar con ellos y
mecanismos adicionales como la detección y el análisis de manejarlos de la mejor manera posible tanto para sí mismos como
emociones en tiempo real. Esto a su vez repercutiría en un mayor para su entorno.
nivel de inmersión por parte del jugador [18].
Para ello nos hemos basado en las propuestas de experiencias
De igual modo, el escenario en el que se lleva a cabo cada escena docentes para enseñar inteligencia emocional en educación
es muy importante a la hora de trasmitir sentimientos al jugador. infantil consultados, y más concretamente en la propuesta de [6].
Dentro del escenario podemos distinguir entre el Principal o
Básico, que ayuda a contextualizar la historia en sí, esto es, la 4.1.1 Objetivos
ambientación general (por ejemplo, la historia trascurre en un En el videojuego nos planteamos dos objetivos principales:
castillo, en la época medieval, por tanto todo el escenario debe Objetivo 1. Va a ser una aventura con la que los pequeños
estar ambientado como tal) y el Adaptado a la situación, que disfrutarán resolviendo problemas, avanzando en el juego,
ayuda a hacer que el jugador se transporte al juego, es decir, el consiguiendo logros y desarrollando súper poderes; así como con
que contribuye a mostrar hechos y escenas concretos (por las actividades paralelas que se plantean.
ejemplo, si hay una escena de lucha, aunque sea dentro del propio
castillo, podría incluir elementos como juegos de luces o algún Objetivo 2. Va a ser una herramienta que educará a los jugadores
otro tipo de alteración sobre el escenario base que ilustre mejor y les ayudará para que desarrollen su inteligencia emocional de
esa lucha). manera natural y divertida, integrándola poco a poco en su vida
diaria, permitiéndoles aprender a identificar y manejar emociones
La música (banda sonora) es uno de los factores más importantes de manera práctica y contextualizada. A este objetivo hemos
cuando se trata de ambientar un hecho, situación o historia, en asociado una serie de objetivos educativos correspondientes con la
este caso se podría clasificar como: de ambiente, general y común primera fase de desarrollo:
a lo largo del juego; adaptada a momentos particulares que
requieren mayor atención por parte del jugador; y efectos de • Enseñar a identificar las emociones. Para reducir la
sonido, que permiten enfatizar las escenas y situaciones concretas. complejidad del problema se han elegido las tres que
aparecen en el primer nivel: alegría, enfado y tristeza.
Cuanto mayor sea la relación entre estos elementos, mayor
sensación de inmersión deben producir en el jugador y, por tanto, • Posteriormente se puede añadir el resto hasta completar
más efectiva será la generación de emociones en el mismo. los tres niveles propuestos. Cuanto más avanzado esté el
juego, más se profundiza en las emociones estudiadas
Pero lo que de verdad distingue un videojuego del resto de medios con anterioridad y más matices se incluyen dando lugar
de entretenimiento es que permite al jugador interactuar a nuevas emociones (estrés, celos…).
realmente con éste. Le hace partícipe directamente de las acciones
del personaje, pudiendo modificar el entorno en el que éste se

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 176 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
• Enseñar a controlar las emociones. Es necesario elegir y otras simplemente retos lúdicos, para mantener la diversión y la
las técnicas que se aplicarán para las emociones elegidas atención del niño.
en el primer nivel. Por ejemplo: relajación mediante la
respiración y la música, esto se consigue mediante la 4.1.6 Retroalimentación: Recompensas y Puntuación
imitación. Proponemos ofrecer varios tipos de recompensas (Tabla2). Por
una parte las puramente lúdicas, con el objetivo de divertir y
4.1.2 Requisitos de Jugabilidad motivar por el simple hecho de conseguirlas. Por otra parte
El hecho de que sea un videojuego con propósito educativo no mixtas, con el objetivo de motivar y producir realimentación
evita que sea un producto jugable, es decir, que atrape al niño acerca de las acciones del niño en el juego, pero también de
desde el primer momento, lo haga disfrutar y sentir la historia permitir conocer la evolución de su aprendizaje a los educadores.
sobre la que trata [18][25][26]. Estas últimas se caracterizan porque pueden ser también
Para ello hemos establecido las siguientes características, negativas. Un mal comportamiento también debe verse reflejado
en ellas, de manera que se favorezcan las reacciones y
prestando especial atención a los elementos clave mencionados en
la sección 3 de este artículo: sentimientos positivos frente a los negativos por la propia
mecánica del juego. Para este último tipo hemos propuesto el uso
• Historia, es necesario tener en cuenta tanto el contenido de los poderes a modo de metáfora de su evolución real. Los
como la narrativa y los personajes. poderes no deben cambiar igual para todos los niños, sino que
• Diseño tanto visual como conceptual. Se deben cuidar varían dependiendo de distintos factores como el ritmo al que
los escenarios, la música, los sonidos y las formas supera los retos y la forma en que lo hace.
empleadas en los elementos que aparezcan en el juego. Tabla 2. Recompensas
• Retos. Se plantean como elementos que permiten por Recompensa Descripción Función
una parte avanzar en la historia (y por tanto en el Recolectar de Aparecen en ciertas Motivación
conocimiento subyacente) y por otra motivar al niño objetos situaciones.
para que siga jugando.
Permiten conseguir
• Retroalimentación. Es necesario hacer saber al jugador beneficio extra
cuando hace algo bien o mal en el juego de manera que posteriormente.
sea consciente de ello sin que llegue a afectarle
negativamente pudiendo llegar a desanimarle. Este es un Coleccionar Por ejemplo: frutas o Diversión y
aspecto muy importante en la motivación del niño y piezas juguetes. Motivación
también puede tener repercusión en su estado de ánimo Cuantas más piezas se
y por tanto en sus emociones. Para ello proponemos el tengan, mejor.
uso de Súper poderes: el niño puede ver cómo
evolucionan a medida que avanza en el juego y querrá Súper Van adquiriéndose en Motivación
seguir jugando para comprobar cómo cambian o saber Poderes función del comportamiento Evaluación de la
cuál es el nuevo poder que consigue. del niño en el juego. evolución del
También se pueden perder conocimiento
4.1.3 Control del Juego como consecuencia de
Para facilitar la empatía del niño con los personajes del juego, comportamiento negativo
debe controlar un único personaje, el protagonista. De acuerdo
con [24] el protagonista tiene que tener un aspecto amigable y con
rasgos infantiles, de manera que el pequeño se identifique más En todos los casos se hace uso del refuerzo positivo de la
fácilmente con él y por tanto sea más fácil generar emociones. puntuación y recompensa cuando algo se realiza correctamente
con efectos sonoros agradables, asociados al éxito. Para indicar
4.1.4 Ayuda y Guía de Juego que algo está mal también se emplea un efecto sonoro, pero de
El juego debe tener un menú de ayuda que permita a los jugadores manera que no implique connotaciones negativas. Se identifica
realizar consultas sobre aspectos del mismo que no entienden. con sonidos de fallo que se suelen utilizar en otros videojuegos o
También debe ofrecer guías para superar ciertos retos. en dibujos animados y que en ocasiones tienen un tono cómico o
Para ello proponemos la existencia de la figura de la mascota, que sarcástico para restar negatividad a pesar de la alerta
se trata de un personaje que sirve de guía y apoyo a lo largo de proporcionada.
todo el juego. Le ofrece consejo en situaciones que podrían ser Todo ello está apoyado por un modelo interno de juego que refleja
complicadas para él, y permite que éste exprese los sentimientos a la estructura asociada al programa educativo seleccionado.
través de un diálogo que se facilitará mediante el uso de iconos y Además, durante el juego se mantiene un modelo del jugador
pictogramas representativos para el jugador. Esta mascota es donde se almacenan los niveles y los objetivos que va alcanzando
también la representación del sistema de ayuda del juego. Así, el (tanto educativos como lúdicos).
niño verá reforzada la confianza en este avatar y la acción de
solicitar ayuda será simplemente consultar con la mascota. 4.1.7 Introducción de Conocimiento
Durante el juego hay momentos en que es necesario introducir
4.1.5 Retos y Objetivos conocimiento de forma explícita, pero sin que el niño sea
Es importante plantear un conjunto de retos y objetivos a cumplir. consciente de ello y pierda la inmersión. Para ello se propone la
Unas veces relacionados con el objetivo educativo, aunque inclusión de Actividades Extra. Sirven para presentar conceptos
mostrados como objetivo de juego de forma transparente al niño, en forma de cuentos o vídeos que encajan en la dinámica del

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 177 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
juego. Se utilizan también para añadir ejercicios de evaluación [5] Cubas Mejía, Antonieta; Narváez Rivero, Miriam; Espinoza
extra, que permiten a los educadores realizar un mejor Castro, Liliana; Mouchard Seminario, Teresa; Las
seguimiento de la evolución del niño y saber que, efectivamente Inteligencias Múltiples, Pedagogía Serie 1 para docentes de
está adquiriendo el conocimiento esperado. Secundaria. Nuevos Paradigmas Educativos, 2007
[6] López Cassà, Èlia; GROP (Grup de Recerca en Orientació
5. CONCLUSIONES Y TRABAJO FUTURO Psicopedagògica); Educación Emocional. Programa para 3-6
Las emociones no sólo se pueden enseñar, sino también generar.
años, Educación Emocional y en Valores, Wolters Kluwer
Si sumamos los hechos de que los videojuegos son por sí mismos
España, S.A., mayo 2010, ISBN 978-84-7197-880-6
medios de generación de emociones y que hacen partícipe directo
de su desarrollo a quien los juega junto con el hecho de que los [7] Ribes, Ramona; Bisquerra, Rafael; Agulló, Mª Jesús; Filella,
niños ya desde pequeños están en contacto con dispositivos Gemma; Soldevila, Anna, Una propuesta de currículum
tecnológicos como tabletas y móviles, se da la circunstancia y la emocional en educación infantil (3-6 años), Cultura y
herramienta ideal para dirigir hacia este punto la educación de las Educación, Vol. 17-1, Págs. 5-17, 2005, ISSN 1135-6405
emociones. Utilizar un videojuego que además se haya creado [8] Canals, Mireia; Aguilar, Sandra; Emociones, Salvatella 2012,
especialmente para niños con el objetivo de aplicar un programa ISBN: 978-84-8412-635-6
de Educación Emocional puede ser un gran paso para las escuelas,
ya que conseguiría enseñar de las dos formas más eficaces: [9] Puig Ontiveross, Juan Arturo; Colección Emociones,
jugando, que es la manera natural en que los niños aprenden; y Aplicación, Salvatella 2012,
practicando. https://itunes.apple.com/es/app/coleccion-
emociones/id500047073?mt=8
En base a los requisitos identificados en este trabajo, y al diseño [10] Queensland Government – Department of Education and
inicial del videojuego, estamos trabajando en el desarrollo de una Training, The Allen Adventure, The State of Queensland
arquitectura y metodología específica para el diseño de (Department of Education, Training and Employment) 2013,
videojuegos educativos que incluyan la Inteligencia Emocional Aplicación, https://itunes.apple.com/ve/app/the-allen-
como refuerzo del aprendizaje. adventure/id610042067?mt=8
Por tanto, el trabajo futuro se dirige hacia el diseño formal del [11] La Guardia Ediciones, El taller de Reparancio, La Guardia
videojuego propuesto en este artículo, su prueba y evaluación en Ediciones 2014,
entornos reales. Durante este proceso se continuará la http://www.laguaridaediciones.com/noticias/2014/07/23/jueg
investigación en las técnicas de generación de emociones y se o-de-cartas-el-taller-de-reparancio/
usarán las directrices propuestas por el grupo que investigan en el
desarrollo de videojuegos educativos [18][25][26]. De esta forma, [12] Asociación Española Contra el Cáncer (AECC), El Bosque
el objetivo es obtener un juego divertido y educativo, sin perder a Encantado, AECC 2014, juego online,
favor de uno u otro aspecto. Además, se realizarán experiencias http://www.elbosqueencantado.aecc.es
con prototipos que permitan guiar este diseño hacia un producto [13] El Perruco, El Perruco, El Perruco 2014, web,
que realmente sea efectivo tanto desde el punto de vista de los http://www.elperruco.com/
educadores como de los niños. [14] Alava Reyes Consultores S.L., Gomins, Aplicación, Alava
Reyes Consultores S.L. 2014, http://www.gomins.es/inicio/
6. AGRADECIMIENTOS
Este trabajo está financiado por el Ministerio de Ciencia e [15] EA Games, IF…, http://www.ifyoucan.org/, Enero 2015
Innovación, España, como parte del Proyecto VIDECO [16] Olson, Cheryl K., Children’s Motivations for Video Game
(TIN2011-26928), Red de Trabajo Iberoamericana de apoyo a la Play in the Context of Normal Development, American
enseñanza y aprendizaje de competencias profesionales mediante Psychological Association, Vol. 14-2, Págs. 180-187, 2010,
entornos colaborativos y ubicuos (CYTED - 513RT0481) y el 10.1037/a0018984
Proyecto de Excelencia P11-TIC-7486 financiado por la Junta de
Andalucía. [17] Granic, Isabela; Lobe, Adam; Engels, Rutger C. M. E., The
Benefits of Playing Video Games, American psychologist,
7. REFERENCIAS Vol. 69-1, Págs. 66-78, Diciembre 2013, 10.1037/a0034857
[1] Viloria, Carmen de Andrés; La educación emocional en [18] González Sánchez, J. L.; Padilla-Zea, N.; Gutiérrez, F. L.;
edades tempranas y el interés de su aplicación en la escuela, Cabrera, M. J., De la Usabilidad a la Jugabilidad: Diseño de
Tendencias Pedagógicas, Vol. 10, Págs. 107-123, 2005 Videojuegos Centrado en el Jugador, IX Congreso
[2] López Cassà, Èlia; La educación emocional en la educación Internacional Interacción, Albacete, Grupo LoUISE-
infantil, Revista Interuniversitaria de Formación del Universidad de Castilla-La Mancha, Págs. 99-108
Profesorado, Vol. 19-3, Págs. 153-167, 2005, ISSN 0213- [19] PsicoActiva, http://www.psicoactiva.com/emocion.htm, 2015
8464
[20] Psico Vida, http://www.psico-vida.com/2014/10/2570/, 2014
[3] Vera García, Mª del Mar, El desarrollo emocional de los
[21] Goleman, Daniel; Emotional Intelligence, 1995
niños, Innovación y Experiencias Educativas, Vol. 15,
Febrero 2009, ISSN 1988-6047 [22] Gardner, Howard, Inteligencias múltiples. La teoría en la
práctica, Barcelona: Paidós, 1995
[4] Bisquerra Alzina, Rafael; Pérez Escoda, Núria; Educación
Emocional: Estrategias para su puesta en Práctica, Avances [23] Emotional Intelligence Activities for children ages 8-10,
en Supervisión Educativa. Revista de la Asociación de http://www.ong.ohio.gov/frg/frgresources/emotional_intelleg
Inspectores de Educación de España”, Vol. 16, mayo 2012 ence_8-10.pdf, 2003

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 178 |
 Enseñando Emociones a Niños Mediante Videojuegos
N. M. Fuentes García | R. López Arcos | F. L. Gutiérrez Vela | P. Paderewski Rodríguez | A. Torres Soto | N. Padilla Zea
[24] Arce Lara, Roberto Junior; La psicología infantil aplicada a [27] Vallés Arándiga, A; S.l. C.L.E: Siendo inteligentes con las
la creación de personajes, La Libertad, Antiguo Cuscatlan, El emociones, 2000
Salvador, junio 2010 [28] Agulló Morera, M. J. et al., La educación emocional en la
[25] Padilla-Zea, Natalia; Gutiérrez, Francisco L., López-Arcos, práctica. Horsori, 2010
José Rafael; Abad-Arranz, Ana; Paderewski, Patricia, [29] Heil, Marianela, Los Cinco Sentidos Humanos 5 Sentidos del
Modeling storytelling to be used in educational video games, Cuerpo Humano, http://historiaybiografias.com/sentidos/, 22
Computers in Human Behavior, Págs, 461-474, Mayo 2013, de diciembre de 2014
http://dx.doi.org/10.1016/j.chb.2013.04.020
[30] Nicolàs, Cristina, 071 Sé visual, compra una pizarra, Sueldo
[26] Padilla Zea, Natalia; Metodología para el diseño de 3.0, http://www.sueldo30.com/visual-pizarra/, 10 de marzo
videojuegos educativos sobre una plataforma para el análisis de 2015
del aprendizaje colaborativo, Tesis Doctoral, Universidad de
Granada, 2011

XVI Congreso Internacional de Interacción Persona-Ordenador | Child Computer Interaction

| 179 |
INTERFACE DESIGN
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó

CamScan, an application to identify everyday objects for

users with vision impairments
Rodrigo Capa-Arnao Cristina Manresa-Yee Ramon Mas-Sansó
Department of Mathematics Department of Mathematics Department of Mathematics
and Computer Science and Computer Science and Computer Science
Universitat de les Illes Balears Universitat de les Illes Balears Universitat de les Illes Balears
Crta. Valldemossa km 7.5 Crta. Valldemossa km 7.5 Crta. Valldemossa km 7.5
07122, Palma, SPAIN 07122, Palma, SPAIN 07122, Palma, SPAIN
[email protected] [email protected] [email protected]

ABSTRACT Instead of using specialized hardware like in the past for particular
Technology can help users with blindness or vision impairments tasks such as barcode readers, color identifiers, currency
to overcome everyday challenges. Furthermore, the widespread recognizers or talking OCR, we can use only a mobile device. The
acceptance and use of mobile phones, makes it suitable to develop increase of performance and the inclusion of cameras in mobile
accessible technologies on these devices. One common problem devices together with reliable and affordable Internet connections,
that users with vision impairments face is the identification of has led to assistive technologies using only this device.
everyday objects, especially when these objects have a similar Furthermore, the main operating systems on mobile devices
shape as others (e.g. cans, CDs). Different works have tackled this include accessibility services such as screen magnifiers or built-in
problem including object recognition and remote workers on the screen readers (VoiceOver in iPhone, or Talkback in Android)
internet who answer questions on the image sent. This work which support users affected by sight loss.
approaches this problem and presents new features with the aim of We present an application aiming at enabling people with vision
bringing new capture possibilities and then discusses the findings impairments to identify everyday objects by taking captures with
a mobile device (Android-based) and using remote services to
Categories and Subject Descriptors answer the user. Our proposal includes the capability of filtering
H.5.2. Information interfaces and presentation: User interfaces. the captures with the information obtained by a mobile device
K.4.2 Social Issues: Assistive technologies for persons with with an integrated RGB-D camera and also allows the user to
disabilities select among three capture modes (infinity, autofocus, macro).
Finally, we conduct an informal testing to evaluate the object
General Terms identification and observe the use of the added characteristics.
Design, Human Factors
The work is organized as follows: Section 2 reviews works aiming
Keywords at object identification with portable devices. Section 3 and 4
Assistive technologies; vision impairments; object identification; describe CamScan and the new capture enhancements that we
Capture; RGB-D propose. Section 5 reports the results of the evaluation. Finally,
we discuss our findings and outline future work.
1. INTRODUCTION
What is this? Which CD is that? What is inside this? Everyday 2. RELATED WORK
objects identification can be a challenge for users with vision In this section, we review related works that use mobile devices to
impairments both because objects have a similar shape such as identify everyday objects, emphasizing virtues and shortcomings,
CDs, currency or cans, or because the user is not familiar with the with the objective of detecting potential improvements to
object. The continuous failure in the identification can be implement.
frustrating for the user. Mining the literature, we can find works using computer vision
Together with reading (text to transcribe) and description techniques to identify a reduced set of objects previously
(description of visual or physical properties of a depicted object), classified such as currency [2][3] or objects previously recorded
object identification was one of the main categories [1] presented [4]. Liu [2] presented a work that helped identifying US dollars.
in a taxonomy of the types of visual challenges in the everyday LookTel Money Reader [3] is an app for iOS and recognizes
lives of blind people. currency and speaks the denomination. Several currencies are
supported including the US Dollar, Euro, British Pound, Canadian
In a home environment, someone else can label the objects, but Dollar, and Australian Dollar. LookTel also has a version,
this decreases autonomy for the user. To assist in this task and LookTel Recognizer [4] that permits users to store images of
whenever the degree of vision impairment allows it, users can objects in a database, for its later recognition. In this case, it may
make use of magnification tools, either optical or computerized. be interesting to store different points of view of the object in
However, this approach is not effective for users with severe order to improve the identification.
vision impairments. Another solution, more difficult to ensure, is
the correct labeling of the objects at a production stage. This is the Google Goggles [5] is an image recognition mobile app developed
case of the medication packaging, where they normally have by Google. It is used for searches based on pictures or barcodes
information in Braille. taken by handheld devices and it returns a web with information
on the place or object being identified (includes OCR). It is a very

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 181 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó
fast system although it does not always offer a satisfying answer. 3. SYSTEM PROPOSAL
The system is not specially adapted to users with vision We have carried out an informal evaluation with different
impairments, but it can be combined with screen readers. The app available systems (using only mobile devices), and we realized
is no longer supported for iPhone devices and it is not too popular that frequently, the object under study is just occupying a small
on Android mobiles. CamFind [6] is a free app, similar to Google region of the image and sometimes the autofocus does not focus
Goggles, but more functional (according to users’ opinion) due to correctly on the object we want. An example of this situation is
its easiness. As drawbacks, its use in Spanish is not too effective when a person picks up a product in a supermarket holding it in
and it does not include a barcode reader. In this line, we find his or her hand and takes a photo. Behind, other products can be
TapTapSee [7], which uses the same search engine as CamFind, visible in the scene, so the autofocus may not focus correctly.
but the descriptions are more informative. One of the These problems may not be noticed by users with sight
characteristics of this system is that if the image is not identified, limitations.
then it is processed by remote workers on the Internet. The
interface is totally accessible and it can be used in both Iphone The result given by the different systems also differ. As stated
and Android. However, the translation of the interface to Spanish before, all systems work in English and then they use automatic
is not too correct yet and even if the object identification is correct translators to translate it to other languages. Diverse services for
in English, the translation to Spanish is sometimes confusing. This translating are used. Based on [16][17] and our own informal
system has a cost, that it can be a monthly service fee or pay per testing with Yandex, GoogleTranslate and BabelFish, the last two
number of images. have the best translations. Furthermore, the translation to Spanish
improves when the result given for the object identification is
The use of remote workers to assess in the task of identifying included in a complete sentence such as “The image describes +
objects has been used in Vizwiz [8], Vizwiz:Locatelt [9] and in [result of the object identification]” and sent to the translation
Chorus:View [10]. These systems enable the user to add questions services.
attached to the image, which offers more flexibility and
interaction possibilities. VizWiz and VizWiz:LocateIt only allow IPhone is considered very accessible for users with vision
one question, but Chorus:View offers a more conversational impairments [18], but we will work on Android devices basically
crowd assistant. because of its open nature. Further, we can find many different
devices (makes and models) with it and a wide range of prices,
There are other works that attempt to identify objects without making it more accessible economically.
using specifically a mobile device but striving for mobility.
EyeRing [11][12][13][14] is a wearable interface, a ring that Based on the observations done in the initial evaluation, we
allows a person to point at an object to see or hear more propose a system for Android devices with new image capture
information about it. The image captured by the device is sent to a characteristics. The system will aim at identifying an object using
mobile or to a PC so it can be processed by the appropriate a remote service that receives the image, searches for it in a
software. One of the advantages is that the user directs his finger database and if a satisfactory result is not found, then it is sent to a
to the scene and does not have to worry about the position of the remote worker to be identified. First, we will offer the user three
mobile device’s camera. The system is a prototype and needs a capture modes depending on the distance to the object: when an
particular dedicated hardware. OrCam [15] is a commercial extreme close-up photography has to be captured, a macro mode
product consisting of a portable device with a camera mounted on will be used. If the object is a few centimeters away, an autofocus
the frames of the eyeglasses. When the user points at something, will be used. And finally, the third mode will be full-focus, for
this is identified and the information is given through a bone- objects further than one to two meters. Full-focus is the capability
conduction earpiece. The use of this system is more effective for to focus objects from different distances which allows focusing all
users with a minimum degree of sight. objects. The inconvenience using it is that it does not allow
focusing too close. Although not all devices include this feature,
When analyzing the diverse systems, different aspects are taken Android is able to do an infinity focus in these cases.
into account. First, not all of the aforementioned systems are
designed for users with loss of sight or blindness, but the mobile The second proposal is the use of images together with its depth
operating systems accessibility options, can help overcoming map. RGB-D cameras in mobile devices are an emerging feature
these limitations. Internationalization is an important factor too. (e.g. HTC or Google in their TANGO project [19]. There are not
Most systems are planned for English speakers, so the interface is too many devices yet with this characteristic, and the technology
in English and in some cases its translation is not correctly done is still not mature enough. Results are not optimal, but are
(e.g. TapTapSee). Answers are usually also given in English. sufficient to test this functionality and to contribute with the
However, answers can be translated by automatic translators, insights for future developments.
whose translations can sometimes lead to confusion in other
In order to test these two functionalities, we designed and
languages. In the case of those systems where the user records the
developed an app, which allows selecting the capture mode and
object identification, the user is the responsible for the
use the depth information to remove the background of the object.
information given. Another aspect to analyze is whether they need
We assume the use of TalkBack or other screen reader services
specific hardware or not.
for navigation purposes.
The drawback of systems based on reverse image search is that
The usability will be taken into account for users totally blind or
the result may not be always appropriate or related to the original
with sight loss, so the interface will aim for easiness reducing the
image. In the case where remote workers participate, answers are
number of controls and app states. The system must be able to be
usually subjective but are really related to the image given.
used in Spanish and in English, and to be translated to other
However, they can be slower than reverse image search, as it
languages in an easy way, following the Google recommendations
depends on the cost of service and the workers income.
for the app localization Android Developers. Localizaing with

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 182 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó
resources. 4.2 Design
http://developer.android.com/guide/topics/resources/localization.h To achieve an easy-to-use system, all the functions except for the
tml. Last access April 2015.. settings, are accessible from the main screen. There are no
graphical controls, as all the interaction is carried out by different
4. CamScan multitouch gestures: 2-finger swipes or long-press gestures. This
In the following subsections, we will describe the CamScan achieves an app with a reduced number of states, avoids
application, the device and tools used to develop the system.
interaction with graphical controls and facilitates the navigation
4.1 Apparatus and development libraries with TalkBack.
The mobile device used for testing the app and these features will In Fig. 3, a scheme of the functioning of CamScan is shown.
be a HTC One M8, which uses Android Kit Kat 4.4.3 and Initially, the main screen is presented, where TalkBack will read
includes two back cameras (denominated by the manufacturer as the name of the app initiated and a welcome message followed by
Dual Lens) (see Fig. 1) which capture images in the 2D-plus- a specific pattern of vibrations, for the user to be sure that he or
Depth format, that is, a 2D image frame supplemented with a she is in the correct place. Moreover, the name of the app is
greyscale depth map (see Fig. 2). To process the images, written in big characters to help those users with sight loss but not
developers need the HTC Dual Lens SDK which is still under totally blind.
development. Images have a resolution of 2688x1520 pixels and
the image for the depth map has a resolution of 1280x720 pixels. Then, the three different capture modes are shown on screen, each
of them is presented with a sentence and the screen is divided in
To translate the results we will use Google Translate, due to the three regions separated with thick lines and with no boundaries on
number of languages that it supports and the results it achieves for the sides. By placing the fingers on each region, the text will be
Spanish. read aloud.
TalkBack uses two fingers to interact, so all the app has to be The design decision of dividing regions in the vertical axis and the
prepared to interact with two fingers and all elements should have position of the three modes is due to the natural position of the
its description included to be read. user holding the phone vertically and the proximity of each region
to the user: the farthest region (higher region) is the infinity focus,
the middle region corresponds to the autofocus and the closest
region is the macro focus (See Fig. 4). The app is only designed
for the portrait orientation to keep its simplicity; however, after
selecting the mode, the user can rotate the phone to the landscape
orientation to capture the image. In Fig. 5, the use of the app using
one hand is depicted.
The design decision of using the long-press gesture instead of
Figure 1. Cameras detail of the HTC One M8. double tap to capture an image, was due to the initial evaluation
where we observed that using the double tap could cause the
image to be blur specially when using the device only with one
hand. The long-press enables the user to maintain the device more
stable than the double tap.

Figure 2. Image and depth map captured by the mobile

device.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 183 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó

Figure 5. A user holding the device with one hand and sliding
through the different capture modes
The help menu is a basic menu functionality included in the main
screen, as the interface cannot be enriched with graphical
elements. It adds an extra element to the presentation screen, but
is fundamental that users can access the help menu in a simple
way. Therefore, a small region at the lower part of the mobile
screen has been reserved for the menu of help, for it to be read by
TalkBack in the initial screen. The message given is “Slide down
with two fingers to bring out the help menu”. If TalkBack is not
detected when the app is loaded, then the message is changed to
“Slide down with one finger to bring out the help menu”.
Once the capture is done, to access the other functionalities the
user will perform a fast sliding towards different directions. Down
will bring out the help menu, to the right will send the image to
identification, to the left will go to the previous menu or will close
the app and finally, up will bring out the settings.
The capture and identification has been divided into two tasks: the
Figure 3. Functioning of the CamScan application capture and the sending of the image. Sending the image for
identification means time and can have a monetary cost or
increases the user’s data usage on the mobile. Providing these two
steps allows users to repeat the image capture without sending the
image, prevents unintentional sending and avoids multiple image
sending. Expert users would maybe prefer this task to be done in
just one step; therefore, this option can be activated in the settings.
The decision of sliding to the right for identification and going
back with the left movement has been based on the reading
direction for the Spanish (and occidental) population (from left to
right). So going to the right would mean advancing and going to
the left would mean going backwards. However this decision can
be reversed in the settings section. Further, before going out from
the app, the system will ask for confirmation to avoid
unintentional actions.
5. Evaluation
The system was evaluated to analyze three factors: the easiness of
the system, the functioning of the capture parameters and finally
the effectiveness of the filtering using the depth map.

Figure 4. Presentation screen with the 3 capture modes. 5.1 Easiness of use
The system was initially tested by 3 three ‘proxy users’ who
represented actual users [21]. Working with research participants
with impairments. Wiley. These users did not use the accessibility
options on their mobile device and they were new to TalkBack. A
training period of around 15 minutes was dedicated to TalkBack,
however, to be an expert on it, users should train longer periods.
Even though, this initial trial is important to explore the usability
of the system and to analyze the first findings.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 184 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó
Three users with no vision impairments (2 male, 1 female with an receives the answer of the identification, where the remote users
average age of 35) were introduced to the app. The procedure we can notify that the image is blur.
followed was: Finally, the full-focus mode does not focus, there is a compromise
1) Introduction to the app: participants were introduced to the between the distance and the image definition. The benefit of this
app. We explained the aim of the app and its general mode of capture is that if the object is at an adequate distance, it is
working, without showing them any screen. We were the fastest system.
interested in the first contact of users with the app.
5.3 Filtering using the map of depth
2) TalkBack training: users went through a short tutorial about Filtering the image with the map of depth can be suitable in those
TalkBack. Without trying CamScan, they moved around the situations where there are multiple objects in the background such
screens and menus on the mobile. The evaluator highlighted as in a supermarket or at a home.
the concept of “sliding a finger on the screen is like looking
at that zone”, and all the text below the finger is read. To evaluate this functionality, a field trial was held in a
Participants did not need to worry about tapping, as the supermarket. Several captures were taken by holding the object
system only responds at taps with two fingers. For these with one hand, extending it and taking a photo with the mobile.
three users, the use of TalkBack was not intuitive, so they Then, the captures were sent with the original image and the
had problems interacting with it. filtered image to compare the answers given by the system. 81
images were captured, but we select the most interesting cases to
3) Hands-on the app: the mobile phone was then introduced into exemplify different captures.
a cloth bag with CamScan active. The main problems came
due to the interaction with TalkBack, as users were not used When capturing objects with flat surfaces, the filtering achieved
working in that manner and they got confused. This problem correct results segmenting the object using the depth map (See
will probably not be encountered with users who are used Fig. 6). With the original photo, the answer was “Cartón de leche”
working with screen readers. Then, the functioning of the (“milk carton”) and with the filtered image, the answer included
app was shown to users, and then they all found its use the label of the milk carton.
simple and intuitive. When users were working at the When objects are not placed perpendicularly to the camera, the
beginning with TalkBack, as the description given to each filtering is not too effective as there is a compromise between the
region was a complete sentence, users felt it was more distance to capture and the distance to filter. In Fig. 7, we find a
complicated that what it really was. The descriptions for packet of flour. In this case the top of the image (the farthest) was
TalkBack were shortened to: Close, Normal and Far in the removed when filtering. However, answers to both images were
main screen, and more detailed information was given in the not correct: in the first case, the answer was “Harunas comida de
Help menu. perro mallorca” (“Flour dog food mallorca” and the word flour
4) Hands-on the app (2nd attempt): the mobile phone was again was not well spelled) and the answer to the filtered image was
introduced in the bag, and in this occasion users could carry “paquete rojo y blanco de la etiqueta” (“package red and white
out different capture tasks. In this stage, sometimes the user from the label”, the sentence in Spanish does not make sense.).
did not know the exact state of the app: after the capture, and We tested the system with non-flat objects like a bottle (See Fig.
while the depth map is processed, there is a delay of a few 8. There are some errors at the edges, but the object is complete
seconds until the main screen is shown again, and during the and the identification gave good results in both cases: “Persona
sending and identification of the image, there is a delay too. sostiene una botella de bebida Pago” (“Person holding a bottle of
To improve the user experience, verbal and haptic drink Pago) and “Botella de pago” (“Bottle of pago”).
notifications were included: a vibration after the capture and
When testing with small objects, a can of anchovies of 10x5 cm,
a periodic message with a vibration notifying that the
the filtering included part of the shelf behind (See Fig. 9). This
identification is being processed.
fact can be due to the transparency of the shelf or because the
Proxy users cannot simulate completely an end-user experience, depth map considered the shelf to be close enough to the can to
as these users are already used to screen readers and exploring the include it at the same depth distance. In both cases the
apps using them. However, the design has been enhanced due to identification was correct “Anchoas consorcio” (“Anchovies
the findings of these tests. consorcio”).
5.2 Capture, focus and flash
TapTapSee uses an automatic focus and flash, but results are not
always satisfactory as some focus take between several seconds
and one minute and the flash is not always activated when needed.
When the system has focused, an audible notification is triggered.
CamScan in macro mode can require a few seconds to focus.
Every attempt focusing and failing will be notified with a sound
and automatically it tries to focus again until it succeed, then it
takes the photo and emits a sound of capture. Each attempt to
focus is about 2 or 3 seconds, unlike TapTapSee.
The normal mode corresponds with the normal autofocus, but it
has been configured for the focus to be done in the centre of the
image. The focus can need 3 to 5 seconds. Sometimes, the photo
will not have focused, but the user will not notice it until he or she

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 185 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó

Figure 6. Capture of an object with flat surfaces. Figure 9. Capture of a small object.

Figure 7. Capture of an object placed perpendicular to the Figure 10. Capture of a shiny object.
camera

Figure 8. Capture of an object with non- flat surfaces. Figure 11. Capture of a transparent object.

Finally a transparent object was captured too, to test the map of

To analyze how the material could affect the image and the
depth in difficult conditions. In Fig. 11, the capture of a
filtering, a set of cans with shiny surfaces were captured (see Fig.
transparent bottle can be seen. The algorithm to compute the map
9). Filtering achieved good results, but the identification in both
of depth looses effectiveness, but as the label is still readable, the
cases was not too successful due to translation problems. With the
identification was correct answering “Naturis agua embotellada”
original image we obtained “Verde y blanco Malkon 6 latas de
(“Naturis bottled water”).
envase” (Green and white Malkon 6 can”, the label was wrongly
spelled) and with the filtered image “Mahou verde puede” Once tested that the filtered image could achieve successful or
(“Mahou green can”, ‘can’ was translated as the verb can or to be similar results as the ones given by sending the original image, it
able to instead of a can as a recipient). is interesting to observe the decrease of the images sent. The
average image size for the 81 images taken was o 143.113 bytes
and the average size for the filtered images is of 98.157. That
means a difference of 44.956 bytes.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 186 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó

6. FINDINGS AND CONCLUSSIONS [2] Liu, X. 2008. A camera phone based currency reader for the
In this work we presented an app that uses the mobile device to visually impaired. In Proceedings of the 10th international
identify everyday objects by taking a photo and sending it to ACM SIGACCESS conference on Computers and
remote services. This app has three capture modes: macro, accessibility (ASSETS '08). ACM, New York, NY, USA,
autofocus or infinity. Each of these modes can help acquiring 305-306
better photographs. [3] LookTel Money Reader.
Further, several mobile devices offer the possibility to take http://www.looktel.com/moneyreader. Last visited April
photographs and compute the depth map. We use this depth map 2015
to filter the background of the image and keep only the object [4] LookTel Recognizer. http://www.looktel.com/recognizer. .
under study. Last visited April 2015
The user interacts with the app trough a simple interface with big [5] Google goggles.
interaction zones to require easy interactions with finger gestures https://play.google.com/store/apps/details?id=com.google.an
without the need to tap precisely on any graphical control. The droid.apps.unveil&hl=es. Last visited April 2015
system has to use a screen reader such as TalkBack. [6] CamFind. http://camfindapp.com/. Last visited April 2015
The option of offering three capture modes was not appreciated by [7] TapTapSee http://www.taptapseeapp.com/. Last visited April
all users, as some knowledge about photography may be needed 2015
to understand the different captures modes.
[8] Bigham, J.P., Jayant, C., Miller, A., White, B., and Yeh, T.
Regarding the usability of the application, once users understood 2010. VizWiz::LocateIt - enabling blind people to locate
how TalkBack worked and was combined with CamScan, they objects in their environment. In Proc. 2010 IEEE Computer
found the functioning intuitive and simple. It is important to test Society Conference on Computer Vision and Pattern
the app with end-users, as they will not encounter the same Recognition Workshops (CVPRW), 65-72
problems as the proxy users. However, we included a settings
menu to offer a flexible app to adapt to users’ preferences. [9] Bigham, J.P, Jayant, C., Ji, H., Little, G., Miller, A., Miller,
R.C., Miller, R., Tatarowicz, A., White, B., White, S. and
Using the map of depth to filter the original images, we can gain Yeh, T. 2010. VizWiz: nearly real-time answers to visual
speed in the transmission of the image and reduce costs. Further, questions. In Proceedings of the 23nd annual ACM
and maybe with a higher impact than the cost or speed, is that the symposium on User interface software and technology (UIST
user can preserve some degree of privacy of his or her '10). ACM, New York, NY, USA, 333-342
environment, especially at home. By using the filtering option, the
user just sends the object to be identified. [10] Lasecki, W.S., Thiha, P., Zhong, Y., Brady, E. and Bigham,
J.P.. 2013. Answering visual questions with conversational
By sending only the object under interest, we help the remote crowd assistants. In Proceedings of the 15th International
worker to focus only on that object and they may give more ACM SIGACCESS Conference on Computers and
concise descriptions, and if the identification is done by computer Accessibility (ASSETS '13). ACM, New York, NY, USA,
vision techniques, we remove the background which could Article 18
improve the object recognition (if filtering is successful).
[11] Nanayakkara, S., Shilkrot, R., and Maes, P. 2012. EyeRing: a
This assistive app should evolve as a holistic system including finger-worn assistant. In CHI '12 Extended Abstracts on
software, hardware and multimodal input and output. Multiple Human Factors in Computing Systems (CHI EA '12). ACM,
functionalities should be integrated such as OCR, colour New York, NY, USA, 1961-1966
detection, navigation information, local database of objects, or
remote worker assistance. [12] Nanayakkara, S., Shilkrot, R., Yeo, P., and Maes, P. 2013.
EyeRing: a finger-worn input device for seamless
A research line related to this project that has to evolve to improve interactions with our surroundings. In Proceedings of the 4th
the results of the identification is the automatic translation. In our Augmented Human International Conference (AH '13),
testing, answers in English were correct in some cases, but the ACM, New York, NY, USA, 13-20
translation to Spanish was confusing.
[13] Nanayakkara, S., Shilkrot, R., and Maes, P. 2012. EyeRing:
Computer vision techniques could also improve object recognition an eye on a finger. In CHI '12 Extended Abstracts on Human
to be able to recognize objects without the need of remote Factors in Computing Systems (CHI EA '12). ACM, New
workers. However, when including them, it would be interesting York, NY, USA, 1437-1438
to work with people close to the user’s local or national
environment to achieve more accurate responses. [14] Shilkrot, R., Huber, J., Liu, C., Maes, P. and Nanayakkara, S.
2014. FingerReader: a wearable device to support text
7. ACKNOWLEDGMENTS reading on the go. In CHI '14 Extended Abstracts on Human
This work is supported by the Department of Maths and Computer Factors in Computing Systems (CHI EA '14). ACM, New
Science (University of Balearic Islands). York, NY, USA, 2359-2364

8. REFERENCES [15] Orcam. See For yourself. http://www.orcam.com/. Last

[1] Brady, E., Ringel Morris, M., Zhong, Y., White, S. and visited April 2105
Bigham, J. P. 2013. Visual challenges in the everyday lives [16] Hampshire, S. and Porta Salvia, C. 2010. Translation and the
of blind people. In Proceedings of the SIGCHI Conference Internet: Evaluating the Quality of Free Online Machine
on Human Factors in Computing Systems (CHI '13). ACM, Translators. In Quaderns. Rev. trad. 17, 197-209
New York, NY, USA, 2117-2126

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 187 |
 CamScan, an application to identify everyday objects for users with vision impairments
R. Capa-Arnao | C. Manresa-Yee | R. Mas-Sansó
[17] Shen, E. 2010. Comparison of online machine translation [20] Android Developers. Localizaing with resources.
tools. In TcWorld. http://www.tcworld.info/e- http://developer.android.com/guide/topics/resources/localizat
magazine/translation-and-localization/article/comparison-of- ion.html. Last access April 2015.
online-machine-translation-tools/. Last visit March 2015 [21] Lazar, J. Feng, J.H, Hochheiser, H.2009. Research methods
[18] Federal Communication Commission. 2012. First biennial in human-computer interaction. Chapter 15. Working with
report to Congress on the Communications and Video research participants with impairments. Wiley
Accessibility Act
[19] Project Tango – Google Developers.
https://developers.google.com/project-tango/. Last visited
March 2015

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 188 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud

Interacción de los Usuarios con Aplicaciones Web Offline:

un Caso de Estudio
Félix Albertos Marco Víctor M.R. Penichet José A. Gallud
Instituto de Investigación en Escuela Superior de Ingeniería Escuela Superior de Ingeniería
Informática de Albacete (I3A) Informática de Albacete Informática de Albacete
Universidad de Castilla-La Mancha Universidad de Castilla-La Mancha Universidad de Castilla-La Mancha
[email protected] [email protected] [email protected]

RESUMEN más tiempo conectado, a través de un número mayor y más

La interacción del usuario con las aplicaciones web ha sufrido variado de dispositivos, así como en multitud de escenarios y
cambios importantes durante la última década. El avance contextos. Pero también es cierto que existen multitud de
tecnológico permite que el usuario pase de ser un simple escenarios donde esa conectividad no es posible debido a
consumidor de información a producir gran parte de esta. Tanto es numerosos factores como son las zonas rurales o aisladas, los
así que gran parte de la interacción y gestión de la información desastres naturales o situaciones sin cobertura a la red (viajes en
con estas aplicaciones se produzca en el lado cliente, en el tren, avión …).
navegador local. El auge de nuevos escenarios donde las Uno de los campos de aplicación que han experimentado un
aplicaciones web son utilizadas, ha hecho que en la interacción mayor auge dentro de las aplicaciones web es el de las
con las interfaces de usuario se introduzcan nuevos mecanismos. aplicaciones destinadas a gestionar contenidos educativos en la
De entre estas aplicaciones web, las dedicadas al e-Learning son red, los denominados sistemas e-Learning. Si bien se han
cada día más utilizadas, permitiendo la gestión de contenidos estudiado numerosos aspectos relacionados con estos sistemas,
educativos en la red. Pero no siempre existe una conectividad a la como son la colaboración, su utilización en dispositivos móviles o
plataforma web que estemos utilizando. Diversos motivos lo el awareness producido por estos sistemas, el estudio de
impiden: zonas rurales, desastres naturales o situaciones sin escenarios offline adolece de una falta de propuestas y soluciones
cobertura. Es por ello que surgen las aplicaciones web offline o para paliar los efectos negativos que estos conllevan.
desconectadas. El presente trabajo profundiza en la problemática
de la utilización de aplicaciones web existentes en escenarios Es en este punto donde se necesitan estudios que proporcionen
offline. Para tal fin, se introduce la herramienta Proxy Offline y resultados acerca de cómo se ven afectadas las aplicaciones web
se describe un test piloto que analiza cómo se ve alterada la en entornos offline. Pero dentro de estas aplicaciones podemos
interacción del usuario con las interfaces web offline con el distinguir claramente dos aproximaciones. Por un lado, existen las
objetivo de mejorarla. aplicaciones web diseñadas y producidas teniendo en cuenta los
escenarios offline. Por el otro, se encuentra la posibilidad de
Categorías y Descriptores adaptar las aplicaciones web existentes para permitir la
H.5.m Information interfaces and presentation (e.g., HCI): interacción offline.
Miscellaneous.
En este trabajo se analiza mediante la evaluación de un caso de
Términos Generales estudio la interacción de los usuarios con aplicaciones web
Design, Experimentation, Human Factors. existentes que han sido adaptadas para el trabajo offline mediante
la utilización del Proxy Offline. El Proxy Offline es una
Palabras clave aplicación que permite tanto modelar la interacción del usuario
Offline Web Applications, e-Learning, Case Study. con aplicaciones web existentes en escenarios offline como el
soporte para la interacción offline en ellas. Con los resultados
1. INTRODUCCIÓN obtenidos en este caso de estudio se espera comprender cómo se
Las aplicaciones web fueron diseñadas desde su origen para ser ven afectadas las aplicaciones web en estos escenarios y ayudar a
utilizadas siguiendo el paradigma cliente-servidor, pero el avance desarrollar soluciones conceptuales y tecnológicas que permitan
tecnológico abre nuevas formas de interacción con estas extender la interacción de los usuarios con la web hacia entornos
aplicaciones. Por ello, las posibilidades de interacción con las offline.
aplicaciones web ha cambiado radicalmente. En la última década, El presente trabajo está organizado de la siguiente manera: el
y especialmente en los últimos años, los usuarios han pasado a ser capítulo 2 introduce la problemática de las interacción offline en
responsables de la creación y mantenimiento de gran parte de los aplicaciones web, presentando el caso de estudio y las decisiones
contenidos de la red [6][13]. Esto implica que se involucren de tomadas para permitir la interacción offline en aplicaciones web;
manera activa tanto en la creación de nuevos contenido, como en el capítulo 3 presenta la evaluación realizada; en el capítulo 4 se
su modificación. Los usuarios han pasado a ser simples exponen los resultados obtenidos; por último el capítulo 5
consumidores de información a ser productores de la misma. presenta el trabajo relacionado, mientras que en el capítulo 6 se
Este avance ha implicado que gran parte de la interacción con la presentan las conclusiones y el trabajo futuro.
aplicación web y la gestión de la información se produzca en el
lado del cliente. Además, también es cierto que cada vez se está

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 189 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud

2. LA INTERACCIÓN OFFLINE CON nivel de presentación permite que el usuario interaccione con la
totalidad de elementos de los niveles inferiores.
APLICACIONES WEB
Como paso previo a la evaluación de la interacción de los usuarios Sin embargo al estar la aplicación web en modo offline, pueden
en aplicaciones web offline, describimos el escenario donde se no estar disponibles todos los elementos del nivel de dominio de
produce esta interacción. En la actualidad, son pocas las la aplicación. En la Figura 1 se muestran atenuados los elementos
aplicaciones web que permiten la interacción offline. Por este que no están disponibles debidos al modo offline de la aplicación
motivo, a la hora de realizar el caso de estudio, se han tomado una web. Siguiendo las funciones de mapeo anteriormente descritas se
serie de decisiones con el fin de habilitar la interacción offline en observa que los elementos de los niveles superiores tampoco están
aplicaciones web. disponibles en su totalidad. Se puede apreciar como hay
elementos del nivel de hipertexto no disponibles y a su vez
Previamente a las decisiones tomadas, se presenta el caso de elementos del nivel de presentación que tampoco están
estudio utilizado con el objetivo de ilustrar estas decisiones. disponibles. Estas circunstancias condicionan la interacción del
2.1 Caso de estudio usuario con la aplicación web y las acciones que este puede
Las tareas web relacionadas con tareas e-Learning se encuentran realizar, por ejemplo, en la interfaz web (nivel de presentación).
entre las más comunes. Es por ello que para realizar el caso de
estudio hemos seleccionado una de las plataformas web más
populares dentro del ámbito e-Learning que se puede encontrar en
la actualidad: Moodle [14]. Moodle tiene una estructura modular
que permite la definición de cursos. Estos cursos están
estructurados en diferentes partes, pudiendo ser por temas o
fechas entre otros. A su vez, cada tema contiene una serie de
actividades y recursos a disposición del usuario.
Para el caso de estudio se organiza la plataforma de modo que el
usuario pueda acceder a unos cursos con el fin de realizar unas
actividades en ellos. Para cada curso, las actividades consisten en
la lectura de unos textos para posteriormente responder a
preguntas sobre ellos. Los textos se pueden presentar de dos
formas: 1) el texto está en la misma página web que las preguntas
y el cuadro de texto proporcionado para las respuestas; 2) el texto
se presenta como un documento PDF, mientras que las preguntas
y el cuadro de texto proporcionado para las respuestas se
encuentran en una página web independiente.
Cuando el usuario realiza la actividad en el caso de estudio, esto
es, responder a las preguntas asociadas al texto, guarda esta
interacción pulsando el botón de enviar ejercicio. A partir de este
momento, los datos quedan guardados en el servidor para su
posterior consulta o evaluación por parte del profesor del curso.
Cuando la interacción del caso de estudio se produce en modo
offline, sin conexión a internet, la realización de las actividades se
ve afectada, ya que no es posible interactuar con la aplicación. Es
por esto que debemos de tomar las decisiones oportunas de cara al Figura 1. Disponibilidad de los elementos en los niveles de las
diseño de la interacción del usuario y las acciones a realizar. aplicaciones web en modo offline
2.2 Decisiones para la interacción offline Dado que el caso de estudio se basa en una aplicación web real, lo
Cuando la aplicación web está en modo offline, puede darse el ideal sería poder interactuar con ella sin necesidad de
caso de que no toda la información esté disponible para su uso en modificarla. Además, también permitiría extender el estudio en
todos los niveles que componen las aplicaciones web (Dominio de otras aplicaciones web ya existentes. De esta forma, los datos
la Aplicación, Hipertexto y Presentación). Además, estos niveles obtenidos de estos casos de estudio serán más cercanos a la
realidad al utilizar aplicaciones web ya utilizadas por los usuarios.
no son independientes, existiendo una función de mapeado entre
ellos. En la Figura 1 se muestra cómo la función de mapeo Para ello se decide que la mejor forma de interactuar con las
relaciona los elementos entre los niveles de la aplicación web. aplicaciones web ofreciendo el soporte offline será a través de la
Estas relaciones indican cómo están conectados los elementos en adaptación de los niveles de la aplicación web ya existente. Pero
los distintos niveles y sus dependencias. no todos los niveles se pueden adaptar de igual manera. En una
De este modo, cuando la aplicación web está en modo online, con aplicación web normalmente el modelo de datos se encuentra en
conectividad al servidor, todos los elementos del dominio de la el servidor, haciendo más difícil su adaptación. Sin embargo, el
aplicación están accesibles. Siguiendo la función de mapeado nivel de presentación y el de hipertexto son más fácil de adaptar
correspondiente, los elementos del nivel de hipertexto están a su ya que la interacción con el usuario se produce de forma directa
vez disponibles. Esto permite que la navegación en la aplicación en el navegador, de forma local. Es por esto que se decide adaptar
web se realice de modo normal. Siguiendo esta ascensión de esos niveles, dejando intacto el modelo de datos. La principal
niveles, y a través de la función de mapeado de presentación, el consecuencia es que se evita el tener que trabajar con la capa de

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 190 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud
datos, que es la que está más fuertemente ligada con la En la Figura 2 se muestra el diseño realizado con la herramienta
implementación de la aplicación web en el servidor. de soporte desarrollada utilizando la notación para la descripción
Para conseguirlo se establecen los siguientes criterios por cada de la navegación. Se establecen como disponibles tanto la página
nivel en la aplicación web: de entrada a la plataforma Moodle (nodo “Inicio”) y los cursos
que contienen las actividades a realizar ( nodos “Comprensión
• Nivel de dominio de la aplicación: El modelo de datos lectora PRIMERO” y “Comprensión lectora SEGUNDO”). Como
no se modifica en la aplicación web original con el fin no disponibles, se establecen las páginas para registrarse como
de poder trabajar con el mayor número de aplicaciones usuario (nodos “Login” y “Logout”), cuya funcionalidad no está
web posible sin que sea necesaria su modificación. disponible cuando no hay una conexión con el servidor web.
• Nivel de hipertexto: la navegación se podrá modificar
de forma que permita la navegación entre las distintas
páginas web dependiendo del estado de la conexión.
• Nivel de presentación: se adaptará la interfaz de usuario
permitiendo decidir los elementos con los que el usuario
puede interactuar dependiendo del estado de la
conexión.
Con estos criterios como base para establecer la interacción con la
aplicación web offline se presentan los siguientes desafíos:
• ¿Cómo habilitar la interacción offline en aplicaciones
web?
• ¿Cómo diseñar la interacción offline con aplicaciones
web existentes?
• ¿Cómo actualizar la interacción del usuario en modo
offline cuando se retome la conexión con el servidor?
En los siguientes apartados se tratan estos desafíos con el fin de
proporcionar los mecanismos necesarios para habilitar la
interacción con aplicaciones web offline.
Figura 2. Descripción de la navegación offline para el caso de
2.2.1 Habilitando la interacción offline en estudio
aplicaciones web Descripción de la Interfaz
Para habilitar la interacción offline en aplicaciones web se recurre
a la utilización de las tecnologías que ofrece el estándar HTML al La interfaz de una aplicación web se compone de elementos.
respecto. A través de ellas se permite que el usuario interactúe con Estos elementos pueden ser cuadros de texto, botones, enlaces a
la aplicación web cuando esta no tiene conexión a internet. Para otras páginas web, imágenes o formularios entre otros. Estos
ello, se guarda una copia loca de los recursos asociados a la elementos pueden contener a su vez otros elementos. Para
aplicación web. Cuando no exista conexión a internet, los recursos describir la interacción con la interfaz web en modo offline
son leídos de forma local, permitiendo al usuario continuar la necesitamos definir cómo se van a comportar los elementos
interacción con la aplicación web. cuando no exista conexión a internet. Para ello, se realizan
transformaciones y acciones sobre la interfaz web. Se definen las
Pero guardar estos recursos de forma local es sólo parte de la siguientes transformaciones y acciones para la descripción de la
solución al problema planteado. Necesitamos establecer qué interfaz web:
partes de la aplicación web estarán disponibles en modo offline.
Para ello, y según los criterios que se han establecido, necesitamos • Ocultar: el elemento de la interfaz web se oculta de
definir en los niveles de navegación y presentación qué elementos manera que no se muestra al usuario ni se puede
estarán disponibles para la interacción del usuario. interactuar con el cuando la aplicación web está en
modo offline.
Esta descripción de ambos niveles forman lo que definimos como
el Modelo Offline. Este modelo comprende las transformaciones y • Deshabilitar: el elemento de la interfaz web se muestra,
las reglas que definimos para la aplicación web. A continuación se pero el usuario no puede interactuar con el. Esta
presentan los conceptos y la notación utilizada para la descripción transformación afecta a los hiperenlaces que tienen
de los niveles de navegación y presentación. como destino páginas web que no están disponibles en
modo offline. Aunque se muestra el texto asociado al
Descripción de la Navegación hiperenlace, no permite seguir la navegación asociada.
Para la descripción de la navegación en aplicaciones web offline
• Guardar: algunos elementos de la interfaz web enlazan
se utiliza la notación presentada por Albertos [2]. Esta notación
a recursos tales como documentos PDF o imágenes para
permite representar las páginas web como nodos. Las conexiones
ser descargados o visualizados en el navegador.
entre ellos representan la navegación en la aplicación web.
Indicaremos que el recurso se guarda para poder ser
Finalmente, los atributos de cada nodo definen las políticas sobre
utilizado en modo offline.
su disponibilidad en el escenario offline, así como la forma en que
se crea la copia local. • Actualizar: para el caso de elementos que permiten una
interacción directa por parte del usuario que necesita ser

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 191 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud
actualizada cuando se retoma la conexión a internet se implica que si se aplica una acción sobre un elemento, todos los
indica que necesitan ser actualizados. Tal es el caso de elementos contenidos se verán afectados por ella.
cuadros de texto. En ellos, en modo offline el usuario Para el caso de estudio se muestran en rojo los elementos que se
introduce un texto en el. Cuando se vuelve a visitar ese ocultan y no se muestran en la interacción offline. Estos
elemento en modo online, este debe de mostrar el elementos comprenden los foros, las búsquedas u otros elementos
resultado de la interacción producida en modo offline. que por decisiones de diseño para la interacción offline se ocultan
De esta forma la interacción queda registrada y el al usuario. Los elementos marcados en verde comprenden
usuario puede actualizarla en el servidor. recursos que estarán disponibles en modo offline, la anteriormente
Estas transformaciones y acciones en la interacción web se denominada como acción guardar. Para el caso de estudio se
definen directamente sobre la interfaz de la aplicación web. Los muestra un documento PDF que está disponible en modo offline
mecanismos utilizados se presentan en el siguiente apartado. para que el usuario pueda completar la actividad que realiza.
2.2.2 Diseño de la interacción offline con
aplicaciones web existentes
Para diseñar la interacción en aplicaciones web existentes se ha
implementado una aplicación que se sitúa entre el servidor web
que aloja la aplicación web y el navegador del usuario que
interactúa con ella. A esta aplicación se le denomina Proxy
Offline.
El Proxy Offline constituye un elemento software que contiene el
Modelo Offline que es inyectado en la aplicación web existente y
servida al navegador web del cliente. De esta forma se consigue
utilizar aplicaciones web existentes sin ser modificadas en el
servidor web original. En la Figura 3 se muestra cómo el Proxy
Offline se encarga de incluir en las aplicaciones web ya existentes
el Modelo Offline. Para cada petición que realiza el cliente a
través del navegador, el proxy media de forma que solicita esa
petición al servidor web. La respuesta del servidor web es
procesada por el proxy inyectando los elementos necesarios para
que la respuesta que recibe el navegador incluya el Modelo
Offline con las adaptaciones pertinentes.

Figura 4. Diseño de la interacción offline en la aplicación web

Todos los elementos marcados en la Figura 4 hacen que la
aplicación web se comporte de un modo específico cuando se
encuentre en modo offline. En la Figura 5 se muestra el resultado
de las transformaciones en la aplicación web con respecto al
diseño de la interacción offline definido en la Figura 4. Como se
puede apreciar, los elementos marcados como ocultar no se
Figura 3. Funcionamiento del Proxy Offline muestran en la interfaz del usuario. Por otro lado, los enlaces a
Incluir el Modelo Offline en la aplicación web no es el único páginas web o elementos que no están disponibles se encuentran
cometido del Proxy Offline. También se encarga de incluir en ella atenuados y deshabilitados, tal y como se puede apreciar en la
el software que se encarga de la edición del modelo sobre la barra superior de navegación. En ella sólo el elemento “Home” es
aplicación web. Esto permite seleccionar los elementos de la accesible al usuario. El resto de enlaces está deshabilitado, a
aplicación web y aplicar las transformaciones sobre la interfaz excepción de los incluidos en la Figura 2 y que forman parte de la
web y la navegación. navegación offline de la aplicación, como el primer enlace de la
sección “Actividades”. De esta forma se limita la interacción del
En la Figura 4 se muestra como se marcan los elementos de la usuario con elementos que estén disponibles para su interacción
interfaz con las transformaciones y acciones disponibles. Se offline. También se encuentra disponible en este modo el
indica con un recuadro punteado de diferentes colores los documento PDF, al cual se le dio la propiedad de guardar.
elementos que tendrán un comportamiento específico cuando la
aplicación se encuentre en modo offline. Cada color indica la
aplicación de una transformación o acción determinada sobre el
elemento contenido.
Para realizar esta anotación de la interfaz web, se sitúa el cursor
sobre el elemento en cuestión que es resaltado sobre la propia
interfaz. En ese momento un menú contextual permite aplicar las
diferentes acciones disponibles sobre el elemento. Cabe destacar
que los elementos de la aplicación web están anidados. Esto

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 192 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud

Figura 6. Cambios en la interfaz para guardar la interacción

offline de los usuarios
Cuando la aplicación web se ejecuta de nuevo en modo online, el
indicador con el número de interacciones offline se sigue
mostrando. Así se permite al usuario actualizarlas simplemente
Figura 5. Transformaciones en la interfaz de la aplicación web
en modo offline haciendo click en el indicador. La interfaz cambia en ese
momento, mostrando las páginas web que contienen la interacción
Con estas transformaciones también se consigue que el usuario se offline, tal y como se muestra en la Figura 7. Se muestra una lista
centre en la actividad que está realizando. Al eliminar los con estas páginas web y su título. También se incluye información
elementos que no intervienen en la actividad que esté realizando acerca de la fecha y hora en que se produjeron las interacciones
en ese momento y que no están disponibles en modo offline, se offline. Esta información ayuda al usuario a saber cuándo se
facilita la interacción del usuario, esperando reducir el número de realizaron las interacciones en modo offline en las páginas web
errores debido a este modo. indicadas.
2.2.3 Actualizando la interacción offline en
aplicaciones web
Una vez que el usuario puede interaccionar con el sitio en modo
offline, el último criterio establecido que debemos de cumplir para
permitir la interacción offline con la aplicación web es la
actualización de las acciones realizadas por el usuario en ese
modo cuando se retome el modo online.
Para ello, la aplicación que ejecuta el cliente con el Modelo
Offline incluido incorpora mecanismos que permiten estas
acciones. Cuando se establece un elemento con la opción de
actualizar se permite esta actualización offline-online. Por
ejemplo, al establecer un cuadro de texto con la opción
mencionada, al estar la aplicación web en modo offline se
introducen o cambian algunos elementos en la interfaz. Estos
elementos se muestran ampliados en la Figura 6 :
Figura 7. Interacciones offline pendientes de ser actualizadas
• El botón de envío asociado al cuadro de texto cambia
para indicar al usuario que lo que se va a hacer es 3. EVALUACIÓN
guardar el trabajo offline. La evaluación se plantea como test piloto para obtener
información de cómo los participantes interactúan con el sistema.
• Se introduce un indicador en la parte inferior derecha de En la evaluación se tomas datos cuantitativos (tiempos) y
la interfaz que muestra el número interacciones offline cualitativos (opiniones de los participantes).
pendientes de ser actualizadas en modo online, además
de la fecha y la hora en que se realizó la copia local. El escenario utilizado para evaluar la interacción de los usuarios
Este elemento también establece un enlace directo con en las actividades anteriormente descritas se presenta a
las páginas web que contienen las interacciones offline, continuación.
el cual se explicará en detalle a continuación. 3.1 Participantes
El número de participantes es de 10 personas, distribuidas en dos
grupos de 7 hombres y 3 mujeres. El participante más joven tiene
22 años de edad, siendo la media de 24 años mientras que el
participante con más edad tiene 42 años. El 70% de los
participantes son estudiantes de post-grado. El 100% de los
participantes utiliza aplicaciones web a diario. El 100% de ellos

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 193 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud
ha usado Moodle con fines académicos durante el último año, Los tiempos para la realización de las actividades utilizando el
bien como estudiante o como profesor. proxy online se reflejan en la Figura 9.
3.2 Aparatos 12:00
El hardware utilizado para llevar a cabo la evaluación consta de
un portátil “Apple 15’’ Macbook Pro (modelo de finales 2012)”. 9:36
El sistema operativo utilizado es Mac OS X versión 10.9
corriendo una máquina virtual Parallels Desktop versión 9. La
7:12
máquina virtual ejecuta el sistema operativo Microsoft Windows
7. El navegador web utilizado es Chrome en su versión 33. Otro
software disponible para el usuario durante la evaluación incluye:
4:48
Microsoft Office 2010, Microsoft Notepad versión 6.1 y los
accesorios disponibles en una instalación estándar de Windows 7. 2:24

3.3 Procedimiento 0:00

Todos los participantes son voluntarios no pagados. La tarea del 9:10 9:16 9:59 11:41 10:12
usuario consiste en realizar una actividad en una aplicación web.
La aplicación web utilizada es el entorno de enseñanza virtual Figura 8. Tiempos sin la utilización del Proxy Offline
Moodle. La tarea consiste en la realización de una actividad
dentro de un curso. La actividad consiste en la lectura de un texto 9:00
para responder a preguntas relacionadas con el.
A los usuarios se les dan instrucciones sobre las actividades. Estas 6:36
instrucciones consisten en la estructura del curso y en cómo
responder a las preguntas dentro de la aplicación web. De forma
adicional, a los usuarios que realizan la evaluación utilizando el 4:12
Proxy Offline se les explica cómo salvar el curso para uso offline
y cómo salvar y restaurar el trabajo offline. 1:48
3.4 Diseño
En la evaluación de la propuesta se compara la interacción de los
usuarios realizando una tarea en una aplicación web utilizando el 7:10 8:14 8:40 8:54 6:41
Proxy Offline y sin utilizarlo. Para ello, las actividades de la tarea
se realizan en dos escenarios: con el Proxy Offline y sin el Proxy Figura 9. Tiempos utilizando el Proxy Offline
Offline. Para realizar la tarea en cada escenario, se establecen dos Respecto a los tiempos totales para la realización de la tarea, se
grupos de 5 usuarios, siendo la selección de los usuarios al azar. puede observar que estos son menores utilizando el Proxy Offline.
Un usuario sólo participa en uno de los escenarios de los que Si bien hay tiempos parecidos en ambos escenarios, offline y
consta la evaluación. online, si tomamos la media como referencia obtenemos un
Para la tarea se establecen tres periodos: preparación (p1), tiempo medio de 10:03 minutos sin la utilización del Proxy
ejecución (p2) y sincronización (p3). Estos periodos identifican Offline por una media de 7:55 minutos utilizando el Proxy
las distintas fases por las que transcurre la tarea con el fin de Offline. De este análisis podemos concluir que, si bien la
analizar de esta manera los resultados sobre cada una de ellas. El experiencia del usuario realizando este tipo de trabajo en
periodo p1, denominado de preparación, comprende la fase escenarios offline puede hacer variar la duración de la tarea, de
durante la cual el usuario prepara la información necesaria para forma general se produce una disminución en el tiempo al utilizar
poder realizar la tarea antes de que la aplicación web se establezca el Proxy Offline.
en modo offline. El periodo p2, denominado de ejecución de las Además, una comparación de las medias de los tiempos por
actividades, corresponde a la fase offline en la que el usuario periodo para la realización de las tarea tanto utilizando el Proxy
realiza las actividades. El periodo p3, denominado de Offline como sin utilizarlo se muestra en la Figura 10. En ella se
sincronización, corresponde a la vuelta al modo online. Durante puede observar como en p2 la diferencia no es muy elevada, ya
este periodo el usuario actualiza el trabajo realizado en modo que supone una reducción del 3% en la media de tiempo. Sin
offline para que se actualice en el servidor Moodle. embargo, esta diferencia es mayor en p1 y p3. Mientras que en p3
la reducción es del 46% en la media de tiempos para las
4. RESULTADOS Y DISCUSIÓN actividades en ese periodo, en p1 la reducción es del 89%.
Las medidas para comparar la ejecución de la tarea en los
escenarios online y offline son la productividad y la satisfacción.
La productividad del sistema se analiza en base al tiempo usado
para la realización de las tareas. La Figura 8 muestra los tiempos
de los usuarios en la realización de las actividades sin la
utilización del Proxy Offline. Cada barra vertical representa el
tiempo que cada usuario tarda en cada uno de los periodos
definidos anteriormente (preparación, ejecución y sincronización).
En el cuadro inferior de la figura se muestran estos tiempos junto
con el tiempo total que tarda el usuario en realizar la actividad.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 194 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud

8:24 0,297222
0,307083 Estos datos se pueden aplicar a una problemática concreta: la
7:12 222333 ejecución del caso de estudio en dispositivos móviles. Si se
6:00 realizan estas actividades en dispositivos móviles, como teléfonos
4:48 0,064722 0,047361
o “tablets”, la interacción del usuario todavía se complica más
3:36 0,007083
222 0,026111 debido a las peculiaridades en la interacción con las aplicaciones
2:24
333 111111 en este tipo de dispositivos. De esta forma, la tasa de errores
1:12 con PO podría incrementar, al igual que disminuir la satisfacción de
0:00
sin PO usuario.
Por último, durante el transcurso de la evaluación fueron
significativos los comentarios de los usuarios. A continuación se
recogen algunos de ellos utilizando el Proxy Offline: “Creo que
sería necesario un tutorial previo, aunque no sería necesaria
mucha explicación. Con un poquito quedaría claro”; “Es fácil
pasar los datos trabajados en modo offline cuando ya hay línea”;
Figura 10. Comparativa de medias de tiempos por periodo “La página se queda limpia con la tarea que tienes que hacer; te
según la utilización el Proxy Offline es más fácil concentrarte para resolver la tarea”.
Aún siendo la media de tiempos por periodo menor con la Por su parte, algunos comentarios durante la realización de la
utilización del Proxy Offline, se puede observar tanto en la Figura evaluación sin el uso del Proxy Offline fueron los siguientes: “En
8 como en la Figura 9 que no todos los tiempos por usuario para base a la experiencia que tengo, para realizar la tarea offline
p2 y p3 son menores utilizando el Proxy Offline. En estos descargo todo los documentos y los guardo en un dispositivo de
periodos también interviene la experiencia del usuario y su pericia almacenamiento externo USB. Luego trabajo en ellos en el tren
a la hora de gestionar los recursos y el uso de la plataforma. Sin para actualizarlo cuando llego al trabajo”; “Es complejo de
embargo, en p1 todos los tiempos son menores utilizando el Proxy utilizar porque tienes que abrir todas las tareas y los documentos
Offline. relacionados para poder hacer las tareas una vez que no se
De este análisis de los tiempos que los usuarios tardan en realizar dispone de internet. Hay que tener cuidado de cargar todos los
las tareas utilizando el Proxy Offline y sin utilizarlo, podemos contenidos o te quedas sin hacer las tareas”.
observar que se produce una reducción drástica en el tiempo para 5. TRABAJOS RELACIONADOS
los periodos p1 y p3. Sin embargo, para el periodo p2 esta
reducción es menos drástica, sino inexistente. Además, para el 5.1 Las aplicaciones web y el
periodo p1 la mejora en el tiempo es generalizada y no depende almacenamiento local de la información
tanto del usuario. Sin embargo, en los periodos p2 y p3 el usuario Las aplicaciones web son accesibles a través de clientes ligeros,
determina en gran medida el margen de mejora en el tiempo que como los navegadores web, a través de redes como Internet o el
le lleva la realización de la tarea utilizando o no el Proxy Offline. uso de intranets [1]. Esta definición implica que la aplicación web
Para la evaluación de la satisfacción de usuario se utiliza el test está alojada en una máquina remota, el denominado servidor web,
SUS (System Usability Scale) [4]. La puntuación obtenida sin la que sirve el contenido (como texto, imágenes o video) tanto de
utilización del Proxy Offline es de 65.5, indicando que los forma estática o dinámica entre los usuarios que lo requieren bajo
usuarios no están satisfechos con el sistema. Los resultados demanda por medio de la utilización de un navegador web
obtenidos utilizando el Proxy Offline es de 95.5, indicando que instalado en el dispositivo del usuario. El contenido servido por el
los usuarios están muy satisfechos utilizando el Proxy Offline. servidor web se encuentra empaquetado como una mezcla de
código HTML y código ejecutable. Cada vez que el cliente recibe
Otro de los resultados interesantes encontrados durante la
este código junto con los recursos asociados, se crea una copia
realización de la evaluación es el número de aplicaciones que los
local, o cache, de forma temporal que el navegador procesa y
usuarios utilizan durante la realización de las actividades.
muestra al usuario.
Mientras que utilizando el Proxy Offline el usuario sólo utiliza el
navegador web (al igual que lo haría si no estuviera en un Es habitual la utilización de caches para aumentar el rendimiento
escenario offline), los usuarios que no utilizan el Proxy Offline de las aplicaciones web y mejorar la interacción con la aplicación
utilizan 3 aplicaciones de media. web. Muchas de las aproximaciones para la gestión de la cache se
basan en las tecnologías del lado del servidor como “proxies” o
El uso de varias aplicaciones complica la interacción del usuario.
plantillas. Sin embargo, tecnologías como Gears-monkey [10],
No sólo se ve afectado el tiempo que le lleva realizar la actividad.
HTML5 [17] y Web Store [18] permiten el uso de nuevas
También hay otros parámetros que se ven afectados, como lo son
estrategias para almacenar información de forma local en
el aumento del número de acciones que el usuario realiza. Estas
aplicaciones web. Gears-monkey permite la inyección de código
acciones son del tipo “salvar documento”, “abrir documento”,
en sitios web existentes para ser visualizados posteriormente en
“copiar texto” … Según el estudio realizado, se introducen una
los navegadores. Los scripts del lado del cliente desarrollado por
serie de acciones al no utilizar el Proxy Offline que no se realizan
los usuarios pueden entonces soportar la interacción offline para
en el navegador web, aumentando la probabilidad de que se
la gestión de la información. Sin embargo, esta solución está
cometan errores durante la interacción del usuario. Durante la
limitada a unas pocas plataformas y no puede ser ejecutada en
evaluación se han contabilizado una media 0.4 errores en p1, 0.8
muchas de ellas, por ejemplo en teléfonos móviles. También
errores en p2 y 1.8 errores en p3 cuando no se utiliza el Proxy
requiere de usuarios experimentados para la escritura de los
Offline. Cabe destacar que la media de errores utilizando el Proxy
scripts. También existen soluciones “ad hoc”. Por ejemplo,
Offline es de 0 en p1, p2 y p3.
McAllister [12] propone una aplicación web offline que utiliza
tecnologías web offline para que los pacientes guarden la

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 195 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud
interacción con la aplicación de forma offline para poder trabajar existentes, así como la publicación del funcionamiento y la
con ella sin la necesidad de una conexión de red. arquitectura del Proxy Offline.
Para gestionar la información en las aplicaciones web de forma 7. AGRADECIMIENTOS
local la tendencia actual es la utilización de tecnologías HTML5. Este trabajo ha sido parcialmente financiado por la convocatoria
La más básica se gestiona a través de Web Storage. Esta introduce (2015) GI20142832 de financiación de grupos de investigación y
un mecanismo similar a las cookies de sesión HTTP permitiendo los contratos para la formación de personal investigador en el
el almacenamiento de pares clave-valor. También se pueden marco del Plan Propio de I+D+i, cofinanciados por el Fondo
utilizar mecanismos más complejos que permiten el Social Europeo [2014/10340] de la Universidad de Castilla-La
almacenamiento local de información y su recuperación a través Mancha.
de estructuras complejas, permitiendo consultas avanzadas a
través de Web SQL Database [19] o Indexed Database API [20]. 8. REFERENCIAS
Pero para poder usar todas estas tecnologías es necesaria la [1] Alalfi, M. H., Cordy, J. R., & Dean, T. R. (2009). Modelling
utilización de tecnologías que permiten la interacción con los methods for web application verification and testing: state of
sitios web en modo offline. Actualmente existen dos propuestas the art. Software Testing, Verification and Reliability, 19(4),
para llevar a cabo este fin dentro de la especificación HTML: 1) 265–296. doi:10.1002/stvr.401
Offline Web Applications [21], que permiten declarar los [2] Albertos Marco, F., Gallud, J. A., R. Penichet, V. M., &
contenidos que están disponibles de forma offline; y 2) Service Winckler, M. (2013). A Model-Based Approach for
Workers [22], que permite realizar una gestión más completa de la Supporting Offline Interaction with Web Sites Resilient to
forma en que van a ser utilizados los recursos disponibles en la Interruptions. In Q. Z. Sheng & J. Kjeldskov (Eds.), Proc. of
aplicación cuando estos no estén accesibles o estén disponibles Current Trends in Web Engineering - ICWE 2013
sólo de forma local. International Workshops (Vol. 8295, pp. 156–171). Cham:
Springer International Publishing. doi:10.1007/978-3-319-
5.2 Gestión de la conectividad offline en 04244-2
plataformas e-Learning [3] Albertos Marco, F., Penichet, V. M. R., & Gallud, J. A.
Aún siendo las tareas académicas y la interacción con sistemas e- (2013). Offline Web Applications : A New Model for
Learning unas de las actividades más comunes con plataformas Blended Learning. In 2nd International Workshop on
web [8] no hay trabajos que estudien a fondo la interacción de los Interaction Design in Educational Environments (IDEE
usuarios con este tipo de plataformas en escenarios offline. 2013), ICEIS 2013. Angers, (France).
Hay numerosas aportaciones sobre multitud de temas relacionados [4] Brooke, J. (1996). SUS - A quick and dirty usability scale. In
con software para educación, como puede ser el estudio del Usability Evaluation in Industry.
awareness para educación [7][23][11] o su utilización en
dispositivos móviles [5][15] pero sólo unos pocos abordan de [5] Cochrane, T. (2006). Learning with wireless mobile devices
forma superficial la problemática offline en plataformas e- and social software, 143–146. Retrieved from
Learning [9][16][3]. Estos últimos no aportan resultados http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.11
significativos de estudios con usuarios en escenarios reales donde 6.8457&amp;rep=rep1&amp;type=pdf
se soporte la interacción offline con aplicaciones web existentes y [6] Gerhard Fischer. (2009). Democratizing design: new
que estén siendo utilizadas de forma generalizada. challenges and opportunities for computer-supported
collaborative learning. In Proceedings of the 9th international
6. CONCLUSIONES conference on Computer supported collaborative learning -
En este trabajo se presenta una primera evaluación de un caso de
Volume 1 (CSCL'09), Claire O'Malley, Daniel Suthers, Peter
estudio con usuarios en una aplicación web ampliamente utilizada
Reimann, and Angelique Dimitracopoulou (Eds.), Vol. 1.
adaptada a la interacción offline. Para ello, se presentan las International Society of the Learning Sciences 282-286.
decisiones y acciones a realizar sobre los diferentes elementos que
componen las aplicaciones web en escenarios offline, [7] Gutwin, C., Stark, G., & Greenberg, S. (1995). Support for
identificando los aspectos a tener en cuenta especialmente sobre la workspace awareness in educational groupware. In The first
interfaz de usuario. Una vez sentada esta base, se diseña la international conference on Computer support for
interacción offline a través de la aplicación denominada Proxy collaborative learning (pp. 147–156). L. Erlbaum Associates
Offline, que habilita tanto el diseño como la interacción offline en Inc. Retrieved from
aplicaciones web existentes tanto para la capa de navegación http://portal.acm.org/citation.cfm?id=222126
como para la capa de presentación. [8] Herder, E. (2006). Forward, back and home again : analyzing
La evaluación realizada a través del test piloto arroja unos user behavior on the web. Retrieved from
primeros resultados acerca de cómo el Proxy Offline afecta a la http://doc.utwente.nl/55828/1/thesis_Herder.pdf
utilización de aplicaciones web existentes en entornos offline: se [9] Ijtihadie, R. M., Chisaki, Y., Usagawa, T., Cahyo, H. B., &
reduce el tiempo en la realización de las tareas; desaparecen el Affandi, A. (2010). Offline web application and quiz
número de errores cometidos por el usuario debido a la synchronization for e-learning activity for mobile browser. In
problemática offline; y por último, mejora de la satisfacción de TENCON 2010 - 2010 IEEE Region 10 Conference (pp.
usuario. Además, se ha obtenido información acerca de cómo los 2402–2405). IEEE. doi:10.1109/TENCON.2010.5685899
usuarios trabajan en escenarios offline y su opinión acerca de las
soluciones presentadas. [10] Kao, Y.-W., Lin, C., Yang, K.-A., & Yuan, S.-M. (2012). A
Web-based, Offline-able, and Personalized Runtime
Como trabajo futuro, se continuará con la evaluación del uso del Environment for executing applications on mobile devices.
Proxy Offline en diferentes categorías de aplicaciones web

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 196 |
 Interacción de los Usuarios con Aplicaciones Web Offline: un Caso de Estudio
F. Albertos Marco | V. M.R. Penichet | J. A. Gallud
Computer Standards & Interfaces, 34(1), 212–224. 10th IEEE International Conference on Advanced Learning
doi:10.1016/j.csi.2011.08.006 Technologies, 703–704. doi:10.1109/ICALT.2010.199
[11] Lambropoulos, N., Faulkner, X. and Culwin, F. (2012), [17] W3C. (2014). HTML5 A vocabulary and associated APIs for
Supporting social awareness in collaborative e-learning. HTML and XHTML. W3C Candidate Recommendation 6
British Journal of Educational Technology, 43: 295–306. doi: August 2013. Retrieved from http://www.w3.org/TR/html5/
10.1111/j.1467-8535.2011.01184.x [18] W3C. (2014). Web Storage. Retrieved from
[12] McAllister, P., & Bond, R. (2014). An Offline Web App for http://dev.w3.org/html5/webstorage
the Self-Management of Diabetes and Obesity. In 2014 Irish [19] W3C. (2010). Web Database. Retrieved from
Human Computer Interaction Conference. Dublin. Retrieved http://www.w3.org/TR/webdatabase/
from http://ihci2014.dcu.ie/style/papers/Full/Patrick
McAllister and Raymond Bond_An Offline Web App for the [20] W3C. (2015). Indexed Database API. Retrieved from
Self-Management of Diabetes and Obesity.pdf http://www.w3.org/TR/IndexedDB/
[13] Mcloughlin, C., & Lee, M. J. W. (2007). Social software and [21] W3C. (2008). Offline Web Applications. Retrieved from
participatory learning : Pedagogical choices with technology http://www.w3.org/TR/offline-webapps/
affordances in the Web 2 . 0 era Introduction : Social trends [22] W3C. (2015). Service Workers. Retrieved from
and challenges. Current, 664–675. https://slightlyoff.github.io/ServiceWorker/spec/service_wor
[14] Moodle (2015). https://moodle.org/ ker/
[15] Qian, F., & Nan, D. (2008). An Innovative Model of Mobile [23] Zheng, Y., & Yano, Y. (2007). A framework of context-
Education Application - The Research and Implement of awareness support for peer recommendation in the e-learning
“Palmtop Learning Mate.” In 2008 International Conference context. British Journal of Educational Technology, 38(2),
on Computer Science and Software Engineering (pp. 812– 197–210. doi:10.1111/j.1467-8535.2006.00584.x
815). IEEE. doi:10.1109/CSSE.2008.86
[16] Shrestha, S., Moore, J. P. T., & Abdelnour-Nocera, J. (2010).
Offline Mobile Learning: Open Platforms for ICT4D. 2010

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 197 |
 Elaborating a Web Interface Personalization Process
J. Eduardo Pérez | X. Valencia | M. Arrue | J. Abascal

Elaborating a Web Interface Personalization Process

J. Eduardo Pérez Xabier Valencia Myriam Arrue Julio Abascal
Egokituz: Laboratory of HCI for Special Needs
Informatika Fakultatea. Universidad del País Vasco/EuskalHerriko Unibertsitatea
(+34) 943018067
{juaneduardo.perez, xabier.valencia, myriam.arrue, julio.abascal}@ehu.eus

ABSTRACT systems include repositories for storing user models and adapting
This paper builds the required conceptual model for implementing web content according to their content.
adaptation systems intended to provide a personalized navigation The system proposed by Takagi et al. in [15] is user-type-centric
experience for any user. To this end, aspects such as the users’ as it is devoted to facilitating the web navigation of blind users.
needs, preferences, and navigational behaviour are taken into The system is placed in a proxy, and inserts annotations into the
account in order to select the adaptation techniques to be applied HMTL of target web pages by means of XPATH expressions. The
for each particular user. The validity of the proposed conceptual system performs simplification, rearrangement and text
model to provide the necessary components and mechanisms for alternatives insertion based on crowd sourcing annotation
implementing suitable web personalization systems is shown by techniques. Bigham et al. [2] also proposed a system focused on
means of a case study. improving blind users’ navigation experience. This system is
capable to add the missing alt text of images by applying
Categories and Subject Descriptors contextual analysis, OCR techniques and human-based
H.5.4 [Information Interfaces and Presentation]: crowdsourcing labelling functionalities. Other examples of user-
Hypertext/Hypermedia—User Issues; K.4.2 [Computers and type-centric adaptation systems are Dante [17] and SADIe [7].
Society]: Social Issues—Assistive technologies for persons with Both are oriented to users with visual disabilities. The former
disabilities. implements structural adaptations such as segmentation of a web
page in a number of simplified web pages by identifying
General Terms meaningful content chunks, including a table of content, etc.,
Human Factors whereas the latter implements the rearrangement of content and
the insertion of navigation links in order to improve blind users’
Keywords performance. Both perform semantic annotations: SADIe applies
Automated web personalization, assistive technology, adaptation annotations to CSS elements while Dante make uses of Xpointer
techniques, physical impairments. technology to annotate the web elements. Flatla et al. proposed the
SPRWEB system [4] that adapts the colours of the target web
1. INTRODUCTION page to people with colour vision deficiency. This system applies
Several attempts have been made to automatically adapt web a model to predict the perceived temperature, activity and weight
interfaces for users with disabilities. Performing appropriate of the colours defined in the CSS file related to the web page. The
adaptations requires taking several issues into account, such as the Sasayaki system [14] is devoted to blind users and provides a
techniques to be applied for each specific user’s needs, navigation secondary voice which augments with contextual information the
behaviour and preferences. Most of the currently existing primary voice output of screen readers. In addition, it provides
automated adaptation systems are not comprehensive enough to methods to directly access to “semantically meaningful”
cover the full range of diversity among users.
components, to provide spatial information and to access social
The objective of this paper is to present a conceptual model for information supported by volunteers based on annotation
the automated adaptation of web interfaces. The conceptualization techniques.
of this complex process will benefit from abstract modelling. This A lower number of works related to other user groups are
abstraction will allow a better comprehension of each of its phases available. For instance, Hurst et al. [8] proposed interface
and their interactions. In addition, it will facilitate the design of adaptations for the group of users with physical disabilities. The
modular and interchangeable tools for each phase. A case study techniques employed are related to specific health conditions such
for users with physical impairments based on the results obtained as ataxia, dystonia, tremors, etc. Mankoff et al. [11] proposed web
in our previous work [12] is presented as a proof of concept for adaptations —such as increasing the font size to make links larger
the proposed conceptual model. and reducing horizontal scrolling by means of page
2. BACKGROUND linearization— that can improve accessibility for users with
physical disabilities. Hanson and Richards [6] proposed a proxy
Many automated web adaptation systems have been developed in
system for adapting web content for elderly people. It was later
the last decade. Most of them require the target websites to be
converted to an add-on [13] for enhancing the adaptations
semantically annotated. This method allows referring web
performed in the client-side. The adaptation techniques applied
contents to external metadata, which add semantic information for
are magnification, line-spacing personalization, text size
appropriately transforming a target web page [1]. Some of the
definition, high contrast option, etc. The adaptations are
automated adaptation systems perform user-type-centric
performed at the user’s request by means of dialogs that collect
adaptations based on basic stereotyped transcoding techniques.
users’ preferences. Gajos et al. [5] proposed a system for
Other systems can be classified as generalist approaches. These

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 198 |
 Elaborating a Web Interface Personalization Process
J. Eduardo Pérez | X. Valencia | M. Arrue | J. Abascal
automatically generating personalized interfaces for users with Adaptation_Repository is defined as the data structure that
motor impairments. In order to define a personalized interface an contains all the adaptation techniques:
ability test is performed by the user. However, the adaptation
techniques performed are not based on semantics because these Adaptation_Repository (Core_Adaptation_Technique,
systems do not apply annotation techniques. Preferences_Adaptation_Technique,
Beviour_Adaptation_Technique)
Fewer references to automated web adaptation oriented to
multiple disabilities can be found in literature. Valencia et al. [16] The Adapted_WebInterface is an adapted version of the web
proposed a system for automatically adapting websites according interface obtained through the application of the corresponding
to the user characteristics based on WAI-ARIA annotations. This adaptation techniques for a given user.
system incorporates a comprehensive repository of adaptation Adapted_WebInterface (Adapted_Presentation,
techniques appropriate for different groups of users and an Adapted_Navigation, Adapted_Content)
ontology for user modelling.
The Personalized_WebInterface is a user-tailored web
3. PERSONALIZATION APPROACH interface considering user preferences.
3.1 Conceptual Model Personalized_WebInterface(Personalized_Presentation,
The automated adaptation of web interfaces to accessibility Personalized_Navigation, Personalized_Content)
requirements is a complex process that requires formalization in
order to enable the modelling of the whole process taking into Each of the phases involved in the automated adaptation
account multiple factors, such as user needs, required adaptation process can be defined as the application of a set of
techniques and user preferences. In addition, a comprehensive transformations. Some of them are applied to the original web
model helps to structure and specify the necessary phases of the interface (Web_Interface component), others to the annotated web
process: web interface annotation, adaptation techniques selection interface (Annotated_WebInterface component) and others to the
and the definition of user preferences. adapted web interface (Adapted_WebInterface component). The
formal definition of a transformation is the following:
The conceptual model presented here defines the components and
mechanisms required for performing this process. The basic New_Web_Interface (New_Presentation, New_ Navigation,
components of the model are: New_Content) « transformation (Transformation_Parameters)
Web_Interface (Presentation, Navigation, Content)
Web_Interface, which is a set of elements enabling users to
interact with the web application. A Web_Interface can be defined where Transformation_Parameters are the required
as a set of Presentation, Navigation and Content elements: information for performing the conversion.

Web_Interface (Presentation, Navigation, Content) The transformations applied in each of the adaptation process
phases are the following:
Annotated_WebInterface is a new web interface obtained after an
annotation process performed with the aim of providing semantics 3.1.1 Annotation phase
to the elements contained. Annotated_WebInterface(A_Presentation, A_Navigation,
A_Content) « annotation (annotation_language semantics)
Annotated_WebInterface (A_Presentation, A_Navigation, Web_Interface (Presentation, Navigation, Content)
A_Content)
3.1.2 Adaptation phase
A User_Model contains information about the parameters that
Adapted_WebInterface (Adapted_Presentation,
characterize each user and are relevant to determine the adaptation
Adapted_Navigation, Adapted_Content) « adaptation
techniques to be applied relating to the needs, preferences and
(User_Model (U), Adaptation_Repository
behaviour.
(Core_Adaptation_Technique,
User_Model (User_Needs, User_Preferences, User_Behaviour) Preferences_Adaptation_Technique,
Behaviour_Adaptation_Technique) Annotated_WebInterface
An Adaptation_Technique comprises the components that are (A_Presentation, A_Navigation, A_Content))
applied to obtain the adapted web interface for a given user. These
adaptation techniques can affect one or more of the components of 3.1.3 Preferences definition phase
a Web_Interface: presentation, navigation, content. Personalized_WebInterface (Personalized_Presentation,
Adaptation_Technique (Presentation, Navigation, Content) Personalized_Navigation, Personalized_Content) «
Preferences_Definition (User_Model (U),
The components of an Adaptation_Technique to be applied Adaptation_Repository (Core_Adaptation_Technique,
for a specific user are selected based on the information contained Preferences_Adaptation_Technique,
in their User_Model. Some techniques to be applied are related to Behaviour_Adaptation_Technique) Adapted_WebInterface
the needs of the user due to their abilities (Adapted_Presentation, Adapted_Navigation, Adapted_Content))
(Core_Adaptation_Technique), others are related to their
preferences (Preferences_Adaptation_Technique) and the Note that the annotation and Preferences_Definition
remainder to the set of restrictions imposed by the assistive transformations are manually performed by the system designer
technology used and the set of strategies applied by the user for and the user respectively whereas the adaptation transformation is
the interaction (Behaviour_Adaptation_Technique). automatically performed by the adaptation system. Nevertheless, a
comprehensive adaptation system must include utilities and
assistants for facilitating both manual processes. In addition, the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 199 |
 Elaborating a Web Interface Personalization Process
J. Eduardo Pérez | X. Valencia | M. Arrue | J. Abascal
system must provide features for storing the results gathered in Table 1. Set of adaptations techniques specially focused to
these phases in order to apply them in future interactions improve access to the Web for physically impaired users.
Categories Adaptation Techniques
3.2 Case Study (1) Add hot area around hyperlinks
In our previous work [16] a comprehensive set of 99 adaptation (2) Use stretchtext in main content
techniques were identified from the literature and classified for (3) Incorporate specific scrolling icons on each
application to cognitive, physical and sensory impairments.

Content
page
However, this approach was based on stereotypes and cannot deal (4) Incorporate buttons for browser “Back” and
with users’ specific navigational strategies, preferences and “Forward” functionalities
heterogeneity of each user in a group. Results from recent studies (5) Remove walking menus
about navigational behaviour of users with disabilities are now (6) Avoid “click and drag” elements (e.g., spinners)
considered for obtaining appropriate personalization systems. (7) Avoid double-clicking elements
In addition, one of our previous research projects focused on users (8) Define large clickable area for links and buttons

Presentation
with physical impairments [12] showed that the type of assistive in a visible way
technology used has a significant impact on their navigational (9) Define links and buttons well separated
behaviour. Therefore, the defined conceptual model provides (10) Hierarchically structure content
mechanisms for classifying adaptation techniques in three main (11) Break pages into chunks of no more than one
or two screens
groups: core adaptations, preferences adaptations and behaviour
adaptations. According to this classification, we stored the (12) Provide navigation bars
adaptation techniques appropriate for all users with physical (13) Provide quick access to different topics within
a website

Navigation
impairments as Core_Adaptation_Technique type. Those
techniques appropriate for users using the same assistive (14) Provide an index with information in the page
technology were stored as Behaviour_Adaptation_Technique type. (15) Create a table of contents for website
Finally, optional techniques selected by users according to their (16) Provide internal links to important content on
preferences were stored as Preferences_Adaptation_Technique. the page (e.g., return to the top of page)
This classification is relevant because the adaptation techniques (17) Provide skip links (e.g., to main content)
are applied in different phases of the process. (18) Disable non-recommended links (e.g., banners)

In this sense, the total number of 18 adaptation techniques (see • Content: (1) Add hot area around hyperlinks and (4)
Table 1) for physical disabilities identified in the literature Incorporate buttons for browser “Back” and
[3][5][9][10] were classified in the proposed three groups. The “Forward” functionalities.
core adaptation techniques identified as beneficial to all users in • Presentation: (8) Define large clickable area for links
the group are: and buttons in a visible way and (9) Define links and
buttons well separated.
• Content: (5) Remove walking menus, (6) Avoid click and
Most participants from this group were not used to keyboard
drag elements and (7) Avoid double-clicking elements.
specific keys for activating scroll functions and preferred using
• Presentation: (10) Hierarchically structure content.
standard scroll bar buttons instead, which requires additional time
• Navigation: (12) Provide navigation bars, (13) Provide for pointing and clicking. For this reason, the next adaptations
quick access to different topics within a website, (17) considerably improve the performance of these users and were
Provide skip links (e.g., to main content). also included in the behaviour adaptation techniques group:
Observations made during the exploratory study about the
• Content: (2) Use stretchtext in main content and (3)
pointing time revealed that users of specific pointing devices such
Incorporate specific scrolling icons on each page.
as joystick, trackball or reconfigured standard input device needed
• Navigation: (16) Provide internal links to important
more time to reach and click on targets due to their lack of
content on the page (e.g., return to the top of page).
accuracy. Therefore they would benefit from the following
specific adaptation techniques which should be included into the • Presentation: (11) Break pages into chunks of no more
group of behaviour adaptation techniques: than one or two screens.
Our results indicated that the performance of numeric keypad
users’ when moving the cursor depends on the distance and
number of keystrokes needed to reach a target. This condition had
even a greater impact among participants using a head wand than
those participants directly accessing the numeric keypad with their
hand. The following adaptations will benefit the navigation
experience of this group of users by reducing cursor movement or
the number of total keystrokes needed to reach a target:
• Navigation: (14) Provide an index with information in
the page, (15) Create a table of contents for the website
and (18) Disable non-recommended links (e.g.,
banners).
Therefore, these techniques are included in the corresponding
behaviour adaptation techniques group.
The preference adaptation techniques group include the following:
• Hide/show specific scrolling icons

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 200 |
 Elaborating a Web Interface Personalization Process
J. Eduardo Pérez | X. Valencia | M. Arrue | J. Abascal
• Hide/show buttons for browser “Back” and “Forward” [35] Gajos K. Z., Wobbrock J. O., Weld D. S. 2008. Improving
functionalities the performance of motor-impaired users with automatically-
• Define the preferred location for scrolling icons generated, ability-based interfaces. Procs. SIGCHI Conf. on
• Define the preferred location for browser “Back” and Human Factors in Computing Systems (CHI '08). ACM, NY,
“Forward” functionalities USA, 1257-1266
• Hide/show accesskeys for scrolling/browser “Back” and [36] Hanson V. L., Richards J. T. 2004. A web accessibility
“Forward” buttons service: update and findings. Procs. 6th Int. ACM Conf. on
• Hide/show accesskeys for main content sections in the Assistive Technologies, ASSETS’04. NY, USA. Pp. 169-
website 176.
The phases of the adaptation process were performed as follows:
[37] Harper S., Bechhofer S. 2007. SADIe: Structural semantics
• Annotation phase: WAI-ARIA annotations were applied for accessibility and device independence. ACM Trans. on
to add semantics to the presentation, navigation and Computer-Human Interaction, 14(2). Article 10.
content elements in the web interfaces. This process is
thoroughly described in [16]. [38] Hurst, A., Hudson, S. E., Mankoff, J., Trewin, S. 2013.
• Adaptation phase: the core and behaviour adaptation Distinguishing Users by Pointing Performance in Laboratory
techniques were applied to the annotated web interface and Real-World Tasks. ACM Trans. on Accessible
in order to perform adaptations based on the users’ Computing 5(2). Pp. 1-27.
abilities and navigational strategies related to the [39] Knutov E., De Bra P. Pechenizkiy,M. 2009. AH 12 years
assistive technology used. later: a comprehensive survey of adaptive hypermedia
4. CONCLUSIONS AND FUTURE methods and techniques. New Review of Hypermedia and
Multimedia, 5(1). Pp. 5-38.
WORK
The conceptual model presented in this paper provides a high [40] Kurniawan S. H., King A. D., Evans G., Blenkhorn, P. L.
level abstract description of the automated web adaptation 2006. Personalising web page presentation for older people.
process, including the management of the necessary components Interacting with Computers, 18(3). Pp. 457-477.
and mechanisms considering users’ needs, navigational strategies [41] Mankoff, J., Dey, A., Batra, U, Moore, M. 2002. Web
and preferences. To show its validity for designing complex accessibility for low bandwidth input. Procs. 5th Int. ACM
systems for web adaptation a specific case study for users with Conf. on Assistive Technologies (Assets '02). ACM, NY,
physical impairments based on this model is defined and USA. Pp. 17-24.
developed. Currently, the adapted interfaces, the personalized [42] Pérez J. E., Arrue M., Valencia X. and Moreno L. 2014.
interfaces and the preferences definition features are being Exploratory study of web navigation strategies for users with
evaluated with users. Results obtained in these evaluations will be physical disabilities. In Proceedings of the 11th Web for All
analyzed and included in future works. Conference (W4A '14). ACM, New York, NY, USA
5. ACKNOWLEDGEMENTS [43] Richards J. T. and Hanson V. L. 2004. Web accessibility: a
The authors gratefully acknowledge the support of the Spanish broader view. In Proceedings of the 13th international
Ministry of Economy and Competitiveness through Project conference on World Wide Web (WWW '04). ACM, New
eGovernability (TIN2014-52665-C2-1-R). The EGOKITUZ York, NY, USA, 72-79.
Research Laboratory is supported by the Department of
[44] Sato D., Zhu S., Kobayashi M., Takagi H., Asakawa C. 2011.
Education, Universities and Research of the Basque Government
Sasayaki: augmented voice web browsing experience. Procs.
(Eusko Jaurlaritza/Gobierno Vasco) through Grant # IT-395-10.
SIGCHI Conf. on Human Factors in Computing Systems
6. REFERENCES (CHI '11). ACM, NY, USA. Pp. 2769-2778.
[31] Asakawa C., Takagi H. 2008. Transcoding. In Harper S., [45] Takagi H., Asakawa C., Fukuda K., Maeda J. 2002. Site-
YesiladaY. (eds.). Web Accessibility: A Foundation for wide annotation: reconstructing existing pages to be
Research. Springer. Pp. 231-261. accessible. Procs. 5th Int. ACM Conf. on Assistive
Technologies (Assets'02). ACM, NY, USA. Pp. 81-88.
[32] Bigham J. P., Kaminsky R. S., Ladner R. E., Danielsson O.
M., Hempton G. L. 2006. WebInSight: making web images [46] Valencia X., Arrue M., Pérez J. E. and Abascal J. 2013. User
accessible. Procs. 8th Int. ACM SIGACCESS Conf. on individuality management in websites based on WAI-ARIA
Computers and Accessibility (Assets'06). ACM, NY, USA. annotations and ontologies. In Proceedings of the 10th
Pp. 181-188. International Cross-Disciplinary Conference on Web
Accessibility (W4A '13). ACM, New York, NY, USA.
[33] Brajnik G. Barrier Walkthrough Method. In:https://users.
dimi.uniud.it/~giorgio.brajnik/projects/bw/bw.html. [47] Yesilada Y., Stevens R., Harper S., Goble C. 2007.
Evaluating DANTE: Semantic transcoding for visually
[34] Flatla D. R., Reinecke K., Gutwin C., Gajos K. Z. 2013.
disabled users. ACM Transactions on Computer-Human
SPRWeb: preserving subjective responses to website colour
Interaction, 14(3). Article 14.
schemes through automatic recolouring. Procs. SIGCHI
Conference on Human Factors in Computing Systems (CHI
'13). ACM, NY, USA. Pp. 2069-2078.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 201 |
 Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison
A. L. Carrillo | J. Falgueras

Goal Driven Interaction (GDI) vs. Direct Manipulation (MD),

an empirical comparison
A. L. Carrillo J. Falgueras
Department of Computer Sciences Department of Computer Sciences
ETSI Industriales. Campus Teatinos ETSI Informática. Campus Teatinos
29071 Málaga, Spain 29071 Málaga, Spain
+34 95195239 +34 952132792
[email protected] [email protected]

ABSTRACT The fundamentals of GDI trace back to the works of Newell and
This paper presents a work in process about Goal Driven Simon [5] that were devoted to the mechanism of human
Interaction (GDI), a style of interaction intended for reasoning for problems resolution. Their vision of problem
inexperienced, infrequent and occasional users, whose main solving (as in GDI) was based in the breaking up of the main or
priorities are to use a system and achieve their goals without cost general goal in a hierarchical tree of sub-goals, whose branches
in terms of time or effort. GDI basic philosophy is to guide the would have different lengths depending on the degree of their
user about the "what" to do and the "how" to do it in each moment fragmentation into sub-goals. The leaves of the tree would be
of the interaction process, without requiring from the user a elementary actions or sub-goals.
previous knowledge to use the interface. This interaction style was Based on these works, Card, Moran and Newell [6, 7] developed
introduced in previous work, where a description of its the most important of the existing cognitive models, the Human
characteristics and the most appropriate user interface for it, were Processing Model, whose initial paradigm (as in GDI) consisted in
described. Those works included a methodology for the analysis conceiving the interaction as a problem resolution task, and
and synthesis of the whole interactive process through a language described a psychological model of humans formed by three
of specification. This paper presents partial results we are interactive systems: perceptive, motor and cognitive, each one
collecting in real user testing, with the main aim of comparing would have their own memory and their own processor.
GDI with direct manipulation interfaces (MD), nevertheless the
most extended and commonly regarded as the most suitable for This vision of the user as an information processing system,
novice and experienced users. allows for the formalization of all the activities (both physical and
mental) that take part in that task, and gave origin to the methods
for modelling, specification and evaluation of the user interface
Categories and Subject Descriptors that are widespread today, the GOMS (Goals, Operators,
D.2.2 [Design Tools and Techniques]: User interfaces; Methods, and Selection rules) methods [8, 9]. Among other
H.5.2 [User Interfaces] things, that methods allow the description of the sequences of
behavior and knowledge that the user need to correctly interact
General Terms with the system and accomplish his goals. The models themselves
Human Factors, Experimentation, Design. are framed in the set of techniques that allow for a hierarchical
task analysis, as their main goal is the decomposition of those
tasks, so that the resolution method can be followed step by step.
Keywords Authors extended the NGOMSL notation [10] to use it as a source
User testing, Empirical usability evaluation, Guided interaction, specification language, which after a compilation process,
Interaction styles, User interfaces generates the corresponding GDI user interface.
1. INTRODUCCIÓN Therefore, because of this extended NGOMSL models include the
Although Direct Manipulation (DM) with WIMP (Windows, hierarchical knowledge the user must have (the tasks to do, and
Icons, Menus and Pointer) elements [1] is currently the most the procedures to be followed), the aim of GDI is to preclude the
extended desktop user interface paradigm in use, there are still user from having to devote time to acquiring such knowledge.
many users that need training and learning period, manuals and/or GDI’s main strategy is to let the interface gradually provide the
expert support to become efficient users. For this reason, in user with such knowledge, and to guide him in a hierarchical and
previous works [2, 3, 4], the authors proposed and presented a progressive way through the whole interaction process, not only
new and alternative style of interaction: Goal Driven Interaction as far as the tasks and goals approach are concerned, but also
(GDI). GDI was meant to become the interaction style of choice about the sequence of steps to follow or the choices that can be
for applications where the main priority is ease of use and made at any moment to achieve those goals.
minimal learning time for a user to interact with the program (as
occasional, infrequent or inexperienced users), even if sacrificing This strategy is, in many respects, the opposite of the MD in the
speed in task achievement, the ability of running parallel tasks, sense that the user has no freedom of interpretation or any
and other advantages of WIMP interfaces. possibility of experimentation with a metaphor. As wizards

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 202 |
 Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison
A. L. Carrillo | J. Falgueras
oriented interfaces, GDI gives the control to the system, as
opposed to MD that gives the control to the user.
The user interfaces based on this interaction style will need an
area (as seen in Fig. 1) called Goals Driver Window (GDW) that
will be the place where the user will be presented with either the
steps of the method to be followed or the different alternatives to
choose from, to satisfy the specific goal at any moment. Then,
GDW becomes the substitute (or alternative) to the typical menus,
icons, toolbars and those elements in WIMP interfaces, that are
not necessary in GDI (as seen in Figure 1).

2. TEST CONDUCTING
2.1 Introduction
To perform the comparison between the two interaction
techniques (GDI vs MD-WIMP) a set of tests involving real users
is being conducted [12, 13]. For these tests a familiar task was
chosen, a task for which is likely the user has some experience or
at least will understand it quickly. In particular, the theme chosen Figure 2. DM-WIMP interface screenshot
for this testing is the design and furnishing of kitchens, both
because it is an infrequent scenario, except for professionals in
this field; and, as we have said, is a familiar task domain for Each user should try to achieve the same goal with each of the
almost all users, at the same time requiring manipulation of two interfaces (GDI vs. MD-WIMP).
diverse objects, reasons which, in principle, could be seen
favouring MD interaction style. Data are collected both quantitative and qualitative, the former as
time lengths, number of issues of assistance, number of errors
Two prototypes in Java language, one with the GDI interface (categorised by severity), etc., as well as qualitative data from the
(Fig.1) and the other with a classic MD WIMP interface (Fig.2), subsequent questionnaires.
both sharing most of the code except, of course, the sections in
which user interface and style of interaction are involved. 2.2 User features
Although simplified systems, both offer the same functionality By the end of the study we will have tested around 25
and allow the user perform the same tasks. The application is participants. This number will include EyeTracker analysis for
inspired by the desktop version of the similar one offered by the some of them. If any conclusions would deem it appropriate, a
company IKEA [11]. second phase with more users would be conducted.
Before starting, all participants sign an Informed Consent, and are All users, except a small percentage of them, will be
meanwhile advised that the tests are taken voluntarily, free to inexperienced users. The rest will be users with an intermediate
leave any time without any justification. This Informed Consent computer level, and a very small percentage of them will be
also clarifies that these tests are not intended to make any personal kitchen design professionals, used to using specialised
or psychological assessment, but for the evaluation of the applications for this purpose.
computer user interfaces involved.
The age range is planned to span from 10 to 65 years (with an
average of 45 years) without any disabilities described.

2.3 Test conditions

The tests are performed individually, in controlled spots, without
distractions, thus allowing both techniques Thinking Aloud as
well as screen and sound recording. The user does not feel any
other influence but the presence of the moderator and the
applications under test.
The estimated time per user (including questionnaires) ranges
between 45 to 75 minutes.
The whole process of interaction is collected with two recordings.
Figure 1. GDI interface screenshot The first one is taken with a external video camera, located next to
the user recording the user hands and the screen, and any
incidences, assistances, etc., for further re-examination, if
necessary. And the other recording, even more useful, is that
which is taken with computer screen recording software.
A common laptop computer is used for all tests, which facilitate
their relocation and is less intimidating and more familiar to users
than any other complex installation. External video recording is
also more conveniently done with a small camera.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 203 |
 Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison
A. L. Carrillo | J. Falgueras

2.4 Test development tendencies can be drawn from those data: 1) GDI is more efficient
The tests follow the next steps: in time (took less time in accomplishing the whole tasks), and 2)
GDI needed fewer moderator warnings or assistances. On the
1. The user fills out a preliminary questionnaire containing other hand, from the questionnaires is explicit that all users have a
personal information such as their age, genre, experience with sharp preference for the GDI interface. This might not be
technological and digital devices, domain knowledge of the surprising among inexperienced users, but it was not expected for
task, etc. users of an intermediate computer level, and especially surprising
in the case of professionals on kitchen design, who are used to
2. The user receives a single sheet briefly describing the three
more complete and complex MD interfaces. In addition, all users
group of tasks to do (Fig. 3). This sheet with the tasks is
in the final questionnaire, that includes to compare both styles of
available to the user throughout all the process.
interfaces, clearly opt for the GDI guided interface.
3. Then, following a balanced strategy within-subjects, each user
is made to use both versions of the application, alternating the
order among users. Thinking Aloud technique is used in a
relaxed way, not making at any time the user to explain what
is being doing, but leaving them freedom to give explanations
when necessary. The moderator tries not to interfere at any
moment, or if so, only in critical or blocking situations. In any
case the incidents are reflected and graded as part of the data
collected.
4. At the end of each of the two tests (each one corresponding to
a type of interface), and before moving on to another type of
interface, the user fills out a questionnaire with 10 questions
on specific points of the process that has just experienced.
This covers important aspects for the final evaluation.
5. Finally, after the two tests and the corresponding
questionnaires are done, the user fills out a quick
questionnaire comparing both styles of interaction. Among
the questions asked are: with which interface have you had
less hesitation about what the next step was; with which was it
easier to know how to take every step; for which do you
consider that more aided should be included; which one
seemed easier to use and required less training; with which
was it faster to operate; and, naturally, their final preference.

3. EVALUACIÓN DE RESULTADOS
HASTA EL MOMENTO
The Table 1 displays the partial empirical results of the first 8
users. The last columns of the Table 1 display the types of
incidences and, perhaps, the need of assistance by the moderator.
There are three types of incidences: 1) slight, 2) non-locking, and Figure 3. Sheet with the tasks handed out to users
3) severe or locking. The most severe is the third type of
incidence, which takes place when the user is stuck with
something in the interface. This would prevent the user from 4. CONCLUSIONS
finishing the task. This will require the assistance of the The work in process presented here complements the theoretical
moderator in order to exploit the collected data and analyse the study of the GDI. This style of interaction is intended for
rest of the planned tasks for that user. applications that require the minimization of the learning time up
to a point that makes them appropriate for inexperienced,
Non-locking issues correspond to detected or undetected mistakes infrequent or occasional users. The essential point here is no
that somehow affect the final results. For example a common requiring previous knowledge. With this aim, these interfaces
error is repeatedly confusing the ‘undo’ action with the deletion of should guide the user towards their goal. To do so, indications of
the last object added; or forgetting performing a task, or initially both the “what to do” and the “how to do it” must always drive
not choosing the correct geometry, erroneously keeping the user at all times.
throughout the process the wrong shape of the kitchen, and
perhaps requesting a warning by the moderator. The partial results obtained so far show that, compared with a
MD-WIMP interface, the total execution time is shorter, there is a
Finally, slight incidents could not be properly considered errors fewer number of incidences, and finally the users show a clear
because could be sorted out by the user, perhaps with some delay. preference for this guided interaction style.
They are non-locking and did not need any moderator assistance.
An initial analysis of the empirical data obtained until this
document was sent may be premature, nevertheless, two apparent

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 204 |
 Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison
A. L. Carrillo | J. Falgueras
Table 1. Partial empirical results (time and incidences) of the first 8 users.

Evaluation Task 1 Task 2 Task 3 Total Incidences

User / (category)
order Time Time Time Time Slight
Non-
Locking
locking

GDI 6:20 9:20 8:11 23:51 � 1

USER 7
(inexperienced)
WIMP 14:23 11:26 11:32 37:21 (∞) 1 2

USER 2 WIMP 7:05 11:30 7:00 ∞ 25:35 (∞) 3 3

(inexperienced) GDI 5:30 7:00 3:30 16:02 �

USER 6 GDI 6:11 8:07 6:17 20:35 �

(inexperienced)
WIMP 4:40 5:18 17:55 ∞ 27:53 3

USER 4 WIMP 9:50 13:20 12:50 ∞ 36:00 (∞) ∞ 2 2 3

(inexperienced)
GDI 8:30 4:40 4:46 17:56 �

USER 1 GDI 2:45 3:35 2:30 8:50 � 1

(intermediate)
WIMP 2:30 2:36 4:14 9:20 3

USER 5 WIMP 4:10 4:05 6:45 15:01 1 2

(intermediate)
GDI 2:53 4:07 4:10 11:10 � 1

USER 3 (kitchen design GDI 2:30 4:15 1:45 8:30 �

professional)
WIMP 5:55 1:50 2:00 9:45 1 2

WIMP 4:34 3:40 3:53 12:07

USER 8 (kitchen design
professional) GDI 2:18 2:16 3:23 7:57 �

5. REFERENCES [5] Card, S., Moran, T., Newell, A. 1980 Computer text-editing:
[1] Dix A., Finlay J., Abowd G., Beale R., 1998: Human- An information processing analysis for a routine cognitive
Computer Interaction. 2nd ed. Prentice-Hall. skill. Cognitive Psychology, 12, 32-74.
[1] Carrillo A., Falgueras J., Guevara A. 2005. A notation for [6] Card, S., Moran, T., Newell, A. 1983. The Psychology of
Goal Driven Interfaces specification. In Interacción 2004, Human-Computer Interaction. Hillsdale, New Jersey:
Raquel Navarro-Prieto & Jesús Lores Eds. Springer, Erlbaum.
Dordrecht, The Netherlands. [7] John, B.E., Kieras, D.E. 1996. The GOMS family of user
[2] Carrillo, A., Guevara. A., Gálvez. S., Caro. J. 2002. interface analysis techniques: Comparison and contrast. ACM
Interacción guiada por objetivos. In Proceedings of the 3th Transactions on Computer-Human Interaction, 3, No.4,
congreso internacional Interacción Persona Ordenador (December 1996), 320-351.
(Leganés, Spain, May 8-10, 2002). INTERACCION'02. 68- [8] John, B.E., Kieras, D.E. 1996. Using GOMS for User
75. ISBN: 84-607-4501-5 Interface Design and Analysis: Wich Technique? ACM
[3] Carrillo A., Falgueras J., Guevara A. 2006. GDIT: Tool for Transactions on Computer-Human Interaction, 3, No.4,
the design, specification and generation of Goals Driven (December 1996), 287-319.
User Interfaces. In Proceedings of the 8th International [9] Kieras, D. 1997. A Guide to GOMS Model Usability.
Conference on Enterprise Information Systems Evaluation using NGOMSL. In The handbook of human-
(Paphos, Cyprus, May 23-27, 2006). ICEIS’06. computer interaction. (Second Edition), M. Helander & T.
[4] Newell, A., Simon, H. 1972. Human Problem Solving. Landauer Eds. Amsterdam, North-Holland, 733-766.
Prentice-Hall. [10] Ikea, 2015. Last access: March, 2015:
http://kitchenplanner.ikea.com/ES/UI/Pages/VPUI.htm

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 205 |
 Goal Driven Interaction (GDI) vs. Direct Manipulation (MD), an empirical comparison
A. L. Carrillo | J. Falgueras
[11] Barnum, C. M. 2011. Usability-Testing-Essentials. Elsevier [12] Dumas, J. S., Redish, J.C. 1999. A Practical Guide to
Usability Testing. Intellect Book.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 206 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck

Generating User Interface from Conceptual, Presentation

and User models with JMermaid in a learning approach
Jenny Ruiz Gayane Sedrakyan Monique Snoeck
University of Holguín KU Leuven KU Leuven
XX Anniversary Ave, Naamsestraat 69 Naamsestraat 69
80100 Holguín, Cuba 3000 Leuven, Belgium 3000 Leuven, Belgium
+53 482672 + 32 16 32 68 83 +32 16 32 68 79
[email protected] [email protected] [email protected]

ABSTRACT (48% of application code; 50% of application time) [17]. These

Because of the pervasiveness of information technology in figures are still considered relevant [7]. For this reason, the study
industry and the tremendous increase in User Interface (UI) of UI development has become an issue of great significance.
design possibilities, the study of UI development has become an UI development is a difficult process [20]. Some authors [16], [8]
issue of great importance. To achieve short turnaround times emphasize that is really hard to understand the user´s
when developing UIs, that moreover are of high quality and can characteristics and the task they perform with the system. Other
adapt to various changes, the UI should be developed using a authors [6] explain the increased complexity by the diversity of
model-driven approach. Such model-driven development, multiple platforms simultaneously, incorporating problems like
generates a demand for IT professionals with model-driven UI the one of splitting User Interfaces in components. Moreover, the
development skills. The technical complexity of model-driven more easy to use the interface should be, the more difficult its
development adds up to the inherent complexity of UI design, and development can be [16]. That´s why it is necessary to base the
makes teaching UI development even more difficult. The development of UIs on models that capture the essence of a UI
evaluation of current environments reveals that none of them design. To achieve short turnaround times when developing UIs,
provides the necessary features to achieve good teaching support. that moreover are of high quality and can adapt to various
Therefore the authors propose to extend the proven conceptual changes, such model-based UI development should preferably be
modelling teaching environment JMermaid by integrating of the expanded to a truly model-driven approach allowing to
W3C standard for Abstract User Interface meta-model and automatically transform models into code. Such model-driven
extending the existing models with presentation and user models. approach allows implementing UI in a professional and systematic
This tool will enable a student to generate a fully functional and way, but it also generates a high demand for IT professionals with
feedback-enabled application with a single click and allow them model-driven UI development skills. Therefore, teaching UI
to study how to design UI according to users preferences and development is difficult due to the high inherent complexity of UI
skills. design and the additional technical complexity of model-driven
development.
Categories and Subject Descriptors
This paper investigates what optimal support to teach UI design
• Human-centered computing~HCI theory, concepts and
would be, evaluates current environments against the necessary
models • Human-centered computing~User interface
requirements for supporting UI design and proposes the
design • Applied computing~Computer-assisted
integration of UI generation concepts in a didactic tool
instruction • Applied computing~Interactive learning
(JMermaid) which is based on a Model-Driven-Architecture
environments • Software and its engineering~Software
(MDA) enterprise engineering method (MERODE) [27]. Based on
prototyping • Computing methodologies~Modeling
the integration of the W3C standard for Abstract User Interface
methodologies
(AUI) meta-model and the enterprise layer meta-model of
General Terms MERODE, the authors propose to generate part of the UI from a
Design, Human Factors, Theory. conceptual model, and to additionally incorporate presentation
and user models. Using this tool, a fully functional application
Keywords could be generated according to users preferences and skills.
User Interface Generation, Model-Based User Interface Moreover, the authors propose method for further expanding the
Development, Model-Driven Engineering feedback features already present in JMermaid to provide a
learner with feedback on the UI aspects to help novices to
1. INTRODUCTION understand the relationship between their design decisions, the
In the last decades, Information Technology (IT) industry has models and the components of the UI.
experienced significant advances that have led to the increased This paper is structured as follows. Section 2 gives a brief
use of software applications with wide ranges of functionalities. description of Model-Based User Interface Development, the
While process automation for software applications has been an additional requirements needed to support a learning approach in
important focus of these advances [10], the range of User UI generation and the analysis of the current approaches. Section
Interface (UI) design possibilities have tremendously increased as 3 presents MERODE and JMermaid and the proposed extension
well. of JMermaid as a didactic tool to teach User Interface design.
Research in the early 1990s found that a significant proportion of Section 4 shows the evaluation of the proposed environment.
software development time was devoted to implement the UI Conclusions are provided in Section 5.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 207 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck

2. MODEL-BASED USER INTERFACE • Coverage of the development cycle: In order to facilitate

learning, the environment needs to offer an integrated support
DEVELOPMENT for the complete software development process, and not only
The generation process of User Interfaces involves the of developed interfaces [7].
development of a conceptual model of the application and its
functionality by the developer, on top of which the UI is then • Generation of application code: [32] explains the importance
implemented, often using one or many metaphors [11]. Different of the application's code generation. To produce an accurate
kind of tools have been created to support UI development based application at low cost, developers expect that both the User
on models, called Model-Based User Interface Development Interface and the application code are automatically generated.
Environment (MB-UIDE) [7]. MB-UIDE is centered around the
premise that a declarative interface model that represents • Concern with user perspective: This allows making more
information about UI characteristics can be used as a basis for flexible UI in order to satisfy the different characteristics and
building interface development environments. skill of users.
MB-UIDE approaches and their supporting tools emerged because • Language and template flexibility: This should allow the
of the need for a comprehensive support of the whole life-cycle implementation of the UI in different ways on different
software development, and the need for a user-centered design delivery platforms.
methodology with corresponding environments [21]. However, in
general, current User Interface tools support only the development • Ease of use: According to [18], MB-UIDEs have a high
phase of the User Interface life cycle. threshold of use because programmers must learn new
languages to define the models. This can be avoided by
A good MB-UIDE environment must meet certain requirements to offering easy to use MB-UIDEs.
satisfy the reasons why they emerged and to reap their potential
benefits. On the other hand, if the environment will be used to • Level of learning support: As this research is intended to
learn UI development, additional requirements appear. Section 2.1 improve the learning of User Interface generation, learning
lists the key advantages of MB-UIDE, Section 2.2 explains the support is also an important criterion to analyze in the
necessary additional requirements to provide effective support in different approach.
teaching UI design, and Section 2.3 analyzes the approaches
following the mentioned criteria. 2.3 Investigating support by existing MB-
UIDE approaches
2.1 Advantages of MB-UIDE In the last decades, different MB-UIDE approaches have been
The model-based approach is instrumental in dealing with the developed. Surveys of the most prominent MB-UIDE approaches
inherent complexity of UI design. By describing the UI through provide in [3],[7] and [33] were used as a starting point. However,
the use of declarative models, developers can implement the UI in for this analysis AME []14, ITS [34], HUMANOID [12] and
a professional and systematic way, more easily than when using UIDE [28] were omitted. For AME and ITS there is not enough
traditional UI development tools. According to [21], [3] and other information available. Mastermind [29] (analyzed in this section)
authors, this paradigm offers a number of key benefits: abstract represents the continuation of the work of HUMANOID and
and centralized UI specifications based on models able to target UIDE. OO-Method [15], Mantra [1] and DB-Use [33] were
different delivery platform [21], [3], [33]; the use of explicitly developed in the last years, and so were added to the analysis.
represented design knowledge [21]; a user centered development
cycle [21]; and facilities for the creation of methods to design and Table 1 lists MB-UIDE approaches and shows the analysis of
implement the UI in a systematic way: (1) model UI using each approach using the mentioned criteria. If the criteria is
different levels of abstraction; (2) incrementally refine the models; achieved, it is marked with “+”; if not with “-”; “+/-” indicates
and (3) re-use UI specifications Da Silva. that there is some, but not entirely concrete evidence that the
criteria is supported.
Since UI designers need to understand the impact of design
choices on the final result, the ability to create a full functional In general, the analyzed approaches lack support for the complete
working application by integrating UI models with other models development cycle. Only OO-Method provides complete support.
that represent a system´s functionality is essential to effectively DB-Use provides partial support with the support for create,
support a teaching process. MB-UIDE approaches make the update, delete and retrieve function. The ease of use is negatively
creation of UI more economical and maintainable by facilitating affected by a lack of documentation, its (lack of) clarity and the
the automatic generation of UIs [7]. The term ‘model-based’ is a low level of integration of tools. The automatic application code
more general denomination that implies the use of declarative generation is only considered by the more recent approaches.
models. In the last few years, MB-UIDEs have evolved towards
Most approaches generate only UI code (Adept [13], Mastermind
model-driven UI development. As [30] explains, there are
[29], Janus [5], Teallach [9]). Fuse generates the user guideline of
comprehensive model-based approaches to develop UI but only a
the application. Mantra is intended to generate the UI for web
few of them are truly MDA compliant [15], [30].
applications and these prototypes are integrated with the
2.2 Additional Requirements application core via web service protocols. DB-Use also generates
Besides their use of models and ability to generate code, a MB- the code for create, update, delete and retrieve functions related
UIDE should satisfy a number of additional requirements in order with the UI. Only OO-Method generates a full application with
to provide effective support in teaching UI design, amongst which commercial tools that are not available for teaching purposes. The
relieving the novice modeler from technical complexities lack of concern of user´s perspective in most approaches shows
associated to MDE and providing learner-oriented feedback: that the end-user of the system is not adequately taken into
account.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 208 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
The criteria that are met the least are learning support and features, facilitating tracing the application’s behavior back to its
template flexibility. Design decisions are difficult to understand origin in the models. The didactic environment furthermore
for novice up to medium developers, and there is no feedback that supports learning analytics based on process-mining of the
helps novices to understand the relationship between design learner’s modelling process. This collection of features allows for
decisions, the models and the different components of the UI. process-oriented feedback rather than outcome feedback only. The
effectiveness of this teaching support has been well tested and
Table 1. Criteria to analyze the MB-UIDE and its evaluation demonstrated by means of scientific experiments [25].
development cycle

3.2 Current limitations

Level of learning
application code

user perspective
Coverage of the

MERODE has no UI development support, but since it starts from

Generation of

Concern with
conceptual models, it has a good potential for extension with UI-

Ease of use
Mb-UIDE

Language
flexibility

flexibility
Template
models. A MERODE-application has three layers: an enterprise
/Criteria

support
layer, an information system service layer and a business process
layer. JMermaid however only supports the development of the
models of the enterprise layer. As a result, all the systems
Adept - +/- + - - - - generated by JMermaid look alike: they have a standard set of
Genius - - - - - - - services, accessible through a standard user interface. For
Mecano - - +/- - - - - example, the tool always generates a menu, in a vertical
Mastermind - +/- - - - - - orientation, with tabs for all the object types in the model. All the
Trident - - - - - +/- - methods can be accessed by buttons. The prototype generated by
Janus - +/- - +/- - - - the JMermaid tool allows handling input and output data.
Fuse - +/- +/- - - - - Although JMermaid has a very good and consistent data
Tadeus - - +/- - - - - validation, it always generates the same kind of representation
format (e.g. textbox for an input) for attributes, no matter what the
Teallach - +/- +/- - - - -
type of the attribute is. The JMermaid tool allows defining
UI Teresa - +/- - +/- - +/- -
additional User Interface functionalities to e.g. search or filter
Goliath - +/- - - - - -
information stored in the database but not in an easy way for the
OO-Method + + - + +/- + - final user (e.g. it is necessary to manually query the database). As
Mantra - +/- - +/- + - - [20] explains, referencing the attribute by name would not be
DB-Use +/- +/- + +/- +/- + - practical or correct if usability is considered a critical aspect of
software quality. However, in JMermaid the name of the labels
Although there are many MB-UIDE approaches, generally they used in the generated User Interface are those from the attributes
do not exploit the benefits of model-driven engineering for and methods of the objects in the class diagram. No possibilities
meeting the growing needs for UI generation. This raises the are offered to specify other names. Furthermore, as it is now,
important issue of assisting software designers and developers in JMermaid generates the labels with the name of the attributes and
building such applications. methods. It also presents the type of the attribute on the label. This
is the way to show the user what kind of data he should provide.
3. LEARNING APPROACH TO This tool always generates a simple input interaction object.
GENERATE UI FROM CONCEPTUAL, JMermaid also manages input errors, so it can guide the user to
PRESENTATION AND USER MODELS enter the correct type of data in each input interaction object (a
Given the lack of support of current environments for teaching number or a character for example). The size of buttons are
User Interface design, and the high need for better support, this proportional to the name of the methods. This causes that not all
paper proposes to extend an existing environment to achieve the button sizes are equal.
better support. In the beginning of this section there is a
presentation and motivation of the choice for MERODE and
3.3 Integrated UI development support in
JMERMAID, the environment that will be extended. JMermaid
Subsequently, this section presents the limitation of the current To improve the way JMermaid supports learning the different
environment in terms of support for UI design teaching. Finally, aspects of application modelling and development, it is required to
an outline of the proposed extensions to the JMERMAID tool is extend the MERODE models with separate UI models. UI models
given. exist at different levels of abstractions. The AUI is an expression
of a UI in terms of interaction units without any reference to
3.1 MERODE & JMermaid implementation and independent of any particular language. In
MERODE is an enterprise engineering approach that follows order to standardize those concepts, the W3C Abstract User
MDA/MDE approach of system engineering. It focuses on the Interface standard has been introduced as a part of Model-Based
development of a conceptual model (close to a Computational User Interface Design to facilitate interchange of designs through
Independent Model) that is sufficiently complete to generate code a layered approach that separates different levels of abstraction. It
out of it [27]. The method is supported by a didactic environment, is a standard meta-model for expressing an AUI as specified in the
JMermaid, allowing model creation and checking their coherence Cameleon Reference Framework Märtin. It defines the main
and consistency. The MDA-based code generator generates a fully concepts for the meta-model, as well as their properties and
functional prototype Java application [23] out of the model relationships, identifying its main requirements, proposing,
information that is stored in an XML file (called mxp-file). The describing and discussing a meta-model that provides a standard
prototype application moreover includes a number of feedback

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 209 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
definition for AUI models Vanderdonckt, J., Tesoriero, R., In MERODE the enterprise layer is captured through 3 models: a
Mezhoudi, N., Motti, V., Beuvens, F., and Melchior, J.. class diagram that defines the domain object types, their attributes
and associations, an object-event table that captures business
It is commonly accepted that simulation contributes to better
event types and relates them to the domain object types by
understanding of modeling decisions offering a new standard of indicating how objects are affected by a business event (created,
learning quality allowing the learner to “learn by experiencing”. modified, deleted), and a finite state machine per domain object
Simulated environments are also known to promote successful type that indicates in which sequence the events types are allowed
transfer of the skills learned in classroom to real-world to happen from the perspective of that domain object type. The
environments by allowing to simulate real-life situations where principal concepts in the enterprise layer meta model are therefore
learners improve their technical and problem-solving skills. the object type and its attributes, the business event types and the
However, the existing standards for simulation technologies also method, which defines the reaction of a domain object type to a
introduce a number of shortcomings. The major disadvantages
business event type.
include being too complex and time consuming to achieve
simulation by novice modelers whose technical expertise is In the AUI meta-model, the AbstractInteractionUnit consists of
limited. Another important disadvantage is connected with the the basic unit for expressing any piece of interaction in a recursive
difficulty of interpreting the simulation results. This makes decomposition of an AUI in abstract terms. This decomposition
MERODE particularly suitable for the context of our study since could be linked to one or many context-dependent definitions like
it provides an integrated environment for modeling and ObjectType and its Attributes. Since any level of the recursive
simulation, which in addition is enhanced with feedback that decomposition is considered as an Abstract Interaction Unit, the
allows facilitating a process of model validation by novices same term is used both for elementary and compound ones. The
[22],[24]. The method has proved to have a significant effect on class Reader is an interface to support a resource to be read which
learning outcomes of novices [26]. are define also by the ObjectType and its Attributes.
This work proposes an extension of MERODE to support DataInputOutputSupport consists of an elementary AIU that is
feedback-enabled UI modeling to improve learning achievements responsible for data input and/or output that could be linked to a
of novice modelers. It is possible to link the concepts applied in domain via a domain reference in order to ensure data binding
UI generation with MERODE’s concepts in a way that not only a giving the associated domain model. UpdateEvent supports the
fully functional application could be generated, but also with a state changes and it is related to the defined methods in the
highest level of usability. enterprise architecture layer that are responsible for making the
updates. Method also is related to InteractionEvent that defines
In order to validate the feasibility of such approach, the authors an interaction event of the AbstractInteractionUnit, not only for
investigated to what extent MERODE-models can be merged with the updates. Finally, although in the enterprise layer meta-model
User Interface models. The MERODE enterprise layer contains the defined EventType is related to the business event that occurs
(amongst others) a domain model that is defined as a description in real life, this EventType also generates the TriggerEvent to
of the classes of objects used in an enterprise, while in UI design a manage all the necessary events to enable the use of the system.
domain model is used to show the description of the classes of
objects manipulated by a user while interacting with a system. The mentioned classes are presented in the JMermaid tool.
These two domain model definitions can be merged to obtain an Nevertheless, the W3C Abstract User Interface standard proposes
improved software in the generation process. Fig. 1 shows the attributes for all the classes like the AbstractInteractionUnit with
connection where the concepts of the AUI meta-model are linked its role, state, frequency, repetition factors and others. Those
with the relevant concepts in the MERODE enterprise layer meta- attributes are not present in the JMermaid tool. It is possible to
model. The green classes are those represented by the enterprise incorporate the complete W3C Abstract User Interface standard to
layer. The others represent the classes in the W3C AUI standard. the MERODE concepts. This would allow generating a better UI.
The red lines are the links between the two layers.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 210 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck

Figure 1. Links between Enterprise architecture and Presentation layers.

Although MERODE uses an MDA approach, and it is possible
to generate different application for different platforms using the
same source models, JMermaid does not yet take into account
information from the user model such as preferences,
capabilities, psychomotor skills, characteristics, of the end-user
to support the UI generation. In order to generate a better UI it is
necessary to incorporate into MERODE and its JMermaid tool
an explicit presentation model capturing (amongst others) those
preferences.
The UI generation process, as explained in [14], generates the
Concrete UI (CUI) from the AUI and then the Final UI (FUI).
The CUI expresses the UI in a more concrete level taking into
account aspects like the platform where the system is going to
run, the program language and the modality of interaction.
However, in this state of the research the incorporated flexibility
to JMermaid is related to the Presentation and User models only.
Currently, the prototype is generated in Java, a multiplatform
language. Since the developed tool is intended as teaching Figure 2. Artefacts and modeling cycle with AUI concepts
support and in order to ensure the feasibility of the project, the incorporated to MERODE.
intent is to maintain the generation of a Java application for
The generation process will use platform independent models
desktop computers as a start. For this reason, in this case, the
according to 3 views: a structural view, that is established
platform, program language and modality of interaction are
through the class diagram; a functional view, defined by a
previously known, allowing skipping the elaboration of CUI and
collection of Finite State Machines and the Object-Event Table;
going directly from AUI to FUI, as shown in Fig. 2.
and an Interface view, defined by a Presentation model and a
User model. After the modeling activity and the code generation
step, in each iteration, developers can change the models in
order to improve the quality of the generated prototype.
The code generation of the extended tool will enable the
automatic creation of a UI. This gives the possibility to validate
user requirements reducing the time and effort required to
implement the UI. UsiXML [11] was the UI description
language selected to generate the UI. Using the MDA approach
a Model to Model transformation will be performed to obtain the
AUI model using as a source the EDG, FSM and OET in the
mxp file.
The proposed extensions will contain new rules for obtaining the
control type of objects that can be added in order to improve the
generated UI by JMermaid based on abstract level models. It
should be possible to add a user model through JMermaid that
defines, for example, based on the experience of the user whose

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 211 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
control type is best in every situation. If there is more than one JMermaid has implemented visualizations both for structural
control type determined then the software could choose one of and behavioral aspects that include feedbacks on failures that
them based on the information in the user model. can result from: (1) Mandatory one cardinality violation: an
object is attempted to be created before the object it needs to
The proposed approach comprises an iterative development refer to, is created or chosen to be associated with the newly
process that enables the automatic generation of a fully created object; (2) Cardinality violation: a create-event
functional prototype, including, of course, the UI code. The execution fails due to a cardinality constraint of maximum 1; (3)
generated code can be obtained from early stages in the Referential Integrity: ending-event execution fails due to
development process, as long as the used models are correct. existing referred (dependent) objects; (4) Sequence constraint
The models will be based on the W3C AUI standard and violation: event execution fails because of the state of the owner
MERODE Enterprise layer meta-models with links between object in which the event execution is not enabled [26].
Enterprise architecture and Presentation layers.
The feedback features currently present in JMermaid are
The tool will select the templates dynamically according to the illustrated by means of an example of a generated system from a
information stored in presentation and user models. These model solution for controlling the information about students.
templates will be used when generating programming code for With this system the students can make an appointment to ask
the graphical components of the application. Following the questions about the courses they are following to the teaching
transformation of the UI generation model, the generator will assistants of these respective courses.
produce different final UIs according to the selected templates.
Using this approach identical data can be displayed in different The following scenario is an example of what is shown to the
ways (within text fields, inside tables, with menus, etc.). This is modeler when testing the prototype, : a teaching assistance is
showed in Fig. 3: the User Model and Presentation Model are attempted to be created for a course before selecting a teaching
stored in the mxp-file, jointly with the enterprise model. This assistant to associated to it. This is the case of a mandatory one
collection of models is transformed into an AUI by means of a cardinality violation. A popup window will explain to the
model-to-model transformation. Using different templates, this modeler why it is not possible to create this object and a possible
AUI is transformed into different prototype UI for the solution according to the model; in this case: creating an
application that is generated from the enterprise model. This instance of a teaching assistant first or selecting an existing one.
figure only shows the generation process of the extension of The feedback can be expanded showing the class diagram and
JMermaid for UI prototyping. As it was showed in Fig. 2, the highlighting the involved classes and associations with the same
persistence layer and the event handler is generated from the colors used in the explanation. This is shown in Fig. 4.
conceptual domain model, as in previous versions of JMermaid.
Similarly as for the Enterprise Model feedback, UI feedback
At the same time, developing such models for the UI features will assists the learners to validate the generated UI in a
development and generating the prototype according to this will fast and easy way. At the same time, it will also allow to
allow iterative changes to the software solution and facilitate compare the impact of different decisions made in the
comparison between each one of them prior to final approval. Presentation, User and AUI models in the final prototype
through experimenting with a concrete form of an information
system.

Figure 3. Extension of the JMermaid prototype generation

using AUI concepts, Presentation and User models.
3.4 Automated feedback for UI
development incorporated in JMermaid
While prototyping in itself is already an efficient instrument
supporting the learning process, adding feedback features can
offer further support of learning [23], [24], [26]. JMermaid
already includes feedback features for the enterprise models.
The feedback features of JMermaid allow to explain reasons of Figure 4. Feedback for mandatory 1 constraint violation.
execution failures with graphical visualization that link the The feedback can show to the modelers the reasons why the UI
failure to the model used for the simulation. For example, the is generated in certain way. Fig. 5 shows the main window of an
user may be informed when trying to execute an event that is not example of a generated prototype. The feedback explains that
allowed because of failed precondition checks. In this case, a the orientation of the options, that could be vertical or
message explains the reason of rejection by indicating what horizontal, is selected according to choice made in the
constraint of a model is violated [22]. Presentation Model. It gives the possibility to see this
Presentation Model and explains how to change it.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 212 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
Table 2. Evaluation of the proposed tool
MB-UIDE/Criteria Proposed tool
Coverage of the development cycle +
Generation of application code +
Concern with user perspective ++
Language flexibility +
Template flexibility ++
Ease of use +
Level of learning support ++

These criteria allow to make a theoretical evaluation of the

proposed approach in two directions: 1) for the User Interface
development (the first four criteria) and 2) learning support
(specially the last three criteria). Nevertheless, as was explained
in section 2, all of them are important to provide effective
Figure 5. Feedback related to the orientation of the options. support in teaching UI development.
Fig. 6 shows a window for the creation of an instance of certain
The major advantages are achieved in level of learning support,
object type. The feedback explains the relationship between the
template flexibility and dealing adequately with the user
control input type and the type of the attributes of this object
perspective. Specifically the two first criteria were met the least
type. This, among other choices like the rules for determining
in the analyzed MB-UIDE approaches. With the proposed
the control type of the final interaction objects, allows to
approach, design decisions should be easier to understand for
generate an UI according to the user´s preferences and skills,
novice up to medium developers, and there will be feedback that
collected in the Presentation and User Models. It gives the
helps novices to understand the relationship between design
possibility to see the rules related with this generated window
decisions, the models and the different components of the User
and the Existence Dependency Graph with the object types and
Interface. More in particular, this extended JMermaid
their attributes.
environment will support the learning and teaching of UI design
in the following way:
• Coverage of the development cycle: This tool offers an
integrated support for the complete software development
process, including UI development.
• Generation of application code: The tool provides
automatic generation of software products (with both UI and
application code) that are functionally equivalent to the
system specification in one language for which a
transformation process from an AUI is defined.
• Concern with user perspective: The proposed tool will
generate the User Interface taking into account the user´s
characteristics and skills in Presentation and User models.
The generation process uses rules for specifying the control
type based not only on the data types, primary key, foreign
Figure 6. Feedback related to the control input type. key, relationships between objects but also the actual needs
of users.
It is possible to show the links between the User Model and the
generated UI, to see how JMermaid can select, based on the • Language and template flexibility: Because the AUI and
experience of the user, for example, which control, among the the rest of models used to generate the final application are
possible ones, is best in every situation. If there is more than one totally independent of the platform, they can be transformed
control type determined for one Abstract Interaction Object, into different final UI for different platforms. The defined
then the software chooses one of them for based on the transformation process uses templates that can be changed in
information in the User Model. order to obtain the implementation of the software in
different languages, platforms and ways of presentation.
4. EVALUATION OF THE PROPOSED
• Ease of use: Only one tool is needed for the development
ARCHITECTURE process. Using this tool novice modelers are able to make
The previous sections presented an integrated architecture for UI the Conceptual, Presentation and User models. Once the
development support and the automated feedback for UI models are in the tool the software is generated with only
development incorporated in the tool. one click. This produces an “illusion of simplicity”, which
Table 2 shows the analysis of the proposed approach using the makes it practical to use.
criteria presented in section 2. • Level of learning support: The tool offers to novice up to
medium developers a running start on UI design, and
XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design
| 213 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
stimulate them to reflect on how come up with better UI Hypertext (1990), ACM, 105-117.
with less effort. It is possible, using an iterative approach, to [6] Florins, M. Montero, F., Vanderdonckt, J., Michotte, B.,
incorporate different aspects to the Conceptual, Presentation Splitting rules for graceful degradation of User Interfaces,
and User models and see impact of the UI by the generation in Proceedings of 8th Int. Working Conference on
process. The tool uses feedback to explain the Advanced Visual Interfaces AVI´2006, (Venezia, 2006),
correspondence between the UI models and the actual ACM Press, 59-66.
design when generating the prototype.
[7] Gomaa, M., A. Salah, and S. Rahman, Towards a better
5. CONCLUSION model based user interface development environment: A
This paper proposes an architecture for an integrated UI comprehensive survey, in Proceedings of MICS 5, (2005).
development support environment. This environment will assist [8] Gould, J.D. and C. Lewis. Designing for usability: key
in involving the novice up to medium developers in real principles and what designers think. Communications of the
development and developing knowledge of what a good User ACM, 28(3), 300-311.
Interface design requires. There are a few criteria that are
necessary for the successful implementation of any new tool, in [9] Griffiths, T., McKirdy, J., Paton, N.W., Kennedy, J.B.,
this case, for teaching the basics of good UI design. Cooper, R., Barclay, P.J., Goble, C.A., Gray, P.D., Smyth,
M., and West, A., An Open-Model-Based Interface
There are a number of approaches and tools that support the Development System: The Teallach Approach. in DSV-IS
different phases of UI development, as was shown in this paper. (1998), Citeseer, 34-50.
In general however, current approaches and tools either focus on
[10] Harrison McKnight, D., V. Choudhury, and C. Kacmar.
the design phase, or on the development phase, and have a
The impact of initial consumer trust on intentions to
number of shortcomings that make interfaces difficult to build.
transact with a web site: a trust building model. Journal of
The solution proposed in this paper is an integrated approach Strategic Information Systems, 11(3),297-323.
that targets the entire software development process, focuses on [11] Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L.,
the User Interface design and requires little technical expertise and López-Jaquero, V. USIXML: a language supporting
on behalf of the students. As the generation of a full-functional multi-path development of user interfaces. in Engineering
prototype pre-supposes the existence of a domain model and its human computer interaction and interactive systems,
generation, to keep the project feasible, the investigation will Springer, 2005, 200-220.
start from a proven existing environment (JMermaid) that
already generates the core functionalities of the application [12] Luo, P., Szekely, P., and Neches, R., Management of
based on a conceptual model. This environment is adapted to interface design in HUMANOID. in INTERACT'93 and
learning goals, and is equipped with feedback framework that CHI'93 Conference on Human Factors in Computing
makes it easier to expand to support UI. The proposed extended Systems, (1993), ACM, 107-114.
JMermaid environment will use UsiXML. It will incorporate a [13] Markopoulos, P., Pycock, J., Wilson, S., and Johnson, P.,
presentation and user models to improve the flexibility of the UI Adept-A task based design environment. in Twenty-Fifth
generation. Hawaii International Conference on System Sciences
(1992), IEEE, 587-596.
6. ACKNOWLEDGMENTS [14] Märtin, C., Software Life Cycle Automation for Interactive
The authors would like to thank Jean Vanderonckt for his
Applications: The AME Design Environment. in CADUI,
excellent feedback and the opportunity of visiting UCL. This
(1996), 57-76.
research has been sponsored by the VLIR-UOS network
program. [15] Molina, J.C. and O. Pastor, MDA, OO-Method y la
tecnología OlivaNova Model Excecution.in I Taller sobre
7. REFERENCES desarrollos dirigidos por modelos, MDA y aplicaciones,
[1] Botterweck, G. A model-driven approach to the (Málaga, 2004).
engineering of multiple user interfaces. in Models in
Software Engineering, Springer, 2007, 106-115. [16] Myers, B.A. Why are human-computer interfaces difficult
to design and implement, DTIC Document. Carnegie-
[2] Calvary, G., The CAMELEON Reference Framework, Mellon Univ Pittsburgh Pa Dept of Computer Science.
CAMELEON Project Retrieved March 26, 2015, from 1993.
http://giove.isti.cnr.it/projects/cameleon.html.
[17] Myers, B.A. and M.B. Rosson, Survey on user interface
[3] Da Silva, P.P. User interface declarative models and programming. in SIGCHI conference on Human factors in
development environments: A survey. in Interactive computing systems, (1992), ACM.
Systems Design, Specification, and Verification, Springer,
2001, 207-226. [18] Myers, B.A. User interface software tools. ACM
Transactions on Computer-Human Interaction (TOCHI),
[4] Daniel, F., Matera, M., Yu, J., Benatallah, B., Saint-Paul, 2(1), 64-103.
R., and Casati, F. Understanding ui integration: A survey of
problems, technologies, and opportunities. Internet [19] Pastor, O., and Molina, J.C.. Model-driven architecture in
Computing, IEEE, 11(3). 59-66. practice. Springer, 2007.

[5] Fischer, G., McCall, R., and Morch, A., JANUS: [20] Puerta, A.R., Eriksson, H., Gennari, J.H., and Musen,
Integrating hypertext with a knowledge-based design M.A., Beyond data models for automated user interface
generation, in BCS HCI, (1994), 353-366.
environment. in Second annual ACM conference on

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 214 |
 Generating User Interface from Conceptual, Presentation and User models with JMermaid in a learning approach
J. Ruiz | G. Sedrakyan | M. Snoeck
[21] Schlungbaum, E. Model-based user interface software Conference on Human Factors in Computing Systems
tools-current state of declarative models. Graphics, (1993), ACM, 375-382.
Visualization & Usability Center, Georgia Institute of [29] Szekely, P.A., Sukaviriya, P.N., Castells, P.,
Technology, 1996. Muthukumarasamy, J., and Salcher, E., Declarative
[22] Sedrakyan, G. and M. Snoeck, A PIM-to-Code interface models for user interface construction tools: the
requirements engineering framework. in Modelsward 2013- MASTERMIND approach. in EHCI (1995), Citeseer, 120-
1st International Conference on Model-driven Engineering 150.
and Software Development-Proceedings, (2013), 163-169. [30] Vanderdonckt, J., A MDA-compliant environment for
[23] Sedrakyan, G. and M. Snoeck. Feedback-enabled MDA- developing user interfaces of information systems. in
Prototyping Effects on Modeling Knowledge. in Advanced Information Systems Engineering, (2005),
Enterprise, Business-Process and Information Systems Springer, 16-31.
Modeling, Springer, 2013, 411-425. [31] Vanderdonckt, J., Tesoriero, R., Mezhoudi, N., Motti, V.,
[24] Sedrakyan, G. and Snoeck, M. Technology-enhanced Beuvens, F., and Melchior, J., MBUI - Abstract User
support for learning conceptual modeling. in Enterprise, Interface Models. Retrieved January 12, 2015, from
Business-Process and Information Systems Modeling. http://www.w3.org/TR/2014/NOTE-abstract-ui-20140408/
Springer, Berlin – Heidelberg, 2012, 435-449. [32] Vi Tran, T.A., Kolp, M., Vanderdonckt, J., Wautelet, Y.,
[25] Sedrakyan, G., M. Snoeck, and J. De Weerdt. Process and Faulkner, S. Agent-based user interface generation
mining analysis of conceptual modeling behavior of from combined task, context and domain models. in Task
novices–empirical study using JMermaid modeling and Models and Diagrams for User Interface Design, Springer,
experimental logging environment. Computers in Human 2010. 146-161.
Behavior, 41(2014), 486-503. [33] Vi Tran, T.A., Vanderdonckt, J., Kolp, M., and Wautelet,
[26] Sedrakyan, G., M. Snoeck, and S. Poelmans. Assessing the Y, Generating User Interface for Information Applications
effectiveness of feedback enabled simulation in teaching from Task, Domain and User models with DB-USE. in 1st
conceptual modeling. Computers & Education, 78,367-382. International Workshop on USer Interface eXtensible
[27] Snoeck, M. Enterprise Information Systems Engineering: Markup Language UsiXML'2010, (Berlin, 2010), 183-194.
The MERODE Approach. Springer, 2014. [34] Wiecha, C., Bennett, W., Boies, S., Gould, J., and Greene,
[28] Sukaviriya, P., Foley, J.D., and Griffith, T., A second S.. ITS: A Tool for Rapidly Developing Interactive
generation user interface design environment: The model Applications. ACM Transactions on Information Systems, 8
and the runtime architecture. in INTERACT´93 and CHI´93 (3). 204-236.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interface Design

| 215 |
INTERACTION DEVICES (I)
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach

Blind Source Separation Performance based on

Microphone Sensitivity and Orientation within Interaction
Devices
Navya Amin, Thomas Gross, Susanne Rosenthal, Markus Borschbach
FHDW, University of Applied Sciences,
Hauptstr. 2, 51465 Bergisch Gladbach, Germany
{ami, tgr,rsu,mbo}@fhdw.de

ABSTRACT are currently done using a smartphone. Using a smartphone as a

The use of smartphones has tremendously increased in the last hearing aid or even in combination with a hearing aid is the focus
decade. The coupling of a mobile phone with multiple sensors has of our research. We intend to develop a smartphone application:
expanded the usage of smartphones into a wider area. A basic SMARTNAVI app [35], [23] which captures audio signals in the
research on pre-requisites of Blind Source Separation (BSS) to be surroundings of the user, separates the sound sources and allows
used in interaction devices is performed. Our focus of research is the user to select his source of interest for hearing. The
the application of signal processing techniques for BSS of audio architecture of this app including a benchmarking of different
signals captured using interaction devices like the smartphones open source BSS tools on three categorized acoustic challenging
with the final aim of recovering the spatial listening experience of situations is presented in [35]. Application of such a BSS app in a
persons with reduced hearing abilities using the proposed app- smartphone is not just restricted to hearing impaired individuals
based system. With this app, the user is able to interactively but also designed for people without hearing disabilities. This is
enhance speech or sound sources in his surroundings using the applicable in case when two individuals speak in a crowded
interactive interface on his smartphone. The sensitivity and environment, which in technical signal processing terminology is
orientation of the smartphone microphones (mics) play a very termed Cocktail party problem [7],[1]. The individuals are able to
important role to achieve a better quality of BSS and hence is one separate out the speech sources of their interest using this app.
of the prime prerequisites for BSS. Therefore, a benchmarking Hearing impairment refrains the affected from recognizing the
regarding the interdependence of the different internal and sounds in the environment and also distinguishing different
external mics as well as the orientation of the smartphones relative sounds. Thus an interactive interface becomes essential in such
to the sources on the overall separation performance is discussed scenarios. The SMARTNAVI app enables the user to interactively
in this paper. Four smartphones are used to examine the influence select the sound sources of interest using his smartphone or any
of mic sensitivity on the BSS quality of captured signals. Further other interactive smart device. The user is also able to recognize
external mics with smartphones are tested for their influence on any dangerous sounds to avoid approaching danger, suppress the
BSS. The captured audio signals are separated using PARAllel noise signals for better speech intelligibility and to overcome
FACtor analysis (PARAFAC) based BSS algorithm. orientation and communication difficulties. This app is designed
such that it is able to capture audio signals in the environment,
Categories and Subject Descriptors separate the sources and provide visualization of the speech and
H.4 [Information Systems Applications]: Miscellaneous; noise sources. The number of separated sound sources depends on
D.2.8 [Software Engineering]: Metrics-complexity measures, the number of mics built in the interactive device. Also the
performance measures sensitivity of the mic influences the quality of audio capture and
source separation thus influencing the final output.
General Terms The audio systems of smartphones are technically designed very
Theory
differently with the consequence of high performance differences.
Keywords The quality of in-built audio capturing systems in smartphones
has improved to a large extent to sustain itself in the smartphone
Smartphone, Internal Microphones, External microphones,
market race. Different phones or vendors provide different levels
Microphone sensitivity, BSS.
of sound quality. Using a smartphone for a BSS app requires a
1. INTRODUCTION very high quality of audio-captures, good enough to enable source
Smartphones have penetrated the mobile phone market in the last separation. In this paper we have captured audio signals in
few years due to cost effectiveness and their applications into a extremely Lower Level of External Noise (LLEN) using four
wider area. People today depend on their smartphones to perform different smartphones. The audios captured using these
a lot of their day-to-day activities. According to Gartner, Inc. smartphones are compared based on the quality of BSS. The
Smartphones accounted for 53.6 percent in 2013 [17] and two- findings of this experiment would clarify the smartphones that are
thirds in 2014 [18] of the total mobile phone sales. With the favorable for their usage as audio devices for capturing sound
rapidly increasing percentage of sales more than two billion signals. The quality of the sound signals depends on the quality of
people worldwide are expected to own a smartphone by 2016 as the inbuilt mics in these smartphones or external mics as well as
stated by eMarketer Inc. in [15]. Equipped with many sensors, a the orientation of the smartphones during the process of audio
vast amount of applications previously performed on a computer signal capturing. This totally depends on the sensitivity of the mic
in the smartphones which varies to a great extent across vendors

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 217 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
as stated in the review in [28]. In this paper, we have tested the visually impaired [39]. The OtoSense Mobile app enables the user
quality of audio recordings captured in 2 different environments to identify and distinguish the source of incoming audio alerts like
giving good quality separation of the mixed sources in the audio- alarms, sirens, timers, bells and other sounds making smartphone
captures into their individual sources. We have also tested the an assistive device for the hearing impaired [26]. Several other
influence of the orientations of the smartphones during audio apps for assistive hearing systems are presented in [31], [32], [8],
capturing. The focus of this paper is to examine interdependence [9], [38] and [16].
of the mic-sensitivity and orientation of the smartphones relative
to sources with the final aim of achieving optimum BSS quality. 3. SMART MIC
The audio data is qualitatively evaluated by informal listening The conventional smartphones were equipped with only one mic
tests regarding a subsequent successful BSS separation. Further, for voice capturing unlike the state-of-the-art smartphones. Today
the influence of the mic orientation as well as the attachment of an the smartphones are equipped with two or more mics: one for
external mic on the separation performance are investigated. capturing and the other for noise cancellation. Nokia Lumia 1520
(NL1520) has four bidirectional mics inbuilt for a better capturing
2. RELATED WORK quality. The iPhone models above 5S are equipped with three
The use of smartphones is extended immensely from its usual mics: one for noise cancellation and two for audio capture. Other
purpose making it an ubiquitous communication technology. It is phones like the Sony Experia Z2 (Z2), Z3 (Z3), Samsung Galaxy
used not just for communication but also in the field of Health- S4 (S4) and above, LG Nexus 5 (N5) and above are equipped with
care, Fitness, Navigation, Entertainment, Banking and Finance, e- multiple mics for similar audio capturing and noise cancellation
commerce, etc.. The reason for this is the variety incorporated purpose. The internal mics of the smartphone are basically meant
sensors in a smartphone allowing the handling of different tasks for voice capturing. However, the smartphone mics do not provide
with one small device. The mics of the smartphone now-a-days the high quality audio-captures like the hand-held recorder or
are not just used to make a phone call or record a voice but for external mics. In such cases, external mics are a better option used
many other purposes like audio signal processing as well. Several to fulfill the requirement of high quality captures for signal
modules for audio signal processing using smartphones are processing. The internal mics are able to capture a restricted
developed for their implementations in a larger application area: frequency range of approximately 300 Hz to 3.4 kHz that is much
Sur et al. in [37] design and implement a so-called Dia system to lower compared to the external mics. However, not every state-of-
overcome the inability of the smartphone to locate the spatial the-art smartphone provides the audio jack for multi-channel
position of the speaker in noisy environments. They use a audio-captures. So far, only Z2 and Z3 provide a 5-pole 3.5 mm
synchronized mic array to determine autodirective audio capturing audio connector to capture stereo audio data. The other Android
and develop a practical speaker tracking and audio beam-forming phones including iPhone still have a 4-pole jack which captures
module. Asakawa reports about sound signals used in different only mono audio data. The mics as used by Onu [25] are directly
application areas like surveillance and artificial systems like BSS connected to the dock like the TASCAM iM2 for the iPhone.
[4]. He exemplifies about Shot Spotter, a system developed by a Several other mics attachable to the micro-USB port of Android
American start-up company which works like a sound phones are used to capture multi-channel audio data. The
surveillance system. This system involves several mics set up in directionality of the external mics is used as an advantage for
the city streets that alert the security companies and police of any audio processing especially in the case of BSS.
abnormal sounds. He also discusses the work of Onu, where BSS
methods are used for separating desired sound signals from a 3.1 Internal Smartphone Mic
jumble [4], [25]. In [25] auxiliary-function-based independent Traditionally, internal smartphone mics work best in the near-field
vector analysis (AuxIVA2) for stereo signals is used for BSS range. For phones, the audio source (i.e. the mouth of the speaker)
which is implemented as an iPhone application. Mukai et al. is mostly close to the mic. However, smartphones now offer far
develop a prototype system which estimates the Directions of more options than just making phone calls. For example, they
Arrival (DOA) of the source signals in addition to the separation serve as a voice recorder or to record the sound when shooting a
of these signals. They used 8 omni-directional mics to separate video. In order to cover all areas satisfactorily, mostly omni-
speech signals in real-time distributed in three dimensional space directional Micro-Electro Mechanical Systems (MEMS) [48] mics
and a complex valued Infomax algorithm [6] for BSS. Pertila et are built into modern smartphones [46]. Omni-directional mics
al. propose a closed form temporal offset estimation technique and record all sound sources in the environment. These include both
estimates the accuracy of this approach using 10 smartphones for foreground sounds like conversations, as well as background
audio capturing in three different rooms [27]. Li et al. develop an noises like distant people talking or street noise. For our project,
algorithm for non-linear multiple-regression based enhancement mics without any directionality could be a disadvantage. For
method for in-car speech recognition [22]. hearing aids, it is proven that devices with directional mics
improve the Signal-to-Noise Ratio (SNR) of speech occurring in a
The uses of smartphones and their sensors have expanded
noisy background [5], [19], [12], [20], [33]. As described later,
tremendously in the field of artificial intelligence and robotics.
directional mics have noise-cancellation characteristics due to
Sensing technologies are now used by people owning a their design which can significantly reduce background noise
smartphone making their day-to-day activities easier. Application during audio capture, without having to perform any subsequent
of SMART technologies for fulfillment of human disability is of software-based noise reduction. Such optimized audio-captures
great interest for the researchers today. Smartphones are allow a better BSS. However, the sound source must be within the
becoming a common mode of assistive technology [14],[13],[11] directivity angle, at which the mic is sensitive. For example -
for the disabled due to their simplistic learning, usage, integration depending on the directional characteristic of the mics - the sound
and support methods. Several applications for blind, deaf, arriving from the sides is captured with a lower volume or is
dyslexic, diabetic etc. are available for different smartphones.
strongly attenuated. However, there are already solutions for this
Peek Vision is one such example as a diagnostic tool for the problem [20]. Another problem of the internal mic is its

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 218 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
sensitivity. Originally telephones are intended to make phone too similar since the mics are placed very close to each other. For
calls. The audio source is normally close to the telephone mic. example, the iPhone has mics placed next to each other. As a
Due to the low sensitivity of audio sources, the sensitivity of the result, the channels of the audio output even though stereo, are
internal mic is no longer sufficient to record sources that are very similar to each other. However, some Android phones
located a few meters away from the mic. As shown in Figure 1, provide a 5-pole output enabling better stereo audio capture with
the level of an acoustic signal decreases by around 6dB (one-half) mics placed at opposite ends of the phone like the Z2 and Z3. The
each time the distance from the source is doubled. In addition, N5 has mics placed also at opposite ends but provides only a 4-
modern audio chips have hardware-integrated components for pole output jack enabling mono audio capture [47]. Some
automatic noise cancellation during the audio capture independent applications that perform stereo audio capture using such
from the used audio capture app [29]. smartphones convert mono audio signals to stereo. External mics
play an important role here. However, external mics for phones
like N5 having a 4-pole output must be connectable to the micro-
USB port of the phone. The human ear is able to differentiate
between different sounds due to its directionality [42]. A similar
directionality approach using the smartphone for BSS is used in
this paper. Two experiments are performed to infer on the mic
sensitivity and orientation of the smartphone enabling better BSS
quality of captured audio signals.

4.2 Performance of Mics for BSS

The aim of the first experiment is to compare the quality of BSS
of captured audio signals using internal and external mics across
different smartphones. The audio-captures for this experiment are
done using the different smartphones presented in Table 1.
Table 1. Overview of smartphones and operating systems used
for the experiments.
Operating System Phones

Figure 1. The volume is reduced by half when the distance to Android N5 S4 Z2 Z3

the mic is doubled. This image is a modified reproduction iOS iPhone 5S
from [21].
Windows NL1520
3.2 External Smartphone Mic Two types of external mics are used for capturing audio signals
There are numerous external mics for smartphones available. with Z3 and Z2 namely STM10 and Edutige EIM- 001. STM10 is
They differ in their directional characteristics, size and other a stereo mic with a 5-pole audio jack as illustrated in Figure 2a,
technical specifications. Their advantage over internal mics is the which is only compatible with the smartphones Z3 and Z2 and
previously described sensitivity. Audio sources from a distance of Edutige EIM-001 is an omni-directional mic with a 4-pole audio
several meters could be recorded significantly better with external jack (illustrated in Figure 2b) and is compatible with all of the
mics compared to the internal mics. A further advantage is the smartphones listed in Table 1. Therefore, STM10 is not usable
significantly greater frequency range obtained using external mics with the other Android smartphones, iPhone or NL1520 and the
compared to internal mics. Frequencies in a range of Edutige EIM-001 is used for these smartphones instead.
approximately 150 Hz to 4000 Hz are necessary for the
intelligibility of the human voice [36]. The full range of the
human voice, e.g. a singing voice, is approximately in the range of
80Hz to 12 KHz [2]. In this frequency range, there are frequency
segments that play a role in speech intelligibility, for example
high-frequency sibilance [10]. Usually the main frequencies in the
range of 200-3500 Hz (Narrowband) have been covered in the
traditional telephones. Newer smartphones however use the HD
voice or HD voice standard with a frequency range from 50Hz to
7kHz or higher [34] (wideband, or better known as Adaptive
Multi-Rate Wideband (AMR-WB)). Modern Smartphone mics
often cover a frequency range from 20Hz to 20kHz (full-band)
[30]. Figure 2. a. SONY STM10 Stereo Mic (Image is taken from
[40]), b. Edutige EIM-001 Omni-directional Mic (Image is
4. EXPERIMENTS taken from [41])
4.1 Technical requirements The apps for audio capture were selected based on the avail-
The hardware set up of smartphones is mostly for capturing less ability of the options to set the Sample rate, Bit Rate, Output
distant sound signals. This makes it challenging to use format, Automatic Gain Control and ability to select channels
smartphones for BSS. As mentioned earlier, some smart-phones (mono/ stereo). The app RecForge Lite [43] is used on the
are equipped with two mics. But the captured audio channels are Android smart-phones (Z2, Z3, N5 and S4). The app provides the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 219 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
additional option to choose between mono and two different types distances. The difference in the distance is a potential advantage
of stereo channels (Native and Mono x 2). [43]. The app for iOS for the separation algorithm. Further, a BSS algorithm based on
platform is Awesome Voice Recorder by the Newkline Co., Ltd. PARAFAC is applied on the captured audio signals. The quality
[44]. THE SOUND RECORDER app [45] from Windows is used of source separation is estimated by informal listening tests for
for audio capturing in NL1520. The audio capturing is done in a both test series. The quantification of the difference between the
room of the size length: 6m x 3m x 3m (length x width x height). channels is done by calculating the difference between the two
The mics are placed at a distance of one meter from two sources channels.
S1 and S2 as depicted in Figure 3 and 4. These experiments
comprise the influence of using two smartphones for simultaneous 5. RESULTS
recordings as a mic array on the BSS quality: Therefore, audio-
captures were done by two N5 mics which are at a distance of 0.5 5.1 Technical requirements
meters from each other and both mics have a distance of one The difference between the channels is very important for source
meter to the sources. Further, a BSS algorithm based on separation algorithms. The audio files with higher channel
PARAFAC is applied on the captured audio signals. difference achieved a better source separation com- pared to the
audios with lower channel difference. However, even the good
4.3 Impact of Mic Orientation separated audio files have some crosstalk of the source which
The second experiment estimates the impact of the mic orientation could be handled using some post-processing methods. The stereo
relative to the sound sources. This was performed in LLEN audio-captures performed using the setting Mono x 2 in the audio
environment. The smartphones used for this experiment were Z3 recorder app gave absolutely no separation using the PARAFAC
and N5 as we are mainly focused on open source Android algorithm. On determining the difference between the two
platforms. In general, the smartphones Z3 and S4 are not further channels it is seen that both the channels are exactly the same and
considered in these experiments since the performance of these in some cases slightly different which is insufficient to determine
phones are similar to Z3. The experimental set ups are depicted in a good separation. This is because of the conversion of mono
the Figure 3 and 4: channel to stereo by the sound recording app. Further, on testing
the Stereo Native option, it is seen that the difference between the
channels also depends on the specific mic sensitivity of the
smartphones used.

Figure 3. Layout of the experimental setup 1: Horizontal

orientation of the mics M1 and M2 of the smartphone relative
to the sources S1 and S2.

Figure 4. Layout of the experimental setup 2: Vertical

Figure 5: Difference between the channels captured using
orientation of the mics M1 and M2 of the smartphone relative
to the sources S1 and S2. internal mics of different smartphones for male and female
speech.
where S1 and S2 are the sound sources and M1 and M2 are the
The source separation of the audio mixes captured using the
internal mics of the smartphones. The mic of the smartphones Z3
internal mics of the smartphone gave very weak source separation
and N5 are placed at a distance of one meter from the sound
with IP5S and NL1520. The NL1520 technical specifications
sources in two different orientations namely vertical and
states four directional mics. However, these mics could be
horizontal as shown in Figure 3 and 4. Two types of audios
accessed only with video capturing and not with audio capturing
captures are used, male speech and female speech (left-right
applications. Also the stereo audio-captures with this device
resp.), male speech and cafeteria noise (left-right resp.). The audio
reached no sufficient source separation. However, for S4, Z2 and
files have a length of approximately 10 seconds. The files are
Z3 source separation was obtained at different quality levels based
captured using 0 gain at 44kHz and 16 bits in stereo native WAV
on the informal listening tests. The boxplots in Figure 5 show that
format. The phone is placed first horizontally and later vertically
the channel differences for N5, IP5S and NL1520 are very low to
to examine if the orientation of the phone has an influence on the
achieve a good quality source separation unlike S4, Z2 and Z3.
difference in the matrices. These set ups are guided by the idea
The mean channel differences for S4, Z2 and Z3 were in the range
that the mics at both ends of the phone in the horizontal position
of 0.01 to 0.015 which was sufficient to achieve source separation
receive signals from the left and right speakers at different
even though the quality of this separation was weak. Further the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 220 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
recordings made using the external mics showed that the usage of within the range of 0.01 to 0.015. A good quality source
external mics had a positive impact on the quality of source separation is achieved using the N5 phones as an array due to a
separation compared to the usage of internal smartphone mics. comparatively higher distance between the mics (Figure 7a). The
However, this positive impact was restricted to the usage of the audio files from AMDecor [3] recorded by Anemueller et al.
STM10 stereo mic. The stereo audios captures using the Edutige using PARAFAC gave a very good quality compared to the best
EIM-001 mic gave no separation of source channels due to a very source separation performing experiment using smartphones. The
low difference between the channels as shown in Figure 6. This mean channel difference for the AMDecor audio is much higher
mic was only compatible with S4, N5, Z2, Z3 and NL1520 and as shown in Figure 8 compared to the smartphone recording due
thus the experiment is performed using only these phones to the mics used for audio recording. The recordings were done
excluding IP5S. This mic only allows good quality mono channel using professional mics by Anemueller et al. and not smartphone
recordings and thus is insufficient for our purpose of BSS using mics.
PARAFAC which requires at least two input channels.

Figure 8: Difference between the channels captured using two

mics for AMDecor.
Figure 6: Difference between the channels captured using
EIM-001 with different smartphones for male and female
5.2 Results for Impact of Mic Orientation
The orientation of the phone and thus the position of the mics
speech.
relative to the sources while audio capturing also has an influence
The audio captured using the STM10 stereo mic gave a good on the separation as noticed during informal listening tests. The
quality source separation due to a good difference between the cafe-speech (C-S) audio files captured Z3 in a horizontal
channels as shown in Figure 7b. orientation in HLEN gave a mean channel difference close to 0.01
which is sufficient to achieve a good quality source separation.
Channel differences for N5 give comparatively lower mean as
shown in Figure 9a but enabling source separation with higher
cross talk compared to Z3 in horizontal orientation. However, the
vertical orientation of Z3 gave lower quality source separation
compared to its horizontal orientation and N5 gave no separation
at all due to a very low mean channel difference as shown in
Figure 9b.

Figure 7: a. Difference between the channels captured using 2

Nexus phones as an array, b. Difference between the channels
captured using STM10 with Z3 and Z2.
This mic is compatible only with the Z2 and Z3 and hence the
experiment is performed using these two phones only. The mean
channel difference for audios captured using Z2 and Z3 with the
STM10 mic was similar to the mean channel difference for the
audio captured using two N5 internal mics as an array both falling

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 221 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
audio-captures for Z3 and Z2 required a stereo mic to achieve a
good source separation compared to their internal mics. The other
smartphones tested have hardware specifications restricted for
stereo audio capturing that is a prime requisite for BSS using
PARAFAC to have a minimum of two channels as an input. Also
the orientation of the smartphone plays a major role. The
horizontal orientation of the smartphone relative to the sources
resembles the human ear positioning thus making it easier for the
source separation algorithm to separate the sources due to the
directional behavior of the smartphone mics. External omni-
directional mics are usable in multiple numbers (for example: as
an array) to achieve multiple channels, kept at sufficient distance
from each other to get a good source separation. It is necessary to
have mics placed at a distance high enough to achieve a difference
in the channel matrices that is a prerequisite for BSS. If the
internal mics of the smartphones are used, then it is necessary to
access both the mics of the device for audio capturing without any
pre-processing.
Figure 9: a. Difference between the channels captured using
horizontal orientation of Z3 and N5 for Cafeteria noise (C) Our main goal is to develop a smartphone app for BSS to enable
and speech (S), b. Difference between the channels captured the people with hearing disabilities to choose the sources of
using vertical orientation of Z3 and N5 for Cafeteria noise (C) interest for hearing. The mics to be used for our purpose as
and speech (S). mentioned previously are the smartphones internal and external
mics or a combination. We intend to use these mics in a manner
Further, an orientation experiment using male and female (M-F) that they resemble the setup of the human ear to achieve better
speech audio files gave similar results. The mean channel separation results. The audio capturing system must be such that it
difference for audio files captured using Z3 was much higher is able to access multiple inbuilt mics of the phone to estimate
(0.01) compared to its vertical orientation (6x10-6) as shown in more sources in the audio mixture.
Figure 10, part a and b. Further, the mean channel difference for
N5 in horizontal orientation was lower compared to Z3 but higher 7. ACKNOWLEDGMENTS
than its vertical orientation as shown in Figure 10, part a and b. This work is a part of the SMARTNAVI-project and funded by
Thus, the audio files captured in the horizontal orientation achieve the German Federal Ministry of Education and Research (BMBF).
a good quality source separation compared to the ones captured in The financial project organization is directed by the VDI/VDE
vertical orientation. Berlin.

8. REFERENCES
[1] Golumbic, I.M.Z., Ding, N., Bickel, S. Peter Lakatos et al.
2013. Mechanisms Underlying Selective Neuronal Tracking
of Attended Speech at a ’Cocktail Party’. Neuron, vol. 77(5).
[2] Alten, S. 2013. Audio in Media Cengage Learning. Boston.
[3] Anemueller, J., Kollmeier, B. 2003. Speech communication.
http://medi.uni- oldenburg.de/demo/demo_separation.html,
[Accessed: 2015-03-01].
[4] Asakawa, H. 2012. Sensing sound: The new sound signal.
Multimedia Sensing, NII, vol. 1, 4-5.
[5] Beck, D., Schum. D.J.. 2005. Directional hearing aids:
Concepts and overview. AudiologyOnline,
www.audiologyonline.com/articles/directional-hearing-aids-
concepts-and-1012 [Accessed: 2015-04-22].
Figure 10: a. Difference between the channels captured using
horizontal orientation of Z3 and N5 for male (M) and female [6] Bell, A., Sejnowski, T. 1995. An information maximization
(F) speech, b. Difference between the channels captured using approach to blind separation and blind de-convolution.
vertical orientation of Z3 and N5 for male (M) and female (F) Neural Computation, 7(6), 1129–1159.
speech [7] Cherry, E.C. 1953. Some experiments on the recognition of
speech, with one and with two ears. The Journal of the
6. CONCLUSION AND FUTURE WORK Acoustical Society of America, vol. 25(5), 975-979.
The outcome of the experiments shows that the mics of different
smartphones have different hardware specifications based on their [8] Clark, N.: BioAid. 2012.
usage and applicability. The applications common across similar https://itunes.apple.com/gb/app/bioaid/id577764716
platforms also perform differently even though having the same [Accessed: 2015-03-01].
operating system. Thus the application of audio processing [9] Clark, N.: Aud-1. 2013. http://www.aud1.com/ [Accessed:
algorithms must be independent of these settings. In our case, 2015-03-01].

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 222 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
[10] Corbett, I. 2014. Mic It!: Microphones, Microphone [27] Pertila, P., Hamalainen, M.S. and Mieskolainen, M. 2013.
Techniques, and Their Impact on the Final Mix. CRC Press, Passive temporal offset estimation of multichannel
Boca Raton. recordings of an ad-hoc microphone array. In IEEE
[11] Craddock, G.M. 2003. Assistive technology: Shaping the Transactions on Audio, Speech, and Language Processing,
future. Assistive technology research series (AAATE’03), vol. vol. 21, 2393-2402.
11, 9-13. [28] phoneArena.com. In stereo: phones with the best audio
[12] Dillon, H. 2001. Hearing aids. Boomerang Press. New recording. http://www.phonearena.com/news/In-stereo-
York. phones-with-the-best-audio-recording_id61698
[Accessed:2015-03-01].
[13] Edyburn, D. 2004. Rethinking assistive technology. Special
Education Technology Practice, vol. 5(4), 16-23. [29] A. S. R. 2013. Production For Mobile Media. Cengage
Learning.
[14] Edyburn, D.L.. 2000. Assistive technology and mild
disabilities. Focus on Exceptional Children, 124. [30] [30] Ramiro,J., Hamied, K. 2011. Self-Organizing Networks
(SON): Self-Planning, Self-Optimization and Self-Healing
[15] eMarketer Inc. 2014. Smartphone users worldwide will total for GSM, UMTS and LTE ITPro collection. John Wiley and
1.75 billion in 2014, Sons, New York.
http://www.emarketer.com/Article/Smartphone-Users-
Worldwide-Will-Total-175-Bil lion-2014/1010536 [31] GNU ReSound, Resound control.
[Accessed: 2015-03-01]. http://www.gnresound.de/Services/ReSound

[16] Hoergeraete, Kindilink. 2014. [32] ControlApp, 2015, Accessed: 2015-03-01. [32] GNU
https://itunes.apple.com/de/app/kindilink/id849118289?mt= ReSound, Resound smart-app.
8, [Accessed: 2015-03-01]. http://www.gnresound.de/Services/Smartapp
[Accessed:2015-03-01].
[17] G. Inc. 2014. Gartner says annual smartphone sales
surpassed sales of feature phones for the first time in 2013, [33] Ricketts, T.A. 2001. Trends amplification. Directional
http://www.gartner.com/newsroom/id/2665715 [Accessed: Hearing Aids, vol. 5(4), 139-176.
2015-03-01]. [34] Roebuck, K. 2012. HD Voice: High-impact Strategies - What
[18] G. Inc. 2014. Gartner says smartphone sales surpassed one You Need to Know: Definitions, Adoptions, Impact,
billion units in 2014, Benefits, Maturity, Vendors, Emereo Publishing, Aspley.
http://www.gartner.com/newsroom/id/2996817 [Accessed: [35] Rosenthal, S., Amin, N, Gross, T., Offiah, M.C. and M.
2015-03-01]. Borschbach. 2014. NUI for an artificial simulation of an
[19] Killion, M.C. 2004. Myths about hearing in noise and interactive sound source enhancement to restore spatial
directional microphones. The Hearing Review. listening experience.In 6th International conference on
Intelligent Human Computer Interaction (IHCI), vol. 39,
[20] Kuk, F., Keenan, D., Lau, C.C. et al. 2005. Performance of a 123–130.
fully adaptive directional microphone to signals presented
from various azimuths. Journal of the American Academy of [36] Salvendy, G. 2012. Handbook of Human Factors and
Audiology, vol. 16(6), 333-347. Ergonomics. John Wiley and Sons, New York.

[21] Lewis, J. 2012. Understanding microphone sensitivity. [37] Sur, S., Wei, T. and X. Zhang. 2014. Autodirective audio
Analog Dialogue 46-05 Back Burner, vol. 46, 1-3. capturing through a synchronized smartphone array. In 12th
annual international conference on Mobile systems,
[22] Li, W., Takeda, K. and Itakura, F. 2007- Robust in-car applications, and services. ACM, 28-4.
speech recognition based on nonlinear multiple regressions.
EURASIP Journal on Applied Signal Processing, 5. [38] Siemens, minitek app. 2015.

[23] Offiah, M.C., Rosenthal, S. and Borschbach, M. 2014. [39] https://de.hearing.siemens.com/products/wireless/minite

Assessing the utility of mobile applications with support for k-app/ [Accessed:2015-03-01].
or as replacement of hearing aids. In 9th International [40] [39] Vision, Peek vision app. 2015.
Conference on Future Networks and Communications http://www.peekvision.org/what-it-does [Accessed:2015-03-
(FNC’14)/The 11th International Conference on Mobile 01].
Systems and Pervasive Computing (MobiSPC’14)/Affiliated [41] [40] Sony Mobile Communications Inc.,
Workshops, 593–600. http://www.sonymobile.com/globalen/products/accessories/st
[24] Nion, D., Mokios, K., Sidiropoulos, D. et al. 2010. Batch and ereo-microphone-stm10/ [Accessed:2015-06-09].
adaptive parafac-based blind separation of convolutive [42] [41] DHARANISYNC OHG,
speech mixtures. In IEEE Transactions on Audio, Speech,
and Language Processing, vol. 18, 1193-1207. [43] http://www.edutige.de/microphones/eim-serie-
smartphone-tablet/eim-001-i-microphone-voice/
[25] Onu, N. 2013. Blind source separation on iphone in real [Accessed:2015-06-09].
environment. In 21st European Signal Processing
Conference (EUSIPCO), 1-5. [44] [42] Blauert, J. 1983. Spatial hearing: the psychophysics of
human sound localization, MIT Press, Cambridge,
[26] OtoSense. 2014. Otosense app. Massachusetts.
http://www.otosense.com/#!about-us/coqt, 2014,5 Accessed:
2015-03-01]. [45] [43] RecForge Lite - Audio Recorder,

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 223 |
 Blind Source Separation Performance based on Microphone Sensitivity and Orientation within Interaction Devices
N. Amin | T. Gross | S. Rosenthal | M. Borschbach
[46] https://play.google.com/store/apps/details?id=dje073.an [51] [46] EETimes: MEMS Mics Taking Over. 2014.
droid.audiorecorderlite&hl=en” [Accessed:2015-06-16]. www.eetimes.com/document.asp?doc_id=1324827,
[47] [44] Awesome Voice Recorder, Newkline Co., Ltd. [Accessed: 2015-03-01].

[48] https://itunes.apple.com/us/app/awesome-voice- [52] [47] Self, D., Duncan, B., Sinclair, I. 2009. Audio
recorder-voice/id892208399?mt=8 [Accessed:2015-06-16]. Engineering: Know It All, Newnes.

[49] [45] The Sound Recorder, [53] [48] Nihtianov, S., Luque, A. 2014. Smart Sensors and
MEMS: Intelligent Devices and Microsystems for Industrial
[50] http://www.windowsphone.com/en-us/store/app/the- Applications, Woodhead Publishing, 494 ff.
sound-recorder/819adab9-44dc- 4fd6-a8a6-4522d0c7e9e3
[Accessed:2015-06-16].

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 224 |
 DIY computer mouse for special needs people
L. Ribas-Xirgo | F. López-Varquiel

DIY computer mouse for special needs people

Lluís Ribas-Xirgo Francisco López-Varquiel
Microelectronics and Electronic Systems Department
Universitat Autònoma de Barcelona
(+34)-935811078
[email protected], [email protected]

ABSTRACT Most of the already existent mouse-replacing devices sense head

People unable to use hands to interact with computers do require movements instead of hand and finger ones to determine pointer
special devices to gain access to information technology services. displacements and mouse-actions. To do so, an accelerometer can
The rise of do-it-yourself (DIY) electronics makes it possible to be placed onto an earphone and data from it read and processed on
release construction kits of a wide range of gadgets, including an electronics board such as Arduino, BeagleBone or Raspberry
hands-free computer mice. In this paper we present one such Pi, to mention a few. At its turn, the electronics board emulates a
building kit for a full-featured computer mouse that is controlled conventional USB computer mouse so that no special drivers or
by head movements. The result kit includes an easy-to-find bill of software has to be installed (see Fig. 1.)
materials, instructions to build the device and to use it. The first The result earphone-mounted computer mouse can significantly
experiments showed that the mouse was working fine with people improve the life quality of those who cannot use their arms and
that have good control on head movements but that its embedded hands, temporarily or permanently, providing an alternative to
program must be customized for other individuals. accessibility to computers and similar devices.
Categories and Subject Descriptors Therefore, the main goal of this work is to make available all the
H.5.2 [User interfaces]: Input devices and strategies (e.g., necessary elements to build a “computer mouse earphone” to
mouse); evaluation/methodology; prototyping, and user-centered anyone, and so, make it available to any other person in need of it,
design. H.1.2 [User/Machine Systems]: Human information without any other requirements.
processing.
The proposed device (section 4) must be cheap, with components
General Terms commercially available and easy to buy. Its assembly has to be
Algorithms, Measurement, Documentation, Experimentation. easy, with no need to have any technical knowledge or specific
tools, and the software must be downloaded for free on the
Keywords Internet. Once built, the device will not need any software
Assistive technologies, accelerometer-based computer mouse, installation in order to use it in any computer or portable device.
DIY mouse, special computer mice. The result device should be easy to use, so that the user will not
need any special training (subsection 4.3).
1. INTRODUCTION
Computer mice are vanishing from portable devices as their
functionality is covered by a myriad of tactile sensors and
accelerometers. While these new interfaces improve human-
machine interaction, they rely mostly on hand movements thus
becoming useless to those with impaired hands (section 2).
There are several interface gadgets that partially or totally cover
the functionality of common computer mice through the use of
elements that can be operated by body parts other than hands.
Unfortunately, they are expensive, require software installations Earphone
on the devices (computers of all kinds) with which they are to be with accelerometer
used, and, besides, they do not offer all the functions of a regular
computer mouse (section 3). DIY module
to read sensor data
On one hand, these mouse-replacing devices are rare and and
expensive because of the inexistence of a big market able to emulate USB mouse
generate enough return. On the other hand, thanks to the large-
scale market of everything related to the information and
communication technologies and, specifically, of DIY electronic
components, we all have the possibility of building devices
adapted to our needs.
In this work, we have taken advantage of the availability of DIY
components to build a hands-free computer mouse at an
affordable prize, as its development has been funded by a prize Figure 1. Earphone-mounted mouse.
awarded to the authors and no economic profits are expected. (Source of model photography:
http://www.coroflot.com/yoonsangkim/portfolio1)

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 225 |
 DIY computer mouse for special needs people
L. Ribas-Xirgo | F. López-Varquiel
Some devices were built to be able to test them with real users to In fact, touchpads and touchscreens have made mice obsolete to
determine its user-friendliness and to be able to adjust both the most people. However, these “mouse substitutes” still rely on
head movement tracking and the identification of gestures of the hands and fingers.
users to their intentions (concluding section 5.)
Fortunately, there are more and more machines that have frontal
2. TARGET USERS cameras, and they can execute programs that transform users’
Anyone that may take advantage of using a head-tracking mouse gestures in front of the camera into mouse movements and actions
instead of a conventional one can use it. However, the focus of [5, 14, 16]. Free cam-based virtual mice include EyeMouse [13],
such kind of devices is on people with special needs willing to HeadMouse [21] and Enable Viacam [12]. However, they do not
access to computing devices and applications like non disabled implement all the features of conventional computer mice.
people. Unfortunately, diversity of impairments makes it difficult EyeCan [10] also uses computer cameras, but requires extra
to know the amount of potential users. components.

According to the Instituto Nacional de Estadística (Spanish With the success of the video game consoles like the Wii of
National Statistics Institute), almost four million people (out of 47 Nintendo and with the commoditization of accelerometers, there
million Spaniards) have some type of disability and almost a 40% are also solutions that incorporate them to build computer mice or
of these have problems with bones and articulations [1]. Many of devices alike. Many of them are commercial and protected by
these people are potential users of the proposed device. patents [22, 3, 7, 20, 18]. However, commercial or not, these
solutions require user adaptation and, besides, have accuracy [11]
A more specific segment of population that might be candidate and fatigue [4] problems.
users consists of those people having paraplegias, which account
for fifty thousand people in Spain [8]. By extrapolating Mexican Anyway, special needs people do require hands-free mice and
statistics from [2], approximately half of them have tetraplegias there are commercial products that offer a similar functionality
and are possible users of this kind of devices. (see table 1 for a short list of them.) Unfortunately, they are pricey
and limited to some platforms.
However, it is worth noting again that the diversity of people’s
conditions makes it difficult for a single product to suit all specific Table 1. Pointer devices
needs and only a fraction of them might be really taking profit of Device Body part Price
the proposed device.
Computer mouse Hand <100€
Particularly those with good control on head movements willing
to use common software on any device may find this gadget NoHands Mouse (Hunter Digital) Feet 250€
useful. Take into account that, apart from computers, mice can be
SmartNav (Natural Point) Head 350€
connected to mobile devices with Android [6], iOS [9] and, even,
Microsoft Windows Phone [19]. Lips, tongue,
QuadMouse (Quadadapt) 550€
chin
Furthermore, as people is getting used to interfaces other than
mice, keyboards and touchscreens, a head movement tracking IntegraMouse (Adapt-it) Mouth 2150€
mouse can be used by any person that might find it interesting to.
Another important drawback of previous mouse replacement
In fact, the use of, for example, Wiis from Nintendo or Kinects options is that they need installing software, which limits the
from Microsoft, have familiarized people with new ways of number and type of computers and other devices they can be used
interaction with computers. And this contributes to see hands-free with.
mice as a complementary form of man-machine interaction.
The proposed device emulates an USB mouse so it can be used
To sum it all up, the target users of the proposed device are some with any computer, tablet or, even, smartphone. Moreover, there’s
of those people that cannot access computer applications because no particular software installation, and, to make it cheap, i.e.,
of a temporal or permanent disability to use hands and fingers to under 100€, it will be released as a DIY project open to even
manipulate a pointing and selection device such as mice or electronics or computer unskilled people.
touchscreens, and some of those that might look for a different
user experience with computers. 4. THE DIY COMPUTER MOUSE
It is also important to highlight that, by using the proposed full- The idea of the work is to provide the information for anyone
featured mouse and virtual keyboards, any user can have access to motivated to build a hands-free mouse for someone else needing
regular computer applications without further ado. Therefore, it. Such information includes the bill of materials for the proposed
users are able to establish and maintain social relations through device, instructions to build it, including how to embed the control
Facebook, Twitter, or any other social network system, can play program, and the user’s guide.
games and work with computer applications, regardless of their
possible disabilities. 4.1 Bill of materials
In order to build the device you need to buy the material, which
3. MICE AND MOUSE-LIKE DEVICES includes a pair of earphones, an accelerometer and an Arduino
The idea of substituting mice by more “natural” alternatives (or, board (Fig. 2). Typically, all materials cost less than 80€.
for disabled people, “possible” alternatives) is not new. For The Arduino Leonardo board processes the data from the
example, back in 2006, Microsoft had foreseen a special glove accelerometer about the head movement and transforms them into
[17] to replace mice and, more recently, Leap Motion offers a mouse movements and actions. A buzzer is required to emit
device to be used like a touchpad [15]. sounds to help the user during initial calibration of the device and

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 226 |
 DIY computer mouse for special needs people
L. Ribas-Xirgo | F. López-Varquiel
to indicate when gesture recognition is on. The board connects to The procedure is quite simple: the user has to keep his/her head in
computers through a standard USB to micro-USB cable. a rest position for 5 seconds, then lean it to the left for 5 seconds
more and, finally, lean it to the right for other 5 seconds. Every
The accelerometer ADXL335 (board GY-61) is used to detect
step ending is signaled by a sound from the buzzer and by a
head movements. This very board can be directly powered from blinking LED on the board.
the Arduino board because it includes a voltage regulator on its
own. Therefore, jumper cables link it with the Arduino board for Note that the rest position is the one in which the user feels
power supply and data transmission. comfortable looking at the screen. The movements and actions of
the mouse will be achieved through the change of this position.
The earphones are used as a support for the accelerometer, and
can be used normally, too. 4.3.2 Operation
Apart from the previous component set, scissors and insulating It is convenient to consider that this device has two operation
tape are the only extra elements to build a mouse. modes: as a pointer and as a gesture interpreter. In the pointer
mode, it transfers the head movement to the pointer movement on
4.2 Assembly guide and software installation the screen. In the gesture interpreter mode, it transforms given
The first step is to attach the accelerometer board to the left head movements into mouse actions.
earphone with the tape and the last one consists of programming To switch from pointer mode to gesture interpreter mode you need
the Arduino board. (The accelerometer could be placed at the to stop the pointer in the desired position. Each time the pointer
right earphone, but device calibration is more accurate if placed at stops on the screen, the buzzer module will emit a sound to show
the left side of the head.) In between, jumper cables have to be it. From this moment on, the user will have 2 seconds to make a
connected among them to form longer cables that connect the movement that matches a mouse action. It is important that the
Arduino board with the accelerometer, and the buzzer must be head movement to perform the gesture starts when the sound is
mounted. over.
To program the Arduino board, people must go to the Arduino's If no motion is detected, the mouse will return to the pointer mode
official web page to download the appropriate software, and to and the user will be able to move the pointer through the screen
our earphone mouse web page to download the program to be following the head movements.
embedded into the Arduino board.
All gestures involve moving the head from the resting position
The device shall be ready to use when a message saying ‘charge and going back to this position once finished.
complete’ appears on the Arduino programming application.
The list of actions and corresponding gestures is shown below:
4.3 User’s guide
Upon connecting the earphone-mounted mouse to a PC it starts a • Left click: tilt head to the left and return to the rest
calibration procedure. After that, the device is ready to emulate a position just afterwards.
mouse controlled by the head movements. • Double left click: tilt head to the left and keep it this
way until the action is performed (or as long as the
4.3.1 Calibration pointer is still.)
The calibration procedure determines head movement range and is
required to correctly interpret the gestures that will become mouse • Right click: tilt head to the right and return to the rest
actions. It is automatically done every time the device is plugged. position.
An accompanying person, possibly the same that has put the • Dragging: lean head to the right and keep it this way as
device on the user, must wait until the calibration process is long as the pointer is still.
completed. Take into account that in case of an incidence, the
device must be reset. Once the calibration is correct, the user can • Scroll up: move the head upwards, as if to look at the
continue to use it in a fully autonomous way. ceiling.
• Scroll down: move head downwards, as if to look at the
floor.
To exit the up and down scroll movements, leave your head in the
rest position for 2 seconds until the device switches to pointer
mode and the pointer starts moving again on the screen.

5. RESULTS AND CONCLUSION

Computer mice emulators that do not require hands are suitable
for special needs people so that they could have access to
information technologies just as average people.
We have proposed one such type of devices that fully emulates a
conventional computer mouse, thus being the first of its class that
does not require any specific driver or software installation.

Figure 2. Earphone mouse.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 227 |
 DIY computer mouse for special needs people
L. Ribas-Xirgo | F. López-Varquiel
We have created a web page (www.earmouz.org) to provide [8] Jara, M. 2011. Afirman que los nervios dañados de la médula
potential users with all the necessary information to build and use espinal ¡pueden recuperarse!. In Discovery DSalud, No. 139.
our earphone-mounted, accelerometer-based, USB mouse. [www.dsalud.com/index.php?pagina=articulo&c=1616]
We have built several mice so they could be used by several [9] Karns, J. 2011. How to Connect a Bluetooth Mouse and
testers. Non disabled people had little problem in using it after a Keyboard to your iPad (or iPhone)! In Tablets, Wonder How
few minutes, but it took them a while to learn the more complex To. [tablets.wonderhowto.com/how-to/connect-bluetooth-
gestures for double click and drag-and-drop. mouse-and-keyboard-your-ipad-or-iphone-0118475/]

Thus far, there have been few candidates with special needs to test [10] Kee, E. 2012. Samsung eyeCAN mouse for disabled folk. In
the mouse because of several reasons: some of them were already Übergizmo, Feb. [www.eyecanproject.org/p/english.html]
using other pointing devices, some others had little control on [11] LoPresti, E.F. 2001. Effect of Neck Range of Motion
their head movements, and others, simply, had other priorities. Limitations on the Use of Head Controls. In CHI'01
However, we had taken their opinions into consideration to make Extended abstracts on Human factors in computing systems.
further adjustments to the device, which include being able to be ACM, New York, NY, USA, 75–76.
operated more slowly and with higher tolerance to “movement [12] Mauri Loba, C. 2008. Enable Viacam. [http://viacam.org]
noise.” With this upgrade, we expect that the proposed mouse can
be helpful to, at least, a handful of people. [13] Mónaco, N.A., Ponieman N.B. 2008. Emulador de Mouse –
EyeMouse. Degree thesis. ORT Argentina. Belgrano.
In the near future, we plan to make calibration as seamless as [ticbelgrano.blogspot.com/2008/12/eye-mouse-6-ao.html]
possible and extend the tests to more persons so to broaden the set
of potential users. [14] Palleja, T., Rubion, E., Teixidó, M., Tresánchez, M.,
Fernández del Viso, A., Rebate, C., and Palacín, J. 2008.
6. ACKNOWLEDGMENTS Using the Optical Flow to Implement a Relative Virtual
This work has been partially funded by the 2013 Fundació Mouse controlled by Head Movements. In Journal of
Agrupació Award to the project Earphone-mounted computer Universal Computer Science, 14, 3127–3141.
mouse for special needs people. [15] Panneta, K. 2013. Why the computer mouse will soon be
obsolete. In ECN Mag., Aug. [www.ecnmag.com/articles/
7. REFERENCES 2013/01/why-computer-mouse-will-soon-be-obsolete]
[1] —. 2008. Encuesta de Discapacidad, Autonomía personal y
[16] Pereira, C.A.M., Neto, R.B., Reynaldo, A.C., Luzo, M.C.M.,
situaciones de Dependencia (EDAD), Instituto Nacional de
Estadística. Nov., 2008. [www.ine.es/prensa/np524.pdf] and Oliveira, R.P. 2009. Development and evaluation of a
head-controlled human-computer interface with mouse-like
[2] Bahena-Salgado, Y., and Bernal-Márquez, J.N. 2007. functions for physically disabled users. In Clinics. 64(10).
Calidad de vida de los pacientes con paraplejía secundaria a
lesión vertebral traumática. In Acta Ortopédica Mexicana; [17] Ranta, C.S., Bathiche, S.N. and Chatterjee, M. Wearable
21(1). Gen.-Feb. 2007. computer pointing device, Microsoft Corp. (US Pat. App.
20080084385. October 6, 2006).
[3] Breen, R. and Gerasimov, V. 2008. Inertial Sensor Input
Device, Pat. App. 20080211768. 09/04/2008. [18] Rodgers, A.G., Higgins, R.F., Gagnon, S.T. and Farr, J.C.
2010. Head-mounted pointing and control device. Alken,
[4] Bureau, M., Azkoitia, J.M., Ezmendi, G., Manterola, I., Inc. Patent No. US 7,710,395 B2. May 4, 2010.
Zabaleta, H., Pérez, M. and Medina, J. 2007. Non-Invasive,
Wireless and Universal Interface for the Control of [19] Schenck, S. 2014. Windows Phone 8.1 could support USB
Peripheral Devices by Means of Head Movements.” In IEEE HID class; get your accessories ready. In PocketNow, 2014.
[http://pocketnow.com/2014/02/13/wp8-1-usb-hid-support]
10th Int’l. Conf. on Rehabilitation Robotics, ICORR, 124–
131. 13-15 June 2007. [20] Schmid, O., Baettig, A., and Schmid, C. Dispositif a fonction
de souris d’ordinateur. Pat.WO2002EP12091, Pri.
[5] Graveleau, V., Mirenkov, N. and Nikishkov, G. 2005. A
CH20010002024, May 15, 2003.
Head-Controlled User Interface. In 8th Int’l. Conf. on
Humans and Computers, 306–311. Aizu. Wakamatsu. [21] Teixidó, M., Palleja, T., Tresánchez, M., Font, D. Moreno, J.,
Fernández del Viso, A., Rebate, C., and Palacín, J. 2013.
[6] Hoffman, C. 2013. How to Connect Mice, Keyboards, and
Optimization of the virtual mouse HeadMouse to foster its
Gamepads to an Android Phone or Tablet. In How-To Geek.
classroom use by children with physical disabilities. In
[http://www.howtogeek.com/164783/]
Advances in Distributed Computing and Artificial
[7] Ishimatsu, T., Irie, N., and Takami, O. 1997. Computer Intelligence Journal, 7, 1-7. [http://robotica.udl.cat/]
Interface Device for Handicapped People Using Head
[22] Tong, K. 2000. Head Operated Computer Pointer. Pat. app.
Movement. In IEEE Pacific Rim Conf. on Communications,
WO0017849 (A1). Mar. 30, 2000.
Computers and Signal Processing, 273 – 276, Aug 1997.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 228 |
 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
E. de la Rubia | A. Diaz-Estrella

Explorando la Viabilidad de un Sistema de Tracking

Inercial para la Mano Basado en un solo Sensor
Ernesto de la Rubia Antonio Diaz-Estrella
Dep. de Tecnología Electrónica Dep. de Tecnología Electrónica
Universidad de Málaga Universidad de Málaga
Málaga, España Málaga, España
34-952 13 71 80 34-952 13 27 31
[email protected] [email protected]

RESUMEN De acuerdo a la tendencia actual, el coste de los sensores

Este artículo presenta un proyecto en progreso en el que se inerciales es cada vez menor mientras que aumenta la frecuencia
exploran las posibilidades de utilizar exclusivamente un sensor de muestreo y se reducen los niveles de ruido.
inercial para estimar la posición y orientación de la mano. El En relación al modo en que se emplean, los sensores inerciales se
sistema sufre errores de deriva que afectan a la estimación de la utilizan principalmente como sensores de orientación. También se
posición y al ángulo yaw asociado al eje vertical. Se emplea incluyen estos dispositivos en aplicaciones para detectar choques,
filtrado de Kalman para reducir el error de las estimaciones caídas y vibraciones. Otro uso especialmente adecuado en el
durante los periodos de tiempo en los que la mano se detiene. ámbito de la IPO, es la implementación de trackers de posición.
También se explora la posibilidad de aplicar la técnica de reseteo
selectivo de la matriz de error de covarianza (CESR) del filtrado Los sensores inerciales se componen habitualmente de tres
de Kalman que ha sido empleada con éxito en el tracking de pies. acelerómetros y tres giroscopios que se distribuyen en ejes
Finalmente se presentan los resultados preliminares de las perpendiculares. Registran la fuerza de la gravedad para
primeras pruebas realizadas y se sugieren diferentes estrategias determinar la inclinación del sensor y proporcionan estimaciones
que pueden contribuir a mejorar el comportamiento del sistema de de orientación que solo sufren errores de deriva al calcular el
tracking. ángulo respecto al eje vertical. A causa de este error de deriva, los
sensores inerciales incluyen con frecuencia tres magnetómetros
Categorías y Temas de Interés colocados sobre ejes perpendiculares entre sí que miden el campo
H.5.2 [Interfaces de información y presentación]: Interfaces de magnético de la tierra actuando como brújulas para determinar la
usuario – dispositivos de entrada y estrategias. orientación del sensor respecto al eje vertical sin errores de deriva.
Sin embargo, el campo magnético de la tierra es muy débil y sufre
Términos Generales una fuerte distorsión en presencia de materiales ferromagnéticos
Algoritmos, medidas, diseño, ergonomía. que se encuentran habitualmente en la estructura de los edificios.
Por ese motivo, se desaconseja el uso de magnetómetros en
Palabras Clave interiores. Este hecho tiene implicaciones en cuanto a la exactitud
Sensores inerciales, procesado de señal, trackers, realidad virtual, de las estimaciones de orientación que proporcionan los sensores
interacción natural, inmersión, presencia. inerciales en interiores.

1. INTRODUCCIÓN Por otra parte, cuando se utilizan los sensores inerciales como
El número de aplicaciones de los sensores inerciales ha trakers de posición, se estima la orientación y con ella se anula la
aumentado sustancialmente durante los últimos años. Se componente de la aceleración asociada a la gravedad. De este
implementan con frecuencia utilizando sistemas electromecánicos modo, se obtiene la aceleración debida exclusivamente al
cuyo diminuto tamaño se mide en micras. movimiento. Integrando dos veces con respecto al tiempo este
vector de aceleración, se obtiene una estimación del incremento
Habitualmente, los sensores inerciales son pequeños y ligeros. Su de posición que ha sufrido el sensor durante el intervalo temporal
coste es reducido así como su consumo. Además, existen modelos aplicado en el cálculo de la doble integral. Las estimaciones de
inalámbricos. Estas características hacen que los sensores posición que se obtienen aplicando este planteamiento se basan en
inerciales resulten especialmente adecuados en el campo de la las estimaciones previas (dead reckoning). De este modo, los
interacción persona ordenador (IPO). errores cometidos en las estimaciones se acumulan dando lugar a
Un hecho relevante relacionado con el desarrollo de los sensores un error de deriva que crece con el tiempo. Este error es el mayor
inerciales, es su presencia en dispositivos portátiles como reto al que se enfrentan los sensores inerciales al ser usados como
smartphones y tablets. La combinación de elementos como trackers de posición.
conectividad inalámbrica, pantalla táctil, GPS, telefonía, conexión Una amplia mayoría de los sistemas que estiman la posición y
a internet, etc., en un dispositivo portable que puede ser orientación de las manos, se basa en sistemas ópticos que emplean
programado fácilmente, da lugar a un planteamiento procesado de imagen y cámaras de color o de profundidad. Su
extremadamente versátil que permite desarrollar una gran mayor inconveniente es que requieren visión directa con la
cantidad de aplicaciones de diversa naturaleza. Esta situación cámara. Este hecho implica importantes limitaciones en cuanto a
favorece la evolución y mejora de los sensores inerciales. la movilidad del usuario.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 229 |
 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
E. de la Rubia | A. Diaz-Estrella
Para resolver este problema, otras estrategias recurren a un error de covarianza. En consecuencia, un reseteo selectivo de
conjunto de sensores inerciales colocados en distintas partes del estos elementos mejorará las estimaciones en el próximo paso
cuerpo para estimar la posición y orientación de los mismos. Se (covariance error selective reset, CESR).
trata de los trajes inerciales de captura de movimientos.
Esta estrategia da lugar a excelentes resultados al estimar la
Eliminar o reducir el error de deriva en la posición que sufren los posición y la orientación de los pies. En este trabajo se estudia la
sensores inerciales, haría posible utilizar un solo sensor como posibilidad de aplicar la misma técnica al tracking de manos.
tracker de posición que ofrecería importantes ventajas frente a los
sistemas ópticos y a los trajes inerciales de captura de 3. IMPLEMENTACIÓN DE TRACKING
movimiento. INERCIAL DE MANOS
El resto del contenido de este trabajo se organiza de acuerdo a la En el presente estudio se parte del algoritmo propuesto por [6]
siguiente distribución. En la segunda sección se presenta el trabajo para tracking de pies y se adapta para abordar el tracking de
relacionado con el tracking inercial de manos. La implementación manos. Se utilizan sensores InertiaCube3 inalámbricos de
del sistema objeto de estudio se describe en la sección tres. En la Intersense y cintas elásticas con velcro en los extremos que
cuarta sección se estudia en qué medida es aplicable al tracking de permiten fijar el sensor y la batería a la mano. Los sensores se
manos una técnica que mejora el tracking inercial de pies. La configuran para funcionar a una frecuencia de muestreo de 120 Hz
quinta sección sugiere diferentes estrategias y técnicas similar a la empleada por [6].
complementarias al planteamiento base que también contribuyen a Existen tres factores fundamentales que dificultan el tracking de
mejorar el tracking de manos. Finalmente se presentan las manos con respecto al de pies. La diversidad de movimientos que
conclusiones y líneas futuras de este trabajo de investigación. se realizan con las manos es mucho mayor. La aceleración en los
pies es en general mucho más intensa que en las manos y por
2. TRABAJO RELACIONADO tanto las señales que se obtienen se ven menos afectadas por el
El número de estudios que abordan el tracking inercial de manos ruido de los sensores. La tercera dificultad está relacionada con la
es muy reducido debido a la existencia de otras tecnologías que detección del estado de reposo que es mucho más compleja en el
resultan mucho más adecuadas como los sistemas de tracking tracking de manos. Esto se debe a que las manos no llegan a
óptico o magnético. Sin embargo, los sensores inerciales aportan apoyarse como lo hacen los pies y adicionalmente, los pies
la ventaja de no necesitar una infraestructura de soporte que cambian de orientación durante su movimiento. Esto no puede
proporcione referencias. Además son los más adecuados para asumirse en el tracking de manos.
realizar ciertas tareas como el tracking de pies y el coste de los
mismos es menor habitualmente. Estas dificultades tienen como consecuencia un error mayor de
deriva en la estimación de la posición. Para controlar este error, es
La mayor parte de los estudios realizados con sensores inerciales posible diseñar un paradigma de interacción que utilice el sistema
abordan el problema de reconocimiento de gestos. Para ello se de tracking propuesto en este trabajo y asuma que existe una
utilizan con frecuencia modelos ocultos de Markov [10], redes posición inicial conocida desde la que parte la mano antes de
neuronales [13, 14] y algoritmos de alineamiento temporal realizar cualquier acción.
dinámico (dynamic time warping, DTW) que determinan el grado
de similitud de dos señales con independencia del tiempo y de su Para detectar la fase de reposo, el algoritmo propuesto por [6]
velocidad de cambio [3, 8]. [9] realiza un estudio en el que compara el módulo de la velocidad angular con un valor umbral.
compara estos tres enfoques. Este criterio no detectaría movimientos en los que no cambie la
orientación del sensor y por tanto no es válido para realizar
Un planteamiento más próximo al tracking de manos es el tracking de manos. Por ello, se utiliza en su lugar el detector
reconocimiento de caracteres, en el que se determina y analiza la óptimo de hipótesis de reposo [12]. Para aplicarlo se utiliza la
trayectoria de la mano [1, 2]. siguiente expresión:
En el tracking de pies sí se utilizan con éxito los sensores 𝟏𝟏 𝟏𝟏
𝒂𝒂
𝒚𝒚𝒏𝒏
𝟐𝟐
𝟏𝟏
inerciales. La clave de este éxito es un hecho que pone de 𝑻𝑻(𝒛𝒛𝒂𝒂𝒏𝒏 , 𝒛𝒛𝝎𝝎
𝒏𝒏 ) =
𝒏𝒏+𝑾𝑾−𝟏𝟏
∑𝒌𝒌=𝒏𝒏 �𝒚𝒚𝒂𝒂𝒌𝒌 − 𝒈𝒈 𝒂𝒂 � + ‖𝒚𝒚𝝎𝝎
𝒌𝒌 ‖
𝟐𝟐
𝑾𝑾 𝝈𝝈𝟐𝟐𝒂𝒂 �𝒚𝒚𝒏𝒏 � 𝝈𝝈𝟐𝟐𝝎𝝎
manifiesto [7]. Durante la marcha el pie se detiene por completo
durante la fase de apoyo sobre el suelo. Entonces, es posible Donde n se asocia a la primera muestra de la ventana de
corregir la estimación de velocidad sabiendo que su valor real es procesado. W es número de muestras de esta ventana. yka es la
cero. Esto permite corregir las estimaciones anteriores de posición aceleración en el instante k. ykω es la velocidad angular en k. g es
y velocidad. Esta técnica es conocida como actualizaciones de la aceleración de la gravedad. zna ≜ {yka }k=n
n+W−1
define la ventana
velocidad cero (zero velocity updates, ZUPT) y permite reducir de muestras de aceleración comprendidas entre n y n+W-1.
drásticamente el error de deriva del tracking de posición Foxlin znω ≜ {ykω }k=n
n+W−1
es la ventana análoga de muestras de velocidad
[7]. angular. σa y 𝜎𝜎𝜔𝜔2 son las varianzas de ruido de los acelerómetros y
2
a 1
giroscopios respectivamente. yn = ∑n+W−1 yka es promedio de
Las correcciones ZUPT se implementan habitualmente mediante W k=n
filtrado de Kalman utilizando una matriz de error de covarianza la aceleración entre las muestras n y n+W-1.
9x9 que relaciona todas las componentes de las estimaciones de
Al resolver la expresión anterior para la muestra actual se
posición, velocidad y orientación [6].
obtendrá el valor 𝑇𝑇(𝑧𝑧𝑛𝑛𝑎𝑎 , 𝑧𝑧𝑛𝑛𝜔𝜔 ) que se comparará con un valor umbral
Sobre el planteamiento anterior, [5] proponen una técnica que para determinar si el sensor está en reposo. En el presente trabajo
mejora la estimación de la posición y de la distancia recorrida. se ha utilizado 400 como valor umbral para la comparación.
Dicha técnica se basa en la hipótesis de que no existe correlación Dicho valor se obtiene tras realizar pruebas en las que se aplica el
entre un paso y el siguiente de ciertos elementos de la matriz de criterio de reposo para distinguir movimientos muy leves de la
mano de movimientos claramente definidos marcados. Además se
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)
| 230 |
 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
E. de la Rubia | A. Diaz-Estrella
emplean 3 muestras de tamaño de ventana y los siguientes niveles traza de datos en la que se repite el movimiento 30 veces. Los
de ruido de los acelerómetros y giroscopios: σa = 0.03 m/s2 , tipos de movimientos son:
σω = 0.02 rad/s. Estos valores de ruido se determinaron
registrando las lecturas de los sensores en reposo y una vez T1: La mano avanza 45 cm y vuelve a la posición inicial.
alcanzada una temperatura estable de trabajo. T2: La mano se eleva 45 cm y vuelve a la posición inicial.
T3: Circunferencia de 45 cm de diámetro en sentido horario.
Es necesario señalar que no es posible crear un detector de reposo T4: Circunferencia de 45 cm de diámetro en sentido antihorario.
libre de errores. Esto se debe a una limitación física por la que el T5: Avance y retroceso de la mano de 45 cm en direcciones que
estado de reposo es indistinguible del de movimiento a velocidad forman ángulos de -45, 0 y 45º con respecto a la dirección frontal.
constante.
La Figura 2 muestra algunos ejemplos de estas trazas.
Adicionalmente existe otra limitación que afecta a los detectores
de reposo que se basan en el módulo de la aceleración como el
empleado en este estudio. Cuando la mano desciende y a la vez se
desplaza lateralmente puede ocurrir que las componentes de la
aceleración vertical y lateral den lugar a un valor de la aceleración
cuyo módulo puede coincidir con el valor de la aceleración de la
gravedad (9.81 m/s2). En ese caso, el detector de reposo daría Figura 2. Ejemplos de trazas procesadas T1 (izq), T3 (centro)
lugar a un falso positivo. y T5 (der).
Con todo, al aplicar el algoritmo original con el detector de reposo Existen 245 patrones de reseteo diferentes para aplicar la técnica
descrito, se obtiene un comportamiento muy satisfactorio siempre CESR. Para determinar el patrón óptimo, se generan
que la mano se detenga con frecuencia para aplicar las aleatoriamente patrones de reseteo que se aplican a las 5 trazas. Se
correcciones asociadas a la técnica ZUPT a través del filtrado de obtiene así la LMC para cada traza y se compara con la obtenida
Kalman. El algoritmo de tracking de manos propuesto en este con el algoritmo básico. Una vez se haya simulado un número
trabajo se ha desarrollado utilizando la herramienta descrita en suficientemente alto de patrones, se recopilarán todos los patrones
[4]. La Figura 1 muestra una traza procesada por esta aplicación de reseteo que mejoran significativamente los resultados del
que corresponde a 10 movimientos horizontales y 6 diagonales de algoritmo básico. De este modo, se podrá realizar un estudio
la mano entre las esquinas de un ordenador portátil. estadístico en el que se determine qué elementos de la matriz de
error de covarianza contribuyen a mejorar la estimación de
posición.
No se han obtenido resultados concluyentes hasta el momento
pero sí se han encontrado algunos patrones de reseteo que mejoran
el comportamiento del algoritmo básico en las cinco trazas de
datos. Este hecho apoya la validez de la técnica CESR aplicada al
tracking de manos.

5. TÉCNICAS COMPLEMENTARIAS
Cuando surgen conflictos entre las distintas modalidades
sensoriales, predomina la modalidad visual. De este modo, se
pueden tolerar ciertas discrepancias entre la información visual y
propioceptiva [11] que permitirán mejorar el comportamiento del
sistema de tracking implementando algunas de las siguientes
estrategias.
Figura 1. Movimientos de la mano (10 horizontales y 6 La primera técnica consiste en detectar cuándo la mano decelera a
diagonales) entre las esquinas de un ordenador portátil. la vez que se aproxima a la posición inicial o a un punto de
Para aumentar la exactitud del algoritmo, cabe plantease la interés. Podrá asumirse entonces que la mano va a detenerse en
posibilidad de utilizar la técnica CESR que supone una mejora una posición conocida y se realizará un guiado progresivo de la
significativa en las estimaciones del tracking de pies. mano hacia dicha posición.
Cuando la mano se detiene se aplican las correcciones asociadas
4. ESTUDIO DE VIABILIDAD DE LA al filtrado de Kalman y a la técnica ZUPT. Esto da lugar a
TÉCNICA CESR cambios bruscos de posición y orientación. Mediante una
aplicación gradual de las correcciones cuando el usuario mueve
La técnica CESR consigue mejorar el tracking de posición de los
la mano, mejorará la sensación visual que produce el algoritmo.
pies y reducir hasta en un 37% la longitud media de las
correcciones (LMC) que se aplican a través del filtrado de Kalman La clave del algoritmo de tracking inercial es la detección de los
cuando el sensor está en reposo (ZUPT). estados de reposo. Con ello se controla el error de deriva
aplicando la técnica ZUPT. En el caso del tracking de manos, no
Para evaluar si también mejora las prestaciones en el algoritmo de puede asumirse que existirán periodos de reposo con frecuencia.
tracking de manos, se eligen 5 tipos de movimientos que Sin embargo, cuando la mano realiza un cambio brusco de
comienzan y terminan en la misma posición. De este modo, la dirección en un movimiento de ida y vuelta, puede considerarse
LMC será aproximadamente el error cometido al estimar la que la mano se ha detenido durante un instante. Así, pueden
posición de la mano. Para cada tipo de movimiento se graba una
identificarse los reposos instantáneos y aplicar las correcciones
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)
| 231 |
 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
E. de la Rubia | A. Diaz-Estrella
ZUPT. Esta estrategia contribuiría a controlar el error de deriva y 7. AGRADECIMIENTOS
aliviaría las restricciones que debe imponer el paradigma de Este trabajo ha sido parcialmente subvencionado por la Junta de
interacción que haga uso del sistema de tracking inercial de Andalucía y por el proyecto CENIT España Virtual liderado por
manos. Elecnor Deimos dentro del programa Ingenio 2010.
La última técnica complementaria que se propone en este trabajo
se denomina procesado retroactivo y tiene como objetivo reducir 8. REFERENCIAS
la latencia que introduce la técnica CESR. El reseteo selectivo se [1] Amma, C., Georgi, M. and Schultz, T. Airwriting: Hands-
aplica 6 muestras antes de que finalice el estado de reposo. Un free mobile text input by spotting and continuous recognition
modo de implementarlo es retrasar el procesado durante 6 of 3d-space handwriting with inertial sensors. Proceedings -
muestras. Esta estrategia aumenta la latencia del algoritmo en 50 International Symposium on Wearable Computers, ISWC,
ms puesto que la frecuencia de funcionamiento es 120 Hz. 2012, 52-59.
[2] Bang, W. C., Chang, W., Kang, K. -., Choi, E. -., Potanin, A.
Para aplicar la técnica del procesado retroactivo es necesario
and Kim, D. -. Self-contained spatial input device for wearable
guardar el estado del algoritmo y los datos de los sensores durante
computers. Proceedings - International Symposium on
las 6 últimas muestras. Cuando se detecta que la fase de reposo ha Wearable Computers, ISWC. 2003, 26-34.
terminado, se restaura el estado del algoritmo 6 muestras antes, se
aplica la técnica CESR y se procesa el resto de muestras hasta [3] Barczewska, K. and Drozd, A. Comparison of methods for
llegar a la actual. Con ello, se consigue un procesado en tiempo hand gesture recognition based on Dynamic Time Warping
real que estima la posición de la mano muestra a muestra sin algorithm. Federated Conference on Computer Science and
retardo alguno. Tras aplicar el procesado retroactivo, se producirá Information Systems, FedCSIS 2013. 2013, 207-210.
un cambio en la posición de la mano debida al reprocesado de las [4] de la Rubia, E. and Diaz-Estrella, A. Analizador de señales
6 muestras siguientes al último reposo. Este reajuste es inerciales para tracking de pies y manos. In Proceedings of
típicamente imperceptible ya que los efectos del reseteo selectivo Interacción 2013. XIV Congreso internacional de Interacción
de la matriz de error de covarianza se manifiestan Persona Ordenador. (Madrid). 2013, 91-94.
progresivamente a lo largo del movimiento y no al principio del
mismo. [5] de la Rubia, E. and Diaz-Estrella, A. Improved pedestrian
tracking through Kalman covariance error selective reset.
6. CONCLUSIONES Y LÍNEAS FUTURAS Electron. Lett., 49, 7 (2013), 488-490.
Se ha presentado el trabajo en progreso que explora las [6] Fischer, C., Sukumar, P. T. and Hazas, M. Tutorial:
posibilidades de usar sensores inerciales para realizar tracking de Implementing a pedestrian tracker using inertial sensors. IEEE
manos. Se parte como base de un algoritmo que utiliza filtrado de Pervasive Computing, 12, 2 (2013), 17-27.
Kalman para tracking de pies que se adapta para realizar tracking
[7] Foxlin, E. Pedestrian tracking with shoe-mounted inertial
de manos. Al hacerlo, se han identificado las principales
sensors. IEEE Comput. Graphics Appl., (2005), 38-46.
dificultades de esta tarea como la necesidad de nuevos criterios
para detectar reposo. Para resolver este problema se ha utilizado [8] Hartmann, B. and Link, N. Gesture recognition with inertial
con éxito un detector de reposo propuesto en la literatura y se han sensors and optimized DTW prototypes. Conference
determinado sus limitaciones al aplicarlo al tracking de manos. Proceedings - IEEE International Conference on Systems,
Man and Cybernetics. 2010, 2102-2109.
Se ha descrito el enfoque y los progresos de un estudio de
viabilidad del uso de la técnica CESR para mejorar el tracking de [9] Lefebvre, G., Berlemont, S., Mamalet, F. and Garcia, C.
manos y con el mismo propósito, se ha presentado un conjunto de BLSTM-RNN based 3D gesture classification. Lecture Notes
técnicas complementarias. in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics),
En relación al trabajo futuro, queda pendiente determinar el patrón 8131 LNCS(2013), 381-388.
óptimo de reseteo de la técnica CESR y validar definitivamente su
[10] Lian, K. Y. and Lin, B. H. Gesture recognition using
utilidad en el tracking de manos. La implementación y evaluación
improved hierarchical hidden markov algorithm. Proceedings
de las técnicas complementarias es otra tarea a abordar. También
- 2013 IEEE International Conference on Systems, Man, and
resultará de máximo interés comparar las estimaciones de
Cybernetics, SMC 2013. 2013, 1738-1742.
orientación y posición de los algoritmos de tracking inercial con
sistemas de captura de movimientos ópticos. Con ello, será [11] Mason, A. H. and Bernardin, B. J. Vision for performance in
posible determinar la evolución en el tiempo del error de deriva en virtual environments: The role of feedback timing. Int. J. Hum.
posición y orientación. -Comput. Interact., 25, 8 (2009), 785-805.
También queda pendiente determinar si el patrón de reseteo [12] Skog, I., Nilsson, J. and Händel, P. Evaluation of zero-
óptimo de la técnica CESR depende del tipo de movimiento. De velocity detectors for foot-mounted inertial navigation
ser así, podría evaluarse si resulta efectivo detectar el tipo de systems. 2010 International Conference on Indoor Positioning
movimiento realizado antes de aplicar el patrón de reseteo and Indoor Navigation, IPIN 2010 - Conference Proceedings.
correspondiente. 2010.
[13] Yang, J., Bang, W. C., Choi, E. S., Cho, S. J., Oh, J. K., Cho,
La evaluación de las técnicas se realizará mediante pruebas con
J. K., Ki, E. K., Kim, D. Y. and Kim, S. R. A 3D hand-drawn
usuarios siempre que sea posible. Con ellas se podrá determinar
gesture input device using fuzzy ARTMAP-based recognizer.
además, en qué medida el comportamiento de los algoritmos de
WMSCI 2005 - The 9th World Multi-Conference on Systemics,
tracking dependen de cada individuo.
Cybernetics and Informatics, Proceedings. 2005, 270-275.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 232 |
 Explorando la Viabilidad de un Sistema de Tracking Inercial para la Mano Basado en un solo Sensor
E. de la Rubia | A. Diaz-Estrella
[14] Zhu, C. and Sheng, W. Online hand gesture recognition International Conference on Intelligent Robots and Systems,
using neural network based segmentation. 2009 IEEE/RSJ IROS 2009. 2009, 2415-2420.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 233 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani

An investigation into the comprehension of map

information presented in audio
Feng Feng Tony Stockman Nick Bryan-Kinns Dena AI-Thani
School of Electronic Engineering and School of Electronic Engineering and School of Electronic Engineering and School of Electronic Engineering and
Computer Science Computer Science Computer Science Computer Science
Queen Mary University of London Queen Mary University of London Queen Mary University of London Queen Mary University of London
Mile End Road, London E1 4NS, UK Mile End Road, London E1 4NS, UK Mile End Road, London E1 4NS, UK Mile End Road, London E1 4NS, UK
[email protected] [email protected] [email protected] [email protected]

ABSTRACT analyse the results from an experiment in which 10 sighted people

The growth in mobile and multimodal Computing is leading to the undertook tasks requiring them to understand and reproduce map
consideration of alternative modes of information presentation, information presented to them in auditory form. The paper
particularly in situations such as driving or walking in unfamiliar concludes with reflections on how the tasks were performed and
locations where the eyes are needed for primary navigation. We ways in which these levels of performance might be improved.
report the results of an experiment in which map information is
presented to 10 normally sighted participants using an auditory
2. BACKGROUND
display. Several measures of performance are reported, including 2.1 Auditory Display
the time to navigate a virtual route, keystroke errors and the Past work on auditory displays has focused on four distinctive
ability to construct a visual representation of the route travelled techniques: auditory icons, earcons, actual (synthetic or pre-
based on audio instructions only. The results show significant recorded) speech and spearcons [13]. This research makes
variability in levels of performance between individuals, though extensive use of the first three but not spearcons -- verbal cues
most participants were able to make sense of the auditory display played back at very high speeds -- as they would have added
and produce a reasonable visual representation of the virtual route learning effort for aspects that are well served by synthetic
i.e. participants were able to comprehend the presented audio speech.
map.
Gaver coined the term “auditory icons” in a 1986 article of the
Categories and Subject Descriptors same title: “auditory icons are caricatures of naturally occurring
sounds such as bumps, scrapes, or even files hitting mailboxes”
H.5.1 [Multimedia Information Systems]: User Interfaces – [6]. Gaver explored this emphasis on `everyday listening' further
audio input/output, evaluation/methodology.5.2 [Information in a 1989 paper on SonicFinder, an auditory interface which he
interfaces and Presentations]: User Interfaces – auditory (non- developed for Apple [7].
speech) feedback, evaluation/methodology.
Blattner et al. [3] developed the concept of “abstract earcons”:
General Terms musical “motives” that can be grouped to form “families, where
Design, Experiment, Human factors. earcons with similar meanings have similar sounds”. The authors
defined earcon motives as `a rhythmicized sequence of pitches'
Keywords [3]. The best length, they argued, is two to four notes as longer
Auditory display, spatial information, audio map system, audio diatonic patterns would tend to form tunes and be likely to annoy
information understanding. users. In 1992, Brewster et al. performed a series of experiments
that investigated the use of earcons in much more detail, [5]; see
1. MOTIVATION also [4]. The finding was that earcons do work, but also that
While most people’s interactions with digital devices are visual, Blattner et al. had been overly optimistic regarding the degree of
there is a growing need for people to consider alternative modes differentiation and hierarchical organization that is achievable by
of communication with devices they use every day and changing motives very slightly: participants were unable to tell
everywhere. This need arises out of the fact that digital one member of a given earcon family from another [5].
interactions increasingly take place in the context of other tasks, As [13] have observed, many auditory interfaces call for a hybrid
and, due to the nature of mobile devices, when sight is needed approach, employing “complex mixtures of different techniques”
elsewhere for driving, pedestrian navigation or other tasks. This
[13]. This raises the issue of what the precise balance between
paper explores the extent to which users with normal sight are speech, auditory icons and earcons should be. Each of the three
able to comprehend and reproduce spatial information presented has the potential to distract or annoy the user, so special care
to them in audio. While the choice to present spatial information needs to be taken when recordings of actual sounds, earcon
in audio may seem counter intuitive, the motivation becomes motives and synthetic speech coincide.
clearer when considering the difficulty of using visually based
map information on small screens and in the context of extreme (Begault, 1994) [2] provides an authoritative guide to head-related
navigation, which is navigation under extreme conditions such as transfer functions, distance cues, headphone distance errors, and
extreme darkness, fog or in the presence of smoke. reverberation in the context of 3D sound for virtual reality
applications. Many of Begault's recommendations find a direct
The next section reviews previous work on the development of equivalent in the OpenAL specification and the effects extension
auditory displays, before examining foregoing research on the that accompanies it.
development of auditory map systems. We then describe and

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 234 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani

2.2 Auditory Map System The system provides an immersive, first-person experience of the
The BATS (Blind Audio Tactile Mapping System) project [12] map. Two modes of navigation of the map are provided. The first
pioneered the use of auditory maps (albeit on the scale of whole allows navigation to any point on the map, the second, which is
countries rather than cities) for use by visually impaired people the default, only permits navigation along existing roads and
and so helped create a context for later studies. The researchers paths. The user’s direction of travel is automatically adjusted as
asked to what extent spatial sound can aid spatial understanding streets bend left or right. The sound that accompanies each step
and learning, and highlighted two techniques in particular: changes slightly when this happens to give audible feedback. This
“spatial auditory icons” (e.g. bubbling water for rivers) and means that it is possible to trace a wide oval shape simply by
“callouts” (spoken labels positioned in the auditory scene, e.g. instructing the system to walk straight ahead. When approaching
“Raleigh”). Using tactile trackballs, gamepads and keyboard sharp turns (that is, turns greater than 45 degrees), however, the
commands, users move a virtual listener across the map whose user is asked to confirm the change of direction. At decision
bearing never changes. The BATS developers were keen to ensure points, the user is given a range of options corresponding to the
that the project should use “standard computer hardware and open options available at that point in the physical environment. The
source software” [12]. It appears that BATS development was fact one has reached a decision point is signaled by an earcon, the
discontinued after the two prototypes presented in the 2003 paper. number of notes in which identifies the number of alternative
The first was written specifically for one individual (whose paths from which a choice has to be made.
feedback was that the interface was somewhat complicated) and All turns are 45-degree turns. At any point, users are able to
evaluation of the second was limited to “a number of users, request the name of the street they are on, a rough absolute
sighted and blind, informally interacting with the system” [12]. position on the map and their bearing. They can also trigger a
Heuten [8] developed an “auditory torch” system that bears some sonar function, which gives the user an overview of the landmarks
resemblance to BATS, but specializing in city maps. The user and points of interest in close proximity. Foregrounding
moves the listener across a map of Brussels using an “absolute connections between ways, landmarks and decision points helps
input device”, a customized digitizer tablet with raised borders users by reducing the number of salient features to be borne in
that correspond to the borders of the virtual map. Auditory torch mind.
walkthroughs” are not entirely immersive experiences in the sense Synthetic speech is used to announce street names, names of
of first-person navigation. Once more, users cannot change their points of interest (POIs), system menus and changes in state of the
bearing, for example. The project team conducted two formal system (such as switching ambient sounds on or off). Auditory
evaluations, the first with eleven blind participants and the second icons are used extensively throughout the system to signal how
with six untrained and presumably sighted participants. The busy specific streets are or the presence of POIs such as parks,
results were impressive: participants were able to reproduce maps playgrounds, pubs, bars, restaurants and train/underground
with considerable accuracy, that is, proportions and landmarks stations.
corresponded closely with the underlying map [8]. Heuten went
on to describe a first person application of the auditory torch To avoid sensory overload, the application allows users to filter
system in [9]. out certain sound sources (e.g. restaurants or playgrounds). The
volume of ambient sounds (e.g. market), non-ambient sounds (e.g.
The Audio-based Environment Simulator takes a very different turn around) and synthetic speech can be controlled individually.
approach. It seems that the project's final findings have not yet However, these settings are fixed when taking part in the
been published, and what is known about it comes from a work- evaluation, to present a uniform interface to all participants in the
in-progress paper published in April 2010 [14]. This system experiment.
focused on the representation of small-scale indoor spaces.
The system eschews haptic input devices in favour of basic
Evaluations are conducted using a relatively small group of keyboard commands. The essential consideration here is that
individuals (seven in the case of the preliminary evaluation) in keyboard commands should be as intuitive as possible and use
controlled circumstances. Here, the evaluation requires access to large and/or easily reachable keys wherever possible. The cursor
more than ten rooms, a `tactile model' of the floor plan with keys are clearly invaluable here as they serve to make walking
Braille labels, the Audio-based Environment Simulator itself as and turning somewhat intuitive: cursor up is mapped to ‘step
well as LEGO and plasticine for modeling. The authors concluded ahead’, cursor down to ‘turn around’ and right/left to ‘turn
that the system allows the user to develop an accurate spatial right/left’. Space (‘where am I?’) and Return (‘sonar’) have the
cognitive map of a large-scale three-dimensional space that can be advantage of being large and prominently placed. Escape is a
manipulated for the purposes of a real indoor navigation task. convenient choice for interrupting lengthy verbal descriptions and
dismissing dialogs. The goal was to make the system as easy to
3. AUDITORY MAP EXPERIMENT learn as possible without unduly hampering the user’s ability to
navigate the map and so contribute to the evaluation. One key
3.1 The TEAM System feature of the system is that it allows the user to explore any
Full details of the auditory map system employed in the
location on the planet, from Alcatraz to Brooklyn, New York. For
experiment reported here are given in (Loeliger, 2014) [11]. A example, entering ‘Camden’ and selecting ‘United Kingdom’ in
summary of the key aspects of the system follows. the download dialog will take the user straight to North London.

From there, they can choose a start point and an end point from a has a stereo sound card and uses the Open AL package to render
self-voicing list. The software was implemented on a laptop which its auditory display.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 235 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani

3.2 Experimental Design route correctly. All participants were naïve to the route being used
in the study. At each decision point, participants were instructed
3.2.1 Task Design by the observer about the next step and needed to find the way to
Participants were asked to navigate a pre-determined route using the next decision point relying only on the auditory display.
the TEAM system. The route was set up in the TEAM system Participants were provided with paper and drawing materials and
before the experiment, and comprised nineteen points, including a were required to draw a sketch of the route that they traveled.
start point, an end point and seventeen decision points. All
participants could only access the auditory representation of the 3.2.2 Participants
route presented by the TEAM system during the experiment. The The route used is in London, all participants were screened to
participants were asked to navigate from the start point to the end make sure they had not used that route before and were not
point, and were required to complete six right turns, three left generally familiar with the area. The demographics of the
turns, and nine decisions to carry straight on to navigate the given participants relevant to the study are given in table 1.

Table 1. The demographics of the ten participants

Participant Participants’ Sex Age Computer game Driving experience Nationality

s specialization experience
P1 Material Science F 27 Rarely play it. No driving Chinese
experience
P2 Sustainable engineering F 23 After 13, never play Experienced driver Pakistani
it again.
P3 Computer Science M 28 Never Experienced driver Mexican

P4 Computer Science M 27 Good player Experienced driver Mexican

P5 Mechanical Engineering M 26 Never No driving Chinese

experience
P6 Computer Science M 36 Never Never Argentinian

P7 Computer Science F 23 Rarely play it. No driving Kazakhs

experience
P8 Software engineering M 26 Never No experience Mexican

P9 Biochemistry F 26 Last time is 7 years Experienced driver Mexican

ago
P10 ICT F 20 Long time ago Never SRI LANKAN

3.2.3 Procedure Participants were also asked to construct a sketch of the route
The following procedure was undertaken for each of the 10 travelled as they navigated the route.
participants separately. Firstly, they were introduced to the TEAM
audio map application and overviewed the actions used to 3.3 Data Collection
navigate a map. Each participant was allowed 15 minutes to During the navigation task, participants’ navigation time and their
familiarise themselves with the system working in audio visual performance in terms of keeping on track, keyboard misoperation
mode (TEAM provides a visual presentation of the map as well as and auditory icon recognition were recorded. Additionally, the
an auditory display). After this period of familiarisation, the researcher made notes about how each participant seemed to be
screen was turned so that it was out of sight of the participant, coping with the navigation and sketching while they navigated the
who continued to operate the laptop using a separate keyboard and route. At the end of the navigation task, sketches of the route were
using the auditory display only. Each participant then used the collected and reviewed by the researcher and the participant.
keystroke to place themselves at the start of the preset route, and
from that point on received scripted human instructions about how
4. Results
to reach the next decision point, which they had to translate into 4.1 Performance Assessment
the appropriate keystrokes to navigate within the TEAM system, We measured the participants’ performance in terms of navigation
and respond to the auditory display of the map accordingly. time, their ability to keep on track, keyboard command accuracy,
distance estimation and route review.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 236 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani
Table 2. Navigation time of the ten participants and the interruption time

Participants P1 P2 P3 P4 P5 P6 P7 P8 P9 P10
Recording time 32:43 16:20 13:42 27:49 18:35 21:28 12:49 17:02 14:08 23:08
Pure navigation time 19:25 15:06 16:07 17:13 17:46 17:20 12:27 16:45 13:03 15:00
Interruption times 10 7 4 4 7 4 1 3 2 9

Table 3. Deviation on decision points

Deviation points DP1 DP2 DP4 DP5 DP6 DP7 DP9 DP11 DP15 DP18
Navigation number 3 2 2 3 1 1 1 1 1 1
Back 2 2
Lost 1 2 2 1 1 1 1 1 1 1

4.1.1 Navigation Time keystrokes for the purpose of finding the orientation and exploring
Table 2 reports the times taken by the participants to navigate the what would happen next.
set route. All 10 participants completed the navigation task (5) Miss-keying: this type of error happened due to incorrectly
successfully, though 2 of them got sufficiently lost on their first interpreting instructions from TEAM’s auditory display.
attempt that they started from the beginning again. The times
reported in table 2 below are taken from the second start time for
these two participants.
During the navigation task, the process were interrupted randomly
by raising question, expressing opinion, losing orientation,
confirming information and offering solutions. The process was
ceased for a while during these interruptions. In order to get more
valid navigation time, the time consuming on these behaviors are
excluded from the whole recording time, show as the ‘pure
navigation time’.
4.1.2 Keeping on track
The route navigated comprised the start point, the end point and
seventeen decision points (DP). We mark each point from start to
end successively, and record all participants’ deviations at each
point in table 3. Figure 1. Keyboard command errors based on error type
4.2 Interaction Measures
4.2.2 Distance estimation
4.2.1 Keyboard Commands During the navigation task, participants were asked to estimate the
We recorded participants’ keystrokes during the navigation task, distance according to the number of cursor key presses used
and based on participants’ performance, we identified five types between each decision point, and represent the distance on their
of keyboard error: sketches. Six of the ten participants were able to do the estimation
on different parts of the route. Only one participant did the
(1) Orientation errors: navigating in the wrong direction or estimation throughout the whole process. Three of the ten
misinterpreting cursor navigation as compass directions rather participants did not attempt the estimation during the navigation
than in relation to self. For example, when participants perceived process. Figure 2 shows the sections of the route which were
the speech of ‘heading south’, they tend to press the down arrow divided by each decision points.
key to move forward.
(2) Omitting error: omitting a keystroke to confirm the selection
of a chosen direction.
(3) Unintentional pressing: keystroke happened without notice
and participants usually couldn’t be aware of this mistake.
(4) Incorrect keystrokes while self-orienting (show in figure 1 as
attempting press): This type of error refer to the tentative

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 237 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani

Figure 2. (a) Navigation route setup Figure 2. (b) Route segments

Figure 4. The average of relative length of 12 sections

Figure 3. Sketches of reproduced map

times of the earcon into the length of the line on their sketch.
Figure 3 shows 4 of the 10 sketches collected from participants.
In order to get more valid comparison between the participants’
sketches and the original route, we didn’t calculated the road
sections divided by nineteen decision points showed in the Figure
2. Instead, the route was divided into eleven segments based on
the turning points, which could better reflected participants’
understanding of the spatial configuration.
It is necessary to compare the absolute segment length directly
since the sketches are all in different size. Therefore we could
compare the relative length by calculate the segments ‘proportion
Figure 5. Comparison between participants’ distance on the route. By comparing the relative length on sketches with
estimation on sketch and the original route the corresponding segments on the map, we can determine how
well the participants were able to judge the relative distances of
the route. Figure 4 shows the relative length comparison of the
Each time when participants pressed the up arrow key to move route segments between the original map and ten participants’
forward one step, an auditory icon was played as the feedback. sketches.
Therefore, the more times the earcons were heard, the longer the
road. When participants estimated the distance, they translated the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 238 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani

5. Observations and Exit Interviews views of the use of earcons and auditory icons. The participants
The following summarises the main findings from observations who were positive about the auditory icons and earcons also
made by the researcher of participants during the navigation task, produced the better visual representations of the route.
and the interviews conducted with participants following 7 out of the 10 participants realised that near the end of the route
completion of the task. came close to the middle section they have just passed, the only
one of these realised that the route did in fact form a complete
5.1 The Sense of Current Location circle not far from the end point. This was not the same as we
It was clear from both the observations and the interviews that expected, though the system offered an explicit clue of coming
most of the participants were not confident about their spatial back by displayed the same distinguishable auditory icons
location at several points during the task. Five participant in the successively. In fact, most participants reported that they did not
route review process mentioned that they lack a sense of catch the second auditory icons, and that’s the main reason they
localization. They expected some repeat mechanism would could not figure out the circle configuration. It is not difficult to
remind the current location, and they believed this was the crucial discover the reason through observation: when participants
factor for spatial understanding. pressed the upper arrow key without release, result in a high speed
Participants were quite used to take diagrams or picture as the moving on the road, which inhibited the generation of auditory
external memory which could be checked back instantly [1]. icon at that point.
However, in this pure audio display circumstances, participants
found it difficult and confusing to try to navigate without the aid 5.4 Unintentional Motion Matching
of a visual representation of the route. They reported that the Since people’s spatial perception is a multi-sensory integrated
auditory icons could possibly play the same external reminder system, and this integration of visual and auditory is an
function just as the graph information did. But this function was experience getting process [10], participants tend to do the
limited due to the hierarchical feature of the auditory matches between the direction keys and the orientations
representation. respectively. Ideomotor theory also proved that when people
process the spatial information mentally, haptic reactions could be
5.2 The Sense of Distance easily measured [15]. This could explain the hearing-motion
Participants’ reproduction of relative distances, as shown on the match phenomenon in the navigation task.
sketches they reproduced from the auditory instructions (Figure
3), were often incorrect and distorted, though their representation 5.5 Individual Difference
of changes in direction made a turning points in the route was Some broad correlations were seen regarding performance and
generally quite accurate. However, during the interview we background demographics. The participants who had driving
discovered that most of the participants were able to figure out the experience were more likely to remember the route and to
longest road without referencing the sketch. They reported that reproduce a more accurate visual sketch. The participants who
pay attention on counting the earcon feedback is the key to were good at computer games had no problems with the keyboard
estimate the distance. But what they actually did was estimate the control. Participants who are doing a degree in computer science
distance by counting how many road sections they have passed. spent less familiarisation time at the beginning of the experiment
For example, when participants doing the navigation and than participants from other disciplines.
estimation, the recording of their dialog is as follows:
6. Discussion
‘Counting the earcons of steps is not an efficient way to get
distance information.’
6.1 Navigation Time and The Interruptions
Table 2 shows the results of navigation recording time and actual
‘I think using blocks instead of steps can avoid distracting at some time respectively. The individual times vary substantially between
extent.’ different participants. In both the recording time and the pure
navigation time, participant 1 spent the longest time during the
This could explain why they tend to reproduce same distance of whole process, which is 32 minutes and 43 seconds and
the sections on their sketches. participant 7 spent the shortest time fulfilling the task, which is 12
Another factor relevant to distance estimation is the auditory minutes and 49 seconds. The results indicate that longer
provision. Getting a general idea of the distance of the section or navigation times in the process tend to be accompanied by more
the walking time in advance is more explicit than counting the interruptions, which include operation mistakes, keystroke errors
earcons. Always focus on the earcon display allocated too much and orientation confusion. The direct consequence of those
concentration on counting the distance, result in the omition of interruptions was the deviation from the main road. Whether the
other spatial information. participants were able to realize the deviation by self-checking or
continue deviating until the auditory mismatching encountered, it
5.3 The Ability of Auditory Information took time to redirect the current orientation and get back to the
Perception main road. To the contrary, participants who spend less time
during the navigation progressed smoothly, with less interruptions
Most of the participants seemed to have few problems interpreting
and confusions.
the speech component of the auditory display. The non-speech
parts of the auditory display however provoked much more mixed 6.2 Keep On Track Measure
responses, with 3 participants reporting they found them helpful
Table 3 show the results of participants’ keep on track
and that they complimented the speech effectively, while 4 performance. In all the ten deviated places, there were one start
participants reported that they found them both unhelpful and point, six turning points for changing direction (DP2, DP4, DP5,
distracting. The three other participants were fairly neutral in their DP6, DP15 and the DP18) and three interim points (DP7, DP9,

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 239 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani
DP11) for continue along the road. The result of those deviations compare to other kind of keystroke errors that they made. This
reflects that the participants are more likely to make a mistake at results was in line with participants’ report in Observations and
the turning points than at the interim points. Following is two interviews.
transcript examples of the participant’s >action< and the system’s
<feedback> at a turning point and an interim point. Turning points The results also show that the attempting press error reached 21%.
involved the consideration of direction judgments, orientation Two situations caused this type of error. One situation is the
checking, also involved more steps of keystrokes to choose the participants getting lost during the navigation. Thus they try to
right road branch options, in this example which is ten steps and find out the heading direction and hope there’s some auditory
the interim point only involves five interaction steps. informing message show up by issue a keystroke action. Another
situation is that when participants were not confident about their
At the turning point: operation, what they did was try to figure out whether the reaction
of the system is identify with their anticipation by press any key
1. <Earcons> (Remind the participant here is the decision point) that they believe would be reasonable.
2. >Right arrow key< (Participant flip road branch options) Miss-keying error and keystroke omitting error reached 10%
3. <Right bear gardens> (Speech audio feedback of the system) respectively, and unintentional press error scored 7%. The
purpose of these key presses during navigation is to obtain further
4. >Right arrow key< (Keep flipping road branch options) feedback information, as participants were finding it difficult to
5. <Turn around Footway> (Speech audio feedback of the system) make judgments about their next navigation action. This result
shows the importance of clear and regular feedback in response to
6. >Think of the destination and direct the orientation< navigation actions.
7. >Left arrow key< (Participant back to former option) 6.4 Sketch measure on Distance Estimation
8. <Right bear gardens> (Speech audio feedback of the system) Figure 4 and Figure 5 shows the comparison of the relative length
of twelve road sections between the ten participants’ sketches and
9. >Enter key< (Participant choose the road branch) the original map. The result shows that most participants were
10. >Up arrow key< (Continue the navigation) able to get general knowledge of the distance through earcons
display. From the figure 5 we could see the turning points on the
At the interim point: sketches were congested at the point 3, 6, 10 and the 12, and the
correlated route section 3, 6, 10 and 12 showed in the figure 4 are
1. <Earcons> (Remind the participant here is the decision point)
the most accurate estimation. This results indicate that participants
2. >Right arrow key< (Participant flip road branch options) were doing well when dealing with single segments estimation.
Figure 5 also shows that the point 1, 7, 8, and 11, have more
3. <Continue ahead Park Street> (Speech audio feedback of the separated distribution, and the route section 1, 7, 8, and 11 in the
system) figure 4 have larger difference value. This indicate that they tend
4. >Enter key< (Participant choose the road branch) to have biased estimation when dealing with continuity segments.

5. >Up arrow key< (Continue the navigation) 7. Conclusions

In this paper we first reviewed the development of audio displays
The results also show that the decision point 1 and the decision
and then introduced the audio map system, which use earcons,
point 5 are the places where the participants are most likely to
auditory icons and synthetic speech to represent spatial
make a mistake. However, these two places are the only places
information. We then present the experimental design with ten
that participants were able to find the way back based on the
spatial information that the system offered. Since the most of the sighted participants, and analyse the results from qualitative angle
deviations happened at the beginning part of the route, it seems and quantitative angle. The measurements of the participants’
that participants need time to make themselves be familiar with performance include performance assessment and interaction
the interaction way of this system. This finding is in line with the measurement. The result show that all the ten participants were
observation results of gaining experience and getting better at able to complete the navigation and reproduce the route
configuration by listen to the auditory representation though the
navigation as they progressed through the route.
leverage of individual difference were vary significantly.
6.3 Keystrokes Analysis This study has identified many features that would facilitate
Figure 1 shows the measurement of five type of keyboard people’s spatial configuration. The keep on track feature facilitate
commands errors. The results indicate that the orientation error is the ability of navigation maintenance, but the passive way of
the most commonly occurring keystroke error with a per centage information exploration confined auditory perception to some
score of 52%. Six of the ten participants were made this type of extent. Keyboards commands has positive effect on the familiarity
error more than two times, another two participants who did not of the system interaction, but the arrow keys have an implicit
make this type of error reported that they tend to press up error misleading on orientation judgment. Most participants were able
key when move south, and they did not feel comfortable when to fulfill the distance estimation by listen to the earcons, but this
moving south by press the up arrow key. task occupied too much concentration especially when their main
This result was expected since ideomotor effect (Stock, A., 2004) task is navigation to destination. Audio information can only be
is a common feature in peoples’ spatial related actions. This result represented hierarchically, thus the audio-only representation lack
indicates that spatial recognition ability has a strong association the instance reference mechanism. This is the main reason of the
with kinesis to a large extent. From each of the participants’ side, deviation and miscalculation.
the hearing-action mismatching error also have a large proportion
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)
| 240 |
 An investigation into the comprehension of map infor-mation presented in audio
F. Feng | T. Stockman | N. Bryan-Kinns | D. Al-Thani
The discoveries and the results of this study will not limited to the [8] Heuten, W., Henze, N., & Boll, S. (2007, April). Interactive
audio map systems since visual modality may not be suitable exploration of city maps with auditory torches. In CHI'07
under visually inaccessible interfaces or situations, such as extended abstracts on Human factors in computing systems
subsize wearable devices or in high visual occupation tasks. We (pp. 1959-1964). ACM.
hope the results and the discussion would shed light on the audio- [9] Heuten, W., Henze, N., Boll, S., & Pielot, M. (2008,
based information cross-modal representation in the future. October). Tactile wayfinder: a non-visual support system for
wayfinding. In Proceedings of the 5th Nordic conference on
8. REFERENCES Human-computer interaction: building bridges (pp. 172-
[1] Bauer, M. I., & Johnson-Laird, P. N. (1993). How diagrams
181). ACM.
can improve reasoning. Psychological Science, 4(6), 372-
378. [10] Jones, B. (1975). Spatial perception in the blind. British
Journal of Psychology, 66(4), 461-472.
[2] Begault, D. R., & Erbe, T. (1994). Multichannel spatial
auditory display for speech communications. Journal of the [11] Loeliger, E., & Stockman, T. (2014). Wayfinding without
Audio Engineering Society, 42(10), 819-826. Visual Cues: Evaluation of an Interactive Audio Map
System. Interacting with Computers, 26(5), 403-416.
[3] Blattner, M. M., Sumikawa, D. A., & Greenberg, R. M.
(1989). Earcons and icons: Their structure and common [12] Parente, P., & Bishop, G. (2003, March). BATS: the blind
design principles. Human-Computer Interaction, 4(1), 11-44. audio tactile mapping system. In Proceedings of the ACM
Southeast Regional Conference (pp. 132-137).
[4] Brewster, S. (1992). Providing a model for the use of sound
in user interfaces. University of York, Department of [13] Peres, S. C., Best, V., Brock, D., Frauenberger, C., Hermann,
Computer Science. T., Neuhoff, J. G., ... & Stockman, A. 2008. Auditory
interfaces. HCI Beyond the GUI: Design for Haptic, Speech,
[5] Brewster, S. A., Wright, P. C., & Edwards, A. D. (1992). A
Olfactory, and Other Nontraditional Interfaces, 147-195.
detailed investigation into the effectiveness of earcons. In
DOI=10.1016/B978-0-12-374017-5.00005-5
Proceedings of ICAD'92 (Santa Fe Institute, Santa Fe)
Addison-Wesley. [14] Sánchez, J., Sáenz, M., Pascual-Leone, A., & Merabet, L.
(2010, April). Enhancing navigation skills through audio
[6] Gaver, W. W. (1986). Auditory icons: Using sound in
gaming. In CHI'10 Extended Abstracts on Human Factors in
computer interfaces. Human-computer interaction, 2(2), 167- Computing Systems (pp. 3991-3996). ACM.
177.
[15] Stock, A., & Stock, C. (2004). A short history of ideo-motor
[7] Gaver, W. W. (1989). The SonicFinder: An interface that
action. Psychological research, 68(2-3), 176-188.
uses auditory icons. Human-Computer Interaction, 4(1), 67-
94.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 241 |
Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (I)

| 242 |
INTERACTION DEVICES (II)
 InclineType – An Accelerometer-based Typing Approach for Smartwatches
T. Götzelmann | P. P. Vázquez

InclineType – An Accelerometer-based Typing Approach

for Smartwatches
Timo Götzelmann Pere-Pau Vázquez
Nuremberg Institute of Technology Technical University of Catalonia
Keßlerplatz 12 Jordi Girona, 1-3
90489 Nuremberg, Germany E-08034, Barcelona, Spain
+49 (0)911 5880 1616 +34 93 413 77 90
[email protected] [email protected]

ABSTRACT to privacy. To overcome this limitation, some smart input

Small mobile devices such as smartwatches are a rapidly growing techniques have been proposed. In this paper we focus on
market. However, they share the issue of limited input and smartwatches, and present a method for input that makes use of a
output space which could impede the success of these devices in 3-axis accelerometer sensor. Our technique lets the user type by
future. Hence, suitable alternatives to the concepts and selecting a key using wrist movements, and single taps. We have
metaphors known from smartphones have to be found. In this performed an initial user study to determine the feasibility and
paper we present InclineType a tilt-based keyboard input that the performance novice users may achieve with such approach.
uses a 3-axis accelerometer for smartwatches. The user may
directly select letters by moving his/her wrist and enters them by 2. RELATED WORK
tapping on the touchscreen. Thanks to the distribution of the Typing in devices without hardware keyboards such as the
letters on the edges of the screen, the keyboard dedicates a low classical buttons of conventional Blackberry devices may lead to
amount of space in the smartwatch. In order to optimize the user low typing performances. This issue can be partially alleviated
input our concept proposes multiple techniques to stabilize the using high profile predictive and correction techniques (e.g.,
user interaction. Finally, a user study shows that users get Minuum, Swype, Fleksy…). However, even the excellent results
familiar with this technique with almost no previous training, of these techniques, they are still not perfect and may erroneously
reaching speeds of about 6 wpm in average. correct properly spelled words such as road names, main names
etc. Moreover, some of them rely on the use of the relative large
Categories and Subject Descriptors sizes of the mobile devices. For smartwatches, the typing is an
H.5.2 [Information Interfaces and Presentation]: User even more difficult task.
Interfaces – Input devices and strategies (e.g., mouse, For small devices such as watches, the techniques tailored to
touchscreen); Graphical user interfaces. improve input follow two tendencies: adding hardware that
makes the interaction richer, or make smarter keyboards. Our
General Terms method falls in the second category. Small devices can be
Interaction Techniques, Fat-finger-problem, Input Concepts, enhanced for input by adding sensitive back covers [1]. However
Keyboards, Mobile Devices, Acceleration Sensor.

Keywords
Small Devices, Smartwatch, Input Techniques, Text Entry, Fitts’
Law, Wrist, Angle, Inclination.

1. INTRODUCTION
Smartwatches and other related wearables are becoming more
popular. The launch of Apple Watch is expected to be the
element that helps these devices to become mainstream. Many of
those devices have a minimalistic UI and rely on speech for more
complex input. Unfortunately, dictation is inherently a non-
private task, and both sensitive information and public spaces are
usually not suitable for speech input. Thus, typing in wearables
is a requirement for many tasks.
The fat finger problem (e.g., [10]) is relevant to smartphones and
even more prominent for smaller devices such as smartwatches.
Since text input is required even for simple tasks such as
answering a message, however, the common input technique
present in Android Wear-based watches or Apple Watch (voice
dictation) cannot be used in a broad range of situations, e.g., due Figure 1: Typing by inclination of smartwatch.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 243 |
 InclineType – An Accelerometer-based Typing Approach for Smartwatches
T. Götzelmann | P. P. Vázquez
these back-of-device interactions would require to take the the wrist along the forearms axis (y-axis), a specific letter can be
smartwatch off. Other proposals include adding joystick sensors selected. The currently selected key is presented in a highlighted
for panning and tilt detection [13]. However, it is unlike that all fashion in order to support the user. If neither in x-axis nor in y-
vendors will include these by default in mainstream consumer axis a significant inclination is detected, the space character is
devices, which are our target devices. chosen. In this case, no letter is highlighted. Beside the alphabet
Tilting ([7]) has previously been introduced to input detection characters, the period is presented and a hat character which
both as a full typing solution with a WiiMote [5]. But this allows to switch to capitalized characters, figures, and special
solution relies on the use of an external screen. It has also been characters. By tapping on an arbitrary point of the smartphone’s
used to disambiguate buttons in old feature phones [12], where touchscreen, the user enters the selected letter. On the contrary,
the users had to tilt the device while pressing a button. It has also long tap induces a backspace to delete accidentally typed errors.
been used in conjunction with specialized hardware [8]. Our Turning the wrist around a given point completely without
approach is similar in spirit to this one, but using a consumer inclination to select individual letters may have multiple
device instead of customized hardware for the key recognition. disadvantages. First, the legibility of the screens contents
Finally, pure soft solutions create advanced keyboards. Similar to considerably decreases when the user’s visual axis diverts too
Minuum, Dunlop et al. [4] divide the screen in regions that group much from the screens orthogonal. Secondly, forcing the users to
several keys and let the user tap on these regions. A powerful hold the smartwatch constantly in an inconvenient position may
disambiguating software makes the rest to type correctly. be tedious. Hence, we implemented a null calibration feature to
Zoomboard is a keyboard that requires two taps for key selection. let the user decide (i) which is the preferred null position, and
The keyboard is presented in reduced size, and the first tap on a (ii) how sensitively the rotation around the x- and the y-axis
region generates a zoom-in on that region. The second, affects the selection of letters.
respectively third tap effectively selects the key. Leiva et al. [6] In our first test runs during the design stage we discovered that
compare the Zoomboard to other techniques such as the offset minor changes in the smartwatches inclination often resulted in
cursor and offset zoom and found that Zoomboard was a solid frequent unattended changes in the selection of two neighboring
proposal for small screens, with speeds of 6 wpm in screens of letters. Since this may severely impede the selection of the
18mm. Another alternative to Zoomboard is Swipeboard [2]. desired letters, our design includes the implementation of a
This last approach uses swipes instead of taps. Initially, the hysteresis for the selection of letters which allows a smoother
screen is divided in regions with three to four keys, and a swipe interaction.
activates one of the regions. A second swipe selects the key
Another point we discovered in the design stage of our approach
among the remaining ones. The main advantage is that it does
was that there was often a difference between the desired letter
not require precision because it is target agnostic, and they
and the letter which was actually entered by the users. This
improve slightly over Zoomboard in wpm.
discrepancy was caused by the inherent design of our interaction,
In contrast, our approach allows to select all letters of the i.e., selection by inclination and tap for entering the letter which
alphabet solely by changing the angle of the user’s wrist and a often induced slight changes in the inclination before the tap was
single tap on the screen. There is no need for extra hardware or detected. A simple but effective technique eliminated this issue:
multiple steps to enter a letter. Additionally, it works reliably as a jitter prevention we recorded the interaction for selecting
without the use of automatic spell checking. Hence, uncommon letters in a circular buffer for the timespan on 100ms. When a tap
strings can be entered easily. on the touchscreen was registered, the letter entered was the one
that had been selected this timespan ago. Thus, more reliable
3. OUR APPROACH inputs were possible even when the user was jittering during the
In our approach we rely on sensors for measuring the devices’ touch input.
inclination which are already built in several consumer
Alternative arrangements of letters
smartwatches. One of our design goals was to facilitate the
intuitive feeling of gravity to select an alphabetic letter. Another Letters are selected depending on the angle computed from the
goal we addressed was to allow direct selection of letters with
only one interaction step.
In our approach the alphabets letters are arranged clockwise
manner along the screen borders (see Figure 1). Depending on
the inclination of the smartwatch caused by raising and lowering
the elbow relatively to the wrist (x-axis) as well as the rotation of

Figure 3: Distribution of letter frequencies and reserved

space in respect of Fitts’ law. Parts of bar representing
according to actual letter frequencies (upper bar), and
Figure 2: Letter frequencies in English sentences according adapted version (lower bar) which maintains a minimum size
to [2]. for letters of low frequency.
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)
| 244 |
 InclineType – An Accelerometer-based Typing Approach for Smartwatches
T. Götzelmann | P. P. Vázquez
inclination of the x- and y-axis. Hence, in the first version (standard version) as well as the distribution depending on letter
introduced so far, we assume that the index of difficulty frequencies (optimized version). Subsequently, the participants
(according to Fitts’ law) to select an arbitrary letter is equal. had the opportunity to test the input concepts for a short period
Beside the conventional approach to distribute the alphabets (≤3 minutes). Additionally, they were free to decide whether they
letters uniformly at the corners of the smartwatches display, we wanted to use an aclinic null position or they wanted to calibrate
also implemented a version which is optimized regarding the the null position to a convenient inclination of their wrist.
Fitts’ law [4]. This principle assumes that the difficulty to hit a Both variants (standard and optimized version) were tested with
target mainly relies on the target’s distance and size. It has the participants. The users had to type in a short sentence
already been considered for a study on tilt-based interaction by whereas the timings of the user input were logged. In order to
MacKenzie et al. [7] based on tablet computers. They used a limit learning effects, we alternated the order of the first version
virtual ball as input metaphor and provided 12 target regions to be tested, i.e., if the standard or optimized version was tested
(virtual holes). The users had to steer the ball towards these in the first or the subsequent turn. We analyzed the automatically
holes in a defined sequence. Whilst varying the size of the recorded data by Student’s paired-sample t-test to evaluate our
targets, they measured the time the users needed. They pointed experiment.
out that Fitts’ law can be applied to tilt-based interaction.
This motivated us to consequently adapt this principle to our
approach. Since the inclination of the smartwatch always selects
one specific letter the distance measure can be neglected.
However, based on the fact that letter frequencies [11] are not
equally distributed (see Figure 2) we adapt the targets’ sizes
(letters) according to their frequency. Directly adapting the letter
frequencies to the size of selection revealed that it is nearly
impossible to select rare letters such as j, q, x, z (see Figure 3
upper bar). Hence, we preserved a minimum size for these letters
which proportionately decreased the influence of frequent letters
(see Figure 3 lower bar). The actual arrangements of letters for
both alternatives can be seen in Figure 4. Figure 5: Performance in words per minute for both the
standard version with uniform distribution of letters (top)
and the optimized version (bottom).
We tested our null hypothesis H0: The optimized version
(according to Fitts) of our implementation is not performing
better (in terms of words per minute) than the standard version
(uniform distribution of letters). The arithmetic mean as well as
the median of the optimized version (M=5.9, SD=2.43) showed a
better performance (see Figure 5) than the standard version
(M=4.7, SD=2.18). In average the optimized version allowed
~25% faster typing than the standard version. This difference in
the performance of both versions of our user study was
Figure 4: Normal target size for letters (left) and adjusted significant: t10=-2.105, p=.030. Hence, H0 can be rejected for this
target sizes (right) depending on letter frequencies. preliminary user study. Most of the users preferred the optimized
version and reported the subjective feeling that this version
allows a more convenient typing. Furthermore, the participants
4. RESULTS mentioned that a blinking cursor would have been advantageous,
As proof of concept our approach has been implemented on a especially when space characters have to be entered. Which was
fully functional Android 4.2.2 smartwatch (Umeox KingKong in fact a shortcoming of our user test design – and could be easily
Pro) with an integrated Bosch BMA050 3-axis accelerometer. Its implemented for future tests. Although they liked the possibility
1.6 inch TFT screen has a resolution of 240x240 pixels. to calibrate the null position to arbitrary wrist inclinations most
We implemented an experimental application including the of them preferred the aclinic setting for the test. A possible
features for null calibration, hysteresis and jitter prevention. The explanation to this fact could be that they are used to recognize
smartwatch app allowed to switch between both alternatives of aclinic surfaces (e.g., water surface in a glass of water). In future
letter arrangements. We extended the experimental app by a studies a visual indicator for the null position should be included.
logging feature which recorded the start and end of individual
tests as well as each keystroke. Using the experimental 5. DISCUSSION
application we tested the feasibility in a preliminary user study. In this paper we introduced a novel input concept for typing by
Our user study was carried out with 11 test participants (3 inclination of smartwatches without the need to press one or
females), with ages ranged between 21 to 37 (avg. 27.6±4.8) multiple hardware buttons concurrently. Our approach proposes
years which obtained a short explanation about smartwatches and multiple techniques for the stabilization of the user interaction.
both the input concepts using the uniform distribution of letters Additionally, we addressed the fact that letter frequencies are not

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 245 |
 InclineType – An Accelerometer-based Typing Approach for Smartwatches
T. Götzelmann | P. P. Vázquez
distributed uniformly and adapted this to our concept according 6. FUTURE WORK
to the Fitts’ law. Our technique is not dependent on dictionaries, We plan to carry out a comprehensive user study giving the tests
i.e., arbitrary character strings can be entered at the same speed users the chance to practice our alternative concept for text input
than well-known words. Another advantage of our concept is that for several hours. As it has been shown in other studies, we
this method occupies less screen space than the conventional expect a significant increase in typing speed by experienced
input and the remaining screen space may stay squared (see users. This user study will include an in-depth comparison of
Figure 6). Compared to voice input there are no privacy issues. both alternatives of character arrangements to verify if our
Finally, the feasibility of our concept has been verified by an implementation of Fitts’ law shows an optimization in users
experimental application as well as a preliminary user study. input performance.
In this paper we only addressed the English language. As there
Watches may be worn either on the wrist of the dominant hand or
also exist studies about letter frequencies of other languages
the other hand. For longer texts, some users could prefer to hold
(e.g., [9]), our concept could be transferred to them as well, as
the smartwatch in both hands. Since these different application
long as a minimum character size can be maintained. The
scenarios could significantly affect the input performance, our
support of languages with significantly more than 27 letters, e.g.,
planned comprehensive user study will compare these variations.
the Chinese language would need to take additional effort (e.g.,
multiple interactions), but this applies also to other input Beside existing smartwatches, currently many other devices
methods. appear on the market which could be used in conjunction with
Our approach has rather high requirements to motoric skills of future smartwatches. One of these devices is called Myo which is
the users. Hence, it is likely that persons with motoric a gesture control armband which uses electromyographic sensors
impediments will not be able use this technique. The same issue to detect arm and hand gestures. Although it is already possible
may exist in very unsteady or turbulent environments. However, to connect such a device by standard Bluetooth, in future
the authors assume that other approaches will have similar smartwatches these muscle sensors could be integrated into the
issues. smartwatches armband. Our future work focusses this
combination of hardware in order to optimize our input concept
Some techniques of the related work need to train the users to an for replacing the touch interaction by muscle actuation in order to
excessive extent. A design goal of our approach was to make it further improve the input performance of our concept.
explicitly intuitive. Our preliminary user study aimed at testing
untrained users and we obtained the unanimous feedback that
this technique is very intuitive. However, testing completely 7. REFERENCES
untrained users resulted in mediocre performance for typing [1] Baudisch, P. and Chu, G. Back-of-device interaction allows
speed, compared to some of the approaches of the related work creating very small touch devices. Proceedings of the
which partially trained their test users intensively. Since our SIGCHI Conference on Human Factors in Computing
approach allows interaction with minimized interaction steps and Systems, ACM (2009), 1923–1932.
the letters are always presented in a consistent manner our [2] Chen, X., Grossman, T., and Fitzmaurice, G. Swipeboard: a
technique leverages people’s spatial memory. Thus, it is likely text entry technique for ultra-small interfaces that supports
that well-trained users will be able to perform much better in novice to expert transitions. ACM Press (2014), 615–620.
typing speed and error rate.
[3] Dunlop, M.D., Komninos, A., and Durga, N. Towards high
Similar to other approaches this technique could be extended by quality text entry on smartwatches. ACM Press (2014),
automatic spell checking. Dependent on the concrete application 2365–2370.
this could significantly improve the typing performance, but as
[4] Fitts, P.M. The information capacity of the human motor
stated before, this may impede the input of arbitrary character
strings. system in controlling the amplitude of movement. Journal
of Experimental Psychology 47, 6 (1954), 381–391.
[5] Jones, E., Alexander, J., Andreou, A., Irani, P., and
Subramanian, S. GesText: accelerometer-based gestural
text-entry systems. Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems, ACM (2010),
2173–2182.
[6] Leiva, L.A., Sahami, A., Catalá, A., Henze, N., and
Schmidt, A. Text Entry on Tiny QWERTY Soft Keyboards.
(2015).
[7] MacKenzie, I.S. and Teather, R.J. FittsTilt: The application
of Fitts’ law to tilt-based interaction. Proceedings of the 7th
Nordic Conference on Human-Computer Interaction:
Making Sense Through Design, ACM (2012), 568–577.

Figure 6: Relatively low space consumption and rectangular [8] Partridge, K., Chatterjee, S., Sazawal, V., Borriello, G., and
input by InclineType (left). Standard keyboard input Want, R. TiltType: accelerometer-supported text entry for
consuming large screen area (right). very small devices. Proceedings of the 15th annual ACM

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 246 |
 InclineType – An Accelerometer-based Typing Approach for Smartwatches
T. Götzelmann | P. P. Vázquez
symposium on User interface software and technology, [12] Wigdor, D. and Balakrishnan, R. TiltText: using tilt for text
ACM (2002), 201–204. input to mobile phones. Proceedings of the 16th annual
[9] Shamilov, A. and Yolacan, S. Statistical Structure of Printed ACM symposium on User interface software and
Turkish, English, German, French, Russian and Spanish. technology, ACM (2003), 81–90.
WSEAS Transactions on Mathematics 5, 6 (2006), 756. [13] Xiao, R., Laput, G., and Harrison, C. Expanding the input
[10] Siek, K.A., Rogers, Y., and Connelly, K.H. Fat finger expressivity of smartwatches with mechanical pan, twist, tilt
worries: how older and younger users physically interact and click. ACM Press (2014), 193–196.
with PDAs. In Human-Computer Interaction-INTERACT
2005. Springer, 2005, 267–280.
[11] Solso, R.L. and King, J.F. Frequency and versatility of
letters in the English language. Behavior Research Methods
& Instrumentation 8, 3 (1976), 283–286.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 247 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro

Basketball Activity Recognition using Wearable Inertial

Measurement Units

Le Nguyen Ngu Nguyen Daniel Rodríguez-Martín Andreu Català

Universitat Politècnica de Technical Research Centre for Technical Research Centre for
Catalunya – BarcelonaTech Dependency Care and Autonomous Dependency Care and Autonomous
Rambla de l’Exposició 59-69, Living Living
Vilanova i la Geltrú, Spain Universitat Politècnica de Universitat Politècnica de
Queen Mary University of London Catalunya – BarcelonaTech Catalunya – BarcelonaTech
Mile End Road, E1 4NS, London, Rambla de l’Exposició 59-69, Rambla de l’Exposició 59-69,
UK Vilanova i la Geltrú, Spain Vilanova i la Geltrú, Spain
[email protected] [email protected] [email protected]

Carlos Pérez-López Albert Samà Andrea Cavallaro

Technical Research Centre for Technical Research Centre for Queen Mary University of London
Dependency Care and Autonomous Dependency Care and Autonomous Mile End Road, E1 4NS, London,
Living Living UK
Universitat Politècnica de Universitat Politècnica de [email protected]
Catalunya – BarcelonaTech Catalunya – BarcelonaTech
Rambla de l’Exposició 59-69, Rambla de l’Exposició 59-69,
Vilanova i la Geltrú, Spain Vilanova i la Geltrú, Spain
[email protected] [email protected]

ABSTRACT
The analysis and evaluation of human movement is a growing
Keywords
Wearable computing, Activity recognition, Sports.
research area within the field of sports monitoring. This analysis
can help support the enhancement of an athlete’s performance,
the prediction of injuries or the optimization of training 1. INTRODUCTION
programs. Although camera-based techniques are often used to Human activity recognition (HAR) has facilitated novel
evaluate human movements, not all movements of interest can be applications in healthcare, life monitoring, entertainment, and
analyzed or distinguished effectively with computer vision only. sports [1]. The main goal of HAR is to identify the activities of
Wearable inertial systems are a promising technology to address one person or a group of people from observations acquired by
this limitation. This paper presents a new wearable sensing sensing devices such as wearable sensors and ambient cameras.
system to record human movements for sports monitoring. A new While vision-based approaches suffers from occlusion, cluttered
paradigm is presented with the purpose of monitoring basketball scenes and changes in illumination, advances in sensor
players with multiple inertial measurement units. A data technology allow us to deploy wearable devices for HAR and to
collection plan has been designed and implemented, and enable continuous long-term activity monitoring beyond arranged
experimental results show the potential of the system in areas. Moreover, the sensors embedded in these devices, such as
basketball activity recognition. accelerometers and gyroscopes, collect signals directly from body
movements, in lieu of inferring from visual data. Multiple types
Categories and Subject Descriptors of wearable devices are utilized to record sensing information,
such as: accelerometers and gyroscopes for movements, GPS
H.1.2 [User/Machine Systems]: User/Machine Systems –
receivers for positions in outdoor areas, electromyography
Human factors, Human information processing.
armbands for muscle activities, etc. Relying on required data,
one sensor or a combination of sensors can be mounted on
General Terms different parts of the human body.
Algorithms, Design, Experimentation, Human Factors.
An important application domain for human activity analysis is
in sports activity analysis. Understanding athletic movements

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 248 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro
helps coaches and managers evaluate their players’ performance, collection session and activity recognition method are described
predict injuries, optimize training programs and support strategic in Section 4 and Section 5, respectively. The experimental
decision making. To do that, a sport activity analysis system can results and analysis are presented in Section 6. Finally, Section 7
identify actions through appropriate motion features, context and concludes the paper.
expert knowledge. Challenges in sport activity analysis include
both generic and specific ones. Generic challenges [2] are: 2. RELATED WORK
• Intra-class variability: The same activity may be performed Multiple types of wearable sensors have been used,
in different ways, depending on internal status of the player. independently or combined, for activity monitoring and
For example, a basketball player moves slowly at the end of performance evaluation in sport, including accelerometers,
the match due to fatigue. Furthermore, each individual gyroscopes, pressure sensors, heart-rate monitors. The generic
player has his/her own styles of movements. system pipeline includes feature extraction, classification and
qualitative evaluation [3].
• Inter-class similarity: Some activities are semantically
different but produce similar characteristics in inertial Several works analyze specific movements for different sports.
sensing data. For instance, in basketball, acceleration For example, Kelly et al. [4] studied techniques to automatically
signals of running with or without the ball may be almost identify tackles and collisions in rugby. Their sensor consists of a
identical. GPS receiver and an accelerometer placed between the shoulder
blades overlying the upper thoracic spine. Support vector
• The NULL class problem: Not all inertial sensing data is machine (SVM) and hidden conditional random field (HCRF)
necessary for sport activity analysis. The existence of were applied to detect collision events. The authors used
irrelevant activities (the so-called NULL class) may confuse AdaBoost to combine the classification results produced by these
an activity classification algorithm. two models. Morris et al. [5] used inertial sensors to discriminate
In addition to the aforementioned challenges, sport activity non-exercise and exercise movements (13 exercise actions), and
analysis possesses its own challenges: then to recognize and count the repetition of activities. Bächlin et
al. [6] presented a wearable assistant for swimmers based on
• Definition of relevant activities: A set of activities of acceleration sensors and real-time feedback devices. The system
interest should be proposed by sports experts. Moreover, extracts swimming parameters to evaluate body measurement
human activities contain spatial and temporal constraints, (angle, rotation, and balance), performance parameters, and
which should be considered when modelling. feedback effectiveness. Harle et al. [7] built an on-body sensing
system to monitor the performance in sprinting training sessions.
• Class imbalance: The same activity might have different
Force sensitive resistors were embedded in athletes' shoes to
durations and different activities generally have a different
collect pressure data with millisecond-level accuracy in ground-
frequency. For instance, a soccer striker spends most of the
contact time estimation.
time moving (e.g. walking and running) while he/she might
shoot only few times in a match. With the purpose of replicating the role of expert evaluations in
climbing sport, Ladha et al. [8] developed a wearable
• Data annotation: Training a supervised HAR system
acceleration sensing platform to record climber's movements and
requires a significant amount of data with annotation. The
assessment parameters including power, control, stability and
difficulty of annotation increases in case of team sports,
speed. Their system was extensively experimented with 53
where multiple players interact.
climbers under competition scenarios and could produce scores
• Sensing data characteristics: Movements in a competitive that strongly correlated with official expert results. Furthermore,
match are generally much faster than those in daily living some works provide a feedback to the athlete with the aim of
activities [13] [14]. Moreover, variations in acceleration in giving a stimulus and ameliorate performance. Bächlin et al. [6]
sport are larger than in monitoring systems for daily living used LEDs to provide real-time feedback to swimmers; whereas
activities. Therefore, parameters should be modified so that the system of Velloso et al. [9] has a feedback mechanism that
they adapt to the characteristics for each sports type. helps weightlifters to manipulate their movements. In baseball,
Ghasemzadeh and Jafari [10] collected physiological data from a
In this paper, we introduce a new sensor design using body sensor network to provide corrective feedback for the
accelerometers for recording and recognizing athletic movements players. They interpreted complex movements to generate motion
in basketball. We are interested in movements such as jumps, transcripts which were used for measuring coordination among
lateral displacements, forward/backward moves and body limb segments and joints.
rotations (pivot). We also consider sport-dependent actions such
as for example dribbling and shooting. The new sensor is 3. SENSOR DESIGN FOR BASKETBALL
attached on a user’s back and lower limbs establishing a multi-
A new inertial measurement unit, which is called BSK board,
sensor system. Activities such as walking, jogging, running,
has been developed and is presented in this paper. The main task
sprinting, jumping, jumpshot, layupshot, and pivot are of this device is to record data from specific movements in
recognized using a Support-Vector-Machine-based classifier. basketball. Its size is 62mm x 35mm x 24mm and weight is 38g
The rest of this paper is organized as follows. Section 249 (and 62g with batteries). The BSK board is an inertial
reviews related work using wearable sensors in sport. The measurement unit (IMU) that captures inertial data that requires
proposed sensor system is introduced in Section 3. Then, the data a long range of acceleration as well as experiments requiring

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 249 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro
barometric information. The BSK board is a development tool, to increase the autonomy of the BSK board. Figure 2 shows the
which includes inertial sensors, a storage unit and a small expected consumption at different work modes.
interface in order to send and receive commands. The BSK board
has four main parts, the power system, the microcontroller (µC)
and its interface, the analog system and the communications
module (Figure 1).

Figure 2. µC average consumption in different work modes

Figure 1. Block diagram with the structure of the newly 3.2 Embedded Sensors
designed BSK board proposed in this paper The BSK board contains different microelectromechanical
The system is powered with two AAA standard batteries. The systems (MEMS) sensors, one the one hand it includes the
average consumption of the system (7 tests of five-minutes each) LSM9DS0, a System in Package (SiP) which contains an
is 56.73±0.21mA, having an autonomy of more than 20 hours in Accelerometer + Magnetometer system and a Gyroscope system
a 1200mAh battery at 3 Volts. With a standard alkaline battery (Figure 3). On the other hand the BSK board includes a pressure
(750mAh), autonomy could be about 13.2 hours. Conditions of sensor as well in order to detect fallings or movements with a
test have been analyzed with all the sensors on, a sampling rate change of altitude (mainly postural transitions).
of 200Hz and storing the inertial captured data within the µSD The LSM9DS0 MEMS provide different interruptions in order to
Card. first notify the µC when data is ready to be read with a
The power distribution is comprised with four regulators: digital configured output data rate and, second, provide interrupts when
regulator, analog regulator, comm regulator, and backup a configured threshold is surpassed with the aim of awaken the
regulator. The digital regulator provides power to the digital µC just in case enough movement is detected. This mode of work
system (microcontroller, inertial sensors I/O, µSD Card, and allows saving much power analyzing data of weak importance
USB interface). The analog regulator supplies power to the such a static movements (sit, stand or lying).
inertial sensors, being separated and isolated from the rest of the The MEMS pressure sensor is a barometric sensor. The main
circuit by means of an own ground plane. The comm regulator feature is the RMS noise, which is 0.02mbar. It is considered
supplies voltage to the Bluetooth module. This device can that the traveled distance of the trunk in a sit to stand (SiSt) or
consume more than 40mA alone, for this reason it should be stand to sit (StSi) posture transitions is 0.53m, which is
isolated from the rest of the circuit in order to avoid peaks of considered to be a difference of 6.1Pa or 0.061mbar [11]. This
currents that affects the voltage stability at the analog or digital means that according to LPS331AP’s RMS noise minimum
system. Finally, the backup regulator keeps a regulated voltage value, a posture transition (SiSt or StSi) could be detected.
to the µC’s backup system and the real-time clock system. Furthermore, fallings, going up/down stairs or elevators could be
also detected. For this reason, the MEMS pressure sensor is an
3.1 BSK Microcontroller interesting tool to be added at any human activity recognition
The microcontroller that manages the internal processes from the system.
BSK board is a STM32F415 from STMicroelectronics. This
microcontroller is a Cortex™-M4 CPU with floating point unit, The LPS331AP offers an ODR up to 25Hz, although the
which lets computing advanced online algorithms. The maximum minimum RMS noise can only be achieved with 12.5Hz.
speed of this device is 168 MHz and contains 1MB flash memory According to Zhou et al. this is a low frequency to catch all
and 192KB of RAM (compared to the 128KB and 16KB, human movements [12], however, and having into account that
respectively of the 9x2’s µC). The BSK board contains two posture transitions [13] and walking bands are below this
external clocks, one to run the internal oscillator circuit and a frequency [14], 12.5Hz is enough to identify all these activities.
Real-Time Clock to count seconds with high accuracy. One of the
main features of this µC is the Direct Memory Access (DMA),
which is able to exchange data among the different peripherals
and between the peripherals and the µC. Finally, the STM32F4
includes up to 15 communication interfaces among which UART,
SPI, I2C, SDIO and USB 2.0 full-speed On-The-Go controller are
of main importance. The BSK µC is a 64-pin device with debug
mode and is able to enter in different low-power modes in order

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 250 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro

Figure 4. Layout of the cameras on the basketball court

Figure 3. BSK board inertial MEMS (LSM9DS0)
At the beginning and at the end of each user data collection, a
visual event is performed in order to synchronize cameras and
3.3 Communication Component inertial systems. Then, after data collection data integrity is
The BSK board has been designed to allow downloading the data checked, inertial signal is synchronized with video signal and,
through USB connection (i.e. without removing any pieces of the then, labeled according to recorded video. We propose two
device). Data is stored in a µSD card by means of the SDIO options:
interface and with FAT32 system file format allowing storing
much more data than the 2GB allowed by FAT16. The device • Sensor falling: When the device falls, the accelerometer
contains a specific hinge socket where the µSD card is inserted. generates a peak and change of axes. However, this step
This socket does not allow the µSD card to move in aggressive must be performed before attaching the sensors to human
execution tasks such as sprinting or jumping, avoiding then body. Thus, there is a significant amount of unusable signals
communication errors. in the recorded data.
• Jumping action: Similarly, when the subject jumps, a
The Full-Speed On-the-Go USB system has been incorporated to sudden change appears in acceleration signals. The
the BSK board including a USB buffer device, which filters the annotator can detect that pattern and match it with the
noise and allows to isolate electromagnetically the BSK USB corresponding jump in videos. One limitation is that we are
circuitry from PC circuitry in order to ensure a robust considered jumping as a class in our activity recognition
communication between the two devices. method. Hence, the annotator should be careful to select the
right jump (usually at the beginning of each data collection
4. DATA COLLECTION session)
We employed the BSK board with sampling frequency 200Hz to
collect data from accelerometer, gyroscope, magnetometer,
temperature and barometric sensor. The five sensors were 5. ACTIVITY RECOGNITION
attached to body as shown in Figure 5. The data collection plan is We aim to classify basic actions in basketball listed in the
required to contain the activities-of-interest in a reasonable order previous section. Although the five sensors have been used to
so that it is feasible for the subjects to perform the activities (i.e. capture data, only data recorded by the two accelerometers on
not feeling exhausted). Our data collection plan is designed with player’s feet has been used for machine learning purposes. Our
the consultation of sport experts and amateur basketball players. activity recognition method has five steps: (i) preprocessing, (ii)
segmentation, (iii) feature extraction, (iv) standing – moving
The test protocol consists of nine activities executed
separation, and (v) moving activity recognition (Figure 6).
continuously, including a jump at the beginning and at the end of
each activity and a standing between the different series. We
have performed the following activities: walking, running,
jogging, pivot, shooting from different locations, layupshot,
sprinting. We also use one more label, undefined, to annotate
irrelevant movements. The dataset activities are repeated to
obtain variability in the data. The test is video-recorded with two
cameras to facilitate the annotation of the data from the wearable
sensors (Figure 4).

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 251 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro

Figure 5. BSK board model and sensors set up

Figure 6. Our proposed activity recognition algorithm
First, preprocessing is performed on inertial signals, including
downsampling and filtering. We downsample the acceleration
signals from 200Hz to 40Hz to balance recognition accuracy and
computational cost [12]. Afterwards, aberrant errors due to
communication errors have been eliminated and signals are then
passed through a low-pass filter with 15Hz as the cut-off
frequency and divided into equal segments with 50% overlap.
Features are extracted from each segment
In each segment of inertial data, we extract features from the
three axes for each accelerometer. The root mean square value
for each accelerometer is calculated. Thus, in each sample of
each accelerometer, four values (three raw values and one
amplitude value) are obtained. From each of them, time and
frequency domain features are extracted, including: range, sum,
mean, standard variation, mean crossing rate, skewness,
kurtosis, frequency bands, energy, and number of peaks above a Figure 7. Actual and predicted labels of standing and
threshold. Moreover, correlation between three axes of each moving. The blue plot is the range value of Z-axis
sensor and each pair of axes on two sensors are also considered. acceleration (best view in color).
In the third step, we use a simple feature to separate standing 6. EXPERIMENTAL RESULTS
from moving activities based on the range values of acceleration We set the window length to 128 raw samples and the
on the Z-axis. In each segment, the difference between maximum overlapping percentage to 50%. The duration of each segment is
and minimum values of acceleration, which is called range, is 3.2 seconds. Then, we split the samples into training and testing
calculated. If the value is higher than an optimized threshold k, datasets. Our dataset contains inertial signals of three subjects.
we classify the segment as moving; otherwise, it is a standing Thus, in the cross-person setting, we use data from two players
segment. To estimate this threshold, we trained a linear SVM for training and one for testing. The final result is the average
classifier. Based on range values, this classifier can discriminate measures from all splitting configurations of subjects. This can
two classes: standing and moving. Then, we visualize the be considered as leave-one-subject-out evaluation strategy. We
classification result and select the value that achieves the highest use LibSVM library [15] to implement the multi-class SVM
accuracy. Thus, it can be modified for different datasets. classifier.
The remaining non-standing segments are used to recognize In Figure 7, the actual and predicted labels of standing and
moving activities. Each segment of inertial signals becomes a moving of one player are depicted. Most of confusion appears
feature vector or sample of the classifier. We feed them to the when the player performs the pivot action. In this action, one leg
classification algorithm for training and testing in two cases, of the player is kept stable and the other can moves. Thus, it
which are called same-person and cross-person. In the former, generates similar signals to standing in one foot. The threshold
we randomly select samples for training and testing from the for separating standing and moving is k = 4 and the accuracy is
same player. In the later, we train the recognition algorithm on a 92%.
group of players and test it on the other one. In both cases,
training and testing datasets are different. However, the latter is Then, the moving activities are divided into two sets, namely
more challenging because players with distinct physical step and jump activities. The step-related activities include
characteristics generate different inertial data, even when they walking, jogging, running, and sprinting while the jump-related
perform the same activities. activities contain jumping, layupshot, and jumpshot. Figure 8
shows the confusion matrices in both evaluation cases. It is

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 252 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro
possible to clearly discriminate step-related activities on the applicability of our proposed system within basketball activity
same player. Nevertheless, due to distinction in physical monitoring. We plan to add new features from accelerometer and
characteristics, two players perform the same activities in include the gyroscope information to enhance the performance in
different ways. Therefore, in the cross-person evaluation setting, activity recognition. In addition to this, a new dataset with more
confusion appears in both step-related jump-related activities and participants and, thus, with more heterogeneity of physical
the precision decreases. In the case of jump-related activities, all characteristics will be acquired.
of them include a jumping action. Consequently, they produce
similar inertial data. 8. ACKNOWLEDGMENTS
This work was supported in part by the Erasmus Mundus Joint
Doctorate Programme in Interactive and Cognitive Environments.
Authors also would like to thank Dr. J.M. Padullés from Institut
Nacional d’Educació Física de Catalunya for his valuable
assistance in the design of the data collection plan.

9. REFERENCES
[1] Aggarwal, J., Ryoo, M.: Human Activity Analysis: A
Review ACM Comput. Surv., ACM, 2011.
[2] Bulling, A., Blanke, U., Schiele, B. A.: Tutorial on Human
Activity Recognition Using Body-worn Inertial Sensors.
ACM Computing Surveys, 2014.
[3] Lara, O., Labrador, M. A .: Survey on Human Activity
Recognition using Wearable Sensors. Communications
Figure 8. Confusion matrices of activity recognition in two
sets (step-related and jump-related) Surveys Tutorials, IEEE, 2013.
[4] Kelly, D., Coughlan, G. F., Green, B. S., Caulfield, B.:
Automatic detection of collisions in elite level rugby union
using a wearable sensing device. Sports Engineering, 2012.
[5] Morris, D., Saponas, T. S., Guillory, A., Kelner, I.: RecoFit:
Using a Wearable Sensor to Find, Recognize, and Count
Repetitive Exercises. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems,
ACM, 2014.
[6] Bächlin, M., Förster, K., Tröster, G.: SwimMaster: A
Wearable Assistant for Swimmer. Proceedings of the 11th
Figure 9. Confusion matrix produced by our recognition International Conference on Ubiquitous Computing, ACM,
algorithm on basic basketball activities 2009.

Figure 9 illustrates the performance of our activity recognition [7] Harle, R., Taherian, S., Pias, M., Coulouris, G., Hopper, A.,
algorithm on all activities (walking, jogging, running, sprinting, Cameron, J., Lasenby, J., Kuntze, G., Bezodis, I., Irwin, G.,
jumping, jumpshot, layupshot and pivot) in cross-person setting. Kerwin, D.: Towards real-time profiling of sprints using
Similar confusion is revealed when we train and test the wearable pressure sensors. Computer Communications,
classification model on different players. Furthermore, when 2012.
performing layupshot, the player dribbles then throws the ball to [8] Ladha, C., Hammerla, N. Y., Olivier, P., Plötz, T.:
the basket. Dribbling (i.e. running with ball), generates ClimbAX: Skill Assessment for Climbing Enthusiasts.
acceleration signals which are similar to those in jogging or Proceedings of the 2013 ACM International Joint
running. This motivates to integrate sensors in other body parts Conference on Pervasive and Ubiquitous Computing, ACM,
(e.g. on the wrists) to distinguish these activities. 2013.
[9] Velloso, E.; Bulling, A., Gellersen, H., Ugulino, W., Fuks,
7. CONCLUSION H.: Qualitative Activity Recognition of Weight Lifting
In the paper, a new sensing system to record and recognize
Exercises. Proceedings of the 4th Augmented Human
human movements in basketball has been presented. The new
International Conference, ACM, 2013.
inertial measurement units are capable to record human
movements at sport with high accelerometer ranges. The system [10] Ghasemzadeh, H., Jafari, R.: Body sensor networks for
has been employed to collect and analyze motion data in order to baseball swing training: Coordination analysis of human
recognize basic activities in basketball. The proposed method is movements using motion transcripts. Pervasive Computing
able to identify four moving types at different intensity (walking, and Communications Workshops (PERCOM Workshops),
jogging, running, and sprinting), as well as to discriminate 2010 8th IEEE International Conference on, 2010.
shooting executions. Moreover, the promising results prove the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 253 |
 Basketball Activity Recognition using Wearable Inertial Measurement Units
L. N. Ngu Nguyen | C. Pérez-López | D. Rodríguez-Martín | A. Samà | A. Català | A. Cavallaro
[11] Massé, F., Bourke, K., Chardonnens, J., Paraschiv-Ionescu, miniature gyroscope and its application in fall risk
A., Aminian, K.: Suitability of commercial barometric evaluation in the elderly. IEEE Trans. Biomed. Eng. 49,
pressure sensors to distinguish sitting and standing activities 843–851, 2002.
for wearable monitoring. Med. Eng. Phys. 36, 739–44, [14] Barralon, P., Vuillerme, N., Noury, N.: Walk detection with
2014. a kinematic sensor: frequency and wavelet comparison. 28th
[12] Zhou, S., Shan, Q., Fei, F., Li, W.J., Wu, P.C.K., Meng, B., Annu. Int. Conf. IEEE EMBS. 1, 1711–1714, 2006.
Chan, C.K.H., Liou, J.Y.J.: Gesture recognition for
[15] Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support
interactive controllers using MEMS motion sensors. 2009
vector machines. ACM Transactions on Intelligent Systems
4th IEEE International Conference on Nano/Micro
and Technology, 2011.
Engineered and Molecular Systems. pp. 935–940, 2009.
[13] Najafi, B., Aminian, K., Loew, F., Blanc, Y., Robert, P.A.:
Measurement of stand-sit and sit-stand transitions using a

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 254 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno

EMG-based biofeedback tool for augmenting manual

fabrication and improved exchange of empirical
knowledge
Guillermo Bernal Dishaan Ahuja Federico Casalegno
MIT Design Lab Massachusetts Institute of MIT Mobile Experience Lab
20 Ames Street Technology 20 Ames Street
Cambridge, MA, USA 77 Massachusetts Ave.
Cambridge, MA, USA Cambridge, MA, USA
[email protected]
[email protected] [email protected]

ABSTRACT centering. This research is part of a wider phenomenon of

It can be time-consuming and frustrating to acquire any new innovation in the process of teaching. Some contemporary
skill, and for one that relies on muscle memory developed examples are MOOCs [13] and the popular video game `Guitar
through observation and repetition it may require hours of Hero' [10]. Our system is more similar to the latter - we provide
supervised training to reach even minimal proficiency. This the user with real-time feedback based on performance of certain
paper explores whether real-time feedback that compares data tasks involving a test of motor skills. It is widely known that
from bio-signals and physical movements of a novice learner and many years of experience may be required before muscle memory
an expert can shorten the learning process via a wearable device. has advanced to the stage where one can be called an expert at a
These signals include measurements of muscle activity using motor skill. In the book `Outliers' [7], it says that it takes roughly
electromyography (EMG) and from sensors that include ten thousand hours of practice to achieve mastery in a field. The
accelerometers, gyroscopes, and magnetometers. The signals are author also claims that most people quit the learning process in
in the form of sets of illuminated RGB LEDs; the learner the first two thousand hours due to the steep nature of the
receives instantaneous performance evaluation that enables learning curve at the start of the knowledge acquisition process.
immediate realization of an error and allows for rapid and easy It is at this stage that our system is most valuable to the learner,
adjustment of movement. Preliminary tests using the wearable and can reduce the time spent on acquiring new muscle memory.
device in pottery making show effectiveness at aiding students to Imitating an instructor is traditionally at the center of methods of
master movements more quickly than on average. Hence, the learning motor skills in sports and crafts. Learners typically
wearable device aids the user in acquiring muscle memory. imitate based on visual clues about speed and timing, the
positions and stances of different parts of their bodies, and
Categories and Subject Descriptors various other elements involved in mastering the skill. Watching
H.5.3 [Interaction, Learning and Teaching]: User Interfaces; someone else, though, leaves little room for catching the
B.4.2 [Input/Output Devices] subtleties of an instructor's movement in real time or
remembering them later. Our system overcomes this limitation
by integrating real-time feedback that shows how the learner's
General Terms motion characteristics and those of the instructor correspond to
Adaptability, Interactive and Cognitive Environments each other, providing a vivid, precise, and concrete way to
compare and contrast movements. This makes it possible for
Keywords learners to align their movements with those of the expert
instructor without having to rely completely on memory.
Haptic knowledge, craft, knowledge sharing, wearable ACM
Classification Keywords H.5.2. Information Interfaces and Many movements have subtleties that cannot be seen by a
Presentation: User interfaces and Evaluation; User Interfaces; learner, only felt. Hence, our system provides a real sense to the
Haptic I/O. General Terms Human Factors; Design; learner, for example how much pressure the expert uses for a
Measurement. particular task and whether, for example, the movement requires
activation of the triceps rather than the bicep. Sports scientists
who have measured such activity using electromyography [22]
1. INTRODUCTION have found quantifiable differences in the EMG activity between
This paper explores the process of motor skill acquisition, with a novice and an expert [2]. Our system therefore augments the
the goal of augmenting the traditional, observation-based standard method of learning through feedback by matching EMG
approach to learning motor skills with a biofeedback loop. The signals of a learner and instructor and displaying this feedback to
effectiveness of this augmentation is explored by conducting user the learner in real time.
studies where novices attempt to use both the traditional method
and our augmented learning system to learn the skill of clay

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 255 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno

2. CONTRIBUTIONS was much closer to correct than in the other group, indicating
This research's main contribution is a system that augments the that the feedback led to the acquisition of proper dribbling
process through which learners acquire muscle memory related to technique. This confirmed the researchers' hypothesis that real-
a particular motor skill. The system first measures motion data time EMG-based feedback is an effective addition to the process
and EMG signals from an expert performing a movement, of motor skill learning.
measures the same data from the learner, and then provides real-
time feedback via a wearable device that can help alter incorrect
4. RESEARCH METHOD IN BRIEF
To develop our project, we first did some initial research on how
movements or reassure the user when movements are correct.
the human body generates natural electric signals and how these
As the clay centering experiment discussed in this paper shows, signals are evaluated in the medical field. The data generates
real-time feedback aids fast and easy aids acquisition of a motor vast, unorganized amounts of data, so we next created a circuit
skill by informing a learner of a mistake immediately and that would enable us to collect data for later organization and
providing the opportunity to iterate until it is corrected. The analysis.
apparatus we describe could contribute to learning a variety of
Having determined that a wearable device would be the most
skills, for instance in sports that involve specific movements such
appropriate medium through which to collect data from body
as tennis (serving), golf (swinging), and basketball (dribbling).
movements, we developed a prototype that fit close to the body.
Our design focused on the ability to monitor specific body
3. RELATED WORK movements and muscle groups.
In our research, we rely on concepts of sensory motor learning
that expand our perceptual framework. For example, expert Next, having decided that the craft of ceramics would be a good
video gamers develop an extraordinary ability to extract domain within which to test our hypothesis, we interviewed a
information and extend their attention over a wider-than-normal master ceramics craftsman to learn how certain tasks within the
spatial frame while experiencing no apparent decrease in craft are habitually performed. We recorded measurements of
attentiveness [8]. Notably, this integration process can account muscular movement when performing these tasks from two
for the learning opportunities provided by external objects such master craftsmen, and also employed EMG to capture bio-
as tools, which means optimal visuo-haptic integration is signals. This data became the data against which we would later
possible even when tactile input comes through a hand-held tool test and compare data generated by our experiment's learner
[18]. subjects.
Researchers have highlighted motor learning processes and the
importance of the type of information that is being learned, rather
than how it is being learned. Wolpert, Diedrichsen, and Flanagan
further explain in their ‘Principles of Sensorimotor Learning’:
“Learning can occur at different levels of the motor hierarchy.
To understand how these changes are implemented in neurons,
we need theories of the processes and representations through
which learning is achieved. The processes of motor learning can
be distinguished by the type of information that the motor system
uses as a learning signal. Although different sensory modalities,
such as vision, proprioception and touch can all play an
important part in motor learning, we focus here on the nature of
the information, independent of modality, that is used during
learning.”[21]
In sports today, this theory has proven to be correct as it has
proven to be beneficial to first determine which skills need to be
improved upon, and secondarily determine how to best teach the
players and other members of the team how to acquire said
skills. EMG-based feedback learning systems have been used for
basketball dribbling training, taking advantage of the differences
in sharpness and timing between EMG signals from a novice and
an expert. Researchers compared the differences in EMG signals
between an expert and a learner and provided the learner with a
visualization highlighting the differences [2].
In this particular experiment, six basketball novices were divided
into two groups of three, only one of which received visual EMG
feedback. Both groups were given dribbling technique instruction
and then trained for 30 minutes, using a metronome to calibrate
dribble period. Performance tests then determined that in the
group that received EMG feedback, the forearm muscle activity Figure 10. Sleeve prototype.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 256 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno
We next developed a computer program to enable the real-time bicep, wrist flexors, and forearm flexors. These patches
comparison and analysis of the data. The program produces the have snap connections that allow the electrodes to snap
appropriate visualizations that allow users to perceive differences onto the sleeve while maintaining a conductive
between the two data sets, which is a prerequisite for connection.
establishing a useful feedback loop. 3. At the top of the bicep near the should joint, the
Needing a group of participants to test our program with the electrodes connect back to the EMG Band, which
wearable device, we found ideal subjects who had very little slides over the sleeve and tightens down with a simple
knowledge and experience working with their hands. We had flexible fabric band similar to an armband one might
these participants work with clay for a limited time while use to secure an MP3 player while jogging. The
wearing the device, performing the same tasks. We collected a program is loaded in this EMG band and the
mean range of data from this activity in sessions approximately 1 computation occurs using a Teensy micro-controller.
hour in duration, with small breaks every 25 minutes. 4. The Glance-Based display at the wrist, on a strap made
In our final step, we inferred a set of conclusions based on the of 3D printed flexible plastic, allows users to visualize
data analysis. Our objective was to determine whether the the data produced by their actions. Glance-Based
wearable device had a significant impact on the speed with interfaces do not demand a user's focal attention,
which the learners in our experiment familiarized themselves providing visualization to the eyes quickly and only
with the technique and the specific tasks of the craft. when needed [3]. An array of LEDs is concealed within
the strap's three narrow slots and therefore the LEDs
4.1 Hardware & Sleeve Prototypes are visible to the subject when illuminated. (Figure 3)
Several researchers have explored the design space of wrist-worn
devices. For example, Augmented Forearm [14] is made from a
row of four interconnected displays worn on the forearm.
DisplaySkin [4] wraps six displays around the wrist, acting as a
visual display for mobile device information. The focus is to
describe the apparatus (hardware components, integration, and
real-time software implementation) as well as some of the
experiments possible under the system architecture. We present
experimental measurements for illustration that compare single
muscle work-loops under zero and non-zero admittance loads, as
well as methods for conducting experiments on controlled and
uncontrolled environments. The primary goal is to provide a
supervised input to the person learning a task in an attempt to
prove the hypothesis that biofeedback signal representation is
suitable for learning. We based our hypothesis on the work done
by Debaere et al [6] who over the past few decades have shown
how providing augmented visual feedback also facilitates Figure 11. LEDs Color interface chart.
learning bimanual coordination patterns [12], [16], [6], [15] and 5. The LEDs on the visual display of the wrist serve as
gives rise to complex multisensory integration mechanisms [17]. the interface between the apparatus and the user;
the LED strips run vertically along the arm (see
4.2 Device Components Figure 6 for clarification on the orientation of the
1. Electrodes provide information to the myoelectric LED array) and measure both the position of the
sensor board. It also has several electronic components forearm as well as the speed of movement when
that allow for adjusting the input and for filtering any performing a given task.
incoming noise, enabling a clear signal quickly and LED Strip 1 – Progress: When performing the specific task, the
thus eliminating the processing load on the plate board. first of the LED strips that is closest to the thumb (worn on
The design of analog filters and amplifications are the right hand) measures accuracy on matches for the
derived from standard EMG practice [11]. The position of the forearm, speed of movement, and the EMG
amplified myoelectric sensor data are connected to the data. The number of lit LEDs increases with proper
analog inputs of the micro-controller Cortex-M4 movement. The LEDs change color to show progress,
Teensy 3.1 [19] which samples the analog inputs at beginning with a purple light and proceeding to blue, yellow,
approximately 10kHz and sends out RMS values to the and then to green to show mastery of the skill. Clearing a
on-board SD card for each sensor with a window size color bar means the user has matched o r s u r p a s s e d the
of 32 values at a rate of 63Hz. This reduces both the data block set from the master recording.
amount of data to be transferred and the required
processing power. LED Strip 2 – EMG Amplitude: The middle strip of LEDs
measures the EMG signals the user's body produces during the
2. A compression sleeve is fitted over the arm of the task. These lights increase in quantity and change color to
participant, from shoulder to wrist of the dominant indicate progress, moving from yellow to orange and then to red
hand. The sleeve features small patches of conductive (signifying the highest level of muscle contraction) to show
fabric placed strategically over the general areas of the improvement.
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)
| 257 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno
LED Strip 3 – Orientation: The final strip of LEDs at the pinky then provide a baseline against which all other experiments can
side of the display compares the current position and EMG be compared.
signals of the forearm to those of the control data. This strip We employed a 12-camera VICON motion-capture system [20] to
operates slightly differently from the other two in that the lights track markers (9 mm in diameter) at a rate of 120 frames/second.
do not progress but instead remain green when the participant's Marker tracking was accurate within 1.5 mm. Markers were
arm position is considered to be 'correct' and turns red when placed on subjects as follows (Figure 3): four on the head, left,
'incorrect.' and right shoulder; one each on the center of the clavicle, the C7
area of the spine, right scapula, left and right elbow mid and
5. EXPERIMENT OVERVIEW ulna, left and right upper arm, left and right lateral side of
In the experiment we recorded how master ceramics craftsmen forearm, left and right radius and ulna, left and right thumb, and
perform a particular task and then compared how closely a left and right middle finger.
learner could replicate the master's force, position, and speed. By
providing the comparison to the learners, they can determine
which alterations they need to make to their own motions to
replicate the master’s techniques more closely.
he learners performed the given tasks during 25-minute intervals
The learners performed the given tasks during 25-minute
intervals and did not find the sleeve and apparatus to be
uncomfortable or fatiguing. The compression sleeve allowed us
to keep the electrical components tight to the body to prevent any
issues with balance. In total, the sleeve and electrical
components weighed 58 grams.
To ensure accurate data from both sources, we created a
controlled environment suitable for work with clay. We set up
Figure 12. Tracker Placement on users
video equipment in the space to record important motion-tracking
information that we could digitize and convert into visual Both analog and digital devices can be connected to the Vicon
simulations. We also used EMG monitoring devices to determine System mainframe, just as a wireless mouse can be connected to
which muscles both the master and the learners were using most a computer. Such versatility allowed continuous storage
frequently. capabilities for 16 Trigno sensors (Delsys's EMG wireless
system) operating at a full bandwidth, with 3 DOF accelerometer
5.1 Master Craftsmen and Subjects data per sensor. Once all sensors were placed on the body, the
We engaged two master craftsmen for the experiments. Master subject could move into the space where the group of calibrated
craftsman 1 is a pottery instructor, aged mid- to late-30s, with infrared cameras was ready to begin collecting motion-tracking
more than a decade's experience in the craft and more than 6 data.
years of experience instructing new students in pottery making. The first users were master craftsmen, so we could collect data
Master craftsman 1's artwork has been displayed in museums from their movements. We asked them to perform the initial
throughout the United States and abroad. actions when working on the potter's wheel for extended periods,
Master craftsman 2, aged mid-30s, has close to 20 years' which allowed us to identify specific patterns for later use. We
experience working with clay, having first apprenticed for many made a total of five recordings, which produced an interesting
years with experienced pottery makers in Japan and India. For result of the translation to the digital space of materiality. By
the past few years, master craftsman 2 has been teaching at a tracking the motions of the hands while working with clay, a
university in Cambridge, Massachusetts. translation of materiality was embedded into the 3D data.
To establish the other end of the control group, we had someone
as a second user (Subject 2) with no pottery experience making
Subjects in the experiments were all students from our university
things by hand. Subject 2 began with a 25-minute demonstration
between ages 18 and 30, selected because students in this age
of the sort one would receive in an introductory pottery course; it
group typically have a close relationship with technology and
emphasized centering the clay on the wheel and coning the clay
thus were less likely to face certain technological challenges that
up and down (thus the demonstration corresponded with this
might have adversely affected the conduct of the experiment.
paper's focus). The subject was also shown how to create a hole
5.2 Control Group: Experiment 1 in the clay's center, open the hole, and make a cylinder. The
explicit information provided by the master craftsman in the
To test the system we proposed in experiment 1, it was crucial to demonstration included key actions to which the subject should
establish a control group and to collect data in a state-of-the-art pay close attention. During the experiment, the master craftsman
facility with proper equipment. By being able to compare data also provided verbal instructions to help Subject 2.Figure 4
collected with the system prototyped for this study from such a shows the representation model of the two subjects while
facility, one can begin to gauge results. Conducting the initial working on the wheel. The raw EMG data are shown as a series
experiment in an ideal environment provided an opportunity to of plotted graphs, each representing a channel, with the length of
record data with the lowest level of noise possible, which will the plot related to the recording's duration. Evidently, there is a
difference between the master's EMG signal on the left and the
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)
| 258 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno
beginner's on the right, which show clearly the contrast between whereas those of the beginner oscillate continuously, showing
the moment of muscular contraction and muscular extension. The nervousness and a lack of knowledge of the technique.
EMG signals from the master show bursts only at intended times,

Figure 13. Master and novice MoCap and EMG visualization

the visual display (i.e, we did not use EMG data), which enabled
5.3 Technique Comparison: Experiment 2 us to isolate the sensors' measuring of arm position over time and
The second experiment tested the EMG sleeve and the bio-
produce a speed and trajectory of the arm's movements -
feedback LED display for accuracy and effectiveness in assisting
information displayed through the first row of LEDs. The test
the user to perform a desired task. Some subjects had no visual
subject was asked to perform the same motion once to compare
feedback from LEDs and had to rely exclusively on the
motions to the previous test run, but multiple test runs became
indications from the instructor to perform the task. Others had to
necessary because the subject was unable to match the control
wear the components while performing the task, comparing the
data set with ease. Despite the subject's belief that the same
two different sets of data afterwards. All participants were
motion was being reproduced, the LED display showed a
recorded to track their motions using the sensors placed on their
different evaluation of the movement. The subject required time
bodies. The ability to track the participants' motions provided an
and effort to slow motions and focus on the display, which
opportunity to compare those wearing and not wearing the device
enabled the arm adjustments needed to match the previous test.
and determine whether the visual feedback had an effect on the
We determined after a few trials that a gradual increase in the
user's technique. Figure 5 shows a direct comparison of a master
accuracy threshold would give the subject greater leniency in
craftsman and a novice performing the same task: creating a
motion and produce a positive result.
cylinder on the pottery wheel. It is clear that the master
craftsman on the left has achieved a high comfort level with the
medium, as seen by the tall, rigid walls of the cylinder. The
student on the right has created an uneven and incomplete
cylinder, illustrating a lack of familiarity with the material and
technique.

Figure 15. Window and Envelop filters

Figure 14. Expert and Novice pulling clay walls

5.3.1 Evaluating the Sensitivity of Sensors 5.3.2 Evaluating System Efficiency

The first test using the devices focused on a simple movement, For the second preliminary test, we asked the subject to wear all
which the subject performed and then had to match to evaluate three components so we could test the device in totality. This
sensor accuracy (Figure 6). It involved the subject wearing only involved a few different experiments with the subject to test the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 259 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno
posture and force applied by to the material. The subject crushed participants adjust their technique to meet the readings of the
a rubber ball using one hand, and we used the electromyography control data set.
to monitor muscular contraction and expansion. The subject then As we developed the feedback display software, it became
traced the outline of a lamp and we monitored the speed and apparent that the strip of LEDs on the pinky side of the wrist
accuracy of movements over time; for the latter, the subject (worn on the right hand) performed extremely well in monitoring
attempted to match a recorded data control set. forearm position. Users could easily tell when their arm angle
differed from that of the control data because of the change in the
LED display from a full strip of green light to a single, small red
light. A slight elbow or shoulder adjustment would correct
participants' posture immediately, changing the red lights back to
green. Obvious shifts from negative to positive feedback made
clear to users that their movements did not match those from the
master craftsman data.
Figure 16. Evaluating sensor sensitivity through repetition
Recording the movements of both the master craftsmen and
5.4 Pilot with Ceramicist: Experiment 3 learners using motion tracking sensors made clear that a
Gauging the accuracy of the EMG sleeve and LED display with combination of motion tracking video and EMG feedback
the pottery wheel became crucial to our experiment's success, so provides an extremely accurate digital representation of the craft
we had the subject perform some initial calibration routines and makes it possible to understand movements without the need
derived from tests mentioned earlier in this paper. As in to watch the action in real time. These data could be used to
Experiment 1, the master craftsman showed each subject how to create a three-dimensional learning environment or an archive of
perform some simple pottery techniques, which the subject then movements made by masters of a particular craft. Digital
tried to replicate. We had to adjust program parameters to get the recordings of both motion tracking and EMG would allow remote
EMG signals to match and increase the threshold that assigns a teaching from anywhere.
numerical value to the degree of accuracy so the subject could
adapt and understand how the system works. The initial goal of 7. IMPROVING THE DEVICE
subject accuracy within 10 degrees of the master's movements At present, our work focuses on improving EMG signal
proved unachievable, but increasing the threshold to 25-percent acquisition and employing more advanced digital signal
accuracy made it possible for the subject to evaluate mistakes processing techniques before comparisons are conducted.
and make corrections with the aid of the LED signals. All Improving the feedback system for learners constitutes an
subjects were given the typical length of a pottery class, one important future area of work.
hour, to perform the experiment. The custom hardware from which EMG signals are acquired,
The master craftsmen were extremely deft at centering the clay processed, and compared works by using an on-board
on the wheel, as would be expected. Master craftsman 1 needed microcontroller. Feedback is provided through colored LEDs.
only 4 minutes and master craftsman 2 only 7 minutes (see This setup, though, may be inappropriate for some physical tasks,
Figure 8). Novice subjects all required the entire 60 minutes and the on-board microcontroller seems to be a limiting element
without feedback, but once they used the biofeedback sleeve they of the system for advanced digital signal processing in real time.
all were able to reduce the time significantly and complete the The user faces some challenges in looking at the LED feedback
clay-centering task. One subject who had been unable to center signals while moving. So, we are looking for ways to decouple
the clay during the first 60-minute period succeeded in 12 the various functions of the current hardware and thus overcome
minutes with the biofeedback these challenges.

6. RESULTS 7.1 Signal acquisition improvements

The preliminary experiments demonstrated the considerable ease We are considering the Myo armband [1] for the task of
with which participants could become familiar with the acquiring and communicating raw EMG signal data, as well as
movements involved in the task at hand without paying close data regarding physical movement such as pose, orientation, and
attention to the visual display; the participants felt they were velocity. This would replace the custom EMG sensor device.
performing the task correctly even when different results were Several characteristics of the Myo are particularly attractive in
displayed on the device. Refocusing on the visual display helped this regard: it is inexpensive, lightweight, and sufficiently
accurate, while also interacting easily with the device using a.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 260 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno

Figure 17. Task completion times of masters and novices both with and without bio-feedback.

number of popular programming languages. The latter allows us

to use a language such as Python, which is particularly well
suited to digital signal processing. We are also exploring
modifying the device so it can be used with muscles other than
those in the arm.

7.2 Signal processing improvements

The EMG signal reported by the Myo undergoes some basic
processing. Currently, this is used mainly for the purposes of
smoothing in the existing implementation. The processing
includes removal of DC offset by de-trending, rectification by
taking absolute value, and smoothing using a low-pass filter. The Figure 10: Processed emg while touching thumb to
specifics of the low-pass filter vary based on the source and middle finger
characteristics of the data. For example, a fifth-order Butterworth
filter with a cutoff frequency of about 1% of the sampling rate
7.3 FUTURE WORK
From our system and from previous research [2] it has been seen
has worked well for data collected using the custom hardware
that a movement that is part of a craft or a sport can be well
described previously in the paper. Figures 9 and 10 show the raw
described using data on motion characteristics and EMG activity.
and processed EMG signals across the same time period when
One possible avenue of further exploration is the generalization
the subject touches the thumb to the middle finger several times.
of the bio-signal comparison and real-time feedback system. This
The current developments in this area include normalizing the
means that a user could access a portal containing files that
signal against the peak or mean values of a subject for purposes
describe a particular motor skill in terms of the bio-signals
of easier comparison among subjects across trials, and if deemed
mentioned previously. A user could interact with this portal by
necessary, addition of pre-processing to remove noise from the
choosing a motor skill to learn, and an instructor whose
raw EMG signal.
movement the user wishes to learn from. The portal would then
interact with a wearable device that is located at an appropriate
location to measure the learner's EMG signals. Feedback would
be provided based on a comparison between the learner's EMG
signals and those of the chosen instructor, after undergoing some
processing [5] and normalization [9]. This could include visual
feedback in the form of LEDs, overlaid graphs, or words.
Another possibility is haptic feedback via the wearable device,
perhaps to nudge the user in a certain direction or to provide
immediately understandable status feedback. This portal would
allow a user to more quickly and easily learn many motor skills
involving muscle memory acquisition, including clay-centering,
Figure 9: Raw emg while touching thumb to middle finger basketball dribbling, swinging a golf club or tennis racquet, or
throwing or pitching a ball. This paper's contribution is an early
version of this system, minus integration with an online portal.
Our current system can thus be seen as a proof of concept of such
XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)
| 261 |
 EMG-based biofeedback tool for augmenting manual fabrication and improved exchange of empirical knowledge
G. Bernal | D. Ahuja | F. Casalegno
an online learning portal to augment the traditional in-class [7] M. Gladwell. Outliers: The story of success. Hachette UK,
learning experience in many fields. 2008.
[8] C. S. Green and D. Bavelier. Action video game modi_es
8. CONCLUSION visual selective attention. Nature, 423(6939):534{537, 2003.
This paper presents a system aimed at easing and hastening the
process through which one acquires muscle memory. The method [9] M. Halaki and K. Ginn. Normalization of EMG Signals:
involves collecting signals about pose, velocity, and muscle To Normalize or Not to Normalize and What to Normalize
electrical activity (EMG) from someone who has mastered a to? INTECH Open Access Publisher, 2012.
given skill and then comparing the signals generated by a learner [10] L. Homann. Learning through games. Communications of
of the same skill in real time. The learner receives real-time the ACM, 52(8):21 { 22, 2009.
feedback based on the comparison through colored LEDs on a
wearable device. The results of our initial experiments show that [11] M. Z. Jamal. Signal Acquisition Using Surface EMG and
the system does augment tradition imitation-based learning and Circuit Design Considerations for Robotic Prosthesis.
make it possible for learners to pick up a skill much quicker than INTECH Open Access Publisher, 2012.
those not employing the system. The promising nature of the [12] T. D. Lee, S. P. Swinnen, and S. Verschueren. Relative phase
outcomes compels us to make enhancements to the system, alterations during bimanual skill acquisition. Journal of
including more convenient signal acquisition, more sophisticated motor behavior, 27(3):263{274, 1995.
signal processing, and the most learner-friendly feedback system [13] A. McAuley, B. Stewart, G. Siemens, and D. Cormier. The
possible. mooc model for digital practice. 2010.
9. ACKNOWLEDGEMENT [14] S. Olberding, K. P. Yeo, S. Nanayakkara, and
We thank all our supporting colleagues at the MIT Medial Lab [15] J. Steimle. Augmentedforearm: exploring the design space of
Biomechatronics Group and MIT SAA who have provided us the a display-enhanced forearm. In Proceedings of the 4th
insights and access to the equipment that enabled us to realize Augmented Human International Conference, pages 9{12.
this project. We would also like to thank Caitlin Kunzle and ACM, 2013.
Scott Cooper for helping us editing the paper.
[16] V. Puttemans, N. Wenderoth, and S. P. Swinnen. Changes in
10. REFERENCES brain activation during the acquisition of a multifrequency
[1] Myo armband. bimanual coordination task: from the cognitive stage to
advanced levels of automaticity. The Journal of
[2] S. Abe, T. Nozawa, and T. Kondo. A proposal of emg-based Neuroscience, 25(17):4270{4278, 2005.
training support system for basketball dribbling. In Human
Interface and the Management of Information. Designing [17] R. Ronsse, R. C. Miall, and S. P. Swinnen. Multisensory
Information Environments, pages 459{465. Springer, 2009. integration in dynamical behaviors: maximum likelihood
estimation across bimanual skill learning. The Journal of
[3] A. Amditis, L. Andreone, A. Polychronopoulos, and J. neuroscience, 29(26):8419{8428, 2009.
Engstrom. Design and development of an adaptive integrated 
driver-vehicle interface: overview of the aide project. In [18] S. P. Swinnen, T. D. Lee, S. Verschueren, D. J. Serrien, and
Proceedings of the IFAQ conference, Prague, 2005. H. Bogaerds. Interlimb coordination: Learning and transfer
under dierent feedback conditions. Human movement science,
[4] J. Burstyn, P. Strohmeier, and R. Vertegaal. Displayskin:
16(6):749{785, 1997.
Exploring pose-aware displays on a exible electrophoretic
wristband. In Proceedings of the Ninth International [19] C. Takahashi, J. Diedrichsen, and S. J. Watt. Integration of
Conference on Tangible, Embedded, and Embodied Interaction, vision and haptics during tool use. Journal of vision, 9(6):3,
pages 165{172. ACM, 2015. 2009.
[5] R. H. Chowdhury, M. B. Reaz, M. A. B. M. Ali, A. A. [20] U. Teensy. Development board, 2012. [20] Vicom. Vicon
Bakar, K. Chellappan, and T. G. Chang. Surface system.
electromyography signal processing and classi_cation [21] D. M. Wolpert, J. Diedrichsen, and J. R. Flanagan.
techniques. Sensors, 13(9):12431{12466, 2013. Principles of sensorimotor learning. Nature Reviews
[6] F. Debaere, N. Wenderoth, S. Sunaert, P. Van Hecke, and S. Neuroscience, 12(12):739{751, 2011.
P. Swinnen. Internal vs external generation of movements: [22] G. Zhang. Detection and extraction of surface emg signal
dierential neural pathways involved in bimanual based on action potential sequence. In Applied Mechanics and
coordination performed in the presence or absence of Materials, volume 608, pages 216{220. Trans Tech Publ,
augmented visual feedback. Neuroimage, 19(3):764{776, 2003. 2014.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction Devices (II)

| 262 |
INTERACTION FOR PEOPLE WITH DISABILITIES
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral

Performing universal tasks on the Web: interaction with

digital content by people with intellectual disabilities
Tânia Rocha Maximino Bessa Luís Magalhães Luciana Cabral
University of University of University of Minho University of
Trás-os-Montes and Trás-os-Montes and and INESC TEC Trás-os-Montes and
Alto Douro and Alto Douro and Largo do Paço Alto Douro and
INESC TEC INESC TEC 4704-553Braga CITCEM
Quinta de Prados, Quinta de Prados, Campus da FEUP, Quinta de Prados,
5000-801Vila Real 5000-801Vila Real Rua Dr. Roberto Frias, 5000-801Vila Real
Campus da FEUP, C Campus da FEUP, 4200 - 465 Porto [email protected]
Rua Dr. Roberto Frias, Rua Dr. Roberto Frias, [email protected]
4200 - 465 Porto 4200 - 465 Porto
+351 912620074 +351 939012174
[email protected] [email protected]

ABSTRACT “just a click away”. As Tim Berner-Lee, stated “The power of the
With this study we intent to better understand how a group with Web is in its universality. Access by everyone regardless of
intellectual disabilities interacts with digital content, namely web disability is an essential aspect” [1]. Although, some users see
content, when performing equivalent tasks from their daily school their Web access hampered, in the particular case of people with
activities, such as: painting, making puzzles, playing games. To intellectual disabilities, these barriers are increased, enhancing
accomplish this we observed how a group with intellectual digital exclusion of this group of people [2].
disabilities, without experience using computers, performed This reality was presented to the research team when they were
universal tasks (selection, manipulation and navigation) when challenged to start working with a group of people with
presented with different activities on the Web such as painting, intellectual disabilities. After the first meeting with their teachers,
playing games or searching. We aimed at evaluating usability and we realized that they had many difficulties in motivating their
accessibility and for this we registered the following variables: students to perform school activities as well as choosing which
successful conclusion of activities, type of difficulties found, learning material resources to use with a group of people that
errors, satisfaction, motivation and autonomy indicators. discourage very quickly and present many unfinished tasks. Such
Participants showed motivation and learning skills when aspects lead us therefore to study not only if resources from the
performing all the three universal tasks (selection, manipulation Web can be a viable option to enhance learning for this group of
and navigation) which is confirmed by the number of participants people as well as an option for special education teachers (and
that was able to conclude the activities. Concerning errors, it was possibly a tool leading to the digital inclusion of a group
observed that despite the large number of errors made by the previously excluded).
participants, their motivation lead them to complete the tasks. Thus, this paper presents a study on how a group with intellectual
When handling the input devices the participants had a good disabilities performs universal tasks on the Web (selection,
performance using the computer mouse. On the other hand, they manipulation and navigation). The activities presented to them
could not use the keyboard alone because of their reading/ writing were based on their daily school activities, such as: painting,
difficulties. making puzzles, playing games and word search. These activities
were chosen to: motivate the group to perform daily school
Categories and Subject Descriptors activities on the Web; enhance their abilities when performing
H.5.2 [Information Interfaces and Presentation]: User Interface selection, manipulation and navigation tasks; and train the use of
– Input devices and strategies, Interaction styles, Training, help the computer mouse and keyboard handling.
and documentation.
We observed and took notes of major barriers found in their
General Terms interaction and their potential technological skills (manage to
Human Factors, Performance. complete successfully digital activities using the usual input
devices), errors and difficulties found, when using the Web. For
Keywords this assessment we allied ethnography and usability evaluation
Digital literacy, intellectual disability, learning disabilities, (user tests).
universal tasks, Web interaction.
2. Background
1. INTRODUCTION According to World Health Organization (WHO), an individual
We live in an advanced technological world and the Web with special needs (“disability”) has impairment (a problem in the
resources provided numerous advantages, professional and social, body’s function and structure), conditioned activity (difficulty in

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 264 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
performing a task or action) and a restrained participation (it is a reading of instructions and understanding that users are in a Web
problem experienced by an individual, on a daily basis situation) environment [19]. One even stated that W3C-WAI guidelines for
[3]. accessibility are insufficient to ensure access to people with
intellectual disability [19]. Other study refers that despite these
Particularly, a person with intellectual disability is characterized guidelines had important issues about learning disabilities, almost
by having an intellectual quotient (IQ) significantly below average all elements regarding this disability are identified as lower
and limitations in the performance of functioning capacities in priorities [20].
daily life areas such as communication, self-care, and social
coexistence and in school activities. Despite this disability, they 3. Web interaction study per people with
can and should learn new competencies and abilities. However,
their development will always be slower when compared to a intellectual disability
child with medium intelligence and adaptive competencies [4]. It In this study we had the responsibility to train a group of people
is therefore crucial to be aware of different pathologies (and with intellectual disabilities (without Web experience) using the
degrees of severity) among the group of people with intellectual Web, by performing tasks related to their daily school activities.
disabilities in order to choose appealing school activities and We aim at assessing their Web learning evolution process, by
appropriate didactic methodologies and thus motivate these using the mouse and keyboard input devices, observing user-
students to learn [2]. interface interaction and performing selection, manipulation and
Several national and international studies indicate that the use of navigation activities.
the computer and other technologies have major advantages in the
learning process, pointing to increased motivation, performance
3.1 Participants
Twenty participants were invited to partook in the pilot study,
and promotion of the use Information and Communication
Technologies (ICT) [5] [6] [7] [8]. Explicitly, ICT provide several whose ages ranged from 19 to 46 years old. These participants
possibilities of communication, new ways to transmit knowledge, were selected by a special education teacher and a psychologist,
motivational tools to enhance learning, access, efficiency and according to the average rate of literacy and primary education
quality of the learning process [9]. Moreover, the International (coincident with the first grade). All were volunteers and had
Institute for Communication and Development (IICD) (2007) permission of their parents or tutors to perform the activities.
study reveals that 80% of the users “felt more capable by their Concerning their intellectual disabilities, individuals were not
exposure to ICT” and 60% said were “direct and positively associated to one pathology, but a set of pathologies (for example,
influenced by ICT” [6]. fetal alcohol syndrome with dysgraphia). These pathologies can
Likewise, there are several studies that recognize the advantages be classified according to severity levels, between mild to
of ICT use with students with special needs and disabilities [10] moderate, only one of the participants presented a high level of
[5] [11] [8] [12]. In these studies it is strengthened benefits disability [4].
provided to these students by the ICT in education. ICT are more Furthermore, fourteen participants have normal vision and six
efficient and effective due to the user’ motivation in the have corrected to normal vision. Participants had no experience
interaction with the computer, used as assistive technology or with digital environments or the Internet.
pedagogical tool [13] [14]. Although the great number of ICT
studies in literature, it is highlighted the importance of further 3.2 Methods
research on users-interface interaction, accessibility of contents, In this study, we allied ethnography (to overcome difficulties
pedagogical approaches using ICT to support inclusion in special found in communication within the group) with user tests. The
needs education [10] and usability of the different applications methods of data collection used were directly related to the
developed. It is also evident a lack of ICT solutions for people research methods adopted: interviews, direct observation and
with disabilities, such as visually impaired people, but when videotaping. The activities defined were similar to the ones they
compared with people with learning difficulties this number performed daily in school, however we they used the Web to
decreases [11]. perform the activities. These activities were chosen to motivate
the group to perform daily school activities, providing
We believe that the Web can be an unlimited source of activity
technological abilities on Web selection, manipulation and
resources to enhance education for people with learning
navigation tasks and also exercise mouse and keyboard handling.
difficulties and for people with intellectual disabilities. However,
there are several accessibility barriers in this environment, 3.3 Activities
particularly raised by their own disability, due to their slower The activities were defined to train users’ selection, manipulation
learning, low reading comprehension, limited fine motor control, and interaction skills in the Web. We intended to increase
reduce spatial perception, low visual acuity, less hand/eye activities’ difficulty to assess their apprentice level related to
coordination, finger dexterity and lowered information overload mouse and keyboard handling and success of goal reached. This
thresholds [20], leading to digital exclusion of these users [15]. In was measured through the increase number of clicks and adding
several studies it is even questioned the possibility of finding a specific objectives to successfully finish the task. All activities
guiding principle when planning Web sites for people with and difficulty levels are specified next:
learning and intellectual disabilities [16], also related to
difficulties resulting from the wide range of disabilities among
such group of people. Other studies highlighted problems found in
Web interaction such as: text entry is problematic and multi-
options are referred as difficult to use [16] [17]; hyperlinks
recognition are difficult [18] [17] and also typing, scrolling,

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 265 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
at least 3 equal balls, they disappear. The player wins when there
are no globes left. If the player does not eliminate all globes
before them pass the bottom central bar the Game is Over. The
game duration time depends on each player and the time it takes
to complete the level or lose it (Figure 3).
The second game, the participants played a cars game called
Extreme Racing 2. In this game, the player must move away of
several obstacles that appear on the road. The full game duration
is 1 minute and 30 seconds. They had to use the same three
keyboard keys (left and right arrow and the space key) to
accelerate. The level difficulty increased with the number of
obstacles that appear in the road and with the limited time to
finish the level (Figure 3).
Figure 1. First activity (Painting). With this activity we aimed to initiate keyboard handling and
In the first activity, participants had to paint two drawings. In the assess users’ interface interaction and navigation and fine motor
first drawing, participants had to paint it with colors random (7 to skills (speed control and precision).
12 clicks) (Figure 1). In the second, had to paint a word (9 clicks)
matching each letter with a specific color (Figure1).
This activity aims to assess the selection task (fine motor skills
aspects with the mouse use) and comprehension (painting with a
specific order and color).

Figure 4. Forth activity (Web search)

Participants performed a forth activity. With this activity we
aimed to train them with both devices, mouse and keyboard, and
user’s interaction with a Web search engine, Yahoo! (Figure 4).
In this activity, users had to perform three searches, with the
Figure 2. Second activity (Puzzles) following keywords: animals, music and sewing tools. First, they
In the second activity, participants must build three puzzles. had to recognize the search field and click in it with the mouse,
Specifically, in the first, participants must build a six pieces then write the word in the search field (previously written in a
puzzle; in the second, a puzzle with twelve pieces and in the third, paper) and click the search button. This task was performed on the
a puzzle with twenty-four pieces (Figure 2). In this task it is Yahoo! Images so that participants can comprehend all search
intended to evaluate manipulation, click, drag and drop results because of their reading and writing disabilities. Note that
movements and fine motor skills, with the mouse use. this group presents a low level of literacy and it is not intended
that they be constrained with text results, once they could not
read. This task aimed to initiate user’s interaction with the Web,
assessing the interaction with Yahoo! layout (by the keywords
insertion on the search field of the web browser engine) and fine
motor skills, using both devices: mouse and keyboard.

3.4 Apparatus
The following material resources were used: a HP keyboard,
optical computer mouse, Chrome Web browser.

3.5 Procedure
Before starting the planned activities, for two weeks
approximately thirty-five hours in total and two and a half hours
per subject) the group was faced with basic and essential issues
Figure 3. Third activity (Games). needed to use the computer and the Internet. The participants were
told how to connect a computer and how to handle the
In the third activity, it was requested to play two games to mouse/keyboard and its use. Also, we presented the basic features
introduce keyboard interaction. The first, participants play a game of the Web browser. It was given a brief explanation on how to
called Puzzle Bobble. This game consists in throwing colorful open/close the browser, use buttons on the browser and their
globes and group them by color, using only three keyboard keys functionalities: the previous and next arrows, maximize, minimize
(left and right arrows and space). Whenever they are grouped in, and close windows, search the Web pages activity in the favorites

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 266 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
options (previously recorded) and click in the Web page to start 3.6.2 Second activity – manipulation
the activity. Furthermore, they were shown how they might In the second activity, users made three puzzles. The first puzzle
recognize a link, or when the content is clickable or not clickable, was made by nineteen participants (one was absent), with an
based on the transformation experienced by the mouse pointer average time of 112 seconds to complete the puzzle. The fastest
icon (e.g. when the mouse pointer is over a link the original icon, participant did the puzzle in only 18 seconds, on the other side,
the arrow, becomes a hand). the slowest took 231 seconds due his/her difficulties handling the
After this preliminary stage, the group performed the four Web computer mouse. Three participants have difficulties in handling
activities, in 16 weeks (approximately forty eight hours in total the mouse and two in the correct position of the puzzle pieces,
and two and a half hours per subject). The activities were however no one quit.
performed individually, in a controlled environment. Each task In the second puzzle, from twenty participants only seventeen
was explained before the participant initiated it. Participants were accomplished the task (three were absent). The average time for
seated correctly in front of the screen. All tasks were displayed in the level conclusion was 277 seconds. The fastest participant
the computer and performed in the Web. The evaluator/observer finished the puzzle in 64 seconds. The slowest user concluded the
did not help the participant on the input device interaction or puzzle in 618 seconds due to him/her difficulties in the mouse
resolution of the activity unless him/her asked for it. handling. Three participants still had difficulties in mouse
handling and five in the correct position of the puzzle pieces. Due
3.6 Results and discussion to mouse handling difficulties, two participants refused to finish
the task.
3.6.1 First activity - selection In the third puzzle, nineteen participate in the task (one was
In the first activity, participants painted two drawings. The first absent). This puzzle had a time average of 557 seconds. The
drawing was painted by nineteen participants (one user was fastest user concluded it in 289 seconds; on the other hand, the
absent from the class). The average time of conclusion of the task slower user finished the task in 1171 seconds, presenting many
was 292 seconds. The fastest participant took 59 seconds to difficulties in mouse handling. Regarding difficulties found, three
conclude the task and the slowest, 601 seconds. During this participants had problems with the mouse handling (specifically,
assignment, it was observed that ten participants use the mouse one presents difficulties in the mouse’ hand positioning) and five
correctly; seven had difficulties with its use and control and two with the correct position of the puzzle pieces. Three users quit the
participants had difficulty in the beginning but overcome those task due to puzzle construction difficulty.
difficulties before finish. No one quit.
In figure 5, it is compared the average time for conclusion of the
The second drawing was painted by nineteen participants (one three puzzles made on the second Web activity.
participant was absent from the class). In this activity we register:
time to finish successfully the task (to match the predefined color 1200
to the letters), errors on color correspondence (an average of
approximately 1.7 errors) and mouse clicks to finish the task, 1000
difficulties and dropout rate. The average time to complete the
800
task was 326 seconds. The fastest participant finished in 90
seconds, with 18 clicks and 0 errors on color correspondence and 600 M…
Time (seconds)

the slowest 600 minutes, 53 clicks and 8 errors on color match.

Errors observed were on color match (a total of 32) and on 400
painting the surrounding area (six participants). Concerning the
difficulties observed, five participants show difficulties in the 200
mouse handling, such as: in the correct hand’s positioning,
correct use of the buttons and in cursor precision. Participants 0
revealed interest and motivation to successfully finish this second
drawing. No participant quit. -200
Puzzle 1 Puzzle 2 Puzzles 3
In this activity we observed a learning evolution on the mouse
handling as, in the first drawing, seven had handling difficulties Figure 5. Average time for conclusion of the three Puzzles.
and in the second, five. Two participants improved their
As expected, users take more time to finish puzzles with higher
performance between the two drawings tasks. Specifically, in the
number of pieces because it increased the complexity of the
first drawing, it was observed difficulties in the mouse input
activity, despite users having more experience with manipulation
device handling when they had to painting small areas. In the
tasks and mouse handling.
second drawing, participants had difficulties on the colour
matching. Despite these difficulties, no participant asked to quit. In the second activity (puzzles) it is noted that previous learning
Another observation made, in the second puzzle, users painted with the mouse input device handling was not forgotten, once the
the surrounding area of the drawing when trying to paint the participants improved their performance. After the first puzzle,
character, they did not had the mouse handling precision needed three participants showed difficulties in the mouse input handling.
to make the selection. When asked why, they answered the area In the second puzzle, the same three participants had difficulties
(character) was too small. Regarding the results, we conclude this and two quit this puzzle. However in the third puzzle, no one
selection activity is ease to perform by the group and they will refuses to perform the task, three had difficulties but no one quit.
have technological abilities to repeat in another context. Regarding puzzle construction (put the puzzles pieces in the
correct places), in the first, two asked for help but no one quiet; in

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 267 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
the second, five users needed help; and in the third puzzles, six task in 249 seconds and the slowest in 803 seconds. The major
participants asked for help and because of the large number of difficulty observed was in the keyword reproduction, i.e.
pieces, users showed some frustration in finishing and three identifying the character to write the right word, seventeen
wanted to quit. Despite the increased number of dropouts (when participants had this difficulty. Regarding input devices handling
compare with the selection activity), we notice although users quit difficulties, no participants had difficulties with mouse input
the task in hands they did not refuse to participate in the next Web device and four had difficulties with the Keyboard (they click in
activities. This was considered an important result by the group’ two keys at the same time). Also it was observed satisfaction on
teachers because they confirm that when participants refused and the search performance once they laugh and clap their hands
quit a school daily task (without using the Web) they never showing great happiness went the result appear in the screen.
wanted to repeat a similar task (in such a short time) and teachers Another observation was that users often clicked on the images
had to insist very hard to make them performed it. Regarding found within the search results. No one quit the task.
manipulation, we notice participants gained abilities to perform
these activities with proper training. In the second Web search, the average time for task conclusion
was 386 seconds. The fastest participant concludes the task in 243
3.6.3 Third activity – navigation seconds and the slowest in 976 seconds. Twelve had difficulties in
In the third activity, participants had to play two games. The first the keyword reproduction and three with the keyboard handling.
game was played by eighteen participants (two were absent). No participants had difficulties with the mouse input device. They
Only eight concluded successfully this level, i.e., eliminated all showed satisfaction and motivation to perform this search, asking
colour balls before “Game Over”. The average time of playing to repeat the Web search. No one quit the task.
was 171 seconds. The fastest user took 57 seconds to eliminate all In last Web search, the average time for task conclusion was 390
balls and the slowest, 310 seconds. We also noticed that from the seconds. The fastest participant concludes the task in 244 seconds
eight participants that successfully concluded the task, five did and the slowest in890 seconds. Eight had difficulties in the
have difficulties in handling the keyboard. From the remaining keyword reproduction and three with the keyboard handling. No
ten, nine had difficulties but two participants overcame their participants had difficulties with the mouse input device. They
difficulties during the task. Thus, thirteen participants showed asked to repeat the Web search, showing satisfaction and
difficulties in this first interaction with the keyboard. The motivation. No one quit the task. Here it was observed that they
difficulties observed with the keyboard handling, were: users use the ENTER keyboard instead using the mouse and click in the
press two keys simultaneously and switch the correct order of the search button.
arrows. Seventeen participants asked to repeat the game. No
participant quit. In figure 6, we compare the average time for conclusion of the
three Web search users made on the forth activity.
The second game was performed by eighteen participants (two
were absent). Nine users did not present any difficulty in the
1100
keyboard handling, one improved during this level (however
he/she did not successfully complete the task- before “Game 900
Over”). Regarding, successfully conclude the task, four
participants did not finished the game (not because of keyboard 700
Time (seconds)

handling difficulties but due to the fact that they did not reach the 500
requested velocities in order to finish before “Game Over”). It is M…
noticed an improvement in the users’ keyboard handling between 300
levels. Eighteen participants asked to repeat the game. No user
quit. 100

Regarding these results, it was notice that in the first game (Puzzle -100 Web Web Web
Bubble) the success conclusion rate was low (when compared Search 1 Search 2 Search 3
with previously activities), ten participants did not finished the
game successfully, i.e., before “Game Over”. In the second game
Figure 6. Average time for conclusion of the three Web
(Extreme Racing 2) only four participants did not finish
searches.
successfully the game (before: GAME OVER). With these
activities, they started to handle the keyboard (started to use the Comparing average time between the three Web searches, it is
arrows, enter and space keys), and the major difficulty was found noticed that the results were very similar.
was that they click in two keys at the same time but they In the fourth activity (Web search), they performed three Web
overcome it during the task. No one quit and just one participant searches and it is noted an evolution in the users performance
did not ask to repeat the first puzzle. They seemed to be very during the activity. Notice here they had to handle both, input
comfortable in the interaction and showed great satisfaction and devices, mouse and keyboard, and it was not observed any
motivation to perform these activities. Concerning interaction, difficulties in the mouse handling, with the keyboard, users
with proper training and time for learning, users showed several times click in two keys at the same time or click
technological abilities to perform these activities. continuously in one key but they overcome during the task. To
3.6.4 Forth activity – characters insertion write the keyword with keyboard to start the Web search was the
Regarding the forth activity, three Web searches, nineteen major difficulty observed, participants struggled with character
participants performed the task (one was absent from the recognition (on the paper and keyboard keys) and with word
classroom). In the first Web search, the average time for task formation. In relation to navigation, after the keyword insertion,
conclusion was 373 seconds. The fastest participant finished the

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 268 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
users naturally navigate on the page, by clicking on the images http://www.dsm5.org/Pages/Default.aspx. [Accessed 27
presented and using the mouse input device. January 2015].
[5] Balanskat, A., Blamire, R., and Kefala, S. 2006. The ICT
4. Conclusion and future work Impact Report: A review of Studies of ICT impact. European
As this group often do not have the opportunity to perform
Schoolnet. Available in: http://ec.europa.eu/education/
activities with the computer and the Internet, because many
doc/reports/doc/ictimpact.pdf . [Accessed: 14 March 2015].
professional (teachers, developers) feel that they have no abilities
or the will to do so, this study pretend to shows that they can carry [6] IICD. International Institute for Communication and
out (with training) basic tasks of selection, manipulation and development. 2007. ICTs for Education: Impact and Lessons
navigation. The results showed that this group had great Learned from IICD-Supported Activities (The Hague: IICD,
motivation and satisfaction to perform Web activities. Despite the 2007). Available: http://www.iicd.org/files/icts-for-
difficulties found with the keyboard input device, they had an education.pdf . [Accessed: 21 March 2015].
excellent performance in handling a normal mouse input device. [7] Gutterman, B., Rahman, S., Supelano, J., Thies, L., and
From the beginning of the activities they showed great progresses Yang, M. 2009. White Paper. Information Communication &
on the mouse and keyboard handling. After performing all Technology (ICT) in Education for Development. Available:
activities they all could correctly handle the mouse. Regarding the http://unpan1.un.org/intradoc/groups/public/documents/gaid/
keyboard, they did not have difficulties using the function keys unpan034975.pdf . [Accessed: 06 March 2015].
(such as: arrows, enter and space) but had many difficulties with [8] BECTA. British Educational Communications and
the character keys. This appended because of their reading and Technology Agency. 2007. The impact of ICT in Schools:
writing difficulties. Thus, we believe it will be very difficult to Landscape Review, (Coventry: Becta, 2007). Available:
these users to use the keyboard input device with autonomy. http://dera.ioe.ac.uk/1627/ [Accessed: 21 March 2015].
Also, we observed when asked users to perform complexes [9] Ribeiro, J., Almeida A. M., and Moreira, A. 2010. A
activities, which are difficult to conclude due to the participants utilização das TIC na Educação de Alunos com Necessidades
own disability restrictions, can discourage the users with Educativas Especiais: resultados da aplicação piloto do
intellectual disability and lead them to quit. inquérito nacional a Coordenadores TIC/PTE. Indagatio
Didactica. 2, 1. ISSN: 1647-3582.
Global results showed performing activities in a Web environment
provided a high group’s success and a low dropout’s rate. We [10] BECTA. British Educational Communications and
believe with regular training they can gain technological abilities. Technology Agency. 2003. What the research says about ICT
supporting special educational needs (SEN) and Inclusion.
As future work, we want to study alternatives solutions for (Coventry: Becta, 2003). Available:
keyboard input devices such as: voice application or search with http://www.mmiweb.org.uk/publications/ict/Research_SEN.p
images, to overcome reading and writing difficulties presented by df . [Accessed: 21 March 2015].
the group with intellectual disability. Also, we intend to increase
[11] Williams, P., Jamali, H., and Nicholas, D. 2006. Using ICT
the number of participants to allow a more detailed analysis of the
With People With Special Education Needs: What The
results with respect to the level/type of impairment. And thus
Literature Tells Us. Aslib Proceedings, 58, 4, 330-345
continue to draw attention to this group of people that is digital
excluded. [12] Liu, Y., Cornish, A., and Clegg, J. 2007. ICT and special
educational needs: Using meta-synthesis for bridging the
5. ACKNOWLEDGMENTS multifaceted divide. Springer-Verlag: Berlin, 18-25.
We thank all people who directly or indirectly helped in this [13] Kirinić, V. Vidaček-Hainš V., and Kovačić, A.
study. Specifically:... and ... in the activity’s execution, … and ... 2009.Computers in Education of Children with Intellectual
for your willingness. and Related Developmental Disorders. In Proceedings of
Computers and Education Conference of 32nd International
6. REFERENCES Convention on Information and Communication Technology,
[1] W3C. World Wide Web Consortium. 1997. World Wide Electronics and Microelectronics. (Opatija, Croatia, May 25-
Web Consortium Launches International Program Office for 29). MIPRO 2009. 39-43.
Web Accessibility Initiative. Available:
http://www.w3.org/Press/IPO-announce [Accessed 28 [14] Ribeiro, J., Moreira, A., and Almeida, A. 2009. An approach
January 2015]. to Inclusion through Information and Communication
Technology. Actas do I Congresso Internacional FAMÍLIA,
[2] Rocha T. 2014. Interaction metaphor for Access to Digital
ESCOLA E SOCIEDADE – Educação Especial. Educare
Information an autonomous form for People with Intellectual (Oporto, Portugal, July 9-11, 2009). 1089 – 1102.
Disabilities. Doctoral Thesis. University of Trás-os-Montes
and Alto Douro. [15] Rocha, T., Bessa, M., Gonçalves, M., Cabral, L., Godinho,
F., Peres, E., Reis, M., Magalhães L. and Chalmers, A. 2012.
[3] WHO. World Human Organization. 2011. Health Topics: The Recognition of Web Pages' Hyperlinks by People with
Disability. Available: Intellectual Disabilities: An Evaluation Study. Journal of
http://www.who.int/topics/disabilities/en/ [Accessed 27 Applied Research in Intellectual Disabilities. 25, 6. 542 -
January 2015]. 552. DOI= 10.1111/j.1468-3148.2012.00700.x
[4] APA. American Psychological Association. 2013. DSM-V: [16] Harrysson, B., Svensk, A., and Johansson, G. I. 2004. How
The Diagnostic and Statistical Manual of Mental Disorders, People with Developmental Disabilities Navigate the
Fifth edition. Available:

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 269 |
 Performing universal tasks on the Web: interaction with digital content by people with intellectual disabilities
T. Rocha | M. Bessa | L. Magalhães | L. Cabral
Internet. British Journal of Special Education.. 31,3. 138- http://mutex.gmu.edu:2068/pqdweb?did=828407271&sid=3
142. &Fmt=2&clientId=31810&RQT=309&VName=PQD .
[17] Rocha, T., Gonçalves, M., Godinho, F., Magalhães, L. and [Accessed: 21 March 2015].
Bessa, M. 2009. Accessibility and usability in the internet for [19] Small, J., Schallau, P., Brown, K., Ettinger, D., Blanchard,
people with intellectual disabilities. Proceedings of 2nd S., and Krahn, G. 2005. Web Accessibility for people with
International Conference on Software Development for Cognitive Disabilities. Technology and Health Care Journal
Enhancing Accessibility and Fighting Info-exclusion. 13, 432–433. Special issue: MedNet 2005 – New tasks in
(Lisbon, June 3-5, 2009). DSAI’2009, 25–29. internet use in health care archive. ISSN 0928-7329.
[18] Roh, S.Z. 2004. Designing accessible Web-based instruction [20] Friedman, M and Bryen, D. 2007. Web accessibility design
for all learners: Perspectives of students with disabilities and recommendations for people with cognitive disabilities.
Web-based instructional personnel in higher education. Technology and Disability. IOS Press. 9. .205-212. ISSN
Unpublished Doctoral dissertation. Indiana University. 1055-4181/07/$17.00.
Available:

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 270 |
 Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and Navigation
N. Tung Ly | J. Hurtienne | R. Tscharn| S. Aknine | A. Serna

Towards Intelligent and Implicit Assistance for People with

Dementia: Support for Orientation and Navigation
Nam Tung Ly1, Jörn Hurtienne1, Robert Tscharn1, Samir Aknine2, Audrey Serna2
1
LS f. Psychologische Ergonomie, Julius-Maximilians-Universität, Würzburg, 97074, GERMANY
{nam.ly-tung, joern.hurtienne, robert.tscharn}@uni-wuerzburg.de
2
LIRIS CNRS UMR 5205, 69622 Villeurbanne, FRANCE
{samir.aknine@univ-lyon1, audrey.serna@insa-lyon}.fr

ABSTRACT develop into other psychological deficits such as anxiety,

To date, most healthcare technologies for orientation and suspicions, illusions and aggression [4]. Frequent results are social
navigation for people with dementia or with cognitive isolation and a reduced quality of life [2]. Moreover, the lives of
impairments have been developed for outdoor environments. relatives and caregivers of PwD are also negatively affected.
These systems require skills in handling smartphone or PDA Healthcare technologies and assistive systems are designed to
applications from users. In this paper, the authors focus on the support everyday life of PwD, improving their autonomy, safety
needs of people with moderate and severe dementia, who are and quality of life. The design and acceptance of such systems
unable to use smartphones and are living in nursing homes or raises real challenges, in particular when considering advanced
specialized fulltime care facilities. A new approach is presented, stages of dementia. Regarding navigation and orientation, most
based on iterative people-centered design processes, explores calm current approaches focus on outdoor environments [5]. And in the
computing and implicit interaction paradigms for guiding people majority of systems, users need the capability to use mobile
with subtle cues integrated in the indoor environment. This devices, which is only appropriate for early stages of dementia,
intelligent and implicit assistance is supported by a three-part e.g. [6, 7]. Rasquin and colleagues [8] observe that even a device
framework: a tracking system, a separate guiding system and an with three buttons may be too complicated for people with
intelligent system. The guiding system approach has been tested moderate-to-severe stages of dementia. In addition, people who
on several people with dementia. The first results are promising are living in care homes mainly perform indoor activities. Given
and the feedback from stakeholders is positive but more this background, the authors focus on designing a system for
participants are needed for an empirical analysis. indoor environments like care homes for people with moderate-to-
severe dementia, who are unable or reluctant to use smartphone
Categories and Subject Descriptors technology.
K.4.2 [Computers and Society]: Social Issues – assistive
The authors suggest taking the opposite view of existing
technologies for persons with disabilities; H.1.2 [Models and
approaches by exploring calm computing and implicit interaction
Principles]: User/Machine Systems - human factors; H.5.2
paradigms. The authors argue that for severe stages of the disease,
[Information interfaces and presentation (e.g., HCI)]: User
it is necessary to free people from interacting directly with
Interfaces – user-centered design; I.2.0 [Artificial Intelligence]:
complex systems. Interaction becomes implicit when actions
General – cognitive.
performed by the user are not primarily aimed to interact with a
General Terms computerized system, but which such a system nevertheless
Design, Human Factors. understands as user input [9]. Applied to assistive technologies,
this means that the system automatically gains an understanding of
Keywords the current states, contexts and intentions of PwD and assists them
Healthcare technologies; dementia; orientation; navigation; calm with simple and intuitive stimuli subtly integrated in the
computing; implicit interaction; intuitive guiding; context-aware environment.
computing. This paper presents a new approach of intelligent and implicit
adaptive assistance for PwD and introduce a three-part framework
1. INTRODUCTION for supporting this approach: a subtle tracking system, an implicit
The growing number of people at old ages entails more age- guiding system and an intelligent part that refers to context-aware
related deficits and diseases. One of which is dementia, a complex computing [10].
neurodegenerative syndrome that by 2050 could affect about 106
Mio. people worldwide [1]. People with Dementia (PwD) 2. RELATED WORK
experience a progressive decline in their cognitive abilities and According to literature reviews of PwD and assistive technology
have serious problems coping with activities of daily living, [5], [11, 12], the number of projects for PwD in navigation is
including orientation and way-finding tasks. They even experience limited, especially for indoor environments. Remarkably, many of
difficulties in finding their way in familiar environments [2]. them are research prototypes without clinical studies involving
Henderson [3] reports that more than one-third of PwD were PwD, e.g. [13], [14]. In the few projects that have been developed
diagnosed with visuospatial disorientation and getting lost as outdoor navigational tools for people with mild to moderate
behavior. The ratio of affected people is even higher in complex dementia, Activity Compass [13], “Walk navigation system” [14]
and unfamiliar environments such as hospitals or healthcare and COGKNOW [7] are considered. These systems are using
facilities, where most residents exhibit moderate or severe stages mobile devices, locating people with GPS and instructing next
of dementia. Being lost or fear of getting lost may consequently steps by images, texts, or voices. In contrast, Context-Aware

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 271 |
 Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and Navigation
N. Tung Ly | J. Hurtienne | R. Tscharn| S. Aknine | A. Serna
Wayfinder [6] is an indoor navigation system. Instead of GPS, the technology into the daily lives of PwD, whether or not they
user’s PDA provides location information by scanning the QR remembered to take the device with them. Another difficulty
code tag, which are placed on a door, a turn or an elevator, and concerns the size and weight of the device or embedded sensors
sending it to the server over Wi-Fi. Then the device receives that should not prevent or hinder the activities of the person.
images of the next waypoint from the server and guides the user Moreover, ethical aspects should be considered as well. The
by overlaying directions. device should not make them feel being monitored and that their
Although such projects show the positive effects to PwD in freedom was taken away. Localization external to the person via
navigation tasks, they are not suitable for people with severe cameras and visual tracking had other problems. Identifying
dementia due to their lack of ability to manipulate mobile devices. individuals and recognizing their activities for the intelligent
Moreover, these devices and guidance cues used (e.g. guidance system based on video-processing techniques can be
photographs, alert) have some drawbacks. First, the users have to computationally expensive. For that reasons, a solution that uses
constantly watch displays or scenes which could be dangerous at iBeacon devices is proposed (section 5.1).
crossroads and on stairs [14]. Second, uncharacteristic photos or Regarding the guiding system, the authors focus on integrating
places with similar scenes (e.g. intersections) may confuse users. guiding cues into the environment of PwD. The main approaches
Third, warning sounds surprisingly have a negative effect on the in static architectural design are providing supportive floor plan
quality of way-finding for people with mild dementia [15]. topologies (e.g., direct visual access to the common room, simple
Some projects tried to overcome the limitations of mobile devices circulation routes, and small-scale settings) and distinctive
by focusing on designing assistive devices. KITE [16], for environmental cues (e.g., signage, furniture, and intimate items
example, proposes two prototypes for two participants: an with personal meaning). Although many of these approaches have
armband for a runner and an electronic notepad for a driver. The shown positive effects on orientation and navigation [20], they are
tracking technology is GPS in cooperation with Global not adaptive. The long-term goal is to turn the traditional means of
communication system (GSM). If PwD get lost, they can push a assistance into flexibly controllable ones using a variety of
button to send a message to their carer. Although the design and distributed interactive low-cost components. This way, the
functionality seem to work well, the devices are too large and too guidance cues are expected to be dynamic, adapted to context, and
heavy for users. Furthermore, the panic button on the device might implicit since many PwD do not understand explicit cues such as
be too easy to press. Without incorporating localization floor plans or symbols. Light and colors are chosen to orient PwD
techniques, Grierson and colleagues [17] introduces a wearable because they can be implemented to meet above criteria.
belt to facilitate navigation for PwD using four small, vibrating 3.2 Preliminary Studies on Implicit Guiding
motors that are corresponding to front, back, right and left.
Cues
3. A NEW APPROACH OF INTELLIGENT To evaluate the relevance of using implicit guiding cues using
AND IMPLICIT ASSISTANCE LED light, the authors conducted two studies with PwD from one
For this project, the authors adopted an iterative and people- of the cooperated healthcare facilities.
centered design approach [18, 19]. The partners are two healthcare Light projections on the floor or ambient spatial sounds are also
facilities and one hospital for PwD in Würzburg and Dresden in considered. But the feedback from PwD showed that the light
Germany. The different stakeholders (i.e. PwD, caregivers, projection was more likely focal point and not ambient as LED
healthcare managers, but also family members) are included in the strip. About ambient spatial sounds, PwD could get confused
design process to ensure the system’s usefulness and acceptance. when hearing unknown and un-located voice.
It also proved vital to visit and observe PwD in their everyday
environments to be sure to understand their difficulties, needs and
requirements. The observation lasted two weeks of full time and
continued in following six weeks during the study of guiding cues.
Along with that, qualitative data (included preferences,
biographies of PwD) are also collected from conversations with
PwD, their relatives and caregivers.
The authors then realized all of the residents experienced
difficulties with orientation, navigation and way-finding. The
caregivers’ added workload by this was usually overwhelming.
The caregivers have to go along with PwD almost every time they
want to go somewhere. In addition, all of the residents in the three
places do not use any technical devices.
Figure 1. Implicit guiding cue experiment – Indoor
3.1 Design Considerations environment
In contrast to previous approaches, the authors devised the
navigation support system as two separate parts using different The first study was conducted to determinate how PwD perceived
techniques and devices: one for tracking and one for guiding. To colors. The authors asked 12 PwD to choose the color they like.
bridge these two systems, a third intelligent system is requisite to The colors were shown randomly under pairs (2 colors per time),
detect the context, determine PwD’s intentions and select a multi-colors (12 colors per time), and different shapes (round,
relevant assistance as and when necessary. rectangle, triangle). As a result, bright colors got PwD’s attraction
and preference.
Choosing a well-suited solution for the indoor tracking system
proved a challenge. The first barrier deals with incorporating

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 272 |
 Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and Navigation
N. Tung Ly | J. Hurtienne | R. Tscharn| S. Aknine | A. Serna
Based on this result, a second study was conducted using bright 4.1 Subtle Tracking System
light for navigation assistance. The experimenters asked four The objective of this system is to track the location of PwD in the
participants with moderate to severe stages of dementia to find a environment in real-time. Location data will then be handled by
place that they did not know before. They started at a decision the intelligent system to determine the current context and will
point where it was mandatory to turn left or right. There were two constitute a history of PwD activities. In order to detect contact
conditions: base line (no light) and illumination (left or right area between PwD and environment (bed, chair, door), ambient sensors
was illuminated by bright light emitted by an LED strip). The light (pressure sensor, reed switch sensor) are set-up. Along with that,
was manually controlled using the Wizard of Oz technique. The iBeacon technology (Fig. 5) is used as tracking device, because of
results show that PwD tended to go towards the brighter area (Fig. their small size and weight that are easily accepted by PwD.
2) and they find the way easier with light (ratting by experimenters
– Fig. 3). As feedback from PwD, the light on the left is not clear The indoor map based on OpenStreetMap data can be accessed,
as on the right (due to a sofa). It could be the reason for the customized, controlled in real-time (Fig. 7) by caregivers.
difference in results between the case light (left) and the case light
(right). These first results reinforce the approach of using implicit
stimuli in the environment for guiding PwD. More studies will be
conducted for empirical analysis and investigation of specific
combinations of color and light. The idea is that lights combined
with a personalized favorite color could become the personal
guidance cue of an individual.

Turning Facilitated
Figure 5. Onyx iBeacon Figure 6. Handrail with light

Behavior Finding…
Left Right Leader Protocoler
50
5062,5
37,512,5
87,5 46 109 99
100
0 20
0
ection (%)

ting [0-15]

BaseLight
Line
Light
(Left)
(Right) BaseLight
Line
Light
(Left)
(Right)
Condition Condition

Figure 2. Turning Behavior Figure 3. Facilitated Finding

Level (the higher level is, the
easier users find the way)

4. A FRAMEWORK SUPPORTING
INTELLIGENT AND IMPLICIT Figure 7. Web-UI for caregivers/managers
ASSISTANCE 4.2 Implicit Guiding System
To support the intelligent and implicit assistance system, the The purpose of this system is to support PwD completing daily
authors are developing a three-part framework, illustrated in Fig. way-finding tasks by providing relevant cues. The ideal guiding
4: the tracking system, the guiding system and the intelligent cues should be implicit and dynamic. For the first iteration, LED
system. strips was used and controlled remotely by a human controller.
Later, the intelligent system will control the guiding system
allowing dynamical adaptation to various situations. Another
advantage is that PwD do not need any skill to manipulate and
control the system. Different configurations and positions to place
the light cues are still being explored. For instance, the light can
be placed on the floor (Fig. 1) as in the above study or attached on
a handrail as in the newest prototype (because handrails are
ubiquitous in care homes, Fig. 6). Other cues like recorded voices
from PwD’s relatives or portrait pictures from their youth days
(that are more likely to be recognised by PwD) are also considered
to enhance assistance efficiency. These cues will be tested in next
studies.
4.3 Intelligent System
The system’s main function is to choose the most relevant
Figure 4. System Architecture assistance in the current context. The authors thus aim at
developing an online intention recognition method and a decision-
making process. The logical-approach model (e.g., [21]) is

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 273 |
 Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and Navigation
N. Tung Ly | J. Hurtienne | R. Tscharn| S. Aknine | A. Serna
considered using to transform low-level sensor data to high-level [7] Boer JD. 2010. Cogknow day navigator: the system in daily
behavior. Then, the decision-making process should choose life. In: CHI '10 Extended Abstracts on Human Factors in
whether and which assistance is needed. The user’s history of Computing Systems. ACM, p 4755-4758.
activities and expert data (e.g. medical history) are also taken into [8] Rasquin SMC, Willems C, De Vlieger S et al. 2007. The use
account in order to offer the most appropriate assistance to the of technical devices to support outdoor mobility of dementia
current situation (Fig. 4). The system suggests and stimulates PwD patients. Technology and Disability, 19, 113-120.
to go to the direction that has been judged as the most suitable for
the situation. In this way, according to the model of automation [9] Schmidt A. 2000. Implicit human computer interaction
levels of Sheridan et al. [22], the system corresponds to level 4. through context. Personal Technologies, 4, 191-199.
[10] Baumgarten M, Mulvenna M. 2010. The Role of Context-
5. CONCLUSION & OUTLOOK Aware Computing in Support of People with Dementia. In:
In this paper, a new approach is presented for offering implicit
Mulvenna MD, Nugent CD (eds) Supporting People with
guiding cues for orientation and navigation assistance for people
Dementia Using Pervasive Health Technologies. Springer
with moderate to severe dementia in indoor environments. Taking London, p 131-143.
the opposite view of current approaches, the authors take
inspiration from calm computing and implicit interaction [11] Gillespie A, Best C, O'neill B. 2012. Cognitive function and
paradigms. This paper also presented the first implementation step assistive technology for cognition: a systematic review.
towards a three-part framework supporting this intelligent and Journal of the International Neuropsychological Society,
implicit assistance. The authors adopted an iterative people- JINS 18, 1-19.
centered design process and have already conducted first studies to [12] Bharucha AJ, Anand V, Forlizzi J et al. 2009. Intelligent
evaluate the potential of the approach. The implicit guiding assistive technology applications to dementia care: current
system, tested thanks to Wizard of Oz techniques, showed positive capabilities, limitations, and future challenges. American
navigation outcomes. In addition, perspectives in terms of journal of geriatric psychiatry, 17, 88-104.
technical feasibility and expert feedback were optimistic. The
caregivers and managers in the healthcare facilities were satisfied [13] Patterson D EO, Kautz H. 2002. The activity compass. Papers
with the results of the studies. They showed excitement to see the presented at the Proceedings of the 1st International
further steps implemented. The platform development is still under Workshop on Ubiquitous Computing for Cognitive Aids.
progress, including exploring other technologies for the tracking [14] Kaminoyama H, Matsuo T, Hattori F et al. 2007. Walk
system and developing the decision-making process. At a longer- Navigation System Using Photographs for People with
term perspective, the guiding cues will be dynamically Dementia. In: Smith M, Salvendy G (eds) Human Interface
controllable, taking into account PwD’s current activities, and the Management of Information. Interacting in
emotional state, and also changing abilities and limitations. The Information Environments. Springer Berlin Heidelberg, p
overall approach will be then tested formally with all relevant key 1039-1049.
stakeholders. [15] Hettinga M, De Boer J, Goldberg E et al. 2009. Navigation
for people with mild dementia. Studies in health technology
6. ACKNOWLEDGMENTS and informatics, 150, 428-432.
The authors thank the Bayerische Forschungsstiftung for funding
the work presented in this paper and Stefanie Ebert and Valeria [16] Robinson L, Brittain K, Lindsay S et al. 2009. Keeping In
Bopp who were involved in the experiments. Touch Everyday (KITE) project: developing assistive
technologies with people with dementia and their carers to
7. REFERENCES promote independence. International Psychogeriatric, 21,
[1] Brookmeyer R, Johnson E, Ziegler-Graham K et al. 2007. 494-502.
Forecasting the global burden of Alzheimer's disease. [17] Grierson LEM, Zelek J, Lam I et al. 2011. Application of a
Alzheimer's & dementia, 3, 186-191. Tactile Way-Finding Device to Facilitate Navigation in
[2] Passini R, Pigot H, Rainville C et al. 2000. Wayfinding in a Persons With Dementia. Assistive Technology, 23, 108-115.
Nursing Home for Advanced Dementia of the Alzheimer’s [18] Orpwood R, Sixsmith A, Torrington J et al. 2007. Designing
Type. Environment and Behavior, 32, 684-710. technology to support quality of life of people with dementia.
[3] Henderson VW, Mack W, Williams BW. 1989. Spatial Technology and Disability, 19, 103-112.
disorientation in Alzheimer's disease. Archives of neurology, [19] Hendriks N, Truyen F, Duval E. 2013. Designing with
46, 391-394. Dementia: Guidelines for Participatory Design together with
[4] Mahoney E, Volicer L, Hurley AC. 2000. Management of Persons with Dementia. In: Kotzé P, Marsden G, Lindgaard
Challenging Behaviors in Dementia. Health Professions G, Wesson J, Winckler M (eds) Human-Computer Interaction
Press. – INTERACT 2013. Springer Berlin Heidelberg, p 649-666.
[5] Vogt J, Luyten K, Van Den Bergh J et al. 2012. Putting [20] Marquardt G. 2011. Wayfinding for people with dementia: a
Dementia into Context. In: Winckler M, Forbrig P, Bernhaupt review of the role of architectural design. Herd, 4, 75-90.
R (eds) Human-Centered Software Engineering. Springer [21] Vidal N, Taillibert P, Aknine S. 2010. Online Behavior
Berlin Heidelberg, p 181-198. Recognition: A New Grammar Model Linking Measurements
[6] Chang Y-J, Tsai S-K, Wang T-Y. 2008. A context aware and Intents. In: Tools with Artificial Intelligence (ICTAI),
handheld wayfinding system for individuals with cognitive 22nd IEEE International Conference, p 129-137.
impairments. In: Proc. 10th SIGACCESS conference on [22] Parasuraman R, Sheridan TB, Wickens CD. 2000. A model
Computers and accessibility. ACM, p 27-34. for types and levels of human interaction with automation.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 274 |
 Towards Intelligent and Implicit Assistance for People with Dementia: Support for Orientation and Navigation
N. Tung Ly | J. Hurtienne | R. Tscharn| S. Aknine | A. Serna
Systems. Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions, 30, 286-297.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 275 |
 Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos
H. Cardona Reyes | J. Muñoz Arteaga | F. Acosta Escalante | F. J. Álvarez Rodríguez | A. E. Muñoz Zavala

Terapia Ocupacional para Personas con Discapacidad

Física utilizando Ambientes Interactivos
Héctor Cardona Reyes Francisco Acosta Escalante Francisco J. Álvarez Rodríguez
Jaime Muñoz Arteaga Univ. Juárez Autónoma de Tabasco Ángel Eduardo Muñoz Zavala
Univ. Autónoma de Aguascalientes, Av. Universidad s/n, Zona de Univ. Autónoma de Aguascalientes
Av. Universidad #940 la Cultura, Col. Magisterial, Av. Universidad #940
Aguascalientes, Ags., México Villa Hermosa, Tabasco, México Aguascalientes, Ags., México
C.P. 20131 +52 (449) 910 8417 C.P. 86040 +52 (993) 358 15 00 C.P. 20131 +52 (449) 910 7400
k6550g, jmauaa{@gmail.com} [email protected] fjalvar, aemz{@correo.uaa.mx}
mailto:[email protected]

ABSTRACT el contexto de uso, identificar las necesidades de rehabilitación y

Este trabajo propone los ambientes interactivos como una condiciones del usuario. Con esto se pretende obtener beneficios
solución accesible y de bajo costo que permita dar soporte al en mejorar la productividad, la portabilidad, la interoperabilidad y
proceso de recuperación de las personas en rehabilitación que la reutilización de los sistemas. Este trabajo esta estructurado en
reciben terapia ocupacional. Se presentan un método para el siete secciones. La siguiente sección presenta varios trabajos
diseño de ambientes interactivos basado en modelos que permite relacionados que defienden el uso de tecnologías para asistir el
identificar las capacidades de interacción que se deben tomar en proceso de rehabilitación de personas con discapacidad física. La
cuenta al producir ambientes interactivos para usuarios con sección de problemática, describe la dificultad de incorporar
distintos niveles de capacidad/discapacidad, que permite ambientes interactivos para asistir el proceso de rehabilitación de
capitalizar las buenas practicas establecidas en la terapia física las personas con discapacidad. La siguiente sección propone una
permitiendo la generación de diversos escenarios digitales de método para establecer las bases para la producción de ambientes
rehabilitación con la tecnología adecuada, y que cubran las interactivos. La sección de caso de estudio, en la cual el método
diversas necesidades de rehabilitación del usuario con propuesto es probado a través de la aplicación de ambientes
discapacidad física. Por ultimo este trabajo presenta un caso de interactivos a pacientes con discapacidad física que reciben
estudio en el cual se comprueba el método propuesto por medio rehabilitación de la mano. Finalmente en la sección de
de la aplicación de ambientes interactivos en personas que reciben conclusiones se presentan los principales resultados así como los
terapia ocupacional para la rehabilitación de la mano en el DIF, trabajos futuros.
Aguascalientes, México.
2. TRABAJOS RELACIONADOS
Keywords El la literatura existen varios trabajos sobre rehabilitación, la tabla
Ambientes interactivos, Discapacidad, Terapia Ocupacional, 1 muestra algunos trabajos que abordan el soporte a la
Actividades de la Vida Diaria, Rehabilitación, Enfoque MDA. rehabilitación por medio del uso de la tecnología.
Tabla 1. Trabajos relacionados que dan soporte a la
1. INTRODUCCIÓN rehabilitación por medio de ambientes virtuales
La discapacidad forma parte de la condición humana, casi todas
las personas en algún momento de su vida sufrirán algún tipo de Trabajo Dominio
discapacidad transitoria o permanente, y las que lleguen a la [12], [10] Modelo de diseño
senilidad experimentarán dificultades crecientes de
funcionamiento. De acuerdo con la Organización Mundial de la [5], [8] Sistema Interactivo
Salud (OMS) entre 110 millones y 190 millones de adultos tienen [6], [7], [9] Juegos serios, video juegos, realidad
dificultades considerables para funcionar físicamente [1]. Existen virtual
diversas técnicas y métodos que forman parte de la terapia física,
[4], [11] Diseño de interfaces
uno de ellos es La Terapia Ocupacional hace uso del termino
"ocupaciones" o actividades tales como las Actividades de la Vida Uno de lo principales problemas que enfatiza este trabajo es que
Diaria (AVD) con el propósito de ayudar a las personas con los usuarios pueden tener problemas para comprender lo que se
discapacidades físicas, de desarrollo o emocionales, a llevar una apoyan en las tareas presentadas o cómo asociar las acciones
vida independiente, productiva y satisfactoria [2]. En los últimos lógicas deseadas con acciones físicas de la interfaz de usuario.
años ha incrementado el interés en el diseño y desarrollo de Esto puede abordarse si se cuenta con métodos para representar
software interactivo. Una de las principales razones es el interés explícitamente modelos requeridos y ofrecer indicaciones sobre
en la necesidad de permitir que el mayor numero de personas las técnicas de interacción y de presentación más adecuadas para
pueda acceder a software para el mayor numero de propósitos [3]. apoyar las posibles actividades para este tipo de usuarios y con
Sin embargo el realizar este tipo de ambientes interactivos ello lograr las interfaces adecuadas que los usuarios con algún tipo
dirigido a personas con algún grado de discapacidad física implica de discapacidad física requieren para alcanzar sus objetivos de
tomar en cuenta un gran numero de factores en el diseño, por lo rehabilitación.
que una de las discusiones de este trabajo se orienta hacia una
identificación y desarrollo de un método para diseñar ambientes
interactivos que puedan ofrecer un soporte adecuado a las tareas,

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 276 |
 Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos
H. Cardona Reyes | J. Muñoz Arteaga | F. Acosta Escalante | F. J. Álvarez Rodríguez | A. E. Muñoz Zavala

3. PROBLEMÁTICA
El presente trabajo plantea la incorporación de ambientes
interactivos al proceso de recuperación de pacientes en
rehabilitación, con el fin mejorar las actividades de rehabilitación
y ser un medio accesible para el paciente en donde pueda
rehabilitarse incluso sin salir de casa y realizando tareas de la vida
diaria en un ambiente interactivo. Esto es una tarea complicada
que requiere considerar un diseño centrado en el usuario para la
creación de software que resuelvan necesidades concretas de sus
usuarios finales, establecer un método basado en modelos que
permita capitalizar aquellas buenas practicas de rehabilitación
para así propiciar la producción sistemática de ambientes
interactivos y que estos se adapten de acuerdo a las necesidades Figura 2. Método para el Desarrollo de Ambientes
de las personas con discapacidad física permitiendo incorporar las Interactivos utilizados en Terapia Ocupacional.
diversas áreas multidisciplinarias y aquellos actores involucrados Este método tiene su base en las Guías Practicas Clínicas (GPC)
tales como especialistas, diseñadores, analistas, programadores, [14] que son un elemento de rectoría en la atención médica en
usuarios finales al proceso de producción de ambientes México y están basadas en recomendaciones sustentadas en la
interactivos. Por lo que es necesario aplicar técnicas de la mejor evidencia disponible, a fin de contribuir a la calidad y la
Ingeniería de Software e Interacción Humano-Computadora para efectividad de la atención médica. De acuerdo con el método
diseño y desarrollo de diversos tipos de usuarios con el fin de propuesto se inicia con la identificación de las situaciones del
desarrollar un entorno interactivo que de soporte a las necesidades mundo real cuyo objetivo consiste identificar la evaluación de la
y cuestiones planteadas. A continuación se presenta un método de funcionalidad del paciente, en establecer una definición de las
producción de ambientes interactivos como solución a la difícil habilidades cognitivas deseadas del paciente y conocer las
tarea de producir ambientes interactivos y tecnologías e limitaciones de los usuarios potenciales. Después se genera un
integrarlas al proceso para la recuperación de pacientes en modelo de conocimiento donde se determina un conjunto de
rehabilitación. actividades a realizar para dar tratamiento a la rehabilitación. A su
4. PRODUCCIÓN DE AMBIENTES vez un modelo computacional permite elaborar una representación
computacional del entorno, sea esta en 2D o 3D considerando las
INTERACTIVOS limitaciones funcionales del usuario identificadas en el modelo de
Uno de los fines del proceso mostrado en la figura 1 es la creación conocimiento. De acuerdo al tipo de actividad ocupacional y
de un ciclo iterativo e incremental en el sentido de avances usuario se requiere de un modelo que capture los requerimientos e
graduales y para cada logro en el paciente establecer un ambiente información que permita que solo pueda se usado por usuarios con
interactivo diferente permitiendo adaptarse con esto a las cierto nivel de funcionalidad física. Una vez establecidos los
necesidades de rehabilitación con el fin de lograr la total parámetros y entidades del modelo, las acciones son captadas para
funcionalidad del paciente y que este pueda reincorporase a sus obtener información vital para el especialista y el paciente
actividades de la vida cotidiana. permitiendo la evaluación y clasificación con fines de
retroalimentación y obtener datos sobre la aceptación del modelo
de diseño insertado en el prototipo de ambiente interactivo
producido y si es o no una representación del escenario de
rehabilitación del usuario. A continuación se explican cada una de
las fases propuestas en la figura 2 con un caso de estudio en el
cual se asiste la rehabilitación motora de los músculos de la mano.

5. CASO DE ESTUDIO
Esta sección aborda la puesta en marcha de ambientes interactivos
en personas jóvenes y adultas con discapacidad física que reciben
terapia ocupacional para la rehabilitación de la mano en el
Figura 1. Proceso de Terapia Ocupacional para Personas con Sistema para el Desarrollo Integral de la Familia Estatal
Discapacidad Física utilizando Ambientes Interactivos. Aguascalientes, México (DIF) [15]. El DIF cuenta con una
Dado que se requiere producir varios ambientes interactivos se Unidad de Terapia Física con diferentes módulos en los que hay
propone una método basado en modelos, tal como se presenta en distintos aparatos que facilitan la rehabilitación de los pacientes.
la figura 2. El método inicia con la definición de las habilidades Este caso de estudio conto con la participación de pacientes y
de acuerdo a situaciones reales tales como actividades de destreza, expertos que acuden a la Unidad de Terapia Física, aquí los
sensoriales, resolución de problemas por medio de movimientos, pacientes presentan diferentes condiciones en cuanto a edad,
recrear situaciones de la vida diaria o del trabajo [13]. discapacidad y tipo de tratamiento a seguir. El objetivo es asistir a
los pacientes por medio de la utilización de ambientes interactivos
además de comprobar cada de las fases que componen el método
para el diseño de ambientes interactivos propuesto en la figura 2.
5.1 Fase del Mundo Real
En esta fase se identifico un grupo de cinco pacientes voluntarios
cuyas edades son de: 9, 50, 58, 63, 68 años respectivamente, con

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 277 |
 Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos
H. Cardona Reyes | J. Muñoz Arteaga | F. Acosta Escalante | F. J. Álvarez Rodríguez | A. E. Muñoz Zavala
diagnósticos que incluye secuela de cirugía, distrofia de la mano y
eventos cerebro vascular principalmente.

Figura 4. Modelo de conocimiento en forma de tarea del

usuario: terapia asignada para la rehabilitación de la mano.
El modelo computacional [13] para el ambiente interactivo
producido se compone de una plataforma Web creada a partir del
lenguaje JavaScript para generar el escenario y objetos 3D,
Figura 3. Rehabilitación de la mano por medio de terapia además un sensor que permite captar el movimiento de los dedos
ocupacional en la unidad de terapia física DIF Aguascalientes, de la mano como medio de interacción del paciente con el
México. ambiente interactivo.
El tratamiento para estos pacientes es llevado por 2 especialistas
en rehabilitación física dentro de una sesión de 30 minutos la cual 5.3 Proyección
consiste en realizar actividades por medio de terapia ocupacional La proyección permite determinar la correspondencia de acuerdo
para recuperar la movilidad en las manos. Ver figura 3. al problema que se presenta y los usuarios objetivo [17]. El
especialista en terapia física involucrado en el proceso de diseño
5.2 Fase de Modelado del ambiente interactivo valida que efectivamente el ambiente
Con ayuda del especialista en terapia física se determina un interactivo producido, cuente con la retroalimentación requerida y
conjunto de movimientos como parte del tratamiento para cada sea el adecuado conforme al tratamiento y actividades de terapia
paciente, estos movimientos tienen su fundamento en las técnicas ocupacional requeridas.
documentadas de terapia ocupacional y se dividen en
movimientos de los dedos y movimientos del pulgar, tal como se 5.4 Exploración e integración
muestra en la tabla 2. Durante esta fase con apoyo del especialista en terapia física se
dieron instrucciones básicas a los pacientes sobre uso del
Tabla 2. Movimientos de la mano [16] ambiente interactivo, se describió en que consisten los escenarios
Tipo de Acción a realizar Imagen virtuales y los objetos 3D en escena con los que se van a
movimiento interactuar, y como usar el sensor de movimiento, por ultimo se
indico como realizar las actividades de terapia ocupacional
Movilidad Con la punta del dedo pulgar,
descritas en la tabla 2. Tal como se presenta en la figura 6.
de los dedos intentar tocar la yema de cada uno
de los otros dedos.

Oposición Llevar el talón del dedo pulgar a

del pulgar la base de los dedos, comenzando
por el índice y terminando por el
meñique

Estos movimientos permiten definir modelo de tareas del usuario

con el conjunto de interacciones a simular en el ambiente
interactivo, e identificar variables importantes para la
retroalimentación del experto y el paciente, tales como, numero de Figura 6. Rehabilitación de la mano por medio de terapia
sesiones programadas, duración de la sesión de terapia, avance ocupacional en terapia física del DIF Aguascalientes, México.
registrado, entre otras. Tal como se muestra en la figura 4. Se estableció una sesión de 20 minutos en promedio por cada
paciente donde realizo las tareas indicadas por el ambiente
interactivo y la información captada en esta sesión permito
obtener información que va desde la percepción de los pacientes a
una nueva forma de llevar su proceso de recuperación por medio
de la utilización de la tecnología hasta la forma de cómo estos
ambientes ayudan a asiste el proceso de rehabilitación y obtener
información para la retroalimentación para el especialista y el
paciente.

5.5 Fase de Resultados

Los resultados permiten definir nuevas variables que permitan la
retroalimentación del modelo con esto se puede garantizan en lo
posible el modelo adecuado para asistir el proceso de
rehabilitación del usuario. Ver tabla 3.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 278 |
 Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos
H. Cardona Reyes | J. Muñoz Arteaga | F. Acosta Escalante | F. J. Álvarez Rodríguez | A. E. Muñoz Zavala
Tabla 3. Tabla de resultados de uso de ambientes interactivos Chavarría del DIF Aguascalientes por las facilidades puestas para
en pacientes del DIF Aguascalientes, México. la realización del caso de estudio presentado en este trabajo.
Paciente Grado Velocidad de Mov. Movimientos 8. Referencias
de movimientos de los de alcance
[1] Organización Mundial de la Salud, Discapacidad y salud,
Fuerza dedos
Nota descriptiva N°352, Septiembre de 2013, (Citado el 29
Paciente Medio Medio 3/4 Completado de agosto 2014), http://www.who.int:
A http://www.who.int/mediacentre/factsheets/fs352/es/
Paciente Medio Medio 3/4 Completado [2] Jackson, J., et al., Occupation in lifestyle redesign: the Well
B Elderly Study Occupational Therapy Program. Am J Occup
Paciente Medio Medio 4/4 Completado Ther, 1998. 52(5): p. 326‐36.
C [3] Paterno, F. Task models in interactive software systems. In
Paciente Bajo Bajo 3/4 Completado IN HANDBOOK OF SOFTWARE ENGINEERING AND
D KNOWLEDGE.
Paciente Bajo Bajo 1/4 Completado [4] Burke, J.; McNeill, M.; Charles, D.; Morrow, P.; Crosbie, J.
E & McDonough, S. Optimising engagement for stroke
rehabilitation using serious games The Visual Computer,
Los resultados obtenidos son producto de la realización de las Springer-Verlag, 2009, 25, 1085-1099
actividades descritas en la tabla 2. Estos resultados son de utilidad
tanto al experto como al paciente para conocer la situación actual [5] Cuppens, E.; Raymaekers, C. & Coninx, K. Gilroy, S. &
en cuanto a la movilidad de las manos, los ambientes interactivos Harrison, M. (Eds.) A Model-Based Design Process for
expuestos en este experimento permitieron que el paciente Interactive Virtual Environments Interactive Systems.
conociera nuevas formas de interacción para poder realizar su Design, Specification, and Verification, Springer Berlin
sesión de rehabilitación. Heidelberg, 2006, 3941, 225-236
[6] Karunanithi, M. & Sarela, A. Yogesan, K.; Bos, L.; Brett, P.
5.6 Evaluación & Gibbons, M. (Eds.) A Home-Based Care Model of Cardiac
Al concluir la fase de exploración con los pacientes, un Rehabilitation Using Digital Technology Handbook of
cuestionario de usabilidad basado en escenarios [18] es aplicado, Digital Homecare, Springer Berlin Heidelberg, 2009, 329-
con el propósito de ser un soporte para medir la satisfacción de los 352
usuarios en relación con la facilidad de uso de los sistemas
informáticos. La aplicación de este cuestionario a los pacientes [7] Krótkiewicz, M.; Wojtkiewicz, K. & Kardas, R. Piȩtka, E. &
participantes en el caso de estudio se realizo por medio de la Kawa, J. (Eds.) ACL Rehabilitation Enhanced by 3D
observación, un punto importante a considerar es que muchas Modeling Software Information Technologies in
veces los usuarios con discapacidad no les es tan sencillo poder Biomedicine, Springer Berlin Heidelberg, 2010, 69, 439-446
contestar un cuestionario como lo haría una persona sin [8] Lin, C.-Y.; Chen, S.-H.; Wu, M.-J.; Liao, Y.-S.; Hsien, S.-L.
discapacidad, ya que en algunos casos se requiere la asistencia del & Guo, C.-H. Zhang, Y. (Ed.) Application of Interactive
profesional en rehabilitación por lo que se deben definir Interface Design on Rehabilitation for Children with
estrategias para la evaluación de la usabilidad para este tipo de Cerebral Palsy Future Communication, Computing, Control
usuarios. and Management, Springer Berlin Heidelberg, 2012, 142,
361-367
6. CONCLUSIONES
[9] Shin, J.-H.; Ryu, H. & Jang, S. A task-specific interactive
Este trabajo resalta la importancia de contar con una método para
game-based virtual reality rehabilitation system for patients
el diseño de ambientes interactivos como una solución donde se
with stroke: a usability test and two clinical experiments
capturan no sólo los aspectos funcionales de los usuarios, sino
Journal of NeuroEngineering and Rehabilitation, BioMed
también aquellos que se relacionan con el contexto de la
Central, 2014, 11
rehabilitación, además de considerar las mejores prácticas
utilizadas por los especialistas en rehabilitación y llevarlas a un [10] Smith, S.; Talaei-Khoei, A.; Ray, M. & Ray, P. Brahnam, S.
espacio de rehabilitación interactivo. El caso de estudio permitió & Jain, L. (Eds.) Agent-Based Monitoring of Functional
probar el método propuesto y dar seguimiento a la problemática Rehabilitation Using Video Games Advanced Computational
planteada. La puesta en marcha de ambientes interactivos en los Intelligence Paradigms in Healthcare 5, Springer Berlin
pacientes del DIF para la rehabilitación de la mano, permitió Heidelberg, 2011, 326, 113-141
obtener resultados para la retroalimentación de los expertos y [11] Yeh, S.-C.; Lee, S.-H. & Fan, Y.-C. Huang, Y.-M.; Chao, H.-
pacientes además de observar un efecto cognitivo de una mejor C.; Deng, D.-J. & Park, J. J. (J. H. (Eds.) The Development
percepción de los objetos en relación con el miembro en of Interactive Shoulder Joint Rehabilitation System Using
rehabilitación. Por ultimo, los ambientes interactivos permiten Virtual Reality in Association with Motion-Sensing
adaptar el entorno a las necesidades de los usuarios y Technology Advanced Technologies, Embedded and
especialistas, lo que brinda a los usuarios contar con un soporte en Multimedia for Human-centric Computing, Springer
su proceso de rehabilitación. Netherlands, 2014, 260, 1073-1082
7. AGRADECIMIENTOS [12] Zhang, S.; Hu, H. & Zhou, H. An interactive Internet-based
El presente trabajo agradece por todo el apoyo para el desarrollo system for tracking upper limb motion in home-based
de esta investigación al CONACYT, a la Lic. Anita Aranda

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 279 |
 Terapia Ocupacional para Personas con Discapacidad Física utilizando Ambientes Interactivos
H. Cardona Reyes | J. Muñoz Arteaga | F. Acosta Escalante | F. J. Álvarez Rodríguez | A. E. Muñoz Zavala
rehabilitation Medical & Biological Engineering & [16] Traumatología Hellín, Ejercicios de mano y muñeca, (Citada
Computing, Springer-Verlag, 2008, 46, 241-249 el 10 de octubre 2014),
[13] Sanchez, J. A model to design interactive learning http://traumatologiahellin.wordpress.com/ejercicios/ejercicio
environments for children with visual disabilities Education s-de-mano-y-muneca/.
and Information Technologies, Kluwer Academic Publishers- [17] Billi, M.; Burzagli, L.; Emiliani, P.; Gabbanini, F. &
Plenum Publishers, 2007, 12, 149-163 Graziani, P. Miesenberger, K.; Klaus, J.; Zagler, W. &
[14] Secretaria de la Salud, Catálogo Maestro de Guías de Karshmer, A. (Eds.) A Classification, Based on ICF, for
Práctica Clínica (CMGPC), (Citada: 30 de marzo 2015), Modelling Human Computer Interaction Computers Helping
http://www.cenetec.salud.gob.mx: People with Special Needs, Springer Berlin Heidelberg,
http://www.cenetec.salud.gob.mx/contenidos/gpc/catalogoM 2006, 4061, 407-414.
aestroGPC.html [18] Lewis, James R., IBM Computer Usability Satisfaction
[15] Sistema para el Desarrollo Integral de la Familia Questionnaires: Psychometric Evaluation and Instructions for
Aguascalientes,DIF, (Citada: 10 octubre 2014), Use, International Journal of Human-Computer Interaction
http://www.aguascalientes.gob.mx/dif/. 1995 v.7 n.1 p.57-78

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 280 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro

Experiencias de evaluación de herramientas tecnológicas

para la asistencia de personas con discapacidad cognitiva

Juan C. Torrado Javier Gómez Germán Montoro

Universidad Autónoma de Madrid Universidad Autónoma de Madrid Universidad Autónoma de Madrid
Escuela Politécnica Superior Escuela Politécnica Superior Escuela Politécnica Superior
c/ Francisco Tomás y Valiente, 11 c/ Francisco Tomás y Valiente, 11 c/ Francisco Tomás y Valiente, 11
+34914972292 +34914972292 +34914972210
[email protected] [email protected] [email protected]

RESUMEN ¿A dónde queremos llegar? ¿Qué tipo de pruebas debemos

Este artículo describe un conjunto de técnicas, recomendaciones y realizar? ¿En qué fase del desarrollo? ¿Incluimos o no a los
métricas para validar herramientas software que tengan como usuarios en el proceso?
objetivo la asistencia a personas con discapacidad cognitiva. A El proceso de desarrollo es el objeto de estudio de la disciplina
continuación se describen tres experiencias reales de evaluación llamada Ingeniería del Software, que proporciona, entre otras
de tres herramientas de este tipo (para guiado en tareas cotidianas, cosas, modelos de proceso que esquematizan y regulan el
desplazamiento en interiores y desplazamiento en exteriores), en desarrollo de software. Estos modelos de proceso distinguen
las cuales se aplican total o parcialmente estas métricas y varias fases a lo largo del desarrollo, entre los que se incluyen la
recomendaciones. Por último, se discute cuáles de ellas han extracción de requisitos, el análisis, el desarrollo, la
resultado más válidas para describir la usabilidad de los sistemas implementación, la evaluación y el mantenimiento. Como vemos,
propuestos, y se señalan posibles vías de mejora de las mismas. estos modelos de proceso incorporan el proceso de evaluación en
el plan, y las distintas fases se llevarán a cabo de una forma u otra
Palabras clave y en un determinado orden dependiendo del modelo de proceso
Evaluación de usabilidad, discapacidad cognitiva, experimento concreto que se haya escogido. Estos modelos de proceso se
con usuarios, métricas, experiencias de uso prescriben teniendo en cuenta el tipo de software que vamos a
desarrollar, y en qué circunstancias.
1. INTRODUCCIÓN
Cualquier tipo de herramienta tecnológica que quiera ser Por tanto, cabe preguntarse a continuación: ¿existen modelos de
difundida o puesta en el mercado ha de ser evaluada previamente. proceso y tecnologías concretas y normalizadas para evaluar
Esto se aplica de la misma forma a tecnologías para la asistencia, software destinado a este tipo de usuarios? Podemos afirmar que
que son “aquellas que facilitan la ejecución de actividades o la es un área poco trabajada desde el punto de vista formal, y
interacción con el entorno físico y social de las personas en entender la tendencia a la hora de evaluar tecnologías para la
situación de discapacidad” [13]. Todo tipo de tecnología utilizada asistencia implica hacer una revisión de las publicaciones al
para ayudar a estos usuarios debe asegurar que ha sido probada en respecto de estos productos y propuestas, pero desde una
las circunstancias suficientes como para que se desempeñe su uso perspectiva diferente, es decir, estudiar cómo han evaluado los
adecuadamente. Por ejemplo, una silla de ruedas deberá pasar investigadores sus propias propuestas, en lugar de centrarse en las
múltiples pruebas de resistencia y ergonomía, al igual que unas mismas.
muletas prostéticas o unas gafas especiales.
En el Laboratorio de Inteligencia Ambiental (AmILab) hemos
En este artículo nos centraremos en un tipo particular de desarrollado tres herramientas software destinadas a asistir a
tecnologías para la asistencia, que son las denominadas “de alta personas con discapacidad cognitiva y situación de inclusión
tecnología”, y que comprenden todas aquellas soluciones laboral en sus tareas de la vida diaria [9]. En concreto, son tres
relacionadas con Tecnologías de la Información (TIC). aplicaciones Android instaladas en sus smartphones que les
Actualmente están creciendo las líneas de investigación que se proporcionan guías interactivas para realizar tareas cotidianas
centran en este tipo de tecnologías para la asistencia, por lo que el (AssisT-Task), guiado en exteriores a partir de localización GPS
método de probarlas y evaluarlas planteará un problema creciente (AssisT-Out) y guiado en interiores a partir de códigos QR
a resolver. Se trata de un problema cuya dificultad radica en la (AssisT-In). A la hora de realizar su evaluación, con el objetivo de
naturaleza compleja de la propia discapacidad cognitiva [5]. difundirlas y mejorarlas, surgieron los interrogantes que
Además, cuando ya se trata de herramientas software, la planteamos anteriormente. Al no encontrar ninguna metodología
evaluación se concibe de distinta manera, ya que un programa no orientada específicamente a validar software para personas con
es un objeto físico que podamos evaluar de la misma manera que discapacidad cognitiva, planificamos esta fase a partir de la
haríamos con otro tipo de tecnologías para la asistencia como las recopilación de técnicas puntuales utilizadas por otros
prótesis. investigadores a la hora de validar sus herramientas.
En estos casos, surgen varias preguntas: ¿Cuándo se considera que En este artículo analizamos las técnicas que se suelen utilizar en la
una herramienta software de estas características está evaluada? fase de evaluación de este tipo de sistemas, así como su nivel de

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 281 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
adecuación a estos casos. Por otra parte, describiremos el conjunto visitar centros universitarios y de investigación. Estos consejos
de técnicas y métricas que hemos utilizado para evaluar nuestras también son válidos para cuando las sesiones de entrenamiento se
propias herramientas, con el objetivo de aportar elementos de base llevan a cabo en un centro laboral que aún no conocen, y donde
a posibles formalizaciones de una metodología para estos casos. desempeñarán unas tareas. Estos centros suelen contar con tutores
propios para estos casos.
2. TÉCNICAS DE EVALUACIÓN
En esta sección analizamos el conjunto general de técnicas y Por otra parte, en ocasiones es necesario realizar las sesiones en
metodologías empleadas en la evaluación de software en cuanto a los propios centros. Esto puede deberse a que el experimento
su adecuación a la validación de herramientas de asistencia para requiere material inmueble de los mismos, o que por razones que
personas con discapacidad cognitiva. Esta adecuación se valora atañen a los propios usuarios, resulta inviable realizar la visita.
teniendo en cuenta el uso que se le ha dado en experimentos Esto supone para ellos un ambiente de confianza, aunque también
evaluativos realizados en la literatura, o bien la opinión de los es recomendable que los educadores estén presentes.
educadores con los que hemos tratado durante nuestros 2.1.2 Selección de usuarios
experimentos y los resultados obtenidos.
En muchas ocasiones la selección de usuarios vendrá motivada
En primer lugar acotaremos nuestro estudio a técnicas de por factores externos como la disponibilidad de los individuos y
evaluación que pertenecen al ámbito de metodologías centradas en sus educadores, o del material necesario para recrear las sesiones
el usuario (MCU [8]). La razón para hacerlo así es que el tipo de (por ejemplo, para realizar entrenamiento laboral, se debe tener el
usuario al que va dirigido. Por lo tanto resulta más adecuado partir mismo material que usarán en su puesto de trabajo). La dificultad
de metodologías de diseño centradas en usuarios, en general, para de lograr planificar una evaluación con un grupo de usuarios con
analizar cómo se pueden adaptar para usuarios con discapacidad discapacidad cognitiva implica que la mayoría de los
cognitiva, concretamente. Además, podemos descartar de inicio experimentos de la literatura cuenten con un número reducido de
técnicas de validación que implican un grado de participación del usuarios (en torno a la decena de individuos por experimento).
usuario medianamente complejo, como el card-sorting, el Cuanto mayor sea el número de participantes, más significativos
recorrido cognitivo con usuarios [10] o las cartas de reacción de serán los resultados de los análisis que se realicen sobre los datos
Microsoft [2]. de las sesiones (e.g. estadística descriptiva). Existen estudios con
grupos mayores de usuarios con discapacidad cognitiva, pero se
2.1 Planificación trata de encuestas telefónicas y experimentos en entornos no
Planificar una evaluación de software a través de controlados [7].
experimentos con usuarios implica planificar un número de
sesiones, unas tareas a realizar, un lugar en el que hacerlo y un Con un grupo numéricamente significativo de usuarios, se suelen
grupo de usuarios significativo. Ahora bien, realizar el estudio con realizar grupos, de tal forma que se puedan hacer comparaciones
entre medidas tomadas. El criterio de agrupación depende
personas con discapacidad cognitiva dificulta el acceso y la
enormemente del motivo que lleve a agruparlos, pero existen
planificación de varios de estos puntos.
varias tendencias:
2.1.1 Lugar de las sesiones • Nivel de discapacidad: los centros de educación especial
Estos usuarios ofrecen unas particularidades de comportamiento cuentan con varias escalas para clasificar la diversidad
que se deben tener en cuenta a la hora de escoger un lugar en el funcional cognitiva de sus estudiantes (nivel de
que realizar las sesiones que conformarán el experimento. dependencia, porcentaje de discapacidad, conjunto de
Podemos planificar las sesiones en un lugar desconocido para destrezas o perfil laboral, entre otros), y es conveniente
ellos (normalmente, el entorno académico que convenga a los que los evaluadores pidan esta información si desean
evaluadores). En un usuario sin discapacidad esto no supondría estratificar los grupos adecuadamente.
ningún problema, ya que el desconocimiento del lugar no suele
afectar a la experiencia de uso de un producto software. Sin • Cantidad de experiencia: si la evaluación requiere el
embargo, para estos individuos, puede suponer un problema manejo de una tecnología concreta (ordenador, teléfono
conductual porque la expresión del miedo o la incertidumbre es móvil, Smartphone o Tablet) se puede realizar un
más acuciada, y la concentración que mantendrá en el sondeo previo (a través de encuestas o entrevistas con
experimento no será la misma. Para paliar este problema personas del entorno) para calcular la experiencia previa
sugerimos varias alternativas. Por una parte, es muy conveniente de los usuarios en el uso de esa tecnología.
que los educadores y terapeutas con los que están familiarizados
estén presentes durante las sesiones. Servirán de ayuda para • Disponibilidad: por motivos de horarios académicos y
centrar a los usuarios y advertir a los evaluadores y observadores laborales, existe la posibilidad de que se tengan que
acerca de qué particularidades conductuales son normales en ese agrupar a los usuarios según su tiempo disponible para
usuario y cuáles son reacciones naturales al experimento. la evaluación, de no ser homogéneo.

A lo largo de experiencias evaluativas propias, hemos observado Aplicar estos criterios no implica necesariamente que los grupos
que también resulta muy conveniente hacer que los usuarios se de usuarios tengan que formarse agrupando individuos con las
familiaricen con los propios evaluadores. Inicialmente, esto se mismas características de cada clase. Puede resultar interesante
puede conseguir visitando los centros previamente para formar muestras estratificadas de usuarios. Por ejemplo, si
explicarles lo que van a hacer y para qué sirve, así como insistir observamos el nivel de discapacidad de una muestra determinada,
en presentarse y recibir preguntas. También es recomendable podemos formar los grupos de tal manera que haya una
planificar visitas previas a las sesiones al centro donde realizarán distribución homogénea de niveles de discapacidad en cada grupo,
las sesiones. Además, los centros de educación especial de tal forma que podamos compararlos como algo equivalente,
agradecerán el aporte académico que proporciona a los usuarios asegurándonos además de que cada uno de ellos presenta

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 282 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
variabilidad interna. Dado que el nivel de discapacidad es uno de 2.2 Métricas
los factores que más influyen en los resultados de este tipo de
experimento [11], es conveniente formar grupos equivalentes a 2.2.1 Métricas objetivas
partir de muestras con variedad de niveles de discapacidad. Si el Las métricas que vamos a presentar a continuación son aquellas
experimento, hecho de esta manera, ofrece resultados positivos que ofrecen resultados derivados de la observación y
(aplicando métricas que veremos en secciones posteriores), se cuantificación de eventos que se producen a lo largo de las
puede afirmar que el software evaluado ofrece soporte para un sesiones de evaluación. Se han seleccionado estas a partir de la
rango amplio de usuarios con discapacidad. Este es uno de los combinación de las métricas clásicas de experimentos de
resultados más buscados a la hora de diseñar estos productos, ya usabilidad con usuarios, recopiladas por Tullis y Albert [1], y
que la discapacidad cognitiva es por naturaleza compleja y difícil aquellas métricas que utilizan varios autores en la literatura en sus
de clasificar en subtipos, por lo que las herramientas que experimentos con usuarios con discapacidad cognitiva. A su vez
presentan este tipo de tolerancia a niveles de discapacidad se señalan algunas medidas útiles que indican los educadores de
dispares son muy apreciadas en la investigación y en el mercado. estos individuos. También es frecuente utilizar medidas ad hoc
que atiendan a la naturaleza de la asistencia que se pretende
También atañe a la selección de participantes saber la manera en proporcionar con la herramienta.
las que vamos a tomar medidas y vamos a realizar las sesiones, en
cuanto a los siguientes modos: 2.2.1.1 Tiempo de completado
Si bien en la mayoría de experiencias con usuario, el tiempo que
• Medidas intra-sujeto (within-subjects): varias sesiones se tarda en desempeñar una tarea es una de las medidas más
por usuario en una misma tarea, para medir aprendizaje importantes a la hora de validar una herramienta software, en el
y tiempos; hay que tener en cuenta que estos usuarios se caso de individuos con discapacidad cognitiva no se trata de una
fatigan en mayor medida que otro tipo de usuarios, por medida tan relevante. Los educadores, a lo largo de nuestras
lo que hay que planificar las sesiones para evitar que experiencias de evaluación, insistieron en que tanto en el ámbito
afecte al rendimiento de las pruebas o buscar otras académico con el laboral, se prefiere que los usuarios completen
alternativas de compensación. También hay que tener en la tarea exitosamente a que lo hagan en un tiempo determinado. Sí
cuenta el aprendizaje acumulado de una sesión para es cierto que se atiende a excepciones en las que el tiempo de
otra, y si ello afecta a otros resultados al margen de la completado es exageradamente amplio, en cuyo caso se
curva de aprendizaje. observarán los motivos a través de otras métricas como la
• Medidas inter-sujeto (between subjects): se trata de confusión o las peticiones de ayuda; u ocasiones en las que los
comparar el rendimiento de varios usuarios para una individuos emplean un tiempo mínimo en completar la tarea, o en
única tarea. finalizarla, creyendo que la han completado exitosamente. Este
último caso es frecuente cuando los individuos presentan una
• Medidas mixtas: comparan grupos de usuarios conducta impulsiva frente al aprendizaje de tareas, que a su vez es
equivalentes a lo largo de varias sesiones, añadiendo un caso muy frecuente en individuos con discapacidad cognitiva.
significación a las medidas inter-sujeto y permitiendo, a
su vez, comparar el rendimiento de cada usuario Por tanto, es necesario medir este factor durante las sesiones pero
consigo mismo a lo largo de las sucesivas sesiones analizar los resultados que se derivan de él a través de la relación
(intra-sujeto). con el resto de métricas, en mayor medida que en las siguientes.

Escoger el modo de realizar las sesiones y la configuración de 2.2.1.2 Fluidez

sujetos depende enormemente del objetivo de la evaluación. Si Ya que, como hemos visto, el tiempo de completado comprende
queremos realizar una comparación entre dos elementos de aspectos muy amplios y contingencias muy variadas de la
asistencia (por ejemplo, el utilizado tradicionalmente en su actividad del usuario con diversidad funcional de carácter
formación en comparación con la propuesta tecnológica del cognitivo, es necesario definir métricas nuevas que reflejen el
estudio en cuestión), es necesario realizar medidas inter-sujeto. Si, carácter temporal del rendimiento de los usuarios de forma más
por otra parte, se desea analizar directamente el aprendizaje de los precisa. Proponemos para ello medir la fluidez con la que se
usuarios con el uso de la tecnología aplicada, se deberían realizar desempeñan las subtareas que comprenden la tarea realizada en
varias sesiones y efectuar medidas intra-sujeto. Las medidas cada sesión. Entendemos fluidez como la distribución homogénea
mixtas, por tanto, son idóneas para cuando se quiere realizar un y consecuente del tiempo empleado en completar dichas
estudio simultáneo de la mejora que produce la aplicación de subtareas.
tecnología en la realización de tareas para estos usuarios y el Esto se puede medir de forma cualitativa, estableciendo una escala
aprendizaje que obtienen con esa ayuda a lo largo de las primeras de fluidez y asignando una puntuación dentro de dicha escala,
etapas de uso. deducida a partir de la observación de las sesiones (teniendo en
Desde un punto de vista formal, tener en cuenta este tipo de cuenta el factor subjetivo que conlleva esta asignación deliberada)
circunstancias e integrarlas en una experiencia de usuario o calcular un valor de fluidez a partir de los tiempos medidos
caracterizan este tipo de validación como de inspección (los objetivamente entre cada par de subtareas. Hacer uso de algún
expertos, que en este caso son los educadores, observan la cálculo de estadística descriptiva como la media o la desviación
herramienta y sugieren cambios), de indagación (hablando con los típica de estos tiempos dentro de una sesión pueden dar una idea
usuarios y obteniendo información directa de ellos y de su de si estos tiempos presentan un carácter homogéneo o si, por otra
actividad usual relativa a la herramienta) y de test (poniendo a los parte, se ha tardado mucho más en desempeñar una subtarea que
usuarios a utilizar la herramienta a lo largo de una serie otra, derivando en un valor de fluidez comprometedor.
planificada de sesiones) al mismo tiempo [10].

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 283 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
Si el análisis de la fluidez evidencia que este valor se altera en una acción que impide el éxito de la tarea, mientras que el segundo es
misma subtarea para varios usuarios, se debe estudiar, a través del la ausencia de acción, durante un intervalo de tiempo prolongado.
resto de métricas o de la observación directa de las sesiones, la Una ofrece un resultado de usabilidad global, y la otra aporta
razón que lo motiva. Debemos tener en cuenta además que, al datos sobre la actitud y el comportamiento concretos del usuario,
igual que ocurre con el tiempo de completado, este tipo de respectivamente.
métricas, de carácter temporal, se ven influenciadas en gran
medida por la diversidad conductual que presentan estos 2.2.1.6 Satisfacción o frustración
individuos, y cuyos efectos emplean tiempo en las sesiones (por Es necesario que los evaluadores observen atentamente los
ejemplo, individuos con trastornos del espectro autista, que comentarios de los usuarios durante las sesiones. En muchas
implican ayuda de los educadores para la comunicación con ellos ocasiones, se muestran muy expresivos (sobre todo, en ambiente
y la interpretación de sus gestos y reacciones a lo largo de las de confianza [12]) y comentan en voz alta las impresiones que van
sesiones). obteniendo de la experiencia (a modo de Thinking Aloud [3]).

2.2.1.3 Tasa de error 2.2.1.7 Peticiones de ayuda

Cuantificar los errores que comenten los usuarios durante las Se incluyen aquí peticiones de ayuda explícitas a los evaluadores
sesiones da una idea de la precisión con la que realizan la tarea. o educadores, o implícitas, cuando necesitan una intervención
Sin embargo, los educadores insisten en que uno de los aspectos pero no llegan a formular la petición, por timidez o bloqueo.
fundamentales que deben trabajar estos individuos es la
percepción del error. Si, al finalizar una serie de sesiones,
2.2.2 Métricas subjetivas
preguntamos a los usuarios si han realizado la tarea bien o mal, las La valoración subjetiva de la experiencia de uso se dificulta
respuestas muchas veces no coincidirán con la realidad. En enormemente en el caso de usuarios con discapacidad cognitiva,
algunos casos, la impulsividad les lleva a acabar cuando antes, y ya que los cuestionarios y las encuestas, que son la forma más
erróneamente, una tarea, afirmando tajantemente que lo han hecho común de obtener este tipo de valoraciones, implican cierto grado
bien. Otras, emplearán mucho tiempo y se bloquearán en el de comunicación con el usuario.
proceso, de forma que aunque finalicen correctamente afirmarán Por tanto, en caso de realizar entrevistas, tendrán que ser
haber completado erróneamente la tarea simplemente por las mediando a través de los educadores, quienes servirán de ayuda
dificultades que han encontrado durante la realización. para entender sus respuestas y para transmitir las preguntas.
Por ello, es importante complementar la medición de los errores Ocasionalmente, aunque el usuario sea capaz de entender y
con la pregunta a los propios usuarios de cómo creen que lo han responder a las preguntas del evaluador, no contesta
hecho. Aunque más tarde los educadores pueden puntuar en una adecuadamente por problemas conductuales o timidez y
escala más amplia cómo de bien lo han hecho, para ellos es desconfianza, en cuyo caso la presencia del educador sigue siendo
conveniente presentarles la pregunta de forma binaria, de manera necesaria. En el caso de realizar cuestionarios escritos, los
que no se añada más dificultad de la que ya les supone la propia evaluadores deben asegurarse de que todos los implicados tienen
autoevaluación. el nivel necesario para realizarlos, aunque incluso así se puede
seguir necesitando a los educadores.
2.2.1.4 Número de intervenciones En cuanto a los cuestionarios concretos, se suelen utilizar los
Como complemento a la tasa de error se propone incluir como cuestionarios normalizados de usabilidad (SUS [4], USE [15]),
factor de evaluación el número de veces que los evaluadores o siempre que se adapte el lenguaje para que los usuarios los
educadores han tenido que intervenir durante el proceso [6] [16], entiendan. Sin embargo, pese a la adaptación que se pueda
por las razones que sean. Recomendamos realizar estas realizar, existen muchos factores que restan utilidad a estos
intervenciones de forma mitigante (cuando el usuario haya cuestionarios. Como hemos señalado anteriormente, problemas
realizado alguna acción que le impida continuar) y no preventiva conductuales como la impulsividad o la dificultad de
(dejarle cometer el error, de tal forma que perciba claramente que comunicación hacen que las respuestas puedan perder valor (un
ha fallado, y que no peligren otras métricas que se basan en el usuario impulsivo responde que sí a todo, u otorga la máxima
error). Muchos investigadores observarán que el número de puntuación a todas las preguntas, con tal de acabar rápido, y otros
intervenciones arrojan resultados mucho más significativos que la usuarios confundirán el entender la pregunta con el entendimiento
tasa de error, ya que el concepto de error, como comentábamos, de la herramienta que han utilizado, sin mencionar la capacidad de
está menos definido con estos usuarios, mientras que la necesidad memorización y razonamiento que requiere contestar a este tipo
de intervenir está muy clara. Este resultado es particularmente de preguntas).
interesante cuando la herramienta que se valide sea una propuesta
de mejora de su inserción laboral, ya que en este ámbito la 3. EXPERIENCIAS DE EVALUACIÓN
autonomía del usuario es el objetivo óptimo. En esta sección describiremos tres experiencias de evaluación con
2.2.1.5 Confusión usuarios con discapacidad cognitiva que llevamos a cabo el
Laboratorio de Inteligencia Ambiental (AMILAB) de la Escuela
Son momentos de confusión aquellos en los que el usuario se
Politécnica Superior de la Universidad Autónoma de Madrid, con
bloquea y no realiza ninguna acción representativa para avanzar
la colaboración de la Fundación Síndrome de Down de Madrid
con la tarea. Su presencia en las sesiones (o, visto de otra manera,
(FSDM) y el Instituto de Psico-Pediatría Doctor Quintero
su ausencia) da una idea de la precisión con la que los usuarios
Lumbreras (IPP).
utilizan la herramienta. Aunque es una métrica de significado muy
simple, los resultados que ofrece son enormemente significativos Los tres experimentos se enmarcan dentro del proyecto AssisT,
en el análisis del rendimiento de estos individuos. Se debe prestar cuyo objetivo es el desarrollo de herramientas software para
atención a no confundir error con confusión: el primero es una móviles que ofrezca una asistencia integral a individuos con

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 284 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
discapacidad cognitiva, abordando tres aspectos de su vida: los comentarios, errores, intervenciones y momentos de confusión), a
desplazamientos en interiores (AssisT-In), desplazamientos en partir de etiquetas que almacenamos en un fichero de registros.
exteriores (AssisT-Out) y tareas cotidianas (AssisT-Task). Se Por otra parte, la propia aplicación almacenaba en otro fichero de
propone ofrecer asistencia en estos tres ámbitos mediante el uso registros los datos de interacción con el dispositivo. Todos esos
de sus respectivas aplicaciones smartphone. datos fueron procesados de manera automática, recopilados y
etiquetados para cada usuario. De esta manera, obtuvimos los
Al tratar de validar las tres herramientas es cuando surgió la resultados de las métricas en comparación para uno u otro grupo,
necesidad de adaptar técnicas de evaluación a las necesidades y el así como su evolución a lo largo de las sesiones. De las métricas
contexto de estos usuarios. Dichas evaluaciones dieron lugar a las comentadas anteriormente, las peticiones de ayuda y la
experiencias que describimos a continuación. cuantificación de comentarios de satisfacción o frustración no
AssisT-Task resultaron significativas, ya que obtuvimos pocas muestras.
AssisT-Task proporciona guías interactivas para la realización de 3.1 AssisT-Out
tareas de la vida cotidiana, como cocinar, poner una lavadora o AssisT-Out asiste a usuarios con discapacidad cognitiva en sus
imprimir un documento. Los usuarios escanean con el smartphone desplazamientos en exteriores. A través del sensor GPS del
un código QR que contiene la etiqueta identificativa de la tarea, y dispositivo, se detecta la posición inicial del sujeto y se le ofrecen
reciben la ayuda para completar la tarea en cuestión. Estas guías imágenes a pie de calle adaptadas para guiarle a lo largo de su
interactivas están enriquecidas con distintas posibilidades, como recorrido hasta el destino seleccionado.
bifurcaciones, repetición de pasos y adaptación de secuencias de
pasos a los usuarios. De nuevo tuvimos acceso a los estudiantes de la FSDM; esta vez,
a 18 de ellos. El entrenamiento que reciben en este centro para el
Para probarla, contamos con un grupo de 10 usuarios de la FSDM guiado en exteriores consiste en el acompañamiento del individuo,
durante su capacitación laboral. A lo largo de su formación para que se va retirando progresivamente hasta que aprende la ruta por
obtener empleo, el entrenamiento que realizaban era a través de sí mismo. En este caso, el interés de validar la herramienta no
soporte en papel, que contenía la lista de instrucciones para residía en comparar nuestra propuesta con este soporte presencial
completar cada tarea, como hacer fotocopias, ordenar folios, (la necesidad de sustituir este soporte, más que por el rendimiento,
encuadernar libros, etc. Por tanto, el interés de evaluar nuestra es por el coste logístico y económico que requiere precisar de una
propuesta residía en la comparación entre el rendimiento con este persona en cada sesión de entrenamiento del usuario), sino con
soporte tradicional y el rendimiento con la herramienta. Por esta otros tipos de herramientas de guiado en exteriores de uso común,
razón separamos a los 10 usuarios en dos grupos (medidas inter- como Google Maps. Por tanto, dividimos a los usuarios en dos
sujeto), ambos con variabilidad interna en cuanto a niveles de grupos y precedimos de la misma manera que AssisT-Task:
discapacidad cognitiva, de forma que los podíamos comparar. De comparación cruzada (Latin Square) entre los dos grupos de
esta manera, un grupo realizaría la tarea con soporte tradicional y usuarios y dos rutas seleccionadas.
el otro con AssisT-Task. Sin embargo, también queríamos probar
con dos tareas diferentes, para evitar sesgar nuestro experimento Previamente, realizamos varias presentaciones ante el grupo de
por la selección de la tarea. Así, un grupo realizó una tarea con estudiantes con el objetivo de generar un ambiente de confianza e
soporte tradicional y la otra con AssisT-Task, y viceversa para el informarles del experimento y sus condiciones. En el inicio de
otro grupo, a modo de Latin Square. Una tarea fue realizar cada sesión realizábamos un recorrido mínimo (andar unos metros
fotocopias (más destreza manual) y la otra archivar contratos por y doblar una esquina) con cada usuario para explicarle el
año, código identificativo y orden alfabético (más destreza funcionamiento de una aplicación, y a continuación se realizaba la
cognitiva). Era interesante, por otra parte, estudiar el aprendizaje ruta completa. Un evaluador utilizó una cámara para obtener una
que acumulaban con ambos soportes, de cara a su capacitación grabación en tercera persona, y el evaluado llevó durante la sesión
laboral, por lo que repetimos esta disposición a lo largo de 7 una cámara GoPro para obtener una vista en primera persona. Esta
sesiones (medidas intra-sujeto). segunda modalidad de registro nos permitía añadir más medidas al
experimento, como cuantificar la atención al móvil o al entorno,
Debido a razones de material, elegimos como lugar de las obtener comentarios y percibir la experiencia del usuario con más
sesiones su centro de estudios, donde se recrearon los escenarios claridad.
de empresa que se van a encontrar en sus puestos de trabajo: un
rellano con fotocopiadora y una habitación de archivo. Para Las sesiones fueron espaciadas, con semanas intermedias sin
minimizar la necesidad de presencia de los educadores por sesiones, pues el experimento era exhaustivo y queríamos mitigar
motivos conductuales de los estudiantes, los evaluadores nos en la medida de lo posible los efectos de la fatiga y el aprendizaje
presentamos previamente a lo largo de varios encuentros en los acumulado sobre nuestras medidas. Después de cada sesión,
que les explicábamos el experimento y el interés que motivaba realizábamos un test corto en el que mostrábamos imágenes a los
validar la aplicación, así como la utilidad en el futuro que podría usuarios; algunas pertenecían a la ruta realizada y otras no.
tener para ellos mismos. Preguntándoles si habían pasado por ahí, obteníamos una medida
adicional del reconocimiento espacial que habían adquirido.
Una vez planificado el lugar, los usuarios, su disposición, y el
esquema de sesiones, llevamos a cabo las mismas. Para ello, dos Se procesaron los datos de la misma manera que AssisT-Task, con
evaluadores observamos in situ la actividad de los usuarios, la salvedad de que introdujimos dos medidas nuevas: si llegaban o
tomando notas y grabando en vídeo la misma. no al destino (no confundir la tasa de error, que es el número de
errores puntuales que cometen a lo largo del recorrido, no miden
Posteriormente, analizamos los resultados a partir de un visionado el éxito global de la tarea) y el porcentaje de atención que
detallado de los vídeos, en los que detectábamos cada prestaban al teléfono y al entorno; esta última es necesaria en
eventualidad en la actividad del usuario (inicio y fin, subtareas, tanto a que una aplicación que ofrece un guiado mientras se

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 285 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
desplazan por la calle no debe acaparar una atención excesiva: no las particularidades de futuras experiencias y casos de evaluación
debemos olvidar que la intención principal de este método de exijan la aplicación de métricas nuevas, o la eliminación de
asistencia sería el aprendizaje de las rutas y la adquisición de algunas aquí propuestas. Como se ha comentado anteriormente, la
autonomía (en definitiva, retirar la asistencia progresivamente). misma naturaleza compleja de la discapacidad cognitiva es un
obstáculo para la generalización de técnicas y afirmaciones
3.2 AssisT-In categóricas.
AssisT-In ofrece guiado en interiores a partir de códigos QR.
Estos códigos se utilizan para etiquetar puntos de un entorno, de Desde el planteamiento de las distintas opciones acerca de dónde
tal manera que la aplicación ofrece asistencia interactiva para realizar el experimento, se sugirió en cualquier caso la presencia
llegar a un destino desde un punto inicial. de educadores o personas conocidas. Ciertamente, en los tres
casos de evaluación este hecho resultó indispensable: en AssisT-
Para la validación de esta aplicación contamos con un grupo de 14 Task y AssisT-Out, los propios evaluadores se dieron a conocer
usuarios del IPP, con índices de discapacidad cognitiva muy previamente a los estudiantes pese a que los entornos eran su
variados, y algún individuo con trastorno del espectro autista. Era centro de estudios y alrededores; y en AssisT-In, el experimento
necesario probar el prototipo en un entorno que no conocieran, no se hubiera podido llevar a cabo sin el acompañamiento de los
para que la asistencia fuera totalmente necesaria, por lo que las educadores a lo largo del trayecto. Esto supone un problema
sesiones tuvieron lugar en la Escuela Politécnica Superior de la logístico al organizar el experimento, pero se ha comprobado que
Universidad Autónoma de Madrid (divididas en dos días amortigua posibles problemas conductuales que afectarían a las
diferentes, para facilitar el acceso a todo el grupo de alumnos al medidas tomadas durante las sesiones.
centro). Los usuarios comienzan escaneando un código QR y
seleccionando un destino, y la aplicación les va mostrando La elección de usuarios también está sujeta a vicisitudes externas
imágenes de los puntos intermedios que deben ir visitando y al experimento, como su disponibilidad, la de sus educadores, y el
escaneando, hasta que llegan al destino. Para facilitar el acceso al número de usuarios. Partiendo de la base de que en nuestros
centro, dividimos las sesiones en dos días, con 7 usuarios y sus experimentos este hecho no supuso gran problema, ya que
respectivas sesiones cada uno. En el experimento contamos con la dispusimos de grupos completos de alumnos, la elección se llevó
presencia de 2 educadoras del centro, que sirvieron como puente a cabo bajo criterios de discapacidad cognitiva. En AssisT-Task y
de comunicación con usuarios con trastorno del espectro autista y AssisT-Out, donde eran más numerosos y además teníamos que
para adaptar los cuestionarios. Antes del experimento, realizamos comparar el rendimiento con otra herramienta, realizamos dos
un cuestionario sobre el uso de las tecnologías para confirmar la grupos, organizamos dos tareas diferentes, y realizamos una
idoneidad de los smartphones como plataforma para estas comparación cruzada. La composición interna de ambos grupos
aplicaciones. estuvo estratificada, de forma análoga, entre ellos. Posteriormente,
esto nos permitió aplicar estadística descriptiva asumiendo que
En este caso, no había soporte previo con el que compararlo, y ambos grupos eran comparables. Sin embargo, para otros
tampoco había un interés por parte de los educadores por que experimentos, sugerimos analizar con cautela si, de cara al
aprendiesen este entorno en concreto, por lo que se realizó una análisis de resultados, la composición interna de cada grupo
única sesión para cada usuario (medidas inter-sujeto). Además, podría ser homogénea. Por otra parte, las medidas intra-sujeto, es
realizamos con cada uno una breve sesión de entrenamiento con decir, la repetición de sesiones a lo largo del tiempo, resultó ser
tres códigos QR para enseñarles el manejo básico de la aplicación. más valiosa, porque nos dio información detallada del aprendizaje
Después, realizaban la ruta de test, acompañados de un de cada individuo, y el análisis conjunto de estos datos de cada
observador, con cámara de video, y una de las educadoras. Tras grupo nos dio ya una información similar a la extraída de los
cada sesión, se pasó un cuestionario SUS adaptado a los usuarios. estadísticos provenientes de las medidas inter-sujeto. No implica
esto que se deba descartar la primera, ya que proporciona datos
El análisis de los datos fue idéntico a AssisT-Task y AssisT-Out:
formales sobre la mejora producida por la aplicación del software
un análisis manual de los vídeos y uno automático de los registros
propuesto.
derivados del mismo y de los generados por la interacción del
usuario con el dispositivo. Además incorporamos las medidas de En cuanto a las métricas, como ya se advertía, existe mucha más
los cuestionarios pre-sesión y post-sesión como factores posibilidad de combinación. En dos de los tres experimentos
subjetivos para esbozar la impresión y la satisfacción de los realizados, añadimos medidas ad hoc que interesaban a raíz de las
usuarios con el sistema. particularidades de los mismos. Por otra parte, se comprobó que
algunas de las propuestas de índole general, no proporcionaron
4. DISCUSIÓN resultados significativos: en primer lugar, los comentarios de
En esta sección vamos a considerar la validez de las técnicas y satisfacción o frustración, ya que los usuarios fueron menos
métricas propuestas en la sección 2 a partir de su aplicación en las propensos de lo esperado a proporcionarlos; y por otra parte, las
tres experiencias de evaluación descritas en la sección 3. peticiones de ayuda resultaron ser muy escasas, de una manera
En primer lugar, es interesante señalar que, aunque los tres casos similar. Podría ser interesante entrenar a los usuarios previamente
de evaluación están aplicados a tres herramientas concretas, en el uso de técnicas como el Thinking Aloud, que proporcionaría
abarcan un rango de posibles ámbitos de asistencia (realización de directamente datos aprovechables para estas métricas, o decidir
tareas, desplazamiento en exteriores y desplazamiento en aplicarlas estudiando inicialmente si el grupo de usuarios del
interiores) capaz de englobar múltiples casos futuros de experimento es particularmente elocuente. También se observó
validaciones de herramientas software para asistir a estos que la métrica de fluidez no proporcionaba datos que aportasen
usuarios. Es por ello que consideramos que, a través de las algo diferente a la de tiempo de completado, por lo que quizá
experiencias descritas, la validez resultante resulta resulte interesante redefinirla, en caso de que se pueda extraer un
considerablemente significativa. En todo caso, no se descarta que dato aprovechable de ella, o directamente omitirla.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 286 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
Los cuestionarios de usabilidad fueron muy difíciles de aplicar a analizando, debe ser flexible y adaptable a las diversas
los usuarios, pese al esfuerzo de adaptación del lenguaje y particularidades que presentan estos individuos, así como
presencia de las educadoras. La impresión subjetiva de cada especificar un conjunto robusto de métricas de carácter objetivo y
usuario acerca de la herramienta es imprescindible, y de ello subjetivo que proporcionen un resultado de validez útil y que
depende en gran medida que posteriormente su aplicación en un reseñe el grado de usabilidad de la herramienta. También
caso real sea exitosa [7]. Por tanto, uno de los puntos más conviene que esta metodología establezca las técnicas particulares
importantes en los que se debe trabajar a raíz de estas impresiones de observación de las sesiones (por ejemplo, nuestro método
es la necesidad de idear un medio por el que obtener el grado de consistía en grabar las sesiones, en tercera persona, y a veces
satisfacción de los usuarios tras un experimento. De nuevo, la primera) así como las de medición y recogida de datos (en
complejidad y diversidad funcional de la discapacidad cognitiva nuestras experiencias, establecimos una serie de eventos a
es la gran barrera que obstaculiza este trabajo. contabilizar, y repasamos los vídeos marcando y etiquetando estos
sucesos).
5. CONCLUSIONES
En este artículo hemos hecho una recopilación de técnicas y Insistimos en que formalizar esta fase del proceso de desarrollo de
métricas que se suelen utilizar para evaluar experimentos de este tipo de tecnologías para la asistencia responde a la necesidad
validación software para usuarios con discapacidad cognitiva. La de estructurar y organizar los experimentos que se están
escasez de estándares y métricas formalizadas nos llevan a adaptar realizando en estas líneas de investigación, que han ido en
las que se utilizan ampliamente en el mundo de la Ingeniería de considerable aumento a lo largo de los últimos años [14]. De esta
Software. Muchas de estas técnicas implican la observación de las forma, los productos que salieran al mercado tras este proceso
características conductuales del conjunto de usuarios de quienes tendrían más posibilidad de éxito, menor tasa de abandono, y su
disponemos y medir el nivel de dificultad para los usuarios que asistencia sería, en definitiva, más efectiva.
supone aplicar algunas de ellas. Muchas de las adaptaciones que
se proponen consisten en adaptar el lenguaje que utilizan muchas
6. AGRADECIMIENTOS
técnicas como plataforma de obtención de información, minimizar Este trabajo ha sido parcialmente financiado por “e-Training y e-
la actividad cognitiva que suponen otras, y contar con la presencia Coaching para la integración socio—laboral” (TIN2013--44586--
R) y “eMadrid-CM: Investigación y Desarrollo de Tecnologías
de personas de confianza, como educadores o familiares; ya sea
bien para la asesoría durante la planificación de los experimentos, Educativas en la Comunidad de Madrid” (S2013/ICE-2715).
como en las propias sesiones. En cuanto a las métricas que 7. REFERENCIAS
proponemos, describimos el tiempo, al fluidez, la confusión, el
[1] Albert, W., & Tullis, T. (2013). Measuring the user
número de intervenciones, la tasa de error, las peticiones de ayuda
experience: collecting, analyzing, and presenting usability
y los comentarios de satisfacción y frustración como guías básicas
metrics. Newnes.
de qué se debería medir en un producto software para estos
usuarios; sin embargo, realizamos algunas especificaciones que se [2] Benedek, J., & Miner, T. (2002). Measuring Desirability:
deberían tener en cuenta a la hora de medir dichos factores. Por New methods for evaluating desirability in a usability lab
ejemplo, las peticiones de ayuda no siempre son explícitas (de setting. Proceedings of Usability Professionals
hecho, debido a la timidez y habilidades sociales de los usuarios Association, 2003, 8-12.
objetivo, casi nunca lo son), y las intervenciones deberían ser [3] Boren, T., & Ramey, J. (2000). Thinking aloud: Reconciling
mitigantes de errores, no preventivas, de manera que no sesguen theory and practice. Professional Communication, IEEE
los resultados de nuestro experimento. Transactions on, 43(3), 261-278.
Para ilustrar estas recomendaciones, hemos descrito también tres [4] Brooke, J. (1996). SUS-A quick and dirty usability
experiencias de evaluación donde tuvimos la oportunidad de scale. Usability evaluation in industry, 189(194), 4-7.
aplicar algunas de estas ideas. Además, cada una de ella ofrecía
[5] Chang, Y. J., Han, W. Y., & Tsai, Y. C. (2013). A Kinect-
particularidades distintas que requerían seleccionar unas técnicas based upper limb rehabilitation system to assist people with
u otras. Por ejemplo, la aplicación para asistir a través de cerebral palsy. Research in developmental disabilities,
smartphone en tareas de la vida cotidiana (AssisT-Task), suponía
la sustitución de un soporte antiguo en panel, por lo que las [6] Chang, Y. J., Kang, Y. S., & Huang, P. C. (2013). An
técnicas se orientaron a estudiar la validez de nuestra propuesta en augmented reality (AR)-based vocational task prompting
un sentido comparativo entre las dos ayudas. AssisT-Out asiste en system for people with cognitive impairments.Research in
los desplazamientos en exteriores, por lo que resultaba developmental disabilities, 34(10), 3049-3056.
particularmente interesante estudiar la atención que los usuarios [7] Dawe, M. (2006, April). Desperately seeking simplicity: how
prestaban a elementos del entorno: utilizamos una cámara GoPro young adults with cognitive disabilities and their families
durante las sesiones, y al final de las mismas hicimos un test de adopt assistive technologies. InProceedings of the SIGCHI
reconocimiento de lugares por los que habían pasado. Para el conference on Human Factors in computing systems (pp.
guiado en interiores teníamos la aplicación AssisT-In, para la cual 1143-1152). ACM.
desplazamos a los usuarios a un entorno desconocido, y les
pusimos a recorrerlo con la ayuda de la aplicación y la presencia [8] FDIs, I. (2009). 9241-210: 2009. Ergonomics of human
de una educadora. system interaction-Part 210: Human-centred design for
interactive systems (formerly known as 13407).International
Un objetivo interesante a partir de estas ideas consistiría en Organization for Standardization (ISO). Switzerland.
formalizar una metodología de evaluación de productos software [9] Gómez, J., Alamán, X., Montoro, G., Torrado, J. C., & Plaza,
orientados a la asistencia de personas con necesidades especiales. A. (2014). AmICog–mobile technologies to assist people
Esta metodología, por múltiples razones que hemos ido with cognitive disabilities in the work place. ADCAIJ:

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 287 |
 Experiencias de evaluación de herramientas tecnológicas para la asistencia de personas con discapacidad cognitive
J. C. Torrado | J. Gómez | G. Montoro
Advances in Distributed Computing and Artificial [13] ISO 9999:2011: Assistive products for persons with
Intelligence Journal, 2(7), 9-17. disability -- Classification and terminology UNE-EN
[10] Granollers, T., Perdrix, F., & Lorés, J. (2004). Incorporación [14] Lancioni, G. E., & Singh, N. N. (2014). Assistive
de usuarios en la evaluación de la usabilidad por recorrido technologies for people with diverse abilities. New York:
cognitivo. Interacción’04. Springer.
[11] Hällgren, M., Nygård, L., & Kottorp, A. (2014). Everyday [15] Lund, A. M. (2001). Measuring usability with the USE
technology use among people with mental retardation: questionnaire. Usability interface, 8(2), 3-6.
relevance, perceived difficulty, and influencing [16] Mortenson, W. B., Demers, L., Fuhrer, M. J., Jutai, J. W.,
factors. Scandinavian journal of occupational therapy, 21(3), Lenker, J., & DeRuyter, F. (2013). Effects of an assistive
210-218. technology intervention on older adults with disabilities and
[12] Harr, N., Dunn, L., & Price, P. (2011). Case study on effect their informal caregivers: an exploratory randomized
of household task participation on home, community, and controlled trial. American Journal of Physical Medicine &
work opportunities for a youth with multiple Rehabilitation, 92(4), 297-306.
disabilities. Work: A Journal of Prevention, Assessment and
Rehabilitation, 39(4), 445-453.

XVI Congreso Internacional de Interacción Persona-Ordenador | Interaction for People with Disability
| 288 |
ENGENDERING TECHNOLOGY (I)
 An ICT experience in Computer Women Role promotion: WikinformáticA! in Aragón
M. T. Lozano | R Trillo-Lado | M. Villarroya-Gaudó | A. Allueva | E. Cerezo

An ICT experience in Computer Women Role promotion:

WikinformáticA! in Aragón. Promoting the use of Wiki
tools and visualizing the role of women in ICT
María Teresa Lozano Raquel Trillo-Lado María Villarroya-Gaudó
Centro Universitario de la DIIS - EINA DIIS - EINA
Defensa de Zaragoza Universidad de Zaragoza Universidad de Zaragoza
[email protected] [email protected] [email protected]

Ana Allueva Eva Cerezo

Facultad de Veterinaria Universidad de Zaragoza DIIS - EINA
[email protected] Universidad de Zaragoza
[email protected]

ABSTRACT [4]. From a sociologist point of view, a minority is considered

below 30% of population, so women in computer science are a
Nowadays, the number of female students that enroll in clear minority. In the case of the University of Zaragoza, our
technical (architectural and engineering) degrees is quite low. home university, the same trend has been observed, considering
So, numerous initiatives have arisen to promote these degrees the same time period, despite in Figure 2 where yearly the
and encourage female students to develop activities related to presence of women has decreased 1%, from 25% to 15% in 10
technical areas. In particular, we have organized the contest years. Recent work from the University of Navarra shows same
called Wikinforma´ticA! in Arago´n. situation [7]. In other countries as the US [5] or other EU ones,
This contest was designed in order to achieve the following same behavior has been observed.
main goals: 1) encouraging female students to study degrees
related to Information and Communication Technology (ICT),
2) disclosing the history of prominent women in the field of ICT
in order to serve as role-model for new generations of female
students, and 3) promoting collaborative work by means of Wiki
tools and the use of search tools for images.
In this paper, we describe in detail how the context was
organized, its development, and the results obtained by the
different teams that participated.

Categories and Subject Descriptors

K.4 [Computers and Society]: Organizational Impacts:
Employment
General Terms
Human Factors
Figure 1: Evolution of students enrolled in computer
Keywords engineering at Spanish Universities (1998-2011). Source
Women in engineering, engineering perception, gender equality, [4].
engineering studies, DokuWiki, outreach activities.
1. MOTIVATION
From the 90’s the number of women enrolled in computer
science has decreased significantly, from almost 30% of
students to an actual 15%. In Spain, this situation comes
together with a change in the degree name from Computer
Science to Computer Engineering, produced at the early
ninety’s. Figure 1 shows the evolution of the students cursing
Computer Engineering at all the Spanish Universities (5 years
degrees, previous to the actual European grade system) Women
rep- resent between 20% and 15% of all the students the data
have been obtained from the Spanish National Statistics Institute

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 290 |
 An ICT experience in Computer Women Role promotion: WikinformáticA! in Aragón
M. T. Lozano | R Trillo-Lado | M. Villarroya-Gaudó | A. Allueva | E. Cerezo
University of A Corun˜a, we focus the organization of
WikinformáticA!
2. WIKINFORMÁTICA!
Early positive experiences with technology are recognized as
important to involve women in these fields. So, some col-
leagues at the University of A Corun˜a decided to organize a
contest oriented to High School students to provide them with a
good ICT experience. Moreover, they also encouraged us to
organize something similar at the University of Zaragoza. After
an initial contact among the members of both institutions and
motivated by the low percentage of female editors in Wikipedia
(currently less than 20%), we decided to promote the use of
Wiki tools and organized the contests Wikinform´aticA! in A
Corun˜a and Wikinform´aticA! in Arago´n, respectively.
These contests were celebrated inside the first edition of the
Semana de Anita Borg in Spain (https://wiki.fic.udc.
Figure 2: Evolution of students enrolled in computer es/semanaanitaborg), which was part of the Anita Borg
engineering at University of Zaragoza. Source [9]. Celebration 2015 (hashtag: #anitaborgbirthday).
1984 is the year considered as the beginning of the decrement of
women presence in programming1, particularly relevant is the 2.1 Anita Borg Birthday Celebration
fact that the women in CS were the first that begin to stop Anita Borg (1949-2003) is the inspiration for the community of
programming. Nowadays, this trend is not going to change, if women in Computer Engineering. Anita received her PhD in
no external action is done. 1981 from the New York University and worked in several
computer companies in the United States such as Xerox
An underline point on the EU agenda attending ICT is the Network Systems Laboratory. She was a researcher in computer
women representation. Citing words by Neeliec Kroes, Vice- science that developed synchronization mechanisms and fault
President of the European Commission and responsible for the tolerance for operating systems and web information systems.
Digital Agenda “This is not just about women’s equality; we She is famous because of her efforts to promote the presence of
need to get more women in digital jobs and ICT research for the women in technological areas. In 1987 she co-founded Sisters,
sake of our future competitiveness and prosperity” [3]. The lack an online community that nowadays has over 4,000 members
of women in ICT has to be considered as a loose of part of the from at least 54 countries around the world.
talent for this field, but also, as the reduction of the perspectives
to face the research and innovation, which would influence As it is said in Systers’ official webpage (http://anitaborg.
dramatically the final results. org/get-involved/systers) this community continues to serve this
purpose by providing women a private space to seek advice
It is necessary to encourage young people to view engineering from their peers, and discuss the challenges they share as
studies as a feasible and rewarding option, which is exciting and women technologists. Many Systers members credit the list for
a valuable contribution to the society. Gender diversity has helping them make good career decisions, and steering them
demonstrated better results in plenty of fields, stronger through difficult professional situations. At the final stage of
development and higher market success. Many companies are her career, in 1987, she founded the Anita Borg Institute for
already aware of these facts [6]. Governments, Universities and Women and Technology (ABI) (http://anitaborg.org), a digital
industry can take important actions to encourage this trend, as community for women in computing, headquartered in Palo
for example the action shown in [8]. Alto, United States. Today, ABI works with women
One of the specific measures to focus is the recruitment of technologists in over 50 countries, and partners with leading
women, in particular female student ambassadors and speakers academic institutions and fortune 500 companies.
from industry as role models [3]. From the UN, girls’ situation She also founded (in 1994) the Grace Hopper Celebration of
in ICT was detected a handicap, thus the Member States of the Women in Technology (GHC), the world‘s largest gathering of
International Telecommunication Union (ITU) agreed in 2010 to women in computing and one of the most important events in
celebrate the International Girls in ICT Day on the 4th relation to computers. It is a series of conferences designed to
Thursday of April, every year [2]. In particular, in the online bring the research and career interests of women in computing to
knowledge generation our contribution detected the lack of the forefront, that every year bring together thousands of
women. Particularly in Wikipedia less than 13% of editors are computer professionals.
women, even worse are the contribution of women researchers,
lower than 7% [1]. Not only the contributions are pour, but the A year after she died, Google took an initiative. Google decided
visibility is still worse. to promote the image of women in computers, creating the
Anita Borg Memorial Scholarship (from 2004). Every year
With all this background, with the objective of recruiting more computer students are prized, based on their excellence. The
women on CS and visibilizing the contribution of many others, principal idea of the Anita Borg Memorial Scholarship is to
counting with the collaboration of our colleges from the promote the labor of women in the computer society.
1
From this community, the Anita Borg Scholar Alumni (ABSA)
http://www.engadget.com/2014/10/20/ was created. It is formed by every woman having received the
what-happened-to-all-of-the-women-coders-in-1984/ prize or being finalist. The ABSA was created with the idea of

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 291 |
 An ICT experience in Computer Women Role promotion: WikinformáticA! in Aragón
M. T. Lozano | R Trillo-Lado | M. Villarroya-Gaudó | A. Allueva | E. Cerezo
continuing with the ideas of Anita Borg and continuing with her organization to formalize participation and the
mission: to create links among all these professionals. The creation of a user account.
ABSA is reachable in Facebook: Anita Borg Scholarship • Groups may be formed as determined by the
Alumni. secondary sch. However, groups from 9 to 15
2.2 WikinformáticA! in Aragón students are recommended. A teacher may be
WikinformáticA! In Aragón is a competition for groups of high responsible for several groups.
school students in which they develop a wiki on prominent • Participating teams may join at any time between the
women in the history of ICT. The objective is the visibility of date December 19th, 2014 and January 23rd, 2015.
women involved in technology. The action took place from However, the work must be completed before the end
December 19th, 2014 to February 6th, 2015, when the awards of January 26th, 2015.
ceremony took place.
• The Wiki should have a page detailing who is part of
It was organized by several members of different groups: De- the team.
partamento de Informática e Ingeniería de Sistemas (DIIS) de la
• The jury will be composed of members of the
Universidad de Zaragoza, Asociación de Mujeres Investigadoras organization. In case of incorporating partners and
y Tecnólogas (AMIT) and Red EuLES (Entornos uLearning en sponsors, their assessment will be considered.
Educación Superior), in collaboration with the Anita Borg
Scholarship Alumni (ABSA) association. • Registration is open from December 19th, 2014 until
one week before the end of the competition.
Each participant group developed a Wiki on prominent women
in the field of ICT. Participants were required, from a • Participating teams must finish its work before
photograph, to guess who was the woman (or who were the January 26th, 2015.
women) on it and to elaborate a biography in a Wiki format. • The winning team will be announced the week of
This biography was published in a Wiki that was created for the Anita Borg and the Wiki will be disseminated through
contest. In addition, it could contribute to Wikipedia in anyone the website of the DIIS of the Escuela de Ingeniería y
of the official languages of Aragón. Arquitectura (EINA) of the Universidad de Zaragoza.
In order to participate, teams were leaded by a teacher. The rest
of the participants in the group were students of a high school, a
2.4 Proposed Women
As it was said, the competition consisted of developing a Wiki
professional school or equivalent academic centers. Groups
on prominent women in the field of ICT. This set of women was
should be composed of at least three students and a
specified by the organization by providing the different student
recommended maximum of 15. The registration period began
teams some photographs of these women. The pro- posed
on December 19th, 2014 and remained open until a week before
pictures included not only well-known women inter- nationally
the end of the competition, i.e. until January 23rd, 2015.
such as Ada Byron and Barbara Liskov but also young
Participating teams were required to complete its work before
researchers and entrepreneurial that develop their activities in
the January 26th. The winning teams were announced on
January 31st. Arago´n such as Alicia Asín, María López Valdés and Belén
Masiá. Nevertheless, the students could also include other
2.3 Contest Rules biographies and photographs of other women related to ICT in
In order to the participation was accepted, a series of rules were their works.
to be accomplished. These roles are presented below: Once the different teams received the photographs provided by
• The competition consists of developing a Wiki on the organization, the students should investigate about who they
prominent women in the field of ICT. were. After that, they looked for the reasons why they were
important in technology and other data of their biography in
• The organization will provide the infrastructure of order to elaborate a Wiki with them.
Wikis and photographs of different women whose
career has been highlighted in the field of ICT. Thanks to the contest, the theme was treated in several High
Schools (http://hendrix-http.cps.unizar.es/dokuwiki/
• Contestants teams must guess who the women in the doku.php/noticias).
photographs provided by the organization are and
develop a biography of each one in a Wiki.
• To enter the contest, it is mandatory to include the
biography of the women proposed by the organization
and possible extra Wiki contributions to be evaluated.
• Although it is possible to include other biographies,
the jury will assess whether taking them into account
or not (especially if the number of members of the
group exceeds widely recommended). (a) 1st prize delivery (b) Winning team
• Groups able to participate in this contest should be 2.5 Awards Ceremony
formed by high school students and/or training courses The winning teams were announced on January 31st. The
students (or equivalent) that will include the awards ceremony was held on February 6th, 2015, at 18h. in the
supervision of a teacher from its center. The Aula Magna of the Paraninfo, Universidad de Zaragoza.
supervisor is the one to directly contact the

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 292 |
 An ICT experience in Computer Women Role promotion: WikinformáticA! in Aragón
M. T. Lozano | R Trillo-Lado | M. Villarroya-Gaudó | A. Allueva | E. Cerezo
At the beginning of this ceremony, the Anita Borg Week was This action took place on April 17 and has received the name
briefly introduced. Later on, awards were given to the winning Editatón por la visibilidad de las científicas en Wikipedia [1].
teams. Three prizes (first, second and third) were granted. The
jury who evaluated the Wikis developed by the different teams, 4. ACKNOWLEDGMENTS
consisted of professionals in the field of ICT. They, after a long Authors would like to heartily thank all the devoted students,
and arduous deliberation, given the high quality of the works researchers and engineers who made WikinformáticA! a
submitted to the contest, decided to award the following prizes: success. Authors also wish to thank FIC, AMIT, Red Eu- LES,
a) 1st prize: Enrique Osso, because of the quality of the content, DIIS, KeYStone Action COST IC1302, Cátedra Banco
b) 2nd prize: J. Lanuza, because of the quality of the content and Santander and Wikimedia for their help and support.
the originality in the format, and c) 3rd prize: Blecua, because of 5. REFERENCES
the quality of the content. In addition, all participants received a [1] Editatón por la visibilidad de las científicas en wikipedia.
certificate and there were gifts and surprises for every es.wikipedia.org/wiki/Wikipedia:Encuentros.
participant at the awards ceremony. Finally, our colleague María
Villarroya Gaudó gave the conference El enigma de las [2] Girls in ict. http://girlsinict.org.
Ingenieras en Informática. [3] More women in informatics research and education.
www.informatics-europe.org/images/documents/more-
2.6 Results women-in-informatics-research-and-education_2013.pdf.
The call for the registration of the teams in the contest was
published on December 19th, 2014, and the teams that [4] Spanish national statistics institute. www.ine.es.
participated had to finish their work before January 27th, 2015. [5] Women computer science grads: The bump before the
Despite the short period to develop the work, 98 High School decline. www.computerworld.com/article/2474991.
students, divided into 8 different teams participated and enjoyed
the experience very much. [6] Committee on Diversity in the Engineering Workforce,
National Academy of Engineering. Diversity in
In this project, most of the participants were teenagers. Thus, Engineering:Managing the Workforce of the Future. The
considering the current legislation issues related to personal data National Academies Press, 2002.
and privacy, we decided to ask only for the name of the teacher,
who supervised each team of students, in order to allow a team [7] G. Perez-Artieda, E. Gubia, E. Barrenechea,P. Sanchis, A.
of students to be registered in the quiz. Nevertheless, the team Lopez Martin, D. Astrain, D. Morato, J. Lopez Taberna,
was asked to put a page where they could provide information and I. Matias. Analysis of women enrollment in
about the members in its wiki. Some teams provided engineering programs at the public university of navarre. In
information about their members but others preferred not to Frontiers in Education Conference (FIE), 2014 IEEE, pages
show their identities. 1–8, Oct 2014.
The team which won the first prize was composed by 20 girls [8] M. Villarroya Gaudó, S. Baldasarri, M. Lozano, R. Trillo
and 5 boys, the team which won the second prize was com- Lado, A. Murillo, P. Garrido, and P. Molina Gaudó. Girls’
posed by 5 male-students, while the team in the third position day experience at the University of Zaragoza: attracting
did no provided information about its composition. The women to technology. XV International Conference on
percentage of female students in the rest of the teams where data Human Computer Interaction, pages 443–450, 2014.
about their composition was available was around 50%. [9] P. Zaragoza, A. Peiró, and M. Villarroya-Gaudó.
Most teams granted publication permission of their wikis. So, Diagnóstico de la Situación Respecto a la Igualdad de
they are available on the website of the DIIS, EINA of the Género en la Universidad de Zaragoza. University of
Universidad de Zaragoza (http://hendrix-http.cps. Zaragoza, 2011.
unizar.es/dokuwiki/doku.php/start). Thanks to this con- test, the
number of biographies of women technologists in the Internet
has been considerably increased. Also Wikipedia expressed its
interest to publish them. At last but not least,
We also would like to mention that every participant team
reached its objective of finding the women who corresponded
with images and prepared good Wikis, working collaboratively
and learning several ICT aspects.
3. CONCLUSIONS AND FUTURE WORK
The contest has been very successful, for the short term
available. There were around 100 participants. Given the good
results obtained there are already some companies interested in
sponsoring the next edition. This was the first edition of a
competition WikinformáticA!, which we plan to continue with,
annually, we hope that for a long time.
In relation to the goals of WikinformáticA!, along with L’Oréal,
the UMYC and Wikipedia, AMIT organized a session of in-
corporation of female scientists to the Wiki and the
improvement of the ones that already were represented there.

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 293 |
 Inclusion and promotion of women in technologies
B. Revelles-Benavente | L. Arroyo Prieto | N. Vergés Bosch

Inclusion and promotion of women in technologies

Beatriz Revelles-Benavente Lidia Arroyo Prieto Nùria Vergés Bosch
Researcher at GENTIC, UOC PhD candidate, UOC Investigadora doctora, COPOLIS UB
[email protected] [email protected] [email protected]

ABSTRACT 2013, according to Idescat, 53.9 % of women between 30-35

In the information and knowledge society gender transformations years-old hold a degree, higher than the Spanish (45 %) and the
have characterized the social changes of recent years, while European (39 %) average. However, there is still an important
information and communication technologies (ICT) have taken an unbalance between the areas of knowledge of graduates. In
essential role for social development. Nevertheless, even if Catalonia, only 23% of students who graduated with an
women have accumulated an incredible talent in order to confront Engineering and Architecture degree in 2012-2013 were women.
today’s challenges, the technological system seems not to be In the case of natural sciences the percentage of women was
prepared to include, retain and promote them through the 62,4%.
technological careers that they deserve. Research about gender
and technology related with female occupation has pointed out
several causes for female exclusion that should be addressed from
an interdisciplinary approach. Most of the research has been
focused on finding out the barriers that make more difficult to
access and promote women in technological occupations.
However, there is still a need to identify and foster enablers of
women’s full inclusion in technologies. Carrying out a mix
methodology in between quantitative and qualitative methods, we
aim at providing different solutions to facilitate and promote
women’s presence and retainment in technological careers.

Categories and Subject Descriptors

H.4.2. [Co-op]: Better practices in the enterprises –
experimentation for the workforce, gender, ICT.

General Terms Moreover, this talent is not translated into women’s occupation,
Management, Measurement, Design, Economics, Human Factors, especially in jobs related to technological fields. While the female
Experimentation, Documentation. workforce represents 47,2% of the total of employed population in
Catalonia in 2014, only 28,6% of people who work in the
Keywords Information and Communication Sector are women.
ICT professions, women’s empowerment, knowledge society,
gender

1. INTRODUCTION: WOMEN IN
TECHNOLOGICAL CAREERS
The information and knowledge society is characterized by the
preeminent role that information and communication technologies
(ICT) have undertaken for the development of social and cultural
practices. Gender transformations in these social and cultural
practices have been paralleled to the social changes of last year
albeit this does not include equal opportunities in determined
professional areas. Having demonstrated by far their talent to
confront today’s challenges, women are not included yet in the
technological system since they retention and promotion has not
been facilitated. (Castaño et al., 2011; Castaño & Webster, 2014;
Cohoon & Aspray, 2006). All this is directly related to the
economic crisis affecting women and those minority groups more This fact generates a serious problem of equality since it is
vulnerable in our societies. Tight economic crisis generates preventing women from occupying creative positions with better
inequality, makes it difficult to access resources, time salaries and more recognition; and it also implies a large talent
management and working places inequity, as well as positions of loss because women have been proved to be a majority among
power where fundamental decisions are made to improve the highly qualified professionals. This also creates problems related
current production model. with efficiency because diverse teams produce better decisions
The majority of Spanish and Catalan women hold a university and products. It also has effects for the economic growth and the
degree (INE, 2012). In fact, the Catalan case is remarkable. In

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 294 |
 Inclusion and promotion of women in technologies
B. Revelles-Benavente | L. Arroyo Prieto | N. Vergés Bosch
well-being of society. Thus, women also distribute knowledge and failure in the education system and the importance of informal
resources better and more widely (Castaño and Webster, 2014). training in the learning of ICT, it is necessary to facilitate their
Research about gender and technology related to female technological training, such as public policies and additional
occupations has pointed out causes for female exclusion which are formative actions (Vergés et al, 2011). Related to this, some
often interwoven. On the one hand, historical factors should be measures have to do with increasing women’s confidence and
included. That is to say, women (as any other minority group empowerment and they create and reinforce, for instance,
excluded from certain privileges), have been traditionally shut out networks and women’s groups relating to technologies (Lee,
from knowledge access and technological occupation, and 2011). To conclude, a final group of proposals deals with
because of that, women rarely appear as a reference for young transforming labour cultures, especially the technological ones.
women (García & Sedeño, 2002; González, 2014). On the other hey look for ways to confront discrimination against women, for
hand, this inherited exclusion effects and reinforces gender example regarding salaries or the implementation and
stereotypes provoking a bigger impact on younger women through development of equality plans for labour opportunities in
family, school, and mass media making younger women distance enterprises. Other measures are planned to modify time
themselves from technological practice and research (Sáinz, management and facilitate work-life balance. These measures
2012). In addition to other questions such as curriculum and ways look at transforming women’s limited flexibility or their familial
of working in some courses of technological itineraries; these responsibilities into an advantage for them, an example being
factors produce that nowadays almost 20 % of the students in high tele-working for this sector and society in general (Sorensen et al.,
school are girls and this continues up until ICT studies (Castaño 2011).
et al., 2011; Vergés et al., 2011; Sánchez et al., 2012). Above all, Following these trends oriented to inclusion, with this project we
it is necessary to take into account that the labour culture of aim at avoiding a paralyzing effect by providing solutions through
technological environments is highly male dominated . Men make research that can be implemented by different agents intervening
up the majority in technological research and professions and in the problem/opportunity under study. In this case, the
those environments are strongly competitive, as well as multiple problem/opportunity is women’s insertion and progression in
other female discriminations have been depicted from unequal technological occupation.
salaries to gender violence (Arulampalam et al., 2007; Vergés, The development of information and knowledge society is
2012). Moreover, technological cultures are marked by ways of characterized, on the one hand, by the inclusion of women in
working that follow a bread-winner masculine model. In this academia and labour and, on the other, by the growing importance
sense, intense working days of 24 hours per day, seven days a of new technologies. However, neither the education nor the
week, and full availability are making time management more technological sectors seem sufficiently prepared to adequately
difficult (Gill, 2002), as well as lineal careers that penalize recruit, retain and promote women into technological jobs, since
interrupted careers (Castaño & Webster, 2010; Valenduc, 2010; gender inequalities are still present and generate equity and
González & Vergés, 2013). Besides, these cultures have a great efficiency problems that hinder economic growth and welfare.
impact finding influential groups and informal networks outside
the work environment (Faulkner, 2009; Kelan, 2009); thus, they In this research, we propose to study the causes of gender
present many different barriers to obtain the conciliation between inequality in the inclusion and advancement of women in
work and family. Therefore, if we take into account that women technological occupations. Furthermore, we seek to find solutions
are still those in charge of the home and family tasks (Domínguez to these by also exploring the factors that facilitate the
& Carrasco, 2003), we find that many difficulties arise to retain recruitment, retention and promotion of women in technological
and promote women in technological occupations. occupations, as well as finding ways of enabling these solutions to
reach both the technology sector and the education sector,
2. AIMING AT THE INCLUSION AND together with women themselves. Apart from considering these
PROMOTION OF WOMEN IN factors for inclusion, we will take into account several dimensions
that affect the processes of inclusion and advancement of women
TECHNOLOGIES in technology, such as family, education, the use of time and
2.1. Perspective of the research conciliation, and labour culture or wage inequalities.
Most of the research has been focused on finding out the barriers
that make it more difficult to access and promote women in 2.2. Methodologies and expected results
technological occupations. Lately, a research body is investigating Methodologically, we will firstly carry out a review and update
factors of inclusion. Even though research is making a huge data related to the participation of women in ICT education and
progress, there are many different aspects to study also in the jobs in the Catalan context, in comparison to Spanish and
Catalan context. Research on female inclusion in the European data. Secondly, based on the experiences of women in
technological world points at multiple factors that can help and ICT, we will distribute an online survey in order to analyse the
facilitate this process. Some of these would only imply to reverse factors that have helped or hindered them to progress in the ICT
barriers, while others will try to move beyond and help to develop sector. However, at the same time, by conducting semi-structured
women’s potentials in the technological world. A group of interviews we will try to connect these with the stakeholders
measures seek to modify technological images traditionally involved in the process of recruitment and promotion of talent,
associated with masculine roles and models, in order to transform such as staff of HR departments, recruitment agencies and
them into images that emphasize women’s interests and orient managers of ICT companies. By working with these hiring agents
marketing strategies in this sense. For example, marketing we also seek to identify potential barriers and opportunities in our
campaigns which attempt to attract girls to study engineering hiring systems, in relation to gender equality. Moreover, we also
degrees, or make visible female presence in the technological aim to understand how stakeholders identify talent and what their
world (Lagesen, 2007, 2011; Vergés, 2012, 2014). Apart from the demands are regarding formal or informal qualifications and soft-

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 295 |
 Inclusion and promotion of women in technologies
B. Revelles-Benavente | L. Arroyo Prieto | N. Vergés Bosch
skills. With them we seek to generate debate about attracting and [4] Castaño, C. i Webster, J. 2014. Género, ciencia y tecnologías
promoting talent in relation to gender. Therefore, considering the de la información. Editorial Aresta.
previous results and further impact on the ICT sector, we will [5] Cohoon, J. M., & Aspray, W. 2006. Women and information
perform an action-research in the form of pilot projects to promote technology: Research on underrepresentation (Vol. 1). The
gender equality in ICT. This part of the research will be carried MIT Press.
out in three companies in the ICT sector and will include the
design and development of a training activity for each. [6] Domínguez, C. C. M. 2003. Género y usos del tiempo:
nuevos enfoques metodológicos. Revista de Economía
The project we are presenting here is at an early stage and, Crítica, (1), 129-152.
therefore, we can only refer to expected results. Taking this into
account and regarding the results of the project and their [7] García, M. I. G., & Sedeño, E. P. 2002. Ciencia, tecnología y
dissemination and impact, we propose to develop, on the one género. CTS+ I. Revista Iberoamericana de Ciencia,
hand, a series of open access material including a tool kit, a Tecnología, Sociedad e Innovación, (2), 5.
training guide, a weblog, a viral video and several academic [8] Gill, R. 2002. Cool, Creative and Egalitarian? Exploring
publications. Since we aim to impact the education sector at Gender in Project-Based New Media Work in Euro.
school level we will design and launch a series of talks in schools Information, communication & society, 5(1), 70-89.
for students and teachers, to facilitate motivations and in doing so,
[9] Faulkner, W. 2009. Doing gender in engineering workplace
the incorporation of women into ICT studies. Apart from that we
cultures. II. Gender in/authenticity and the in/visibility
will produce a video for young students addressing the skills (hard
paradox. Engineering Studies, 1(3), 169-189.
and soft skills) needed to progress in ICT jobs. Finally, we will
seek to intervene in society in general through coverage in the [10] González Ramos, A.M. 2014. Victoria y la ciencia: el papel
press and scientific papers. Therefore, through publicity and de las mujeres en la ciencia contemporánea. A Castaño, C. i
knowledge transfer, as well as replicating these in the future, we Webster, J. (2014). Género, ciencia y tecnologías de la
seek to bring our results to girls and women, as well as to the información. Editorial Aresta
stakeholders involved in the recruitment, retention and promotion [11] González Ramos, A. i Vergés, N. 2013 “International
of women in technology. Finally, we aim to make the project mobility of women in S&T careers: shaping plans for
sustainable in the medium and long-term to promote substantial personal and professional purposes” Gender, Place and
changes in the ICT occupational structures. Culture 20 (5): 613-629.
3. CONCLUSIONS [12] INE. 2012. Nota de prensa. Extret el 15/06/2014 de
With this project under progress, we aim at changing http://www.ine.es/prensa/np712.pdf.
organizational practices in enterprises related with the ICT sector [13] INE. 2013. Nota de prensa. Indicadores del Sector de las
into more friendly environments for women, as well as promoting tecnologías de la información y de las telecomunicaciones.
transparency in recruitment processes. Through a preliminary Extret el 25/05/2014 de http://www.ine.es/prensa/np786.pdf
exploration, we have observed how women are kept under an
[14] Kelan, E. 2009. Performing gender at work. Basingstoke:
in/visible barrier that enhances hostile cultures to reward their
Palgrave Macmillan
career. In occasions, these barriers are not noticed because
organizational cultures heavily rely on a social premise that blurs [15] Lee, L. 2011. Empowering women in ICT through
free will with practical allocations. After a detailed analysis of the professional networks. A Sørensen, K. H., Faulkner, W., &
present data and semi-structured interviews with different agents Rommes, E. (2011). Technologies of Inclusion: Gender in
and participants in this project, we will produce some results that the Information Society. Tapir Akademisk Forlag.
aim at connecting stakeholders involved in the process of [16] Lagesen, V. A. 2007. The Strength of Numbers Strategies to
recruitment and promotion of talent, such as staff of HR Include Women into Computer Science. Social Studies of
departments, recruitment agencies and managers of ICT Science, 37(1), 67-92.
companies. By working with these hiring agents we also seek to
identify potential barriers and opportunities in our hiring systems, [17] Lagesen, V. 2011. Getting women into computer science. . A
in relation to gender equality. Assessing how stakeholders identify Sørensen, K. H., Faulkner, W., & Rommes, E. (2011).
talent and what their demands are regarding formal or informal Technologies of Inclusion: Gender in the Information
qualifications and soft skills, we seek to generate debate about Society. Tapir Akademisk Forlag.
attracting and promoting talent in relation to gender. [18] Sánchez Vadillo, N., Ortega Esteban, O., & Vall-llovera, M.
2012. Romper la brecha digital de género. Factores
4. REFERENCES implicados en la opción por una carrera tecnológica. Athenea
[1] Arulampalam, W., Booth, A. L., & Bryan, M. L. 2007. Is Digital (Revista de Pensamiento e Investigación Social),
there a glass ceiling over Europe? Exploring the gender pay 12(3).
gap across the wage distribution. Industrial and Labor
Relations Review, 163-186. [19] Sáinz, M., Pálmen, R., & García-Cuesta, S. 2012. Parental
and secondary school teachers’ perceptions of ICT
[2] Castaño, C., González, A., Müller, J., Palmen, R., Rodríguez, professionals, gender differences and their role in the choice
A., Sáinz, M., ... & Vergés, N. 2011. Quiero ser of studies. Sex roles, 66(3-4), 235-249.
informatic@. Barcelona: Editorial UOC.
[20] Sørensen, K. H., Faulkner, W., & Rommes, E. 2011.
[3] Castaño, C., & Webster, J. 2011. Understanding Women's Technologies of Inclusion: Gender in the Information
Presence in ICT: the Life Course Perspective. International Society. Tapir Akademisk Forlag.
Journal of Gender, Science & Technology, 3(2).

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 296 |
 Inclusion and promotion of women in technologies
B. Revelles-Benavente | L. Arroyo Prieto | N. Vergés Bosch
[21] Valenduc, G. 2011. Not a job for life? Women's progression, [23] Verges Bosch, N. 2012. De la exclusión a la autoinclusión de
conversion, and dropout in ICT professions. International las mujeres en las TIC. Motivaciones, posibilitadores y
Journal of Gender, Science & Technology, 3(3). mecanismos de autoinclusión. Athenea Digital (Revista de
[22] Vergés, N., Hache, A., & Cruells, E. 2011. Indagando en la Pensamiento e Investigación Social), 12(3).
relevancia de Internet en el acceso, uso y deseos de las TIC [24] Vergés Bosch, N. (2014). De las ausencias a las presencias
por parte de las mujeres en las TIC. Teoría de la Educación. de las mujeres en las TIC. Un acercamiento a sus
Educación y Cultura en la Sociedad de la Información, mecanismos de autoinclusión. In Castaño and Webster (Ed.).
12(2), 105-121. Género, Ciencia y Tecnologías de la Información, 165-190.

XVI Congreso Internacional de Interacción Persona-Ordenador | Engendering Technology (I)

| 297 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch

Moving towards Accommodating Women with ICT: Paying

Attention to Self-inclusion Mechanisms
Núria Vergés Bosch
Universitat de Barcelona
Departament of Sociology and
Organizational Analysis.
Avd. Diagonal, 696 08034 Barcelona
+34934021801
[email protected]

ABSTRACT mutually constitutive and fluid relationship with ICT [52] [63]
Recent developments of new information and communication [67] [53] [62].
technologies, as well as feminisms are essential for understanding To carry out research around the inclusion of women in
the major changes in our societies today. Feminist research of technologies involves techno-social processes that delve into
technology, traditionally, has been focused on explaining the technological and social aspects such as motivations, enabling
exclusion of women from technologies. However, a move towards factors, conditions of use, development and participation of
inclusion is already being carried out intensively. This involves women in the field of technology and even the mechanisms of
investigating the experiences of women in ICT and thereby self-inclusion and how all this, in turn, impacts ICT uses and its
making visible their trajectories and mechanisms towards development. However, to consider these motivations and factors
inclusion, as well as the challenges and opportunities that are of inclusion is not only crucial to women's access to technology,
generated thereby. In this article I aim to explore the processes of but also to their persisting in its use and to allow us to actively,
self-inclusion of women in ICT and specifically to show the main emancipatorily and critically participate in the design and
mechanisms that they have followed and activated to self-include development of ICT. In this article I will only focus on the
in ICT. To do this I have relied on shared experiences of a mechanisms of self-inclusion of women in ICT, which are
purposive sample of artistic technologists and computer elements much less addressed in depth in the literature on gender
technologists in Barcelona, by conducting interviews and focus and ICT. Therefore my main question underpinning this article
groups. As a result I show a series of mechanisms that women concerns what mechanisms women follow and trigger to self-
follow and activate to access, remain in, advance, and even include in ICT?
transform ICT. These mechanisms go beyond learning and
include not only doing gender but also undoing it as well as ICT. In this sense, when referring to self-inclusion of women in ICT
Thus, Women in ICT seek to access and progress in ICT adapting women are positioned as agents conducting their own ICT
to certain circumstances but also changing them to generate and inclusion [45][46][61][62]. This entails considering the agency of
regenerate ICT in order not just to be assimilated but better to women in their process of inclusion, i.e. the mechanisms that
accommodate with ICT. themselves activate and/or decide to follow to progress and even
to contribute and transform ICT. Self-inclusion processes occur
Categories and Subject Descriptors over time, dynamically and interactively, and involve to access,
follow, stay, progress, recognize and be recognized, and
K.7 The Computing Profession; K.4; Computers and Society contribute to ICT [61].
General Terms In this article I aim to explore the process of self-inclusion of
Performance, Human Factors, Theory, women in ICT and specifically show the main mechanisms that
they have followed and activated to self-include in ICT. To do this
Keywords I have reviewed the key literature that investigates the exclusion
Women; Doing Gender; ICT; Inclusion; Self-inclusion and inclusion of women in ICT, but above all I have relied on the
mechanisms; Undoing Gender; Technologies; Accommodation; experiences and discourses of a purposive sample of artistic
Gender technologists and computer technologists in Barcelona.

1. INTRODUCTION In the next section, I will discuss what we have learned from
With the rise of New Information and Communication research that focuses on the exclusion of women in ICT, as well
Technologies (ICT) and the development of third-wave as some negative impacts I think this entails. That leads me to
feminisms, current feminisms of technology are experiencing a strive to turn towards inclusion paradigms and, beyond that, to
theoretical renewal, critical but at the same time optimistic, propose a move towards an inclusion that considers and fosters
developing into new opportunities for research and practice. In self-inclusion of women in ICT. Then I introduce the main
this regard, it is diverging from the analysis of exclusion, methodological strategies undertaken in order finally to present
quantities and the digital gender gap towards research on the main results and conclusions of this research about the self-
inclusion, i.e. the presence of women in ICT and gender impacts, inclusion processes of women in ICT.
qualities, as well as opportunities to and from women in a

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 298 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch

2. FROM EXCLUSION… 3. TO INCLUSION…

Traditionally, much of the research on women and technology has If we take into account some of the impacts of merely focusing on
focused on showing and explaining the absence and exclusions of the exclusion of women in ICT and consider the urge to find new
women in ICT. That is, much of the literature and public reports opportunities for women in ICT, a move towards the paradigm of
have focused on the various digital divides and asking why there inclusion is needed. This move should focus on the presence and
are no women in ICT or why there are so few or why they are experiences of women in ICT and allows us to visualize their
leaving ICT [58][7][13][35]. The main concern is currently contributions and wishes, but also the paths, opportunities and
focused on the underrepresentation of women in the areas of potential for the full participation of women in today's
decision making and design of new technologies, traditionally technological developments. Moreover, as explained by
associated with computing studies and engineering [11][10]. Sorensen[52] to know the reasons why women are excluded from
These investigations point to a number of factors that have led to technology, is enough to understand and promote their inclusion?
the exclusion of women from ICT. Historically, women have been Or is it necessary to develop a more in-depth understanding of the
excluded from the study, research and production of technological presence and mechanisms for inclusion in ICT? Thus it makes
knowledge through legal and social norms [21][32]. This sense that we also ask why there are women in ICT and how they
discrimination has hindered the generation of models for other accessed ICT? What do they do, how and why? And even, what
women and makes invisible contributions by women to the do they want? As well as what does their presence imply?
development of technology. This has fostered an image that Placing inclusion in the center of our research also becomes more
alienates women from technologies and has fueled a stereotypical consistent with feminist research that considers the experiences of
attribution of gender roles for women that result in discrimination women at the heart and point of departure of our research [4]
against women. Gender stereotypes are being reproduced through [1][66]. In fact, feminism of technology research has begun a
family, school, media, social networks or even at the workplace process to investigate the processes of inclusion of women in
[47]. Their influence defines what is considered appropriate for technology seeking a renewed relationship between gender and
women or for men in a binary manner, besides marking the ICT [15][56][53][61][9][54]. These investigations, focusing on
conditions of access to prestige, value and recognition that result the presence and inclusion processes, have brought to light the
in discriminate against women horizontally and vertically [35] [7]. contributions and uses that women make of ICT and thus have
In this sense, gender stereotypes are attributed as "natural" and focused on the worlds and experiences of women [33][43][49]
therefore they are highly resistant to change. In this sense, [57][56][53][10][61]. With this, we have generated role models
technology is considered masculine and masculine means man. for many more, as well as having weakened stereotypes about
This constructs and maintains a masculine culture of technology, gender and ICT. In this sense, research has also questioned the
i.e. a technology associated with men, masculinity and their binary tendency of the study of women in technology, by focusing
values [14][18] which implies that ICT learning and working on women while making visible its diversity and going beyond the
environments tend to be unfriendly to women and, for example, dichotomous comparison to men. These research studies have
show serious deficiencies in the possibilities of balancing work tended to develop a large volume of information that qualitatively
and family life [22][42]. Furthermore, it should be stressed that explores gender and ICT, leaving behind a debate which has
the exclusion of women from technology is also often aggravated traditionally focused on the amounts and percentages in order to
by other forms of social exclusion such as socio-economic status, deepen the quality of the participation of women in ICT. Finally,
place of residence and / or socio-demographic factors such as age, the contributions of these research studies for inclusion have
ethnic origin or sexual orientation [7][3]. Thus, study and action helped to generate new inclusive opportunities for other women
to address the exclusion of women in ICT is still necessary, but it and collectives and have made visible alternative developments of
also has some drawbacks. ICT, or even improvement in business, since women are behind
The results of these studies focusing on exclusion have been most purchasing decisions, including technological ones. In this
crucial to highlighting the situation of gender discrimination sense, then, in addition to claiming the inclusion of women in ICT
against women in ICT, the male dominance of technology as well for gender justice, a greater diversity of voices, views, needs and
as to identifying a need for public action, thus generating wills that have become included are emphasized and, in turn, are
indicators and data for its analysis [58][44][35][7]. However, from creating opportunities to develop new ICT products which are
a paradigm of exclusion, too often narratives of exclusion are still more extensive and adaptable to a wider range of profiles. There
focused on the worlds of men and only encourage a change is a need to avoid losing talented women, since they are becoming
among women, rather than changing structures and the society as the majority of students of our universities. Moreover the
a whole. The view from exclusion also tends to stress the problem inclusion of women in ICT could challenge existing power
of quantities seen in a binary way that compares both sexes. This relations through an alternative and innovative ICT development
has also fostered a technophobic and pessimistic perception of the that is more in line with society as a whole and, finally, women’s
women and technology relationship, has solely focused on the inclusion in ICT could produce a redistributive effect of its
negative, the absences and barriers and has contributed to bring a benefits for their families and local communities
pessimist, victimist and too chilling a message to public policy [11][5][62][61][2][36].
makers, academia and the rest of society (as criticism Feminist research of technology has identified certain motivations
[52][56][15][17][61]. Beyond this, a focus on exclusion has that trigger the interest of women which could be summarized as
distanced the analysis from the post-feminist proposals that appear Sorensen stated [52]; as the duty, which expresses a utilitarian
with new theoretical proposals, many yet to be developed motivation, or as love, expressing a passion behind the use of
empirically and with open doors to critical and transformative technologies. Also, research focused on the study of technological
optimism [55] [30][67] [61]. inclusion has identified a number of factors that facilitate the
inclusion of women and their immersion in technologies [52][15]

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 299 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
[56][24]. The range of factors goes from friendliness to women in develop the key questions and explore and evaluate collectively
ICT environments or tools, to access to education, resources or the topic of research, I carried out two mini focus groups of three
infrastructure. Active encouragement from family, friends, media, participants each [12]. One consisted of artistic technologists and
school or gender specific public policies have also been stressed the other of computer technologists. Thus there were 28
[6][10][53]. participants in this research, half of them women artistic
Although considering these motivations and factors of inclusion is technologists and the other half women computer technologists.
crucial to women's access and progress in technology in this The analysis presented in this article is based on a bottom-up
article I will precisely focus on the mechanisms of self-inclusion qualitative analysis using the software for qualitative analysis
of women in ICT, which are elements much less addressed by Atlas.ti. This has generated several codes and code families that
previous literature, especially in a non English-speaking world. have helped me to answer the research questions by including the
Furthermore, although self-inclusion has been targeted by some words of the participants. In this regard, pseudonyms are
authors few times it has not been treated in depth. Finally, to carry implemented, as anonymity was requested by some participants.
out research on self-inclusion can help us consolidate and find As a result, in this next section, I aim to answer a key question:
new ways to continue generating self-inclusions and, ultimately, what mechanisms have those women followed and triggered to
contribute to the inclusion of more women and other groups in self-include in ICT?
ICT.
Taking all this into account, in the next section I will introduce the 5. SELF-INCLUSION MECHANISMS OF
methodological strategies I followed to explore the mechanisms of WOMEN IN ICT
self-inclusion of women in ICT and then, present the main results As pointed out by the majority of research on gender and
of this work that show a range of mechanisms that a sample of technology, the most obvious mechanism to self-include in ICT is
women followed and triggered to self-include in ICT. through the acquisition of knowledge and ICT skills [11][65][9].
Thus, one of the main mechanisms of self-inclusion, also for
4. METHODOLOGIES women, consists of learning, especially non-formal and informal-
Departing from the paradigm of inclusion, according to feminist formal learning [68][69] since women are not studying computing
epistemology and situated knowledges and thus, considering the in great numbers. This last is acquired by formal education,
need to depart from the ICT experiences of women institutionally structured in a particular country. Non-formal
[4][25][41][66], I selected a purposive sample of women learning is acquired through, for example, participation in
technologists resident in Barcelona. Therefore, these women had workshops, courses and conferences. Finally, informal learning
already become advanced and specialist ICT users. However, develops through interaction with friends, family, communities of
considering that ICTs are increasingly transversal and, therefore, practice or even at leisure times. This rarely results in
their advanced use and development is possible from any area of certifications and often occurs unintentionally. However, as
human activity, we need to turn our gaze to these other areas shown in the following discourse and in line with my previous
where, in addition, women seem to be better represented. This investigations [63][65], the ICT learning arises through the
often involves a qualitative effort given the lack of data in many combination of the various ways of learning that often overlap.
of these areas which intersect with ICT. Taking this into account I
The design of the interaction, again, went back to a fairly
narrowed down the selection of the sample to computer
traditional way of learning. Although there were more kinds of
technologists, but I explicitly open that to artistic technologists.
workshops or projects, every two weeks we had a project. Just as
Computer technologists would study or develop practices that
I'm learning in the master’s degree in digital fabrication. Every
have traditionally been considered ICT [40][7][10] and they
two weeks we have a project on the subject. Scanning, molding,
would also clearly work in masculinized environments, at least
laser cutting, assembly, electronics, programming. Also a mix
quantitatively. In contrast, artistic technologists would study or
between working alone, finding resources on the network,
develop practices in intersection with ICT [34][61], i.e. in this
connecting with other people informally, sharing things and a
case where the artistic and technological components are
form of professional apprenticeship. (Marquès, Arts, Italy, degree
combined and in environments with more gender parity, at least
in Pedagogy, Artist and Technoteacher)
quantitatively speaking. Finally, responding to the need for of an
investigation of proximity and considering the relevance of In relation to learning and already pointed by many authors
Barcelona as a South-European creative area, participants are [59][8] most participants did not study computing or engineering,
residents in the province of Barcelona [27][48]. However, also in or formally followed a linear path from engineering to ICT jobs.
an inclusive manner, and to capture migrated trajectories, I In a way then, through more informal channels, the participants
considered diverse places of origin in an international sense. were creating their own ICT itineraries. The importance of non-
formal and informal learning in women trajectories could be
According to what I revealed about the paradigm of inclusion, I constituted as a potential for their ICT inclusion. However, in
understood that the best way to capture the experiences of women turn, it may represent a handicap for them due to lack of
was through qualitative methodologies. This allows me to analyze credentials combined with the stereotype of the inadequacy of
the meanings, but also to work through conversation and narrative women in technology [14][65].
creation and thus delve deeper into a process, such as the self-
inclusion in ICT one, that quantitatively would hardly be possible Another set of mechanisms for ICT self-inclusion relates to labor
to address [41][16]. With this in mind, I carried out a combination issues. In this sense, the participants sought and accepted jobs
of qualitative techniques as the most appropriate way to approach, directly related to ICT as a way to enter or to advance in the ICT
understand and analyze the research topic and answer the research practice, as well as to access some higher income. While artistic
questions. I conducted 22 episodic interviews that combine technologists showed a higher level of multi-occupation, even
narrative and semi-structured interviews [16]. In addition, to simultaneously, among computer technologists another

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 300 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
mechanism related to labor appeared. Many engineering students think that in general, I'm not good at selling myself and yes it is
started to work even before finishing their studies. This seems to likely that women will be less selling themselves, but I think it is a
be a feature of engineering courses that have close links to cultural issue. It is not that we are born without the gene for the
particular companies and should also be a feature of other degree ability to self-promote. (Fité, Computing, Catalunya, PhD in
courses. computer science, university professor and researcher)
I was finishing my third year and I wanted to get some experience, Even if they were aware of the need for self-promotion, they
a bit, if I started to work the following year, I needed to build actively avoided trying to prove themselves superior. Instead, they
some CV... or maybe just to remain in the same company after performed a kind of horizontal self-promotion and networking,
finishing my degree. There, at the University, there is the “job perhaps to avoid unwanted gender discriminatory effects as
campus” where they seek a job for you. Well, they’ve got job women are heavily penalized if they self-promote in certain ways
offers, you register there and get offers. Normally, what they used and less if they do so through others [51].
to send me was more related to computing than to Related to this, another mechanism that has been evident consists
Telecommunications and so... And finally, I went to an interview. of collaboration and sharing. Through establishing partnerships
Well, I went because there was another offer and you had to know and processes of sharing knowledge, software, artworks etc. they
how to program databases or whatever. When I was there in the were self-including in ICT. On the one hand, through sharing and
interview I was told, no, but you do not know Oracle, and I said ... collaboration they learned and helped others to learn informally.
no, but when I get the offer it did not say that I needed to know On the other hand, they were promoting themselves and their
this, right? And they said, well, we keep your CV in case we have abilities and, therefore, this has also to do with self-promotion.
any other job. Then they called me and that is how I started to Finally, it is worth noting that through using free licenses, many
work in this. (Daigenault, Computing, Catalunya, of the participants allowed that program, work or knowledge to be
telecommunications engineering degree, upper computing distributed and, in a sense, be inserted into other codes, works or
engineering student and researcher in computer science at a applications. In this sense not only were they released and could
university) be recognized, but also they were able to generate an effect of
Almost all of the participants, when coming across obstacles or self-inclusion as the following discourse shows.
observing opportunities in another field or context, opted for I need software that I can use as I like, to make inventions with it,
mobility, both geographically and labor mobility, thereby also to do research, to know how it’s done ... That I can copy and
contributing to break the linearity of their careers. That is, as recommend to my friends. That operates on any mobile phone, on
suggested by some literature [26][20] women tend to change any computer, that runs on any router. Practically I can’t use
company, sector, discipline or even country, seeking new proprietary software, which is telling you: do not help others. (...)
opportunities, new ICT challenges and also more friendly I use public communication as a guarantee that this work is mine
environments as a way to advance their ICT careers. and a free license that leads others to use it. Now I use two
Some of the participants opted for entrepreneurship as a means of licenses, before I only used the free license GNU, Free
ICT self-inclusion and labor, especially in the case of artistic Foundation License, which was the one used by the Wikipedia,
technologists who generally showed more job complexity. They because it allowed that my work, part of my work, could be copied
organized themselves, alone or with other partners, to perform to Wikipedia, directly, or vice versa. All documentation,
their ICT practices. Sometimes their ICT practice was done in documentation of all GNU programs are registered under this.
similar or better conditions as when being an employee. However, Since the Wikipedia happened to have a dual license, I kept the
in many other situations this involved risk and insecurity as FL and added the creative commons, attribution and share alike,
pointed out by previous literature in relation to freelance work and like Wikipedia. (Expósito, Info, Catalunya, degree in psychology
ICT [19]. In spite of that, some participants perceived free-lance and audiovisual communication, Hacker and PhD student in
work to have some advantages, such as the flexibility and Computing)
freedom, but also for the challenges and opportunities involving Another mechanism that is still closely related to labor involves
technological innovation, because there were not so many seeking the balance between different areas of life. In fact, the
directions already set compared to working in a specific project or literature on gender and ICT has been interested in this issue and
company. in claimed measures that balance work life and family life as well
Another self-inclusion mechanism identified consisted of self- as labor with other areas of life, since the ICT sector appears to
promotion, expressed by some participants as a major dilemma. involve intense working schedules demanded by 24/7 [26][60].
Thus, clear self-promotion was used only on rare occasions, i.e. in Some participants, albeit with difficulties, sought working time
situations or ways socially acceptable to women such as sending arrangements that gave them enough time for family life or even
curricula, business cards or information about the next shows on technological experimentation, but also to spend time with friends
the web in the case of artistic technologists, as previously pointed or to practice leisure activities. It is worth noting that, in this
out by authors that explore the careers of women and self- regard, ICT and its ways of communication helped them to
promotion [38]. Therefore, in turn, most participants showed great recover some family or social life. At the same time, the
reluctance for self-promotion, especially an exaggerated one, such possibilities of teleworking made fluid the boundaries between
as they saw being performed by some of their male peers and home and work, as well as private and public spaces as shown in
called in spanish "autobombo". the discourse by St- Arneault.
In my research group there are more boys, many more boys and I work at home. This means that somehow I have a very large
we talk about this things among girls very often. They have a living room, partly as the living room, partly as the kitchen and
capacity of Me, that I am, I am .... And we are so modest ... And partly to work. Then there is no border in the same room nor
the problem is that people believes so much on your own nothing stops my body movement. Sometimes I get the computer to
propaganda.... And don’t think it's only in the computer field. I choose a movie to download or to play an on-line series, leisure
XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)
| 301 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
absolutely, then I'm doing four things at once, right? In the in several situations, something already explored by the literature
meantime, the washing machine is working, I hang the clothes, on inclusion. Similarly to traditional women only groups, the
while I am rendering a video for work, all at once. (St- Arneault, participants gathered with other women being aware of their
Artist, Aragon, degree in fine arts, video director and editor, difficulties for progression in certain areas or situations. They did
university professor and PhD student in communication) so, on one hand, by participating in activities already promoted for
Another set of mechanisms would be more transversal and would women as, for example, a women only course or participating in a
go beyond the learning and the labor, but also relate to them, since conference, association or festival of women. Moreover, on the
the mechanisms are strongly associated with each other. First, I other hand, they have even been more active in doing gender
refer to what are called self-regulative processes [70][50]. Thus, while being feminist. For example, participants consciously
participants worked to advance and promote themselves through suggested other women as candidates for job offers in their
their own efforts, organization and work well done and, if current jobs to create a gender balance in their work
possible, testable and demonstrable accumulation of merits. For environments. Similarly, some participants formally or informally
example, they established for themselves some working patterns, offered themselves as mentors, as well as created content on
some disciplines, a specific organization of time, a number of gender and technology, or, indeed, created and / or organized
qualifications to reach etc. This allowed them to overcome a activities, associations or networks together around feminism and
difficult curve when they started learning as well as various technology. As Margolis and Fisher [35] noted regarding access to
obstacles they encountered along their career, besides encouraging the Carnegie Mellon University, to have women can also lead to
themselves and progressing in ICT. having other women. In this sense, the actions that they trigger
can produce an effect of a protective environment, as well a
Another major mechanism of self-inclusion is related to promotional one. In this sense, they might create a women’s self-
networking. Participants built their networks with people with inclusion multiplier effect, especially when there is gender
ICT interests, but at the same time, diverse enough to provide consciousness.
expertise or influence in another specific area. This was not
always expressed as intentional but came up repeatedly in the Because my girl friends were working there and they have been
narratives of the participants. Thus, it is confirmed what Morgan introduced by other girl friends, I introduced my girl friends.
et al.[37] and Lee [31] pointed out, that networking becomes a Wherever I go, in later years, an increase in the female
crucial mechanism of progression in ICT and women activate it in population occurs. Before, there were only two or three girls
various ways, from participating in existing networks to creating there, she, me and another friend. After I left the center they were
new ones. It is worth noting that this can play as a mechanism for mostly girls. And here, when I started last year, we were only 4 or
informal learning as well as a self-promoting one, collaboration 5 girls doing the master. But there are many more this year. And I
and empowerment that facilitates the generation of new projects think it's still less than 20%, but much higher than last year.
and even new jobs. (Codina, Iran, Software Engineer and PhD student in Computer
Science)
We already met in Barcelona, and I had already worked with
some of the members here ... Someone said, why do not we gather However, on other occasions, the participants were undoing
together and share a space? They proposed that to me and I gender as a mechanism of self-inclusion. Contrary to what might
thought it was fantastic. I really had no need for a space, because seem a priori the participants did not want to be "one of the boys".
I already had my house, then this means to me now an extra Still, many adopted and performed some of the characteristics
economic cost to pay the rent together, but brings me to have associated with masculinity because they felt they had to, or rather
relationships with other people who work like me and from there in order not to be treated as "the girl" and so make progress in
to get many projects. This latest exhibition came to be here, go certain situations and ICT environments. Thus, some participants
talking about this and now this project with this one, the other diluted or made less obvious their feminity. In this sense, they did
with the other and so on. And from here we generate new not seek to be or play the most aggressive, the most rational or
dynamics. (Tatxé, Artist, Catalunya, Multimedia Engineering strong, but just stood actively in an intermediate position with
Degree. Artist and Cultural Manager) respect to gender. In a way, this active position in the middle
becomes a very powerful mechanism of self-inclusion because it
A little further and more related to collective participation, some allows them access to the benefits of doing gender and,
of the participants were members of, or participating in, ICT simultaneously, to the benefits of undoing it. When, on the
collectives and associations, from professional to non-profit contrary, placed in this position passively they could run the risk
entities, groups, or communities of a specific ICT practice. In this of suffering the drawbacks of both, as when doing gender means
sense, women may be playing a role in a third sector related to accepting discrimination or undoing gender by adopting
ICT often unexplored [36][23][24]. Through collective exaggerated masculine behaviors that often they claim to reject.
participation they sought to learn but also to interact, give support,
empower, generate new projects or simply make themselves be Beyond the doing and undoing gender it is worth noting the
known. At the same time, most of them had participated in ICT possibilities of transformation of ICT that were observed
events, meetings, , festivals, seminars or conferences. In some throughout the investigation and, indeed, could be appointed as an
occasions these ICT associations and / or events also had to do additional mechanism of self-inclusion in ICT. This involves not
with gender and activism simultaneously. only contributing to ICT, but also adapting ICT to women and
their needs and interests [5]. In this regard, participants were
Finally, to overcome situations of discrimination and also to adapting to ICT tools and given scripts, but some explicitly
facilitate their progression, women could also be doing gender or expressed that they were seeking to undo them. In this sense, not
undoing gender [28][29][14][39][64]. In this sense, they were only by modifying the codes or forms of the tools they used, but
doing gender, i.e. emphasizing feminine behaviors and performing also by changing their own uses, for example, by applying
according to the traditional characteristics associated with women hardware or software typically used in arts or “internet of things”

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 302 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
(like sensors or arduinos) in other areas of human activity where The participants of this research have build new networks, as well
they were also working, such as early childhood education. This as integrated old ones to further progress in ICT and generate new
potential was more pronounced among activists and in many of projects, while learning and promoting themselves. In relation to
the artistic technologists. The latter, quite logically, showed a that, they have been collaborating and sharing knowledge but also
great ability to imagine other ICT or other uses and meanings for works, codes and resources that could help and be used by others.
technologies by developing them themselves. Thus, women in Through collective participation they learn, interact, rely, are
ICT could be developing a process of accommodating with ICT empowered, or generate new projects as well as new inclusions.
more than just getting assimilated by it, especially if we consider Sometimes, in addition, they create new groups or events that, on
the possibilities of other self-inclusions that they open up and the occasions, foster women and technology at the same time. When
changes they might be bringing in their contexts. In this process of doing so they have been doing gender, as well as feminism. Other
accommodating with ICT they seem to seek to feel comfortable times they have been undoing gender, for example when they
and progress in their environments and practices, adapting to have diluted gender marks in their communication or behavior.
specific circumstances but also changing the same circumstances However, neither doing nor undoing gender dynamism appeared
to be more comfortable and regenerate their own ICT trajectories in an extreme form, mostly being softly performed in order to take
and practices. advantage of that. Finally, the participants not only have been
doing ICT but also have been giving new uses and meanings to
6. CONCLUSIONS ICT in a way of undoing ICT. Thus, women can be doing and
Research on women in technology has traditionally concentrated undoing gender and even be doing and undoing ICT as self-
on explaining their exclusion from it. However, feminist research inclusion mechanisms and not just to integrate in ICT but to
of technology has recently strengthened investigations of the accommodate with it.
processes of inclusion of women in technology. By doing so, we
are looking for a critical but renewed relationship between gender Taking this into consideration, women in ICT could be developing
and ICT. an accommodating process rather than just being assimilated by
ICT, especially if we take into account the possibilities of other
From the paradigm of inclusion the advantages of knowing, self-inclusions that they might open up as well as the potential to
exploring and facilitating processes of inclusion and self-inclusion regenerate ICT and the relationship between gender and
of women in ICT have been pointed out. Our inclusion would not technology.
only help us to advance in gender equality and maximize the
potential of talented women, but also we might see improvements 7. ACKNOWLEDGEMENTS
in terms of business, welfare and, of course, in the development of I want to specially thank all the participants of this research for
ICT generating new and more suitable technologies for the society sharing their experiences with us, as well as Dr. Ana M. González
as a whole. Ramos, Dra. Elisabet Almeda Samaranch and Dra. Cecilia
Castaño Collado for their kind inputs in this research. Many
Therefore, departing from this inclusion paradigm I have explored thanks to the reviewers that helped me to improve this work and
the mechanisms of self-inclusion of women in ICT by conducting especially to Dr. Scott for his English review. Finally, this
interviews and focus groups with women artistic and computing research was possible thanks to the Internet Interdisciplinary
technologists. To focus on women that are already technologists Institute funding.
and to follow the proposal to turn to the paradigm of inclusion has
the limitation of omitting those women that are still excluded from 8. REFERENCES
ICT. Similarly, to focus on the mechanisms of self-inclusion [1] Ackerly, B. & True, J. (2010). Doing feminist research in
involves certain overlook to the factors of exclusion. However, political and social science. Hampshire: Palgrave Macmillan.
those have received a lot of attention in previous research and so [2] Antonio, A., & Tuffley, D. (2014). The Gender Digital Divide
have been reviewed in the introduction. Finally, this research in Developing Countries. Future Internet, 6(4), 673-687.
consisted of qualitative and exploratory research and, therefore, its [3] Arroyo, L. (2014). La inclusión digital de las mujeres de clase
results cannot be generalized and were just presented in a trabajadora. Explorando caminos hacia una mayor igualdad. In
descriptive level. In spite of that, this has opened the door to Castaño & Webster, J. Género, Ciencia y Tecnologías de la
studying in depth each mechanism in the future, also in new Información, 115-140. Barcelona: Aresta.
contexts beyond what is traditionally considered ICT. [4] Bartra, E. (1998). Debates en torno a una metodología
The main self-inclusion mechanism commonly stated by the feminista. México: Universidad Autónoma Metropolitana–
participants consists of learning. However, many women do not Xochimilco.
learn in formal education, nor study engineering, but engage in [5] Buechley, L., & Mako, B. (2010). LilyPad in the wild: how
non-formal and informal learning. In addition, participants seek hardware's long tail is supporting new engineering and design
and access ICT jobs, but show important dilemmas when dealing communities. In ACM Proceedings of the 8th ACM Conference on
with self-promotion. When finding barriers in a given context or Designing Interactive Systems,199-207.
seeking new opportunities most of the women have opted for [6] Burger, C. J., Creamer, E. G., & Meszaros, P. S. (2007).
mobility, both occupational and geographic. Some of the Reconfiguring the firewall: Recruiting women to information
participants have even become entrepreneurs, alone or with other technology across cultures and continents. New York: CRC Press.
partners, to maintain and continue their ICT practices, as well as [7] Castaño, C. (2008). La segunda brecha digital. Madrid:
set several self-regulating practices as another mechanism of self- Cátedra.
inclusion [8] Castaño, C., & Webster, J. (2011). Understanding women’s
presence in ICT: the life course perspective. International Journal
I would like to highlight a set of other self-inclusion mechanisms, of Gender, Science and Technology, 3(2), 364-386.
especially because they can bring new self-inclusions for women.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 303 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
[9] Castaño, C., & Webster, J. (2014). Género, Ciencia y [28] Kelan, E. K. (2009). Performing Gender at Work. New
Tecnologías de la Información, 165-190. Barcelona: Aresta. York. Palgrave Mcmillan.
[10] Castaño, C., González, A.M., Müller, J., Palmen, R., [29] Kelan, E. K. (2010). Gender logic and (un) doing gender
Rodríguez, A., Sáinz, M., Vázquez, S., Vergés, N. (2011). Quiero at work. Gender, Work & Organization, 17(2), 174-194.
ser informatic@. Barcelona: Editorial UOC. [30] Landström, C. (2007). Queering feminist technology
[11] Cohoon, J. M. & Aspray, W. (2006). Women and studies. Feminist Theory, 8(1), 7-26.
Information Technology: Research on Under-Representation. [31] Lee, L. (2011). Empowering women in ICT through
Massachusetts: MIT Press. professional networks. In Sorensen, Faulkner and
[12] Edmunds, H. (1999). The focus Group Research Rommes.Technologies of Inclusion. Gender in the information
Handbook. New York: Mc Graw-Hill Professional. society, 87-105.Trondheim: Tapir academic press.
[13] European Comission, (2010). Women and ICT Status [32] Lerman, N., Oldenziel, R., Mohun, A.(2003). Gender
Report 2009. Brussels: European Comission. and Technology. A reader. Baltimore: John Hopkins University
[14] Faulkner, W. (2009). Doing gender in engineering press.
workplace cultures. I. Observations from the field. Engineering [33] Light, J. S. (1999). When computers were women.
Studies, 1(1), 3-18. Technology and Culture, 40(3), 455-483.
[15] Faulkner, W., Lie, M. (2007). Gender in the Information [34] Malloy, J. (2003). Women, Art & Technology.
Society: Strategies of Inclusion. Gender Technology and Massachusetts: MIT Press.
Development, 11(2), 157-177. [35] Margolis, J., Fisher, A. (2003). Unlocking the
[16] Flick, U. (2006). An Introduction to qualitative research clubhouse. Women in Computing. Massachusetts: The MIT Press.
(3rd Ed.). London: Sage Publications. [36] McQuillan, H. (2010). Technicians, Tacticians and
[17] Gansmo, H.J., Lagesen, V.A. & Sorensen, K.H. (2003). Tattlers: Women as Innovators and Change Agents in Community
Forget the Hacker? A critical re-appraisal of Norwegian studies of Technology Projects. Special Double Issue. Gender in Community
gender and ICT. En Lie, Merete. (Ed.) He, She and IT Revisited. Informatics. 5 (3) & 6(1).
New Perspectives on Gender in the Information Society, 34-68. [37] Morgan, A. J., Quesenberry, J.L., Trauth, E. M. (2004).
Oslo: Gyldendal Akademisk. Exploring the importance os social networks in the IT workforce.
[18] Gil-Juárez, A., Vitores, A., Feliu, J., & Vall-llovera, M. Experiences with the Boy’s club. Proceedings of the 10 AMCIS,
(2011). Brecha digital de género: una revisión y una propuesta. 165, 1313-1320.
Teoría de la Educación. Educación y Cultura en la Sociedad de la [38] Moss‐Racusin, C. A., & Rudman, L. A. (2010).
Información, 12(2), 25-53. Disruptions in women's self‐promotion: the backlash avoidance
[19] Gill, R. (2002). Cool, creative and egalitarian? model. Psychology of women quarterly, 34(2), 186-202.
Exploring Gender in Project.Based New Media Work in Europe. [39] Nentwich, J. C., & Kelan, E. K. (2014). Towards a
Information, communication and Society, 5(1), 70-89. topology of ‘doing gender’: An analysis of empirical research and
[20] González, A. M. & Vergés, N. (2011). "Moving for its challenges. Gender, Work & Organization, 21(2), 121-134.
what? International Mobility Strategies of Women in ICT [40] OCDE (2010). Information Technology Outlook. OECD
Careers". International Journal of Gender, Science and Publishing.
Technology, 2, 501-516. [41] Olesen, V. L. (2000). Feminisms and Qualitative
[21] González, M.; Pérez, E. (2002). Ciencia, tecnología y Research At and Into the Millennium. En Denzin, Norman. K. &
género. CTS+ I: Revista Iberoamericana de Ciencia, Tecnología, Lincoln, Yvonna. S. Handbook of qualitative research (2nd Ed.),
Sociedad e Innovación, (2), 5. 235-278. London: Sage Publications.
[22] Griffiths, M., Moore, K., Richardson, H. (2007). [42] Pérez, O. (2010). Capítulo III Mujeres, innovación y
Celebrating heterogeneity? A survey of female ICT professionals políticas de conciliación e igualdad en las empresas TIC
in England. Information, communication and Society, 10(3), 338- españolas. Género y TIC. Presencia, posición y políticas, 3, 107.
357 Barcelona: Editorial UOC.
[23] Hache, A. (2011). Under the Radar: The Contribution [43] Plant, S. (1997). Zeros + Ones. Digital women + the
of Civil Society and Third Sector Organisations to eInclusion. new technoculture. New York: Doubleday.
IPTS. European Comission. Sevilla: Publications Office of the [44] Rasmussen, B., & Håpnes, T. (1991). Excluding women
European Union. from the technologies of the future?: A case study of the culture of
[24] Hache, A., Cruells, E. & Vergés, N. (2013). Yo computer science. Futures, 23(10), 1107-1119.
programo, tu programas, ella hackea: mujeres hackers y [45] Rommes, E. (2011). Websites for women: new routes to
perspectivas tecnopolígicas. In Natansohn. Internet en Código digital inclusion. In Sorensen, Faulkner and Rommes.
Femenino. Buenos Aires: La Crujía Ediciones. Technologies of Inclusion. Gender in the information society, 87-
[25] Haraway, D. (1988). Situated knowledges: The science 105.Trondheim: Tapir academic press.
question in feminism and the privilege of partial perspective. [46] Rommes, E., Slooten, I., Oost, E., & Oudshoorn, N.
Feminist studies, 14(3), 575-599. (2004). Designing Inclusion: The development of ICT products to
[26] Hewlett, S., Luce, C.B., Servon, L. J., Shiller, P., include women in the Information Society.
Sherbin, L., Sosnovich, E. & Sunberg, K.(2008). The Athena [47] Sáinz, M., Pálmen, R., & García-Cuesta, S. (2012).
factor: Reversing the brain drain in Science, engineering and Parental and secondary school teachers’ perceptions of ICT
technology. Cambridge, MA: Harvard Business Review Research professionals, gender differences and their role in the choice of
Report. studies. Sex roles, 66(3-4), 235-249.
[27] Hospers, G. J. (2008). Breeding places in the knowledge [48] Schiermeier, Q. (2008). Catalonian Powerhouse. Nature,
economy. Knowledge, Technology and Policy, 16(3), 143-162. 454, 248-249.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 304 |
 Moving towards Accommodating Women with ICT: Paying Attention to Self-inclusion Mechanisms
N. Vergés Bosch
[49] Scott-Dixon, K. (2004). Doing IT. Women working in [61] Vergés, N. (2012). De la exclusión a la autoinclusión de
Information Technology. Toronto: Sumach Press. las mujeres en las TIC. Motivaciones, posibilitadores y
[50] Sitzmann, T., Ely, K. (2011) A meta-analysis of self- mecanismos de autoinclusión. Athenea Digital-Revista de
regulated learning in work-related training and educational pensamiento e investigación social, 12(3), 129-150.
attainment: What we know and where we need to go. [62] Vergés, N. (2014). De las ausencias a las presencias de
Psychological Bulletin, 137(3), 421-442. las mujeres en las TIC. Un acercamiento a sus mecanismos de
[51] Smith, J. L., & Huntoon, M. (2013). Women’s Bragging autoinclusión. In Castaño and Webster (Ed.) Género, Ciencia y
Rights Overcoming Modesty Norms to Facilitate Women’s Self- Tecnologías de la Información, 165-190. Barcelona: Aresta.
Promotion. Psychology of Women Quarterly, 0361684313515840. [63] Vergés, N., Cruells, E., Hache, A. (2009). Viejos Retos
[52] Sorensen, K.H. (2002). Love, Duty and the S-curve: An y Nuevas potencialidades para las mujeres en la participación del
Overview of Some Current Literature on Gender and ICT. 1–36, desarrollo de la sociedad de la información. Revista Feminismo/s,
SIGIS. Deliverable Number: D02_Part 1. 14, 163-182
[53] Sorensen, K. H., Faulkner, W., Rommes, E. (2011). [64] Vergés, N., González, A. M.., & Almeda, E. (2014).
Technologies of Inclusion: Gender in the Information Society. Doing and Undoing Genders and Information and Communication
Tapir Akademisk Forlag. Technologies. ACM Proceedings of the XV International
[54] Sorensen, K.H. (2011). Changing perspectives on Conference on Human Computer Interaction, 80.
gender and technology: From exclusion to Inclusion. In Sorensen, [65] Vergés, N., Hache, A., Cruells, E. (2011). Indagando en
Faulkner and Rommes. Technologies of Inclusion. Gender in the la relevancia de Internet en el acceso, uso y deseos de las TIC por
information society, 87-105.Trondheim: Tapir academic press. parte de las mujeres en las TIC. Teoría de la Educación.
[55] Sveningsson, Malin & Sunden, Jenny. (2007b). Educación y Cultura en la Sociedad de la Información, 2, 105-
Cyberfeminism in Northern Lights: Digital Media and Gender in 121.
a Nordic Context.Cambridge: Cambridge Scholars Publishing. [66] Vergés, N., Hache, A., Cruells, E. (2014).
[56] Trauth, E., Quesenberry, J. L., Huang, H. (2009). Ciberfeminismo de investigacción con y entre tecnoartistas y
Retaining Women in the US IT Workforce: Theorizing the hackers. Athenea digital: revista de pensamiento e investigación
Influence of Organizational Factor. European Journal of social 14, 153-180.
Information Systems. Special Issue on Meeting the Renewed [67] Wajcman, J. (2010). Feminist Theories of Technology.
Demand for IT Workers(18), 476-497. Cambridge Journal of Economics, 34(1), 143-152.
[57] Trauth, E.M. Howcroft, D. (2006). Critical empirical [68] Werquin, P. (2007). Moving Mountains: Will
research in IS: an example of gender and the IT workforce, Qualifications Systems Promote Lifelong Learning? European
Information Technology & People, 19(3), 272 – 292. Journal of Education, 42(4), 459–484.
[58] Turkle, S. (1988). Computational reticence: Why [69] Werquin, P. (2008). Recognition of Non-formal and
women fear the intimate machine. In Krauarae, C. Technology Informal Learning in OECD Countries: A Very Good Idea in
and women’s voices: Keeping in touch, 41-61. New York: Jeopardy. Lifelong Learning in Europe, 3 2008, 142-149.
Routledge and Kegan. [70] Zimmerman, B. J., & Kitsantas, A. (2005). The Hidden
[59] Valenduc, G. (2011). Not a job for life? Women’s dimension of personal competence: Self-Regulated Learning and
Progression, Conversion and Dropout in ICT Professions. Practice. In Elliot, A. J. & Dweck (Ed.) Handbook of Competence
International Journal of Gender, Science and Technology, 3 (2), and Motivation, 204-222. New York: Guilford Press.
483-500.
[60] Vázquez, S. (2010). Los dilemas de las jóvenes
ingenieras en el sector TIC. In Castaño et al. Género y TIC.
Presencia, posición y políticas, 251. Barcelona: UOC Ediciones.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (I)

| 305 |
 ENGENDERING
TECHNOLOGY (II)
 Influencia del Género en el Pensamiento Computacional
E. E. Espino Espino | C. S. González González

Influencia del Género en el Pensamiento Computacional

Elisenda Eva Espino Espino Carina Soledad González González

Instituto Universitario de Estudios de Instituto Universitario de Estudios de
las Mujeres (IUEM) las Mujeres (IUEM)
Universidad de La Laguna Universidad de La Laguna
[email protected] [email protected]

RESUMEN como un condicionante en la adquisición de conocimientos y

Este estudio tiene como objetivo analizar la influencia del habilidades en programación. Cuando hablamos de pensamiento
pensamiento computacional en relación al género, ya que es computacional, nos referimos a un concepto que implica organizar
importante conocer las causas por las que en la actualidad la y representar información de forma lógica, resolver problemas y
presencia de las mujeres en las carreras de ingeniería informática automatizar soluciones con ayuda del pensamiento algorítmico,
sea cada vez más escasa. En este artículo se presenta un análisis diseñar sistemas y comprender el comportamiento humano,
sobre las competencias y estrategias que se relacionan con el haciendo uso de los conceptos fundamentales de la informática,
pensamiento computacional efectuado en la competición nacional tal y como expone Wing (2006) [9].
de robótica educativa FLL (First Lego League). Los resultados Teniendo en cuenta los estudios anteriores sobre la brecha de
confirmaron que hay mayor presencia masculina que femenina y género en los estudios de ciencias, ingeniería e informática y la
una clara diferenciación de género. Sin embargo, en cuanto a necesidad de que las niñas aprendan competencias tecnológicas,
procesamiento y aprendizaje de información, existe igualdad. en concreto, la competencia de pensamiento computacional [9],
este trabajo de investigación tiene como objetivo analizar si
Categories and Subject Descriptors existen diferencias de género en la enseñanza-aprendizaje de la
H.1.2 [User/Machine Systems]: Human Factors, Human misma [10]. Para ello, necesitamos conocer cómo se está
Information Processing, Software Psychology. enseñando esta competencia en los colegios. En este sentido,
existen distintas iniciativas que se están llevando a cabo a nivel
PALABRAS CLAVES internacional, nacional y local, y casi todas tienen como punto en
Género, Brecha de género, Pensamiento Computacional, común la robótica educativa y la programación de videojuegos.
Enseñanza de la Programación Entre estas iniciativas, destacaremos la FIRST LEGO League
(FLL), que es un torneo que desafía a jóvenes de 10 a 16 años con
una temática del mundo real relacionada con la ciencia y la
1. INTRODUCCIÓN tecnología. Cada año FLL propone un nuevo desafío, que tiene
El avance científico y tecnológico es uno de los principales tres partes: el proyecto científico, el diseño y programación de un
desafíos que la comunidad mundial debe afrontar en este nuevo robot y los valores FLL (inclusión, descubrimiento, coopertición,
milenio. Sin embargo, cuando se habla de mujeres, la reacción respeto y trabajo en equipo). Para participar en el desafío los
inmediata es la de indicar la poca presencia de éstas en su equipos se inscriben y organizan sus torneos microFLL, que dan
desarrollo. Lo que sucede es que a lo largo de toda la historia y acceso a los Torneos Clasificatorios FLL, realizándose
hasta la actualidad, ha existido una cultura androcéntrica [1] que competiciones en diferentes niveles: dentro del propio colegio, en
ha prevalecido en la conciencia individual y colectiva de muchos la localidad, en la región, a nivel nacional y a nivel internacional.
y muchas, repercutiendo en la invisibilidad de destacables Por ello, como punto de partida, en este trabajo, nos planteamos
contribuciones femeninas [2] y abriendo una profunda brecha de estudiar las características y estrategias seguidas por los equipos
género [3] en este campo. En primer lugar, cabe plantearse si las que habían sido capaces de superar los diferentes torneos hasta
mujeres llegan a la educación superior en igualdad de condiciones llegar al torneo nacional. Para ello, hemos analizado las opiniones
que los hombres, dada la distinta socialización que experimentan del profesorado de los grupos de estudiantes ganadores del
[4]. En segundo lugar, hay que prestar atención a los estereotipos campeonato nacional (España) de la feria de robótica educativa
sexuales presentes en nuestras vidas desde el momento en que FLL, llevado a cabo en Tenerife en marzo de 2015 (Figuras 1, 2 y
nacemos hasta que llegamos a la etapa adulta [5]. Por último, 3). Asimismo, se estudiaron las opiniones de los árbitros, del
atender a la discriminación jerárquica, según la cual, mujeres equipo de voluntariado y jueces que participaban en dicha
competición.
capaces y brillantes son mantenidas en los niveles inferiores en la
escala política, en negocios o en la ciencia; o topan con un “techo
de cristal” que no pueden traspasar en su propia profesión [5].
Son muchos los estudios, nacionales e internacionales como el
informe PISA 2015 [6] o la encuesta de Competencias de la
Población Adulta de la OCDE de 2012 [7], los que revelan el
ascendente rendimiento que experimentan las niñas y jóvenes en
materias como matemáticas, física o tecnología; siendo incluso
mayor que el de sus compañeros varones [8]. Por ello, en este
estudio se atiende a la influencia del pensamiento computacional

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 307 |
 Influencia del Género en el Pensamiento Computacional
E. E. Espino Espino | C. S. González González
• En el procesamiento y el aprendizaje de la
información, no hay una diferenciación relevante
que justifique que varían según el género (H2)
Este artículo se organiza de la siguiente manera: primero,
presentaremos algunas iniciativas relacionadas a nivel
internacional, nacional y en Canarias donde se utilizan programas
como el Scratch que proporciona una iniciación dinámica en
programación a los niños y niñas, luego presentaremos el estudio
que hemos llevado a cabo en la FLL Nacional 2015 y sus
principales resultados. Finalmente, se discutirán dichos resultados
y se presentarán las conclusiones del estudio.

2. MÉTODO
Diseño
Figura 1. Equipo en la compentencia. Programación del robot
defendida por chicas. Se ha realizado un estudio de tipo cualitativo, selectivo y co-
relacional por muestreo probabilístico aleatorio estratificado.
Participantes
La población encuestada se compone de 53 sujetos: 39 docentes, 5
árbitros y 9 personas dedicadas al voluntariado en eventos sobre
programación educativa. Aludiendo al género (Tabla 1), las
distintas agrupaciones que encontramos son: género de
Formadores, género de árbitros y género de voluntariado;
denotándose una mayor asistencia masculina salvo en el
voluntariado, donde existe una pequeña diferencia.
Tabla 1. Género de participantes
Formadores Árbitros Voluntariado
Masculino 56% 80% 40%
Femenino 44% 20% 60%
A su vez, el grupo de Formadores, el de árbitros y el de
voluntariado, provienen de las siguientes Comunidades
Autónomas, destacando, por su mayoría, a Canarias y Catalunya
Figura 2. Árbitros evaluando la programación del robot (Tabla 2):
durante la competición. Tabla 2. Comunidades Autónomas
Formadores Árbitros Voluntariado
Andalucía 3 - -
Canarias 10 4 6
Catalunya 16 - 2
Castilla y León 5 - -
Navarra 1 - -
País Vasco 4 - 1
En formación adquirida, la mayor parte de los sujetos pertenecen
a ramas técnicas. El 41% de los docentes son de Ingenierías
Superiores; seguido de Ingenierías técnicas con un 20%,
quedando detrás estudios de: magisterio, otras diplomaturas,
licenciatura en matemáticas, física; licenciatura en
psicopedagogía, pedagogía, arquitectura o doctorado. En el caso
de los árbitros, se sitúan con un 80% en Ingeniería Técnica
seguido de Ingenierías Superiores. Por último, el voluntariado se
aleja de estos datos, caracterizándose por sumar un 66% de
estudiantes y un 34% de diplomados en áreas no especializadas en
Figura 3. Diferentes equipos de la competición nacional. contextos tecnológicos. En relación al grupo docente, es el que
que acumula mayor experiencia profesional. Se sitúan con un 69%
en más de 5 años. Sin embargo, los árbitros y voluntariado no
De este modo, exploramos dos hipótesis, siendo estas: presentan años de docencia, salvo un 2% del primer grupo con
• Existen diferencias de género en el aprendizaje del menos de 1 año. A su vez, las personas con más titulaciones afines
pensamiento computacional siendo mayor la al campo técnico y tecnológico son los docentes con un 72%,
presencia de varones. (H1) seguidos por los árbitros que presentan un 60%. Haciendo
referencia al voluntariado, pertenecen a otras áreas en un 90%.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 308 |
 Influencia del Género en el Pensamiento Computacional
E. E. Espino Espino | C. S. González González
Con respecto a las estrategias para la programación que aparecen
Tabla 3. Área de especialización por agrupamiento en la Tabla 6, destacan: ensayar y depurar y la enseñanza de
sensores, seguidos de ser incremental e iterativo y la
Especialización Formadores Árbitros Voluntariado programación lineal.
SÍ 72% 60% 10%
NO 28% 20% 90% Tabla 6. Estrategias de programación
Instrumentos Estrategia ni fi
Se confeccionaron tres cuestionarios (uno para el profesorado, Ser incremental e iterativo 9 60%
otro para los árbitros y el último, para el voluntariado) con 6 Ensayar y depurar 14 93%
apartados. Uno se dedicó a los datos referenciales (género, edad, Reusar y re-mezclar 5 36%
residencia, formación, área de especialización y docencia). Otro, a Abstraer y modularizar 2 13%
Bloques 7 47%
la composición de los grupos de alumnado en el evento. El
Sensores 10 67%
tercero, a los conceptos de programación que se trabajan con el
Programación lineal 9 60%
alumnado. El siguiente, a las estrategias y metodología de Programación por bloques 7 44%
programación adoptadas. Posteriormente, se aludió a las Profundizando en la metodología adoptada por los responsables
competencias trabajadas en el proceso de enseñanza-aprendizaje de la docencia, se menciona la dedicación de un trimestre a la
con esta actividad. A continuación, se introdujo una pregunta de enseñanza de los conceptos básicos y, posteriormente, se acogen
respuesta múltiple para conocer opiniones acerca de la inserción a un sistema totalmente práctico con creación de Apps, Robótica,
de la asignatura de programación en el currículo educativo. Y un etc. También destaca el trabajo por proyectos y el trabajo
último apartado con una pregunta cerrada y otra abierta dedicado cooperativo. En relación a las competencias que pueden
a valorar las posibles diferencias de género en el desarrollo del favorecer el proceso de enseñanza-aprendizaje de la
pensamiento computacional. Además, se efectuaron entrevistas programación comprobamos en la Tabla 7 las que más destacan:
informales con los tres grupos citados: formadores, árbitros y autonomía e iniciativa personal, aprender a aprender y el
voluntariado; y se realizó una observación procesual no tratamiento de la información, competencia digital e innovación.
estructurada del comportamiento de los alumnos y alumnas. Hay que aludir también a la importancia que los encuestados/as
Procedimiento dan al trabajo en equipo y la educación en valores.
Los cuestionarios, junto con una explicación del estudio y una Tabla 7. Competencias en programación
invitación a participar en el mismo, fueron entregados a los
entrenadores responsables de los equipos participantes de la FLL Competencia ni fi
Comunicación Lingüística 11 69%
el día del registro (día anterior al comienzo del evento).
Matemáticas 10 63%
Asimismo, durante su desarrollo se puso a disposición de los
Conocimiento e Interacción con el mundo físico 12 75%
formadores/as, árbitros y voluntarios/as. Una vez Tratamiento de la información, competencia digital e
cumplimentados, se recogieron y se registraron telemáticamente 13 81%
innovación
en una base de datos que, posteriormente, se sometió a los Social y ciudadana 10 63%
pertinentes análisis estadísticos. Cultural y artística 8 50%
En relación a las entrevistas informales, se realizó un registro de Aprender a aprender 14 88%
las mismas y con respecto a la la observación no estructurada, se Autonomía e iniciativa personal 15 94%
tomaron anotaciones del comportamiento del alumnado durante el En cuanto a la inserción de la programación en el currículo
desarrollo de la liga FLL Nacional 2015. educativo, podemos ver reflejado en la Tabla 8 como los/las
docentes en un 70% coinciden en que la materia se convierta en
3. RESULTADOS obligatoria, al igual que los árbitros con un 60%. El voluntariado,
Los primeros datos, indican que la composición del alumnado de por el contrario, opina en un 89% que sería más conveniente que
los equipos participantes es, mayoritariamente, masculino (64%). la asignatura sea optativa, sobre todo, por gustos y preferencias
Haciendo referencia al segundo apartado, los conceptos de y/o capacidades individuales.
programación que más se suelen trabajar con los alumnos/as, por Tabla 8. Inserción de la programación en el Currículo
orden descendente son: los bucles, los condicionales y las Educativo
secuencias. Además, se hace alusión al trabajo de distintos
Formadores Árbitros Voluntariado
lenguajes de programación y a la comunicación entre dispositivos. Inserción
ni fi ni fi ni fi
Podemos comprobarlo en la Tabla 4 de este artículo en la que ni Obligatoria 27 70% 3 60% 1 12%
son valores absolutos y fi frecuencias relativas: No obligatoria - - - - - -
Tabla 4. Conceptos de programación Optativa 11 28% 2 40% 8 89%
Extraescolar 2 5% - - - -
Concepto ni fi
Por último, los tres agrupamientos indicados en la Tabla 9,
Secuencias 11 73%
Ciclos 8 54%
coinciden en que existen diferencias de género en programación,
Paralelismo 2 13% aunque en los/las formadores/as se observan más diferencias.
Eventos 6 40% Tabla 9. Diferencias de Género en Programación
Condicionales 11 73%
Operadores 9 60% Formadores Árbitros Voluntariado
Diferencias de Género
ni fi ni fi ni fi
Datos 6 40%
SÍ 25 64% 5 100% 9 100%
Bucles 12 80%
NO 14 36% - - - -
Otros 6 40%

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 309 |
 Influencia del Género en el Pensamiento Computacional
E. E. Espino Espino | C. S. González González
A su vez, se realiza un desglose por parte de las tres agrupaciones su participación; mientras que si se alude al punto débil donde
de los puntos fuertes y débiles según el género, masculino o todos/as coinciden, destaca la organización. Por el contrario,
femenino. En la Tabla 10 encontramos los resultados del grupo de estudiando al género femenino, el punto fuerte donde todos los
formadores, de los árbitros y, del voluntariado. grupos coindicen es la organización. En el caso negativo, se
Tabla 10. Diferencias de Género formadores, árbitros y detecta en ellas falta de participación, dominio y liderazgo. Todo
voluntariado ello unido a las anotaciones recopiladas mediante las entrevistas
efectuadas y la observación etnográfica, descubrimos que los
Punto Fuerte Punto Débil chicos suelen destacar en aspectos relacionados con la práctica y
Formadores taller, la competitividad y el liderazgo pero fallan en la
- Constancia
+ Rapidez y Motivación organización y el trabajo en equipo. Por el contrario, las chicas
- Organización
Masculino
+Prácticos
- Orden son buenas organizadoras, saben trabajar en equipo, son más
+Competitividad constantes y creativas pero descienden en liderazgo. Si
- Trabajo en Equipo
+Liderazgo reflexionamos acerca de este asunto y esta clara inversión,
- Focalización de intereses
+Atención - Poca práctica planteamos que la creación de grupos mixtos podría ser
+Constancia - Liderazgo beneficiosa porque habría una notable complementariedad de
Femenino +Creatividad - Gustos cualidades entre ellos y ellas, algo que podría aumentar el
+Organización rendimiento de dichas agrupaciones. A su vez, tenemos presente
+Trabajo en Equipo que todos los resultados pueden verse modificados por la
Árbitros
existencia de variables que influyen considerablemente como son:
Masculino +Participación - Organización
las diferencias individuales, las inteligencias múltiples, los estilos
+Liderazgo - Participación
Femenino
+Organización de aprendizaje, entre otros [4].
Voluntariado Por último, no se han encontrado diferencias notorias que
Masculino +Prácticos - Organización justifiquen que el procesamiento de la información y el
+Trabajo en Equipo - Carácter aprendizaje varía según el género. Asimismo, los diferentes
Femenino
+Organización agentes estudiados han manifestado que no han observado
desigualdades en su práctica profesional, sin embargo, esto puede
4. DISCUSIÓN Y CONCLUSIONES ser debido a que no se ha estudiado directamente este fenómeno ni
se ha prestando atención en profundidad. Por ello, planteamos la
En términos generales, se observa que el grupo de docentes y
necesidad de continuar estudiando e investigando este asunto y
árbitros, conocedores muchos y muchas en profundidad del área
plantear otro tipo de estudios que ahonden en este fenómeno, con
técnica, consideran que la asignatura de programación debería
instrumentos subjetivos como los que hemos utilizado en este
estar implícita en el currículo educativo según los datos obtenidos
estudio, así como otros instrumentos objetivos (por ejemplo, eye
en la variable que alude a la inserción de la programación, en el
tracking, brain computing, etc.). En relación a esta investigación,
apartado de resultados de este artículo. Sin embargo, aquellos/as
nos ha abierto puertas para poder seguir trabajando en aspectos
no especializados en este ámbito, como es el voluntariado,
relacionados con la temática de enseñanza de programación desde
exponen que debe incluirse pero de forma optativa. Las razones
una perspectiva de género, como puede ser la relación de
dadas por parte de este último grupo, no expertos en tecnología,
competencias que se podrían desarrollar con el fin de convertirlas
tiene que ver con su complejidad y con que los intereses del
en fortalezas para el proceso de enseñanza-aprendizaje,
alumnado pueden no estar orientados a esta temática. Tras un
previniendo problemática estereotipada y mejorando resultados. A
proceso de entrevista con estos sujetos, se denotó que relacionan
su vez, consideramos que aporta datos susceptibles de seguir
la programación con conceptos puramente matemáticos y
profundizando ya que su abordaje es justificable en otros
abstractos. Sin embargo, los docentes y árbitros, expertos en
contextos educativos, siendo solo un eslabón más de una larga
tecnología, abordan la programación como una materia con gran
cadena con gran proyección de futuro. De este modo, estaremos
contenido transversal y multitud de posibilidades y beneficios con
encantadas de que pueda servir de base para posteriores estudios.
respecto al resto de asignaturas [7]. Respecto a a la cantidad de
alumnado en este tipo de competiciones sobre robótica y
programación, encontramos una mayor presencia de sujetos
masculinos que ascienden a un 64% del total según los datos de 5. REFERENCIAS
este studio [4]. Por ello, consideramos que aún hay que trabajar [1] Facio, A., Fries, L. (1999). Feminismo, género y patriarcado.
profundamente en el fomento de vocaciones científicas y de Bibliovirtual: http://cidem-ac.org
ingeniería en las niñas. En el transcurso del estudio, observamos [2] Kulis, S., Flavio, F., Nagoshi, J. (2010). Gender Roles,
que en las etapas de infantil y primaria existe mayor Externalizing Behaviors, and Substance Use Among
homogeneidad entre niños y niñas, mientras que a medida que Mexican-American Adolescents. Journal of Social Work
aumenta el nivel educativo disminuye, quizás por posibles Practice in the Addictions, 10 (3), 283-307
intereses o estereotipos implantados.
[3] Orengo, R. (2014). Estereotipos Sexuales y su relación con
En relación a la existencia de diferencias de género, las conductas. Rev. Puertorriqueña Psicología, 23 (2), 48-61
comprobamos que los puntos fuertes de los chicos son,
mayoritariamente, los débiles de las chicas, y con los puntos [4] Ortiz, Elena. (2007). Inteligencias múltiples en la educación
de la persona. (6ª Ed.) Argentina: Ediciones Bonum
débiles ocurre algo similar. Si se analiza el género masculino, hay
coincidencias entre las aportaciones de los/las docentes y el [5] Sarrió, M. (2002). La Psicología de Género a través del
voluntariado en el punto fuerte que alude a que los chicos son más “Techo de Cristal”. Tesis Doctoral. Universitat de València.
prácticos para realizar las tareas destacando los árbitros, además, Dirigida por Ester Barberá. Manuscrito no publicado

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 310 |
 Influencia del Género en el Pensamiento Computacional
E. E. Espino Espino | C. S. González González
[6] [Informes PISA (2015). Accesible en: http://bit.ly/1cXFBAh [9] Wing, J. (2006). Computational thinking. Communications
[7] Informe OCDE (2012). Accesible en: http://bit.ly/1C38VBA of the ACM, 49(3) , 33-35

[8] Waksman, N. (2005). El papel de la mujer en la ciencia. [10] Justina, H. (2014). Gender differences in Scratch Game
Universidad Autónoma de Nuevo León, México: Ciencia design. 3rd International Conference on Information,
UANL Business and Education Technology (ICIBET 2014), 100-
103.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 311 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera

Dos sentidos de lo tecnológico en relatos de vida de

mujeres tecnólogas
Adriana Gil-Juárez Joel Feliu Montse Vall-llovera
Universitat Rovira i Virgili Universitat Autònoma de Barcelona Universitat Oberta de Catalunya
Campus Sescelades Departament de Psicologia Social Estudis de Psicologia i Ciències de
Departament de Psicologia Edifici B l’Educació
+34977558207 +34935814502 Rambla del Poblenou, 156
[email protected] [email protected] +34933563886
[email protected]
Ester Conesa
Universitat Rovira i Virgili
Campus Sescelades
Departament de Psicologia
+34977558207
[email protected]

RESUMEN 1. INTRODUCCIÓN
Los relatos de vida tecnológica nos ayudan a comprender la El descenso en las personas matriculadas en ingenierías
relación entre la experiencia con la tecnología y los múltiples informáticas en el mundo occidental sigue produciéndose, hasta el
significados que le otorgamos. Presentamos los resultados de un punto que Caroline Clarke Hayes [1] llega a afirmar que: “If this
proyecto de investigación que elaboró las trayectorias de vida trend were to continue at the rate experienced from 1986 to 2006,
tecnológica de 15 mujeres del sector, de edades comprendidas there will be no women bachelor degree graduates in computer
entre los 19 y los 46 años, con la finalidad de conocer los science by 2032”. Se cumpla o no, lo cierto es que las dificultades
recorridos que pueden facilitar o inhibir un uso experto de la que siguen teniendo muchas mujeres a la hora de tomar la
tecnología. En sus relatos, observamos dos sentidos de lo decisión de estudiar una carrera informática siguen estando ahí.
tecnológico. El primero es producto de un contacto experto tardío Muchas de estas dificultades se explican a través de los itinerarios
con la informática, que no suele producirse hasta el momento de biográficos de las mujeres en relación a las tecnologías a lo largo
llegar a la universidad o al mundo laboral, lo que marca un punto del ciclo vital. En la abundante literatura académica dedicada a
de partida proclive a las dificultades y frustraciones, esta cuestión, encontramos que las trayectorias tecnológicas que
especialmente durante la carrera: genera sentimientos de pueden seguir las mujeres pueden dividirse en dos grandes rutas
competencia o autoeficacia ambivalentes, y conduce a caracterizar que se entrecruzan en cada persona: una de Exclusión y otra de
la tecnología como compleja, de difícil acceso y necesitada de Inclusión. Es decir aquellas rutas que tienden a expulsar a las
gran dedicación. Sin embargo, en sus explicaciones encontramos mujeres del mundo tecnológico experto y aquellas que tienden a
también una visión de la tecnología como un mecanismo de acercarlas, no sin dificultades, a él.
transformación social, que sirve tanto para mejorar su entorno
inmediato como para generar algo útil. Sus relatos nos ofrecen la 1.1 Rutas de la exclusión
posibilidad de resignificar las tecnologías, pensándolas como Esta línea de investigación cuenta con una gran cantidad de
socialmente útiles, potenciadoras del cuidado y garantes de literatura, es una literatura que suele reproducir la idea de la
transformación. Focalizar esta resignificación de lo tecnológico existencia de diferencias de género estables al explorar los
podría ayudar a cambiar la situación actual de fractura digital de diferentes usos e intereses que mujeres y hombres tienen y sus
género creciente. Al mismo tiempo, también ayudaría evitar causas.
adoptar las trayectorias masculinas como referente, ya que esto
En el ámbito escolar se han explorado las políticas escolares, el
contribuye a destacar las dificultades, suele llevar a una visión de
diseño del currículum, la optatividad y los procesos para escoger
la trayectoria de las chicas como deficitaria e invisibiliza las
materias como espacios donde se produce discriminación
“rutas” diversas de las chicas y sus aportaciones en el ámbito
estructural (Aspray, 2006). Destacan también los estudios de las
tecnológico.
interacciones que exploran las relaciones maestras-alumnas y
Categorías y descriptors temáticos como se transmiten las creencias y prácticas sobre la informática y
K.4.0 [Computers and Society]: General sus profesiones como dominio masculino. Es decir, el efecto de la
transmisión de expectativas y la falta de apoyo o promoción de los
Términos Generales intereses y aptitudes informáticas. Así mismo, se han explorado
Human Factors las relaciones maestras-ordenadores: lo que conocen o sienten
hacia los ordenadores (desconfianza y desinterés en gran medida);
Palabras Clave el poco uso de los ordenadores para usos educativos; su poca
Brecha digital de género; Relatos de vida; Trayectorias formación; y, la falta de promoción del interés en las chicas. [3, 4,
tecnológicas; Significado de Tecnología. 5, 6, 7].

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 312 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
Varias autoras [2, 8, 9] han comparado la presencia y uso de modelos [34, 32]. Sin embargo, para ellas, la experiencia y
ordenadores en casa y en la escuela, encontrando que: conocimientos adquiridos en casa son escasos, los primeros
contactos suelen producirse en la escuela [35, 36]. Un hecho que
• Los chicos dominan el uso de los ordenadores en la seguramente contribuye a que incluso en las chicas que estudian
escuela y escogen más cursos optativos en informática informática la confianza en las propias habilidades sea menor que
que las chicas. la de los chicos [37, 38, 26, 39, 40, 41, 42].
• Los chicos usan más los ordenadores en casa, para más Los estereotipos sobre el estudiante de informática no ayudan, y
cosas y más tiempo para ellas acaba siendo una necesidad distanciarse del estereotipo,
• Los chicos empiezan de más pequeños. casi ninguna quiere ser vista como geek [43, 39, 35, 27, 28]. En
muchos casos la afición “hardcore” a los videojuegos no es
• Los usos en la casa y la escuela se influencian determinante puesto que aparece a posteriori, provocada por la
mutuamente. relación con las amistades informáticas [34].
En general, la familia y la comunidad local (vecinos, amigos) Estas experiencias, sin embargo, no son comunes a todas las
suelen apoyar más los intereses en informática de los chicos que estudiantes de informática, en algunas minorías, por ejemplo en
de las chicas.[6, 10, 7]. Las aspiraciones profesionales y vitales de afroamericanos, no existe una brecha digital de género, y entran
las chicas se suelen corresponder con las expectativas de los en la misma cantidad hombres y mujeres. En el caso de migrantes
padres, entre las que no se pueden obviar las expectativas sobre la la informática se percibe como un posible camino de ascensión
vida familiar futura y las dificultades que puede suponer para ella social [29, 42].
un posible futuro entorno laboral absorbente. Los iguales
transmiten también las creencias de las chicas y los chicos sobre Aunque algunas chicas llegan a la informática de manera parecida
lo inapropiados que son los cursos de informática para las chicas. a los chicos (por enamoramiento de la máquina o pasión por los
Además, los chicos del entorno se apropian físicamente de los videojuegos), existen lo que podemos llamar rutas invisibles [30].
espacios y aparatos [2]. Dichas rutas nos muestran otros caminos, alternativos a los de los
chicos, que las chicas pueden recorrer para relacionarse con las
También se ha explorado abundantemente el imaginario que tecnologías. Son invisibles porque ni las estudiantes de
transmite la cultura popular, a través de la televisión y las revistas, secundaria, de bachillerato, el profesorado o los padres y las
que refuerzan sistemáticamente los estereotipos de quiénes son los madres las conocen ni difunden, impidiendo de esta manera que
expertos, los que tienen conocimientos avanzados de informática. las chicas que las siguen puedan ejercer de modelos de otras. No
El estereotipo del geek es una figura masculina [12, 13, 14, 15, se ven no solamente porque sean chicas quiénes las siguen, sino
16]. Como parte de esta cultura ha sido también explorada porque no se suele considerar que lo que hacen sea informática
abundantemente la cultura del videojuego, a nivel de creadores, propiamente, como por ejemplo cuando una médico crea su
jugadores y contenidos de los propios videojuegos [17]. propio programa informático porque no encuentra el que necesita
o una diseñadora crea programas para sus producciones
Todo esto conlleva diferencias en las actitudes, la confianza, las audiovisuales.
habilidades y las competencias percibidas [18, 10, 19, Sabine
2009; 20, 21], y por lo tanto en el interés y las experiencias de Así, según Gil-Juárez et al. [30], se puede llegar a cursar una
chicos y chicas. En las chicas se describen menos actitudes ingeniería informática por alguna o varias de las siguientes
positivas (especialmente a partir de los trece), menos confianza razones: por querer hacer algo útil para los demás; por curiosidad
(los chicos sobreestiman sus habilidades mientras que las chicas de saber cómo funcionan las cosas; por motivos políticos – como
las subestiman), más atribuciones internas de los errores o fallos formar parte de Comunidades Hackers [44]; por llegar a la
(que suelen ser externas en los chicos [22], en definitiva, menos informática desde otras ocupaciones: medicina, arte, cine,
interés y por lo tanto menos experiencia. [23, 24]. animación, química, otras ingenierías, [45, 46].

1.2 Rutas de la inclusión Diferentes rutas, experiencias diversas, que nos muestran que
De forma paralela a la poca cantidad de mujeres presentes en los debe evitarse adoptar las trayectorias masculinas como referente
sectores tecnológicos existe también menos literatura respecto a la único porque esto contribuye a destacar las dificultades, suele
forma en qué han llegado ahí las que están. Esta literatura muestra llevar a una visión de las trayectorias de las chicas como
los porqués del interés de las chicas, cuando lo hay, por la deficitarias e invisibiliza sus “rutas” diversas.
informática y los ordenadores. Algunas autoras, como Judy Wajcman, han propuesto que el
En la opción por la informática tanto chicos como chicas invocan significado de lo tecnológico debe ser también considerado.
el gusto y el placer, pero ellas añaden factores prácticos y Según estas autoras la asociación de lo tecnológico con lo
oportunidades, mientras ellos se centran más en considerarlos la masculino lleva también a excluir a las mujeres de este terreno al
consecuencia natural de su interés [25, 2, 26, 27, 28, 31]. En el no considerarlo como algo propio ni apropiable. En esta
interés y atractivo de la informática ellas enfatizan las comunicación exploramos el sentido que tiene lo tecnológico para
aplicaciones y los usos, mientras que ellos “sueñan con código” un grupo de mujeres que han desarrollado una carrera en el
[28, 27]. Porque para ellos la afición surgió de un “amor a primera ámbito tecnológico, y que por lo tanto han seguido alguna de las
vista” con la máquina, mientras que en ellas el interés surge de rutas de inclusión. En este grupo encontramos que lo tecnológico
forma moderada y gradual [32, 33]. tiene sobre todo dos sentidos, en primer lugar como algo
complejo, de difícil acceso y necesitado de gran dedicación, y en
Los promotores de su interés siempre acaban siendo hombres, segundo lugar, también como mecanismo de transformación
destacándose la ausencia de modelos de rol mujer. Las chicas social y por lo tanto demasiado importante como para dejarlo de
mencionan a padres o hermanos mayores o amigos como apoyos o lado. Si el primer sentido puede interpretarse como efectivamente

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 313 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
más “masculino” en su sentido tradicional, el segundo sentido Tabla 1. Muestra
puede permitir a más mujeres ver la tecnología de forma
compatible con las exigencias del cuidado tradicionalmente
impuestas a las mujeres.

2. METODOLOGÍA
A partir de la revisión bibliográfica, preparamos un guión de
entrevista en profundidad, abierta y semi-estructurada con la
finalidad de comprender la relación entre la experiencia particular
con la tecnología y la conformación de la identidad de género
femenino. Buscábamos conocer en detalle no solo la trayectoria
que habían seguido estas mujeres en su relación con la tecnología
sino especialmente el sentido que para ellas tenía esta relación y la
tecnología misma.
La pregunta inicial era sobre el recuerdo del primer contacto con
un ordenador y/o una consola de videojuegos, siguiendo de ahí
una conversación sobre sus usos durante la infancia y
adolescencia (tiempos, espacios y personas implicadas), los
modos de aprendizaje, el contexto tecnológico familiar, escolar y
de amistades, la decisión por el estudio, sus vivencias iniciales en
la carrera y a lo largo de ella, los primeros contactos con el mundo
laboral y el desarrollo de su carrera.
Pusimos a prueba el guión en una experiencia piloto durante el
año 2012 en la que entrevistamos a 2 estudiantes de los últimos
cursos de la carrera de ingeniería informática y a 5 trabajadoras en
activo del sector tecnológico. Después de algunos cambios
menores en el guión, utilizamos una estrategia muestral en “bola
de nieve” a partir de contactos personales.
Entre 2013 y 2014 llevamos a cabo quince entrevistas a mujeres
con relaciones distintas con el mundo de la TIC, y de edades
comprendidas entre los 19 y los 46 años (ver Tabla 1)
Puesto que las entrevistas se centraron en las
trayectorias de estas mujeres el objetivo era conocer los relatos de
vida tecnológica que producían, con ello, pudimos conocer los
usos de las tecnologías por parte de estas mujeres, analizar sus
experiencias, percepciones y valoraciones, y, escuchar como las
relatan para darle sentido.
Tras la transcripción literal de todas las entrevistas, llevamos a
cabo un análisis cualitativo de contenido. El proceso de
realización del análisis consiste en transformar las “datos brutos”
(las manifestaciones directas recogidas en las transcripciones de
las historias de vida) en “datos útiles”. Es decir, sistematizar una
información manejable que permita una nueva comprensión de la
realidad.

Para el análisis seguimos las fases que describen Strauss y Corbin Cuando se les preguntó por sus primeros contactos con la
(1998): 1. Fase de segmentación; 2. Fase de codificación de informática, estos aparecen como diversos. En algunos casos se
apertura; 3. Fase de clasificación e interrelación de códigos de producen en la pre-adolescencia o la adolescencia, en casa, en la
apertura; 4. Fase de codificación focalizada; y, 5. Fase de escuela, el instituto o a través de alguna actividad extraescolar.
(re)contextualización. La finalidad del análisis es clasificar Sin embargo no es un contacto que se describa como intenso ni
temáticamente el contenido de la entrevista, establecer relaciones apasionante:
entre sus diferentes partes, destacar los elementos que se “...y el primer contacto fue, si no recuerdo mal, fue en plan
relacionan con los objetivos de la entrevista e interpretar el actividad extraescolar, que me apuntaron a la típica academia que
sentido de estos de forma sistemática.. hay al lado de la escuela, y recuerdo que había mecanografía e
informática.”
3. RESULTADOS
El análisis de los relatos de las mujeres entrevistadas nos mostró “Yo, o sea, no tengo así mucha conciencia de cuándo utilicé la
las maneras diversas en que éstas llegaron a las tecnologías, las primera vez el ordenador, pero yo no tuve ordenador hasta
usan, se las apropian y se relacionan con ellas.
XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)
| 314 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
supertarde. Y quizás las primeras veces fue en la escuela, en el durante la carrera. Consideran que en la carrera les enseñan poco a
instituto o así yo creo, sí.” programar, y que el profesorado no las tiene en cuenta y a menudo
tienen que recurrir a sus compañeros.
A menudo porque el ordenador no es para ellas o bien porque se
comparte de manera que no se puede acabar usando mucho: “Quiero decir a, Informática Básica, quiero decir lo que es
Programación nadie te, nadie te enseña, quiero decir, nadie te dice
“Bueno, mis padres compraron el primer ordenador a mi hermano, nada ... ellos hacen sus clases y básicamente son más a nivel
cuando él teníaaaa... él tenía 10 años y yo tenía 6 años, pero no era teórico [...] realmente no se preocupan de que tú sepas o no
para mi el ordenador, no? yo lo ví pero... era cosa que no... o sea sepas.”
yo no tenía mucho que ver con esto.”
“...para aprender las cosas nuevas que iban… incluso ahora no?
“…pero si, los primeros contactos que tengo yo con ordenador que van surgiendo... es un poco aquí te pillo aquí te mato,
pues es muy joven, yo creo que es como a los 16 o 17 años. Los “¿explícame como se hace esto?,¿ dime cómo has buscado lo
primeros ordenadores que salen, pues estamos en el instituto. Y
otro?”, mmm, “Esto para qué sirve?...¿cómo puedo guardar
bueno que nos lo ponían delante como si fueran una cosa… una esto?”. Mmmm, es un poco quizás más los compañeros, los
especie a visitar, no? El ordenador grande… y amigos, los que me han ido formando.”
claro...participábamos poco …”
Uno de los aspectos a destacar es su parte autodidacta. Tanto en
El ocio tecnológico produce más entusiasmo en algún caso: las mujeres que se habían formado en el ambiente universitario,
“Con videoconsolas sí!!!! Con ordenadores no tocaba mucho, como aquellas que usan la tecnología en sus profesiones, pero sin
no!!! la verdad….” formación universitaria en informática. Algunas sienten
curiosidad y experimentan con ella, ya sea por placer o por querer
Sin embargo, aunque todas en algún momento han jugado a saber.
videojuegos, y utilizaron consolas, algunas de ellas lo hicieron
muy poco, y otras lo hacían con amigos o con miembros de la “Una parte en el colegio, y yo creo que un 80% experimento yo
familia (hermanos/as y/o primos/as). sola.”
“...si sí...pues bueno a lo mejor con 16, o por ahí en casa de “…me pongo hasta que lo consigo. Me da igual no saber….yo,
algunos amigos míos que si tenían videoconsolas de éstas de las busco, busco...hasta que lo hago.”
antiguas, pues eran de los primeros videojuegos que salieron que “Pues claro, ahí ya como que con Arch Linux tienes que aprender
eran como especies de comecocos, y también había de coches.” estas cosas no? Y... y realmente me enamoré de este sistema y...
“…con mi prima sí nos enganchamos a las máquinas estas no sé, en dos semanas con Arch Linux tuve que aprender mucho
grandes de bar, en el Tetris, eso sí que recuerdo ...” más que en dos años con Ubuntu, no? que era como una cosa que
te exige, o sea…”
Aunque en la actualidad refieren jugar poco, la experiencia del
juego suele contarse en pasado, algunas han seguido jugando, sin En cuanto a sus sentimientos de competencia o autoeficacia, a
por ello considerarse muy hábiles. veces pueden resultar ambivalentes y, en ocasiones, autodefinen
su relación con la tecnología por comparación a sus compañeros
“No sé jugar a ningún juego, y aunque le ponga voluntad…, pero masculinos y no se ven a sí mismas como genios, más bien como
se me da fatal, pero me gusta mucho, puedo pasarme horas…” alguien que ha tenido que esforzarse.
La familia juega un papel importante, en algunos casos, a la hora “…bueno yo siempre me veo que, que sé muy poco. Pero ... pero
de proporcionar los medios materiales (o apuntar a la academia, luego ves que hay gente que sabe menos que tú y es el ... tiene un
como hemos visto) o constituirse en referentes que no siempre son nombre, eh, eso! y nunca me acuerdo ... como el ... como un
los más esperables: complejo de ... de que tú no sabes, no? pero es que es imposible
“A veces he culpado a mi abuela. Sí, porque ella ... aunque saber mucho y ... yo no sé si sé mucho o poco, pero ahora sí tengo
evidentemente la tecnología le queda super lejos, pero siempre que reconocer que hace muchos años que las toco, y eso quieras
digo que es algo que le atrae mucho, no? Yo recuerdo que se que no te da ... el experiencia es ... no tiene precio, no?”
compraron un vídeo y que buscaba el manual del vídeo para “No sé, cuando están montando un ordenador. Ellos tampoco
programarlo, era mi abuela. Y entonces pues, a veces haciendo la saben cómo hacerlo realmente! […] Claro, saben un poco más!
broma no? digo, "ehh !! vi esto no ?!", no tuve un referente eh, pero... al principio no lo sabían, no? O sea, no lo sabían. No lo
super masculino o lo que sea, sino que veía mi abuela pues sabían! Te metes ahí, desmontas un cable, montas otro cable, y
mirando manuales de instrucciones de cosas que no eran "funciona, mira bien", "no funciona, pues mejor, así aprendo".
ordenadores …”
Este contacto tardío con la informática a nivel experto,
“... y mis padres también lo encontrarían interesante pues porque desapegado a menudo, y ambivalente respecto a sus capacidades
... para estudiar o lo que sea, no? y me compraron el PC este y la, “naturales”, en el que se destacan los referentes y apoyos
y una impresora enorme y ... y ya está…” externos, supone un punto de partida proclive a pensar más en
“Y hubo, sí, cuando era adolescente hubo un ordenador en la casa términos de dificultades y frustraciones, y en consecuencia
que era de mi mamá y sí que me acuerdo que para arrancar un conduce a caracterizar la tecnología como compleja, de difícil
juego tenía que escribir un comando, no?” acceso y necesitada de gran dedicación.

Las mujeres con formación universitaria en informática, casi “He tenido la sensación de ir ahí a toda prisa en toda la carrera en
todas, con alguna excepción, comienzan desde cero y se plan "venga, tochos tochos, tochos, profes que te dan materia
encuentran con dificultades y a veces frustraciones, especialmente materia, materia" y que de partida no me parecen conceptos

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 315 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
fáciles de entender. O sea que son cosas que si eres un crack sí, la transformación social a la que se puede llegar mediante la
pero para personas normales .. que es super abstracto eso. Y apropiación de las tecnologías, tanto para mejorar su entorno
bueno, ... pues eso.” como para que sean útiles.
“Cuando empecé la carrera fatal. [...] Bueno, estudié, y saqué un De cara a futuras investigaciones, es necesario explorar como a
cero, me decían que no sabía nada, no? Y me fue ... quiero decir, pesar de encontrar múltiples y significativas motivaciones las
fue muy, muy duro no? para mí…” mujeres no se suelen pensar como genios informáticos y se
encuentran con dificultades y frustraciones generadas por su
Esta caracterización de dureza de lo tecnológico se asocia con lo entorno, que en muchos casos les generan la sensación de no ser
masculino, una asociación cultural de la que son plenamente suficientemente competentes. Como destacan recientemente
conscientes las entrevistadas: Leslie et al. [47], las disciplinas donde existe la imagen de que se
“Tengo la imagen de ella metida en un ordenador bastante requiere ser un genio para triunfar en ellas son las que más brecha
presente. …Así que también, bueno, o sea, mi madre estudió de género presentan.
Matemáticas, que creo que es bastante significativo, no? Que ella Encontramos en nuestra investigación que a las mujeres se les ha
como que hizo el primer camino dentro de algo que se supone que negado el acceso práctico, pero también simbólico y placentero a
es un poco más masculino, no? y... cosas lógicas, o sea... los ámbitos tecnológicos, de forma que para muchas mujeres no
Matemática y Informática van bastante juntas, no?” es pensable encontrar ahí «su lugar». Si acceden, el camino que
Sin embargo, en sus explicaciones encontramos también una tienen que recorrer es otro y es más largo. Si intentan adoptar la
visión de la tecnología como un mecanismo de transformación forma del camino masculino pero sin haber seguido la misma
social, que sirve tanto para mejorar su entorno inmediato como trayectoria, se encuentran más frustración. Pero existen otras rutas
para generar algo útil. Hay un aspecto que destaca en particular, el de inclusión en el mundo tecnológico que no son las mismas que
de la transformación social a la que se puede llegar mediante las las de los chicos ni su reverso, que nos muestran otras prácticas,
tecnologías, lo que se puede hacer con ellas, tanto para mejorar su otras relaciones con la tecnología, sin embargo son rutas invisibles
entorno y que sean útiles, como desde un punto de vista más dado que no son la ruta normalizada, la marcada como estándar
activista. por el modelo masculino. Todo esto suele llevar a una visión de
las trayectorias de las chicas como carentes, sin embargo esto no
“… la única idea que sé es que quiero aplicar la informática a es así porque no hay carencia sino aportación: la resignificación
hacer algo bueno, porque la informática está como muy enfocada de la tecnología, pensándola de una forma más completa y
hacia el capricho, hacia el consumo. Quiero que la informática compleja, resaltando lo que pueden hacer para ayudar a la gentes,
ayude a la gente […] yo quiero ... aplicar la informática a la para ser útiles, para cuidar y buscar la transformación social.
medicina, a las operaciones de corazón, por ejemplo, de física Generalizar esta resignificación podría cambiar la situación actual
cuántica ... O a algo que ayude, no sé, quiero hacer algo que tenga de fractura digital de género creciente.
relevancia, que ayude a la gente.”
“... Veo una manera de poder usar cosas que yo he aprendido de
5. AGRADECIMIENTOS
otra manera, aprender cosas nuevas también, pero poderlas utilizar La investigación que se presenta ha sido posible gracias a la
para cosas que sirvan ... mmmm el rollo social, no? o sea con financiación del proyecto Trayectorias tecnológicas desde una
movimientos sociales…” perspectiva de género: factores implicados en el alejamiento de las
mujeres de las especializaciones tecnológicas Ministerio de
Sus relatos nos ofrecen también la posibilidad de resignificar las Economia y Competitividad de España (FEM2011-23086)
tecnologías, pensándolas como útiles, potenciadoras del cuidado y
garantes de transformación social. 6. REFERENCIAS
[1] Clarke, C. (2010). Computer Science. The Incredible
4. CONCLUSIONES Shrinking Woman. En J. T. Misa (Ed.), Gender Codes. Why
Los relatos de vida tecnológica de estas quince mujeres, nos Women Are Leaving Computing. Hoboken, New Jersey:
permiten comprender la relación entre su experiencia y los Wiley-IEEE Computer Society Press.
sentidos que lo tecnológico puede adquirir. Es destacable que en [2] Cohoon, J. M., & Aspray, W. (2006). A critical review of the
las mujeres con conocimientos expertos apareció una research on women’s participation in postsecondary
comprensión de la tecnología que se aleja del modelo masculino computing education. In J. M. Cohoon, & W. Aspray (Eds.),
dominante y que puede facilitar a otras mujeres acercarse a ella. Women and information technology: Research on
Sus primeros contactos con la tecnología son diversos, pero tienen underrepresentation (pp. 137-182). Cambridge, MA: MIT
en común no mostrar un gran apego a esta. En algunos casos se Press.
producen en la preadolescencia o la adolescencia, en la escuela, el [3] Ilomäki, L. (2011). Does Gender Have a Role in ICT Among
instituto o a través de alguna actividad extraescolar. Han jugado a Finnish Teachers and Students?. Scandinavian Journal of
videojuegos, y utilizaron consolas, algunas de ellas muy poco sin Educational Research, 55(3), 325-340.
embargo, y otras lo hacían con amigos o con miembros de la
familia (hermanos/as y/o primos/as). Destaca su formación [4] Jenson, J. & Brushwood Rose, C. (2003). Women@Work:
autodidacta en no pocos casos. Las mujeres con formación Listening to gendered relations of power in teachers’ talk
universitaria en informática, con alguna excepción, comienzan about new technologies. Gender and Education, 15, 169–
desde cero y se encuentran con dificultades y a veces 181.
frustraciones, especialmente durante la carrera, que llevan a [5] Meelissen, M. R. M., & Drent, M. (2008). Gender
sentimientos de competencia o autoeficacia que a veces pueden differences in computer attitudes: Does the school matter?.
resultar ambivalentes. Sin embargo, una motivación importante es

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 316 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
Computers in Human Behaviour, 24, 969–985. studies. Journal of Vocational Behaviour, 80(2), 486-499.
DOI=10.1016/j.chb.2007.03.001. DOI=10.1016/j.jvb.2011.08.005.
[6] Sáinz, M., Pálmen, R., & García-Cuesta, S. (2011). Parental [21] Cakir, O. (2012). Students’ self confidence and attitude
and Secondary School Teachers’ Perceptions of ICT regarding computer: An International analysis based on
Professionals, Gender differences and their role in the choice computer availability and gender factor. Procedia - Social
of studies. Sex Roles, 66(3-4), 235-249. and Behavioural Sciences, 47, 1017–1022.
[7] Vekiri, I. (2010). Boys’ and girls’ ICT beliefs: Do teachers DOI=10.1016/j.sbspro.2012.06.772.
matter?. Computers & Education, 55(1), 16-23. [22] Koch, S. C., Müller, S. M., & Sieverding, M. (2008). Women
DOI=10.1016/j.compedu.2009.11.013. and computers. Effects of stereotype threat on attribution of
[8] Kent, N., & Facer, K. (2004). Different worlds? A failure. Computers & Education, 51(4), 1795-1803.
comparison of young people’s home and school ICT use. DOI=10.1016/j.compedu.2008.05.007.
Journal of Computer Assisted Learning, 20(6), 440-455. [23] Anderson, N., Lankshear, C., Timmsa, C., & Courtney, L.
[9] Simsek, N., & Erdogdu, F. (2012). Current trends in (2008). Because It’s Boring, Irrelevant and I Don’t Like
computer use of elementary school students: an international Computers: Why High School Girls Avoid Professionally-
comparison. Procedia - Social and Behavioral Sciences 47, Oriented ICT Subjects. Computers & Education, 50(4),
1058–1063. 1304-1318.

[10] Vekiri, I., & Chronaki, A. (2008). Gender issues in [24] McKenney, S., & Voogt, J. (2010). Technology and young
technology use: Perceived social support, computer self- children: How 4–7 year olds perceive their own use of
efficacy and value beliefs, and computer use beyond school. computers. Computers in Human Behavior, 26(4), 656-664.
Computers & Education, 51(3), 1392-1404. [25] Cohoon J.M. (2006). Just get over it or just get on with it:
[11] Varma, R. (2010a). Computing self-efficacy among women Retaining women in undergraduate computingm. In J. M.
in India. Journal of Women and Minorities in Science and Cohoon & W. Aspray (Eds.), Women and information
Engineering, 16(3), 257–274. technology: Research on underrepresentation (pp 205–237).
DOI=10.1615/JWomenMinorScienEng.v16.i3.40. Cambridge, MA: MIT Press.

[12] Cox, A. (2009). Visual representations of gender and [26] Engelman, S., McKlin, T., & Guzdial, M. (2011). Conditions
computing in consumer and professional magazines. New that encourage participation in computer science.
Technology, Work and Employment, 24(1), 89–106. GaComputes! Technical Report. Atlanta: The Findings
DOI=10.1111/j.1468-005X.2008.00220.x. Group, LLC.

[13] Gannon, S. (2007). Laptops and Lipsticks: Feminizing [27] Fisher, A., & Margolis, J. (2003). Unlocking the clubhouse:
Technology. Learning, Media and Technology, 32 (1), 53-67. The Carnegie Mellon experience. ACM SIGCSE Bulletin,
DOI=10.1080/17439880601141260. 34(2), 79-83.

[14] Johnson, N. F., Rowan, L., & Lynch, J. (2006). [28] Margolis, J., Fisher, A., & Miller, F. (2000). The anatomy of
Constructions of gender in computer magazine interest: Women in undergraduate computer science.
advertisements: Confronting the literature. Studies in Media Women's Studies Quarterly, 28(1/2), 104-127.
& Information Literacy Education, 6, 1–9. Retrieved from: [29] Lopez, A. M., Jr., & Schulte, L. J. (2002). African American
http://ro.uow.edu.au/cgi/viewcontent.cgi?article=1019&conte women in the computingsciences: A group to be studied.
xt=edupapers. doi:10.3138/sim.6.1.003. SIGCSE Bulletin, 34(1), 87-90.
[15] Misa, T. J. (2010). Gender codes. In T. Misa (Ed.), Gender [30] Gil-Juárez, A., Feliu, J., Vall-llovera, M. & Biglia, B. (2014).
codes: Why women are leaving computing (pp. 1-23). Trayectorias de vida tecnológica y género: Factores
Hoboken, NJ: IEEE-CS Press/Wiley. psicosociales implicados en el acceso a las titulaciones de
[16] Mercier, E. M., Barron, B., & O'Connor, K. M. (2006). ingeniería informática. Informe técnico. Intituto de la mujer
Images of self and others as computer users: The role of y para la ingualdad de oportunidades. Retrieved from:
gender and experience. Journal of computer assisted http://www.inmujer.gob.es/areasTematicas/estudios/estudiosl
learning, 22(5), 335-348. inea2014/docs/Trayectorias_vida_tecnologica_genero.pdf.

[17] Gil, A., Feliu, J., & Vitores, A. (2010). Género y TIC: en [31] Tillberg, H. K., & Cohoon, J. M. (2005). Attaching women
torno a la brecha digital de género. Athenea Digital. Revista to the CS Major. Frontiers: A Journal of Women Studies,
de Pensamiento e Investigación Social,12(3): 3-9. 26(1), 126-140. DOI=10.1353/fro.2005.0016.

[18] Meelissen, M. (2008). Computer attitudes and competencies [32] Serapiglia, C. P., Lenox, T. L. (2010). Factors affecting
among primary and secondary school students. In J. Voogt & women’s decisions to pursue an IS Degree: A Case Study.
G. Knezek (Eds.), International handbook of information Information Systems Education Journal, 8 (12). Retrieved
technology in primary and secondary education (pp. 381– from: http://isedj.org/8/12.
395). New York: Springer. [33] Varma, R. (2010). Why so few women enroll in computing?
[19] Sieverding, M., & Koch, S. C. (2009). (Self)Evaluation of Gender and ethnic differences in students’ perception.
computer competence: How gender matters. Computers & Computer. Science Education, 20(4), 301–316.
Education, 52(3), 696-701. [34] Lenox, T., Jesse, G., & Woratschek, C. R. (2012). Factors
[20] Sáinz, M., & Eccles, J. (2012). Self-concept of computer and influencing students' decisions to major in a computer-related
math ability: Gender implications across time and within ICT

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 317 |
 Dos sentidos de lo tecnológico en relatos de vida de mujeres tecnólogas
A. Gil-Juárez | E. Conesa | J. Feliu | M. Vall-llovera
discipline. Information Systems Education Journal, 10(6). attitudes toward computers. Computers & Education, 37(3),
63-71. 363-375.
[35] Varma, R. (2009). Exposure, training, and environment: [42] Varma, R. (2002). Women in information technology: A case
Women’s participation in computing education in the United study of undergraduate students in a minority-serving
States and India. Journal of Women and Minorities in institution. Bulletin of Science, Technology & Society, 22(4),
Science and Engineering, 15, 205–222. 274-282.
[36] Kahle, J., & Schmidt, G. (2004). Reasons women pursue a [43] Sader, J. L. (2007). Beyond the first “click:” women
computer science career: Perspectives from women from a graduate students in computer science. Disseration Thesis.
mid-sized institution. Journal of Computer Science in Graduate College of Bowling Green State University.
Colleges, 19(4), 78–89. Retrieved from ProQuest Dissertations and Theses database.
[37] Beyer, S., DeKeuster, M., Walter, K., Colar, M., & Holcomb, [44] Hache, A., Cruells, E., y Vergés, N. (2013). I code, you code,
C. (2005). Changes in CS students’ attitudes towards CS she hacks - Women hackers and technopolitical insights from
over time: an examination of gender differences. ACM LelaCoders. Donestech. Retrieved from: www.donestech.net.
SIGCSE Bulletin,37, 392–396. [45] Vergés, N., Cruells, E., & Hache, A. (2009). Retos y
[38] Creamer, E. G., Lee, S., & Meszaros, P. S. (2007). Predicting potencialidades para las mujeres en la participación del
women’s interest in and choice of a career in information desarrollo de la sociedad de la información. Feminismo/s, 14,
technology: A statistical model. In C. J. Burger, E. G. 163-182. Retrieved from:
Creamer, & P. S. Meszaros (Eds.), Reconfiguring the http://www.donestech.net/files/feminismos14donestech.docV
firewall: Recruiting women to information technology across on.
cultures and continents (pp. 15-38). Wellesley, MA: AK and [46] Vergés, N. (2012). De la exclusión a la autoinclusión de las
Publishing House Peters, Ltd. mujeres en las TIC. Motivaciones, posibilitadores y
[39] Irani L (2004) Understanding gender and confidence in CS mecanismos de auto-inclusión. Athenea Digital. Revista de
course culture. SIGCSE Bulletin, 36(1), 195–199. Pensamiento e Investigación Social, 12(3), 129-150.
[40] Scragg, G. & Smith, J. (1998). A Study of Barriers to [47] Leslie, S. J., Cimpian, A., Meyer, M., & Freeland, E. (2015).
Women in Undergraduate Computer Science. SIGCSE Expectations of brilliance underlie gender distributions
Bulletin, 30(1), 82-86. across academic disciplines. Science, 347(6219), 262-265.
[41] Shashaani, L., & Khalili, A. (2001). Gender and computers:
Similarities and differences in Iranian college students'
[48]

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 318 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea

Acercando las mujeres a la ingeniería: iniciativas y

estrategias que favorecen su inclusión
Patricia Paderewski Maribel García Arenas Rosa Gil Iranzo
Universidad de Granada Universidad de Granada Universidad de Lleida
[email protected] [email protected] [email protected]

Carina González González Eva M. Ortigosa Natalia Padilla-Zea

Universidad de La Laguna Universidad de Granada Universidad de Granada
[email protected] [email protected] [email protected]

RESUMEN Para ilustrar este descenso basta ver algunas cifras a nivel estatal
En este artículo se estudian las iniciativas que se han llevado a así como los casos particulares de nuestras universidades, para
cabo tanto en España como fuera de ella para resolver el problema entender la preocupación que provoca la poca presencia femenina
del bajo número de mujeres que realizan estudios técnicos o en las titulaciones técnicas.
trabajan en empresas dentro del área de las tecnologías. Una vez En el artículo de [2] las autoras exponen que, en 2º de Bachillerato
revisadas estas iniciativas, y constatado que actualmente no se ha del curso 2009-2010, el 53,64% hombres y 46,36% de mujeres
mejorado este número, se proponen otras actuaciones o iniciativas optaron por la modalidad de Ciencia y Tecnología y sin embargo
que podrían mejorar la situación, y por tanto, reportar todos los los datos de los y las estudiantes que se matricularon en el primer
beneficios que esto conllevaría, entre otras se propone la co- curso de grados universitarios en el curso 2010-2011 reflejan que
creación empática que se llevaría a cabo desde la educación en solo 23% de las mujeres eligieron la rama de Ingeniería y
primaria. Arquitectura, mientras que el resto de las mujeres se decantaron
por otros estudios pertenecientes principalmente a las Ciencias de
Palabras clave la Salud.
Género, Ingeniería, Estrategia.
A modo de ejemplo ilustrativo y real, analizaremos los datos de
1. INTRODUCCION nuestras Universidades en los últimos cinco años, tanto en las
Actualmente se puede apreciar un número excesivamente bajo de titulaciones relacionadas con la Informática, (Ingeniería,
mujeres en las carreras técnicas, como son, por ejemplo, la Ingeniería Técnicas y Grados), como en las titulaciones
Ingeniería Informática o la Ingeniería de Telecomunicación en las relacionadas con la Telecomunicación. La primera de ellas, la
que se centrará este artículo principalmente. Este problema está Universidad de Granada, aporta los siguientes datos:
presente en nuestra sociedad y preocupa, tanto que existen
numerosos artículos tratando y discutiendo este tema, no sólo a • El porcentaje de alumnas matriculadas en el Grado en
nivel nacional, sino a nivel mundial. Ingeniería Informática oscila entre 8.84% y 13.57% y en el
Grado en Ingeniería de Tecnologías de Telecomunicación
Nosotras, las autoras de este artículo, impartimos clase en estas oscila entre 14.53% (curso 2014-2015) y 22.95%.
ingenierías. Casi todas estamos tituladas en estas carreras técnicas
y consideramos que es importante intentar abordar este problema • La tasa de abandono en el curso 2014-2015 fue 12.85% de
y realizar una revisión de las distintas iniciativas que se están mujeres y 87.15% de hombres en el Grado en Ingeniería
llevando a cabo, junto con las que nosotras sugerimos que se Informática. En el Grado en Ingeniería de Tecnologías de
hagan, tanto desde las Universidades como desde las empresas Telecomunicación la tasa de abandono fue del 32% de
que podríamos denominar TIC. Esta ausencia de estudiantes mujeres y 68% de hombres. Son números sobre el total de los
femeninas es tan importante, que los enfoques, sin una visión alumnos, no solo mirando los alumnos de primer curso, sino
femenina, son totalmente diferentes, lo que hace perder diversidad teniendo en cuenta el Grado completo que incluye cuatro
y puntos de vista complementarios para encontrar la solución a un cursos en esta universidad.
determinado problema. Por otra parte, también estas estudiantes, • El porcentaje total de mujeres matriculadas, a día de hoy, en
aportarán mucho en sus futuras empresas al finalizar sus estudios. el curso 2014-2015 en el Grado en Ingeniería Informática es
En el último informe [1] que presenta el Ministerio de Educación 9.63% y en el Grado en Ingeniería de Tecnologías de
de forma anual, y donde se analizan los datos básicos del Sistema Telecomunicación es 16.81%.
Universitario Español, se comenta la bajada de estudiantes en el • El porcentaje de egresadas en el curso 2014/2015 en el Grado
ámbito de ingeniería y arquitectura, pese a la demanda existente en Ingeniería Informática fue de 9.68% y en el Grado en
con una tasa de variación anual del -5,3% y decenal del -23,3%, lo Ingeniería de Tecnologías de Telecomunicación fue de
que significa que esta rama ha perdido en los últimos 10 años uno 6.67%. Estas cifras realmente no son muy significativas
de cada cuatro estudiantes femeninas. porque este curso fue el primero en el que se impartió el
último curso de ambas titulaciones, y tenía algún que otro

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 319 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
problema adicional que impidió que muchos alumnos Para intentar paliar esta deficiencia, hemos buscado iniciativas o
terminaran sus estudios, en concreto, solo 31 en el Grado en actuaciones relacionadas con este problema intentando ver si el
Ingeniería Informática y 30 en el Grado en Ingeniería de bajo porcentaje de matrículas femeninas en estas titulaciones está
Tecnologías de Telecomunicación. sólo presente en las tres universidades mencionadas, o por el
contrario, se trata de un problema generalizado. En concreto, la
Si comparamos estos datos con los de la Universidad de Lleida, sección 2 revisa las iniciativas nacionales e internacionales que se
utilizando el mismo periodo temporal, los de la Universidad de están llevando a cabo para paliar este problema, y la sección 3
Lleida son incluso más alarmantes: presenta un conjunto de propuestas que se deberían llevar a cabo
• El porcentaje de alumnas matriculadas en el Grado en (o seguir llevándose a cabo) con el fin de llegar al objetivo de
Ingeniería Informática oscila entre 3.2% y 14% (curso 2012- aumentar el número de mujeres que se adentran en el mundo TIC,
2013). examinando el por qué algunas no han funcionado. La sección 4
presenta nuestra opinión a modo de conclusiones, incluyendo una
• El porcentaje de tituladas en las carreras universitarias reflexión personal de las autoras de este artículo sobre la
relacionadas con la informática (I.T. Sistemas, I.T. Gestión, problemática presentada, intentando aportar nuevas soluciones y
Ingeniería Informática y Grado en Informática) fue, en el puntos de vista alternativos.
curso 2013-2014, del 14% frente al de titulados que fue del
86%. 2. ESTADO DEL ARTE
En este punto queremos presentar, para después reflexionar sobre
En el caso de la Universidad de La Laguna, se han incluído no
ello, las iniciativas que se están llevando a cabo tanto en España
sólo los últimos cinco años, sino los 10 últimos años de las
como en otros países para intentar aumentar el número de mujeres
titulaciones relacionadas con Informática para que se aprecie más
que se dedican a las TIC (ingenierías), tanto en el ámbito
la evolución de los números. En este caso, el porcentaje de
académico como laboral.
mujeres es aproximadamente el 17% de la matrícula (Figura 1).
Asimismo, vemos una ligera disminución cuando la titulación En este sentido, hay iniciativas en centros docentes tanto
cambio de nombre de Licenciatura a Ingeniería (desde el año nacionales, como en la Universidad de Lleida, la Universidad de
1998). Sin embargo, el porcentaje de egresadas de las carreras de Granada y la Universidad de La Laguna, como internacionales
informática en los últimos 10 años es superior, en promedio, al como son el centro Rochester Institute of Technology o la
17% (Figura 2). Universidad de Michigan. Además, existen otro tipo de acciones,
a otros niveles y con otros puntos de vista que se la iniciativa se
están expandiendo por todo el entramado TIC, implicando
también a grandes multinacionales como IBM [3], Google [4] o
CISCO [5], Administraciones Públicas o Medios de
Comunicación.

2.1 Iniciativas nacionales

En España se están llevando a cabo diversas iniciativas que hacen
pensar que el problema de que las carreras técnicas estén faltas de
mujeres es generalizado en todo el territorio nacional.
Concretamente hay continuas referencias al problema en
Figura 1. Datos de la evolución de la matrícula por sexo de los
diferentes foros y medios de comunicación, ya sean de más o
últimos 10 años en la Universidad de La Laguna
menos tirada. Por ejemplo, si se escribe en el buscador más
utilizado el texto “carreras técnica mujeres” aparecen 340.000
resultados, dato que da una idea de que el problema está siendo
abordado en diferentes ámbitos, niveles y canales.
Una de las entradas más antiguas que aparecen en la red, en un
ámbito más o menos formal es el informe de Salud y Género del
ministerio de Sanidad en los años 2007-2008. En este informe se
comenta que es significativo el incremento en las especialidades
Científico-Técnicas, aunque éstas solamente son perceptibles en
los hombres. Además, son menos mujeres las que eligen esta rama
Figura 2. Datos de la evolución de egresados/as por sexo de los en 2007 que en 2001. Es decir, ya en un estudio del 2007 con
últimos 10 años en la Universidad de La Laguna datos recopilados desde años atrás, se observa que el número de
mujeres dentro de las carreras técnicas desciende paulatinamente.
Estos datos demuestran que la presencia actual de las mujeres Iniciativas dentro del campo de la investigación como la de [2] se
dentro de las ingenierías, al menos dentro de la Universidad de basan en realizar estudios sobre las causas que hay tras esta
Granada, de la Universidad de Lleida, y de la Universidad de La problemática del bajo número de alumnas en las carreras técnicas.
Laguna es cada vez menor, lo que hace que la diversidad y el Sus autoras realizan una buena revisión de los estudios
enriquecimiento que aporta una visión desde diferentes puntos de relacionados con las motivaciones sobre la baja presencia de las
vista se esté perdiendo, ya no sólo dentro del ámbito docente, sino mujeres en las empresas TIC y en los estudios de ingeniería
también en el ámbito laboral al que se irán incorporando todas las relacionados con las TIC.
personas egresadas de estas titulaciones.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 320 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
Concretamente, presentan un estudio llevado a cabo en la Escuela seleccionan al principio del curso. La controversia de esta
Politécnica Superior (EPS) de la Universidad Carlos III de Madrid iniciativa viene por el hecho de que la iniciativa va orientada sólo
(UC3M) donde realizaron una encuesta preguntando sobre la a chicas en edad pre-universitaria, por lo que ha recibido diversas
motivación de las estudiantes para seleccionar una titulación críticas de algunos sectores indicando que, al ser un curso
técnica, y se destaca que sus motivaciones principales son: tener gratuito, los chicos también deberían poder asistir. Sin embargo, si
interés por las matemáticas y la existencia de posibilidades se admitieran chicos en dicha propuesta, sería muy difícil alcanzar
laborales. Son muy interesantes también algunas respuestas como: el objetivo de la misma. Esta iniciativa ha sido financiada por
el hecho de que han tenido referencias cercanas, tanto masculinas diversas entidades y el equipo organizador busca financiación
como femeninas, en su ámbito familiar. También indican que a la para que las chicas puedan hacerlo de forma gratuita, por lo que la
mujer le interesa dedicarse profesionalmente a dominios de acogida ha sido bastante numerosa en las dos ediciones. Algunas
aplicación de temas sociales. de las entidades que financian la propuesta son, Google, la
Delegación TIC de la Universidad de Granada, la Oficina de
Otros estudios como el de [6] analizan los resultados y decisiones Software Libre de la Universidad de Granada y la propia Escuela
que toma el estudiantado del Bachillerato Científico-Tecnológico Técnica Superior de Ingenierías Informática y de
y Biosanitario que realiza la prueba de acceso universitaria (PAU) Telecomunicación donde se imparte.
en la Universidad de Oviedo durante un largo intervalo de tiempo
(2006-2010). De las conclusiones obtenidas, destacamos que las Women TechMaker Granada [9] es un evento que se organizó en
mujeres eligen materias científicas contextualizadas en la vida abril de 2015 por el Google Developers Group (GDG) de Granada
cotidiana y orientadas a cubrir las necesidades de las personas. y la Escuela de Ingenierías Informática y de Telecomunicación de
Las mujeres eligen como opción preferente de estudios la rama de la Universidad de Granada. En ella la idea fue explicar lo que
Ciencias Sociales y Jurídicas y se confirma la fuerte significa ser ingeniero o ingeniera, es decir, que la función de un
esquematización por género en la elección de los estudios de ingeniero/a tiene mucho de creatividad (ideando soluciones a
Ingeniería donde los hombres son una amplia mayoría. Sin problemas reales de las personas), de trabajo colaborativo, de
embargo, exponen que no se puede hablar de rendimiento innovación y de sociabilidad. En este evento se invitaron a
diferencial significativo entre hombres y mujeres con relación a personas relevantes tanto del ámbito académico como profesional,
las materias científicas y tecnológicas. todas mujeres, con el fin de dar mayor visibilidad a las mujeres
profesionales en el mundo TIC y hacerlas así referentes para que
Las iniciativas de las universidades españolas son numerosas y las jóvenes que asistieron al evento puedan animarse a entrar en
diversas. A continuación se van a detallar algunas, aunque se
este mundo.
quiere dejar claro que no son, para nada, una recopilación de todas
las existentes, sino más bien una pincelada de lo que está vigente En el artículo [10] se describe una experiencia, llamada Girl’s
actualmente. Day, que se está llevando a cabo anualmente en la Universidad de
Zaragoza desde 2008. La idea es llevar a chicas que estudian
Per què no puc fer-ho? [7] es una jornada destinada a presentar al secundaria a la Universidad y dedicarles un día donde se acercan
alumnado de cuarto de ESO de la demarcación de Lleida los tanto al mundo empresarial (o de negocio) como al mundo de la
estudios de la Universidad de Lleida sin estereotipos de género, investigación. En cinco años, han aumentado la visibilidad que
con el objetivo de fomentar, en el seno del alumnado masculino y tienen las mujeres sobre la ingeniería, las estudiantes conocen en
femenino, la elección de titulaciones en las cuales cada género lo que consiste una ingeniería y se ha establecido una red de
está menos representado. El programa está coordinado por el
ingenieras en la región. La actividad central consiste en mostrar
Centre Dolors Piera d’Igualtat d’Oportunitats i Promoció de les ejemplos de productos de ingeniería desarrollados por equipos que
Dones que nace fruto del esfuerzo del SIED (Seminari incluyen mujeres. Tal como las autoras explican en el artículo un
Interdisciplinari d’Estudis de la Dona) para fomentar la igualdad gran problema es que los jóvenes no saben lo que es ingeniería ni
de oportunidades entre hombres y mujeres. La normativa de en qué consiste el trabajo de un ingeniero. Por tanto hay que
funcionamiento del centro se aprobó por Consejo de Gobierno en acercarlos a este mundo desde distintos puntos de vista. El Girl’s
dicha universidad el 7 de marzo de 2006. En esta jornada, chicos y Day fue una iniciativa que nació en Alemania pero actualmente se
chicas realizan diversos talleres para que visualicen los trabajos está celebrando en más de 10 países.
que se llevan a cabo en las diferentes titulaciones. Desde su puesta
en funcionamiento las chicas realizan talleres en la Escuela Tech & Ladies [11] es la iniciativa de las mujeres del grupo
Politécnica Superior y los chicos en Psicopedagogía. Suele Google en España en colaboración con expertas mujeres
agradar la experiencia a nivel general, quedando cortos los talleres tecnólogas. El objetivo del proyecto es dar visibilidad a las
en cuanto a su duración, pues sólo se dispone de una mañana. mujeres que crean y trabajan en tecnología. Para ello, en este
proyecto se busca reconocer los problemas de las mujeres a la
Campus Tecnológico UGR para Chicas [8] es la iniciativa de la hora de conectar con la tecnología y buscar solución a sus
Universidad de Granada en 2014 y 2015. Esta propuesta es desafíos, brindar formación especializada, divulgar patrones de
pionera en el ámbito nacional y ha sido objeto de controversia en mujeres en la tecnología, tener referencias femeninas y conseguir
muchos sentidos. El objetivo principal es promover y difundir qué
que chicas jóvenes se interesen por estudiar una carrera técnica.
se hace en carreras técnicas como la Ingeniería Informática o la Como primera acción del proyecto, se ha creado un mapa
Ingeniería de Tecnologías de Telecomunicación a chicas en edad tecnológico, este mapa va a servir para que todas las chicas que
pre-universitaria. Para ello se organiza un curso de dos semanas tengan un perfil técnico puedan darse visibilidad y hacer
de duración totalmente gratuito para las asistentes donde se les networking.
muestra qué se hace en realidad en dichas carreras. Para ello se
imparten clases totalmente prácticas donde las alumnas pueden ir Se realizan también jornadas de científicas y tecnólogas para
construyendo por ellas mismas, utilizando tanto software libre promover nuevas vocaciones científicas, tales como como la
como hardware libre, diversos proyectos que ellas mismas realizada por el Instituto Universitario de Estudios de las Mujeres

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 321 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
(IUEM) de la Universidad de La Laguna en mayo de 2015 donde con los conocimientos, habilidades y valores para ayudarlas a ser
se llevaron mujeres tanto de la parte académica (profesoras e capaces de crear futuras oportunidades para la innovación y
investigadoras) como de las empresas para incentivar a las chicas empleo de calidad relacionado con las TIC. Dentro de este ámbito
a estudiar y trabajar en el ámbito de las TIC. En estas jornadas se de proyectos europeos está también el proyecto FESTA (Female
invitaron a estudiantes de Bachillerato, tanto chicos como chicas. Empowerment in Science and Technology Academia) [19], el
proyecto Genis-Lab (The Gender in Science and Technology Lab)
e-igualdad.net es un espacio promovido y financiado por el [20], el proyecto HELENA (Higher Education Leading to
Instituto de la Mujer y para la Igualdad de Oportunidades, que Engineering and Scientific Careers) [21], el proyecto TWIST
cuenta también con la cofinanciación del Fondo Social Europeo. (Torwards Women in Science and Technologies) [22], o el
Este espacio forma parte de los objetivos y líneas de actuación del proyecto WISAT (Women in Global Science and Technologies)
Plan de Acción para la igualdad entre mujeres y hombres en la [23], todos orientados en el mismo sentido.
Sociedad de la información (2009-2011) aprobado en el Consejo
de Ministros el 18 de diciembre de 2009. Como se puede apreciar, no es una situación que se intente paliar
sólo en algunas de nuestras universidades, sino que existen
2.2 Iniciativas internacionales numerosas iniciativas, no todas ellas recogidas en este artículo,
A continuación, se presenta una recopilación de iniciativas que donde la temática es siempre el fomentar las titulaciones técnicas
han surgido fuera de nuestro país y se detallarán algunas entre las mujeres.
características importantes:
2.3 Otras formas de acción
En Rochester Institute of Technology [12] pretenden incentivar la Este apartado revisa iniciativas con enfoques más originales o que
participación de mujeres en carreras técnicas (STEM, Science, difieren de los mencionados en el apartado anterior, pero que
Technology, Engineering, and Mathematics) ya que dan por creemos que son importantes por los resultados que obtuvieron y
sentado que un equipo de trabajo y desarrollo se comporta mucho que, en muchos casos, siguen vigentes:
mejor cuando está formado por hombres y por mujeres. Se apoyan
para esto en estudios que afirman que un equipo con diversidad es Iniciativas pasadas provocaron la aparición de movimientos de
capaz de generar ideas más innovadoras y por lo tanto creativas. asociación entre mujeres relacionadas con la informática. Así
Para ello se fomentan proyectos de investigación en el ámbito surge la llamada “Asociación para mujeres en Computación”. Esta
educativo y en el lugar de trabajo para las científicas e ingenieras. asociación curiosamente, tiene un director hombre que intenta
Además, desarrollan programas de extensión que buscan fomentar fomentar la visibilidad de mujeres dentro de este mundo que
el interés por la ciencia y las matemáticas entre las mujeres y las principalmente está compuesto por hombres. Otros proyectos
niñas en todos los niveles, primaria, secundaria o universidad. relacionados con asociaciones de mujeres que intentan impulsar la
participación de las mujeres en la tecnología son Girls who code
La Universidad de Michigan [13] lidera la propuesta titulada [24] o Agile-girls [25], entre otras muchas asociaciones que
“Women in Science and Engineering” (WISE) que incluye actualmente están trabajando en este sentido. O los ejemplos de la
programas para chicas de primaria, pregraduadas e incluso American Association of University Women (AAUW) que vela
postgraduadas o estudiantes de doctorado y postdoctorado por la integración de las mujeres y publican diversos informes
ofreciéndoles una vía por la que obtener plazas en la universidad u sobre la situación de la mujer como por ejemplo: Why So Few?
oportunidades de empleo. Women in Science, Technology, Engineering, and Mathematics
Existe iniciativas conocidas como: IGNITE (Inspiring Girls Now (2010) [26] o Graduating to a Pay Gap: The Earnings of Women
In Technology Evolution) [14] con una función de mentoría e and Men One Year after College Graduation (2012).
información como la anterior, Engineer your life [15] donde, de La publicación de artículos de opinión en medios influyentes
una forma divertida, intentan explicar por qué es interesante ser como el New York Times, donde Lina Nilsson (innovation
ingeniera y convencer a las chicas que hagan grados de ingeniería. director at the Blum Center for Developing Economies at the
También se realizan reuniones, a nivel mundial, de mujeres University of California, Berkeley) [27] muestra que sólo
tecnólogas como The Grace Hopper Celebration of Women in alrededor del 14 por ciento de los ingenieros/as trabajadores/as
Computing organizada por el Instituto Anita Borg en asociación son mujeres. Ella piensa que se puede incrementar este número de
con la Association of Computing Machinery (ACM). forma simple y cuenta que una experiencia llevada a cabo en la
Universidad de California, Berkeley, donde da clases, sugiere que
La Comisión Europea para la Investigación y la Innovación lidera si el cometido de los trabajos fuera socialmente más
una propuesta para el llamado horizonte 20/20. Esta iniciativa significativos, las mujeres se inscribirán en ellos en masa. Nilsson
intenta fomentar la aparición de mujeres líderes en investigación expone que en su centro, comenzó un programa que logró en sólo
para así fomentar la igualdad en número en mujeres que participan un año académico que un 50 por ciento de la matrícula fuera de
en posiciones influyentes en el campo de la investigación. Esta estudiantes femeninas. El curso (2014) estrenaba un nuevo Ph.D.
iniciativa insta a los estados miembros de la Unión Europea a en ingeniería de desarrollo para estudiantes que iban realizan su
desarrollar la legislación necesaria para fomentar la aparición de trabajo de tesis orientado a crear soluciones para comunidades
estas líderes [16]. desfavorecidas (soluciones para obtener de forma asequible agua
Technovation Challenge [17] es un programa mundial sólo para potable, equipos de diagnóstico médico para enfermedades
chicas de 13 a 17 años que enseña a usar la tecnología con un tropicales desatendidas o creación de procesos de fabricación
propósito, promoviendo el espíritu emprendedor. local en las regiones pobres y remotas). Según Nilsson, las
mujeres se sienten atraídas por proyectos de ingeniería que tratan
ICT-Go-Girls! [18] es un proyecto europeo, cofinanciado por la de mejorar la sociedad. Otras experiencias parecidas están en
Comisión Europea, dentro del programa Comenius LLP. Su MIT, la Universidad de Minnesota, Penn State, la Universidad de
principal objetivo es capacitar a las niñas de la escuela secundaria Santa Clara, Arizona y la Universidad de Michigan donde han

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 322 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
propuesto programas dirigidos a reducir la pobreza y la • Realizar estudios que evidencien cuáles son las principales
desigualdad y han logrado resultados similares. Esto demuestra barreras o problemas por los que las mujeres no eligen una
que la clave para aumentar el número de ingenieras puede no solo carrera técnica.
hacer programas de tutoría o resolver problemas de conciliación,
aunque son importantes. Se pueden reformular los objetivos de la • Programas de mentoría.
investigación en ingeniería y en los planes de estudio haciendo • Mejorar la visibilización de referentes tanto en el mundo
que sean relevantes a las necesidades de la sociedad potenciando académico como empresarial o laboral de mujeres con
el compromiso social, y no enfocándolo sólo como un tema formación técnica.
relacionado con la igualdad de género.
• Realizar cursos, jornadas o eventos para explicar la función
En [28], los autores proponen usar juegos educativos para enseñar de un ingeniero/a.
habilidades básicas de informática de manera contextualizada y de
forma interactiva, generando así entusiasmo e interés por el tema. Todas estas iniciativas llevan años realizándose, a veces de forma
Este enfoque, además, debe adaptarse a un método de enseñanza parcial y, sin embargo, no se han visto todavía resultados
neutral del género. Los resultados fueron alentadores, mejorando satisfactorios ni en España ni en el resto del mundo respecto al
el interés por la informática en estudiantes de ambos sexos. bajo número de mujeres con una formación técnica o tecnológica.
En [29] presentan un estudio donde se refleja, basándose en una Uno de los principales escollos que parece evidenciarse es
serie de experimentos realizados con mujeres y hombres, que la precisamente la percepción sobre el rol de las mujeres en trabajos
inclusión de objetos o elementos estereotipados en el diseño de las técnicos. Ya sea bien por las propias mujeres como por sus
aulas virtuales y reales tiene una gran influencia tanto en la congéneres masculinos. Por eso, muchas de las iniciativas que se
intención de matriculación en estos estudios como en la proponen van en ese sentido. Hay que cambiar esa percepción, y
permanencia en ellos. Los autores concluyen con la idea de que el eso se debe realizar desde que los niños y niñas se escolarizan.
diseño físico de los entornos de aprendizaje, tales como aulas, Como dice Virginia Valian [32] el interés de las mujeres en
laboratorios de computación y departamentos universitarios, matemáticas y ciencias aumentará si tienen un sentimiento de
puede proyectar mensajes basados en la identidad de quién pertenencia y una expectativa de éxito. Es necesario conseguir que
pertenece o no a este mundo y con ello disuadir a algunas las mujeres no se vean excluidas de las áreas tecnológicas, sino
poblaciones de entrar en este campo y creer que no pueden lograr que sientan que son parte de ellas.
éxito en él.
Las iniciativas presentadas a lo largo del artículo se consideran
Por otra parte, las empresas también se preocupan por el bajo indispensables pero se deberían completar con las siguientes, que
número de ingenieras. En el estudio [30], se encuentran algunas si bien, se han evidenciado en algunos estudios, no se están
claves para entender y solucionar, en parte, algunos problemas llevando realmente a cabo:
asociados que encuentran las mujeres que quieren acceder a un
puesto técnico elevado en una empresa o corporación. Por un lado • La información de lo que es ser ingeniero/a y sobre todo,
se reconoce en las grandes corporaciones que poseen hombres y cómo es el trabajo (creativo, social, colaborativo, etc.) y los
mujeres en puestos de gran responsabilidad proporciona más objetivos, los productos y servicios que crea debe llegar antes
versatilidad, flexibilidad y apertura a la innovación. Sin embargo, de que el estudiante decida qué rama va a elegir, es decir,
la percepción de las mujeres es que deberán extender su horario desde la educación primaria, puesto que ya en la educación
laboral o bien tendrán que tener redes de influencia o poder para secundaria deben elegir unas asignaturas que condicionan el
ascender en la empresa. Estas son las barreras que perciben las abanico de posibilidades de formación superior posterior.
mujeres, además de creencias sexistas como que la mujer en un • Se debe introducir alguna asignatura de forma temprana
puesto de responsabilidad es vista como menos competente que (primaria, por ejemplo) en el currículum del alumnado los
sus compañeros masculinos. También se apunta a parámetros conceptos básicos de la informática de forma amena y
como la raza o la etnia. Las mujeres latinas (0%) y de raza negra divertida. Existen lenguajes y métodos de libre acceso que se
(1.6%) apenas aparecen en puestos de responsabilidad. pueden usar y que creemos que crearía una familiaridad de
Incluso las Administraciones a diferentes niveles, con el objetivo todos los estudiantes con la disciplina de ingenierías y
de conseguir más mujeres al frente de la investigación, emprende carreras técnicas evitando la animadversión que muchos
acciones. Por ejemplo, la Comisión Europea desarrolló un informe estudiantes, principalmente femeninos, tienen con la
donde se proponen medidas para promover la igualdad de género informática. Actualmente existen iniciativas positivas como,
en el ámbito que nos ocupa detallando propuestas país a país y por ejemplo, el hecho de que los alumnos en Andalucía
proponiendo buenas prácticas [31]. podrán cursar una asignatura más de tipo tecnológico en
secundaria para el curso 2016-2017 además de la que
3. ALGUNAS PROPUESTAS CONCRETAS actualmente existe en el Bachillerato (noticia que salió
recientemente en la prensa).
PARA FOMENTAR LA PRESENCIA DE
LAS MUJERES EN LAS TIC • El liderazgo del futuro depende de la formación de hoy, por
Hemos visto que existen muchas iniciativas con el fin de fomentar ello, se deben promover programas formales en todos los
la presencia de las mujeres en las ingenierías, casi todas ellas niveles educativos ajustados a las demandas de los trabajos
están orientadas a mujeres jóvenes que están a punto de elegir una que necesitarán en el futuro, teniendo en cuenta que la
carrera universitaria que les llevará a ejercer su profesión dentro tecnología es y será fundamental para este desarrollo. El
de las TIC. Estas iniciativas se basan principalmente en: profesorado actual no está preparado para enfrentarse a este
desafío y necesitará para ello una formación especializada en
didáctica de la informática. Esta formación no existe

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 323 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
actualmente, y debería incorporarse en los planes de información para llevar a cabo el proyecto común. Esta
formación de profesorado, como una carrera especializada. forma de trabajar debería trasladarse a secundaria y a la
universidad. Y en el caso español a todos los niveles. De esta
• Se debe asociar un carisma o proyección social a la función manera, son los alumnos los que demandan los
del ingeniero/a. Esta profesión ayuda a mejorar la sociedad, conocimientos, siendo proactiva la forma en que los
el mundo en el que vivimos, porque aporta soluciones a consumen. Estos equipos deberían estar balanceados en
problemas que tenemos habitualmente. En general, un cuanto a nivel intelectual/género/ habilidades/personalidad
ingeniero mejora el mundo para que sea un poco más en todo lo posible. Así pues, cada uno tiene un rol dentro del
cómodo para todos los que habitamos en él. equipo, fomentando el respeto y reconocimiento de cada uno
• En las aulas, los maestros deben usar la tecnología, los de los integrantes del mismo.
computadores, de una forma innovadora para que los • Co-creación empática: Los contenidos para ser realmente
estudiantes sepan disfrutarla y les guste la tecnología, asimilados deben ser redescubiertos, entender sus relaciones
evitando que surja un sentimiento negativo hacia ella. Esto se y entenderlos dentro de un contexto, por eso se partiría de los
puede conseguir integrando nuevas formas de diseñar PBL (Project Based Learning) desde primaria, donde los
actividades y gamificándolas. También es importante contenidos deben ser estudiados porque son necesarios para
personalizar en todo lo posible el aprendizaje centrándose en alcanzar uno o varios objetivos, de manera que se fomenta la
la persona, y para ello ayudarse de la tecnología para poder motivación por aprender y los contenidos no quedan aislados
gestionar ese gran volumen de trabajo adicional que se en asignaturas sino ligados por un desarrollo y fin común.
genera. Sin embargo, esta propuesta puede ser mejorada si cabe, con
• Se deben evitar los estereotipos, y esto se puede conseguir múltiples actores/roles (co-creación) donde los integrantes
con la educación tanto en las personas mayores deben pasar por diferentes roles (que preferiblemente
(progenitores, maestros/as, tutores, etc.) como para deberían ser antagonistas, como jefe-empleado) para
adolescentes y menores. desarrollar empatía. A ser posible deberían aparecer figuras
que respalden estos proyectos como asesores o consultores.
• Hay que incentivar que las propias mujeres no se vean menos Dichos roles podrían ser llevados a cabo por otros profesores
capaces y se pongan barreras a sí mismas (autoexclusión). Se o profesionales. Los grupos que participan en el PBL
les debe explicar a niños y niñas que ambos géneros deben deberían ser mixtos y fomentar proyectos con vertiente más
ser capaces de conciliar vida laboral y personal con sus técnica/científica y otros más sociales. De forma, que se
parejas. Siendo obligación de ambos componentes de la trabaje con personas que puedan desarrollar sus habilidades
pareja. En este sentido el camino está iniciado, pero hay que en un entorno favorable a ello. Consideramos la empatía un
fomentarlo y seguirlo, qué es lo realmente difícil. punto clave para la implicación social y emocional de los
integrantes del grupo. Así como un potente catalizador para
• Es necesario dar cursos a las niñas para despertarles la
evitar conflictos en el seno del grupo. Por lo tanto, se
curiosidad en la ciencia y la tecnología. Por ejemplo,
fomentaría que tanto mujeres y hombres pudieran dirigirse a
explicarles cómo está relacionada la ingeniería química con
sectores profesionales que habitualmente no eligen tal y
la cocina o con los tejidos textiles y que la ingeniería como se ha puesto en evidencia en el artículo.
informática está relacionada de forma transversal con todas
las disciplinas. 4. CONCLUSIONES
• Hay que incorporar, presentar y mencionar, empezando por En este artículo se ha descrito la situación actual en cuanto al tipo
los libros de texto, a mujeres que sean referentes en los de iniciativas que tanto dentro como fuera de España se están
distintos campos: académico, investigación y empresarial, llevando a cabo para paliar un problema importante: el bajo
para que sean tomadas como ejemplos para las futuras número de mujeres que se encuentran en el campo de la
tecnólogas. Esto, desde educación infantil, de forma que tecnología, de la ingeniería en cualquier ámbito (académico,
siempre que se nombre un tecnólogo, también haya un laboral, investigación).
ejemplo de tecnóloga que lo acompañe. Una vez que se ha visto que las acciones realizadas hasta ahora no
• Hay que enseñar a las mujeres a arriesgarse más dentro del están dando los frutos esperados, se realiza una propuesta más
campo empresarial en el mundo de las TIC, a no temer tener amplia de iniciativas que creemos que faltan para completar las
puestos de responsabilidad porque lo harán igual que sus que ya se están realizando y que intentan paliar muchos de los
colegas y a llegar a un compromiso entre su vida familiar y problemas ya detectados y conocidos. Estas iniciativas
laboral sin auto-asignarse más tareas de las que debe realizar. básicamente se basan en modificar la educación tradicional
Esto también entra dentro del punto de educación, muchas empezando a trabajar desde muy pequeños y evitando cualquier
veces es la propia mujer la que no sabe delegar funciones sesgo referente al género.
familiares en los otros miembros de la familia o de su Como opinión más personal, creemos que, a la vista de los datos y
entorno. conforme a nuestra experiencia, las mujeres tendemos más a
• Fomentar el trabajo en equipo con mezcla de géneros como elegir carreras en las que nuestras expectativas de futuro esté más
un valor añadido que permita afrontar problemas complejos relacionado con la sociedad en general. Sin embargo, el
de una manera más eficiente. En primaria en EEUU, se crean estereotipo que representa a las personas que estudian las carreras
equipos mixtos donde se les presenta un reto, cada integrante de ingeniería, es un estereotipo poco social, con poca relación con
debe desarrollar una estrategia que luego deberá compartir su entorno, del que, sin ser totalmente conscientes, las mujeres
intentamos escapar. Sin embargo, no captamos que las carreras de
con sus compañeros de manera que entre todos buscan
ingeniería cumplen un servicio a la sociedad en todos los ámbitos.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 324 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
Sobra decir que hoy en día, un ingeniero informático puede ISSN 1989-7448.
prestar sus destrezas en cualquier ámbito ya sea relacionado con la doi:http://dx.doi.org/10.5944/reop.vol.25.num.1.2014.12016.
Ciencia o no. La prueba la podemos encontrar en algunos campos [7] Per què no puc fer-ho? Revisado por última vez mayo 2015
más que otros, por ejemplo, ¿qué serían hoy todas las técnicas de http://www.cdp.udl.cat/home/index.php/ca/docencia-
diagnóstico sin la ingeniería informática o la ingeniería recerca/jornades/per-que-no-puc-fer-ho/772-per-que-no-puc-
electrónica? o ¿qué serían, incluso de los estudiosos de la lengua fer-ho-2014.
sin una buena base de datos por detrás que les almacene toda la
información existente respecto a la etimología, la pronunciación o [8] Campus Tecnológico UGR para Chicas. http://cs4hs.ugr.es/
la traducción de cierto vocablo? [9] Women TechMaker Granada. http://wtm.gdggranada.com/
Por otra parte, la ausencia de mujeres dentro de las ingenierías, [10] Maria Villarroya-Gaudó, Sandra Baldassarri, Mayte Lozano.
causa una ausencia de diversidad en las propuestas de solución Girls’ Day experience at the University of Zaragoza:
que los ingenieros, informáticos o de cualquier otro tipo, attracting women to technology, Actas del Congreso de
proponen a los problemas del mundo real. Pero, ¿queremos esta Interacción 2014, Tenerife, España, pp. 443-450.
ausencia de diversidad?, en nuestra opinión, esta ausencia de [11] Proyecto Tech & Ladies. techandladies.com
puntos de vista diversos es, en todos los casos, perjudicial, no sólo
para la ingeniería, sino en la sociedad en general, puesto que la [12] Rochester Institute of Technology.
ingeniería está al servicio de la sociedad. Siendo cada vez un valor https://www.rit.edu/kgcoe/women/highlight/encouraging-
en alza la parte emocional, baste mencionar el ejemplo de la more-girls-choose-career-engineering
ingeniería informática donde ya no es suficiente que una [13] Universidad de Michigan. http://www.wise.umich.edu
aplicación sea funcional y usable, debe emocionar, debe conectar
[14] IGNITE. Inspiring Girls Now In Technology.
de forma hedónica con el usuario. Es en este punto, donde el
http://www.igniteworldwide.org/
sector femenino puede aportar valor a estas propuestas.
[15] Engineer your life. http://www.engineeryourlife.org/
En cualquier caso, está claro que las iniciativas que se están
siguiendo en las diferentes universidades, tanto españolas, como [16] Informe de la Comisión Europea.
extranjeras, están orientadas a paliar esa falta de diversidad en el http://ec.europa.eu/research/science-
alumnado. Y en ningún caso, estas iniciativas deberían dejar de society/document_library/pdf_06/mapping-the-maze-getting-
crearse, fomentarse, subvencionarse o llevarse a cabo, aunque sea more-women-to-the-top-in-research_en.pdf
sólo por las pocas mujeres que estamos dentro de estas [17] Technovation Challenge.
ingenierías, puesto que somos las que, en nuestra opinión, vemos http://www.technovationchallenge.org/
más el sesgo presente en las clases que impartimos día a día.
[18] Proyecto europeo ICT-Go-Girls!
5. REFERENCIAS http://www.ictgogirls.eu/Informe de la política de la
[1] Datos básicos del sistema universitario español. Curso 2013- Comisión Europea respecto a la igualdad de género.
2014. Ministerio de educación, cultura y deporte. http://ec.europa.eu/research/swafs/index.cfm?pg=policy&lib
=gender
http://www.mecd.gob.es/dms/mecd/educacion-mecd/areas-
educacion/universidades/estadisticas-informes/datos- [19] FESTA (Female Empowerment in Science and Technology
cifras/DATOS_CIFRAS_13_14.pdf. Academia http://www.festa-europa.eu/
[2] Lourdes Moreno, Yolanda González, Isabel Segura y Paloma [20] Genis-Lab (The Gender in Science and Technology Lab).
Martínez. 2014. Mujeres, Ciencia y Tecnología. Encuesta http://www.genislab-fp7.eu/
sobre la percepción de las dificultades de las mujeres en los [21] Proyecto HELENA (Higher Education Leading to
estudios universitarios técnicos. Actas del Congreso de Engineering and Scientific Careers). http://www.fp7-
Interacción 2014, Tenerife, España, pp. 464-471. helena.org/
[3] IBM. How Can We Encourage More Women to Pursue [22] Proyecto TWIST (Torwards Women in Science and
Technical Careers? Accedido ultima vez mayo 2015. Technologies) http://www.the-twist-project.eu/
https://www.ibm.com/developerworks/community/blogs/f2d
c1eeb-b60e-458b-8f32- [23] Proyecto WISAT (Women in Global Science and
ff540d77f273/entry/how_can_we_encourage_more_women_ Technologies). http://wisat.org/home/
to_pursue_technical_careers?lang=en [24] Girls who code. https://girlswhocode.com/
[4] Google anima a las mujeres a estudiar Ingeniería. [25] Agile-girls. http://www.agile-girls.com/
http://www.ticbeat.com/innovacion/google-anima-mujeres-
[26] Why So Few? Women in Science, Technology, Engineering,
estudiar-ingenieria/.
and Mathematics (2010).
[5] CSR Program: Girls Power Tech. Accedido ultima vez mayo http://www.aauw.org/research/why-so-few/
2015. http://csr.cisco.com/casestudy/girls-in-ict-day
[27] http://www.nytimes.com/2015/04/27/opinion/how-to-attract-
[6] Rodríguez Menéndez, Mª del Carmen; Inda Caro, Mª de las female-engineers.html (27/4/2015)
Mercedes; Peña Calvo, José Vicente. 2014. Rendimiento en
[28] M. Zeffertt, H. Thinyane, M. Halse (2014). Investigating the
la PAU y elección de estudios Científico-Tecnológicos en
use of games for creating a contextualized and gender
razón de género. REOP - Revista Española de Orientación y
inclusive computer science curriculum, ICERI 2014
Psicopedagogía, [S.l.], v. 25, n. 1, p. 111-127, abr. 2014.
Proceedings, pp. 4797-4804.

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 325 |
 Acercando las mujeres a la ingeniería: iniciativas y estrategias que favorecen su inclusion
P. Paderewski | M. García Arenas | R. Gil Iranzo | C. González González | E. M. Ortigosa | N. Padilla-Zea
[29] Sapna Cheryan, Andrew N. Meltzoff, Saenam Kim. [31] Informe Comisión Europea. Mapping the maze: getting more
Classrooms matter: The design of virtual classrooms women to the top in research
influences gender disparities in computer science classes. http://ec.europa.eu/research/science-
Elsevier. Computers & Education. Volume 57, Issue 2, society/document_library/pdf_06/mapping-the-maze-getting-
September 2011, Pages 1825–1835. more-women-to-the-top-in-research_en.pdf
[30] Climbing the technical ladder: obstacles and solutions for [32] Virginia Valian. Interests, Gender, and Science. Perspectives
mind-level women in technology. http://anitaborg.org/wp- on Psychological Science (Impact Factor: 4.89). 03/2014;
content/uploads/2013/12/Climbing_the_Technical_Ladder.p 9(2):225-230. DOI: 10.1177/1745691613519109
df

XVI Congreso Internacional de Interacción Persona-Ordenador | Workshop Engendering Technology (II)

| 326 |
GAMES
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela

Desarrollo de experiencias lúdicas interactivas

geolocalizadas
A. J. Soriano Marín J. L. González Sánchez F. L. Gutiérrez Vela
GEDES Universidad de Granada GEDES Universidad de Granada GEDES Universidad de Granada
C/ Periodista Daniel Saucedo Aranda, C/ Periodista Daniel Saucedo Aranda, C/ Periodista Daniel Saucedo Aranda,
s/n · E-18071 GRANADA (Spain) s/n · E-18071 GRANADA (Spain) s/n · E-18071 GRANADA (Spain)
+34 958 242 812 +34 958 242 812 +34 958 242 812
[email protected] [email protected] [email protected]

ABSTRACT El siguiente trabajo tiene como objetivo presentar diversas

El presente trabajo tiene como objetivo principal ofrecer una técnicas para mejorar la experiencia interactiva y la calidad en uso
visión específica sobre los retos de diseño de experiencias lúdicas para el desarrollo aplicaciones geolocalizadas en base a diseños de
geolocalizadas y cómo aplicarlas para la exploración turística. elementos lúdicos que mostraremos en un prototipo desarrollado.
Este campo ofrece nuevos retos y desafíos para conseguir el éxito Durante el segundo punto nos adentraremos en los avances de la
a nivel de interacción, los cuales ayudan a diseñar mecanismos realidad aumentada y la computación móvil y su uso en
para trasmitir una correcta experiencia de usuario en base a unir aplicaciones geolocalizadas o basadas en el contexto. En el tercer
elementos lúdicos basado en sistemas de juegos y nuevos punto analizaremos conceptos como la jugabilidad y la
paradigmas de interacción como son la computación móvil, la gamificación, narrativa interactiva o métodos de exploración y así
realidad aumentada y mecanismos de geolocalización y sus posibilidades como elementos para generar una experiencia
posicionamiento tanto en interior como en exterior. En este interactiva positiva a la hora de interactuar o de realizar una tarea.
trabajo mostramos un ejemplo de desarrollo de experiencia A través del cuarto punto se presentará una propuesta de diseño de
lúdicas geolocalizada a través del Campus de Cartuja (Granada) aplicaciones geolocalizadas, donde la experiencia interactiva se
usando el factor social como una herramienta principal para la basará en el uso de técnicas vistas en apartados anteriores, unidas
interacción entre los usuarios/as que hagan uso del sistema. a elementos de diseño de videojuegos del género de las “aventuras
gráficas” integrando actividades de geolocalización con realidad
Categories and Subject Descriptors aumentada. De esta forma, visitando las distintas localizaciones
que nos propone el juego, el usuario realizará un recorrido por
H.1.2 [Information Systems]: User/Machine Systems - Human diferentes enclaves de interés pero con una experiencia distinta,
factors. pues esta será diseñada en base a una aventura interactiva en
forma de juego. Finalmente mostraremos conclusiones y trabajos
General Terms futuros derivados del contenido de este artículo.
Performance, Design, Human Factors. 2. INTERACCIÓN MÓVIL BASADA EN EL
Keywords CONTEXTO
Gamificación, jugabilidad, contexto, realidad aumentada, Los dispositivos de computación móvil han propiciado que
experiencia de usuario, computación móvil, geolocalización. nuevas técnicas de interacción sean aplicadas a la hora de realizar
tareas, permitiendo al usuario la libertad de movimiento a la hora
1. INTRODUCCIÓN de poder realizarlas y permitiendo el intercambio de datos con el
Hoy en día los dispositivos móviles (teléfonos o tabletas) son entorno y con otros usuarios y creando ambientes cooperativos y
elementos que habitualmente usamos para estar informados e colaborativos donde la información fluye mejorando la
intercambiar información. Organismos importantes, como es la experiencia interactiva. Todos los avances técnicos hacen que los
UNESCO considera que el uso de nuevas tecnologías, nuevos dispositivos existentes sean una herramienta fundamental
especialmente paradigmas móviles, permitirá una mejora en la para los sistemas de información basados en el contexto (suelen
creación de contenidos cada vez más atrayentes y completos para ser los sistemas de computación móvil, la posición, el lugar, los
el usuario creando un ambiente de intercambio colaborativo de la objetos que lo rodean, etc., para proporcionar información con la
información, y en definitiva originando nuevas experiencias de que el usuario puede interactuar), así como plataformas idóneas
usuario como la adecuación al contexto de uso y la para la ejecución de juegos y mecanismos de interacción basados
geolocalización de los elementos con los que actuar. en realidad aumentada y computación ubicua dadas las
características y elementos que ellos ofrecen.
El desarrollo de la computación móvil unida a nuevos paradigmas
como la realidad aumentada ayuda a que aparezcan nuevos retos La realidad aumentada (RA) se puede definir como la
interesantes, como es hacer cada vez más eficaz los procesos de combinación de objetos de un entorno real y objetos virtuales
interacción y colaboración y generar experiencias más positivas y generados por computadora (objetos, imágenes, texto, etc.) en
placenteras para los usuarios que ayuden a producir un aumento tiempo real mediante un conjunto de dispositivos que añaden
de la calidad en las aplicaciones desarrolladas. información virtual al entorno real ya existente. El usuario puede
interactuar y realizar distintas acciones [35] siendo una de las
disciplinas más innovadoras dentro de la investigación en el

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 328 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
campo de la IPO. A diferencia de la realidad virtual, la realidad descripción de otras formas de diversión incluyendo aplicaciones
aumentada no pretende aislar al usuario del mundo real, sino interactivas.
poder interactuar de forma natural, con una mezcla de un mundo Nosotros, definimos la jugabilidad como el conjunto de
real y virtual usando los objetos generados. La realidad aumentada propiedades que describen la experiencia del jugador ante un
se encuentra entre un entorno real y un entorno virtual puro. sistema de juego determinado, cuyo principal objetivo es divertir
Existen distintas técnicas de interacción para sistemas de realidad y entretener “de forma satisfactoria y creíble” ya sea solo o en
aumentada [38]: basada en marcadores, basada en imágenes, compañía. Es importante remarcar los conceptos de satisfacción y
basada en movimiento corporal, basada en geolocalización [34] o credibilidad. El primero es mucho más difícil de conseguir en un
basadas en interactuar con otros sentidos como el tacto (sistemas sistema interactivo tradicional (por ejemplo Word) debido a sus
hápticos), el oído (sistemas hipermedia o multimodales) o el objetivos no funcionales; el segundo dependerá del grado de
olfato (simuladores). asimilación e implicación de los jugadores en el juego [39]. La
La interacción basada en el contexto es aquella en la que el diferencia en los objetivos de la experiencia de usuario a
conocimiento del contexto y de los elementos socioculturales en conseguir en base a la jugabilidad (experiencia del jugador, Player
los que se desarrolla la tarea se utilizan como parte y producto del eXperience) viene descrita en la Tabla 1. La jugabilidad se
propio proceso interactivo, ayudando a llevarlo a cabo [37]. De caracteriza por el grado de consecución en los siguientes
esta manera el dispositivo móvil actúa como mediador para atributos:
realizar la tarea, siendo el usuario el que realiza la tarea de manera • Satisfacción: Agrado o complacencia del jugador ante el
inmersiva en el medio, percibiendo un mayor grado de videojuego y el proceso de jugarlo.
participación y credibilidad en las acciones a realizar. La
interacción basada en el contexto acerca la tarea al usuario usando • Aprendizaje: Facilidad para comprender y dominar el
componentes y mecanismos de interacción que son familiares a sistema y la mecánica del videojuego: objetivos, reglas y
este, generando una experiencia positiva que facilita la formas de interaccionar con el videojuego.
adquisición de nuevas habilidades y mejoras en la productividad • Efectividad: Tiempo y recursos necesarios para ofrecer
de la tarea a realizar al reducir el proceso de aprendizaje de la diversión al jugador mientras éste logra los objetivos
misma. propuestos en el videojuego y alcanza la meta final de éste.
El contexto aumentado se define como “la conjunción de
información espacial, temporal y social que provoca que se • Inmersión: Capacidad para creerse a lo que se juega e
produzcan determinados eventos dentro del proceso interactivo integrarse en el mundo virtual mostrado en el juego.
para determinar qué tipo de actividad se debe realizar y qué • Motivación: Característica del videojuego que mueve a la
contenidos de dicha actividad son válidos de acuerdo a las persona a realizar determinadas acciones y a persistir en ellas
características del usuario y de los elementos interactivos para su culminación.
existentes a la hora de realizar una tarea, o lo que es lo mismo, un
contexto enriquecido con información adicional que ayuda a • Emoción: Impulso involuntario originado como respuesta a
mejorar su asimilación y mejora la experiencia inmersiva [43]. los estímulos del videojuego, que induce sentimientos y que
Para ello se hace uso de elementos gráficos como texto, objetos desencadena conductas de reacción automática.
2D/3D, videos, sonido o cualquier contenido multimedia con el
• Socialización: Atributos que hacen apreciar el videojuego de
cual el usuario puede interactuar directa o indirectamente.
distinta manera al jugarlo en compañía (multijugador), ya sea
3. DISEÑO DE EXPERIENCIAS LÚDICAS de manera competitiva, colaborativa o cooperativa.
Los videojuegos son sistemas interactivos concebidos para La gamificación o ludificación (gamification en el ámbito
explotar al máximo la experiencia del usuario a nivel lúdico anglosajón) es la aplicación de los elementos propios de los
cuando se hace uso de ellos, pues su principal objetivo es explotar juegos en contextos que no son estrictamente lúdicos con el fin de
distintas emociones del usuario y asegurar el entretenimiento. conseguir que las personas adopten ciertos comportamientos
El estudio de los videojuegos y sus virtudes ha permitido dar un deseados. La aplicación de la gamificación busca potenciar la
grado más de calidad a la experiencia del usuario, aumentando la motivación, la concentración, el esfuerzo, la fidelización y otros
motivación y el compromiso del usuario y provocando una valores positivos comunes al mundo del juego. Desde el punto de
experiencia distinta en aplicaciones “no de ocio”. Para vista de los videojuegos, la gamificación está relacionada con el
comprender cómo podemos enriquecer la experiencia de usuario uso de elementos del diseño de videojuegos en contextos que no
en un sistema interactivo, usando las características propias de los son “juego” para hacer que un producto, servicio o aplicación sea
juegos, no hay nada mejor que analizar qué es lo que hace que los más divertido, atractivo y motivador. Esta definición nos interesa
juegos sean divertidos y qué mecanismos podemos usar en bastante ya que nos habla de cosas que nos acercan mucho a los
nuestros diseños. Para ello es importante hacer referencia a los objetivos planteados por la experiencia de usuario (UX) [46] .
conceptos de jugabilidad y de gamificación.
La jugabilidad es un término empleado en el diseño y análisis de
juegos que describe la calidad del juego en términos de sus reglas Tabla 1. Diferencias entre objetivos de UX vs PX: Usabilidad
de funcionamiento y de su diseño como juego. Se refiere al respecto a Jugabilidad
conjunto de las experiencias que siente un jugador durante la UX-Objetivos de Usabilidad: PX-Objetivos de Jugabilidad:
interacción con sistemas de juegos. Este concepto surgió junto con Productividad Entretenimiento
las teorías de diseño de juegos en los años 80, el término 1. Realizar una tarea 1. Entretener el máximo tiempo
jugabilidad era usado solamente en el contexto de los videojuegos, eficientemente posible
aunque ahora por su popularidad ha comenzado usarse en la 2. Eliminación de posibles 2. Divertir superando obstáculos

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 329 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
errores 3. Recompensa Interna: Diversión
3. Recompensa Externa: 4. Nuevas cosas por Aprender y
Trabajo Realizado Descubrir
4. Fácil de Aprender e 5. Incrementar mecánicas y metas de
Intuitivo juego
5. Reducción de la carga de 6. Asume que el jugador tiene que
Trabajo ser retado por la tecnología
6. Asume que la tecnología
debe ser humanizada
Los elementos básicos que componen la estructura básica de una
propuesta de gamificación son las dinámicas, las mecánicas, los
elementos o componentes de juego y los perfiles de usuario.
Las dinámicas están muy relacionadas con objetivos, efectos,
deseos y motivaciones que se pretenden conseguir o potenciar en
los usuarios. Tienen que ver con las necesidades e inquietudes
humanas que motivan a las personas. No existe una lista cerrada
de las dinámicas y pueden clasificarse de muchas formas. Una
posible clasificación es la propuesta por Kevin Werbach que las
clasifica en: Restricciones (limitaciones, reglas o compromisos
forzados), Emociones (curiosidad, competitividad, frustraciones,
Figura 1. Definición de un ciclo de actividad típico en un
felicidad), la Narrativa (la historia que da coherencia), la proceso de gamificación.
Progresión (el crecimiento del jugador a lo largo del juego) y las
Relaciones (interacciones sociales, estatus, altruismo) [45] . No todos los jugadores son iguales, tienen los mismos intereses, ni
les divierten las mismas cosas. Partiendo del estudio realizado por
Las mecánicas son las acciones básicas que hacen que progrese la Richard Bartle en 1996 sobre los diferentes jugadores que
acción de jugar y que llevan al jugador a involucrarse en el juego.
podemos encontrar en un MUD (Multi User Dungeon) [36]
Las mecánicas las podemos ver como la forma en la que podemos
podemos decir que existen 5 tipos de jugadores clasificados en
“inyectar” dinámicas en las actividades para hacerlas más
dos ejes: por un lado, se encuentran aquellos que se centran en la
divertidas y motivadoras. Algunas de las más importantes son: los
consecución de los objetivos y en el resultado final del juego, y
desafíos, la competición, la cooperación y colaboración, la suerte
por el otro, los que se centran más en el valor participativo y
o la aleatoriedad, la realimentación, las recompensas y el
lúdico que entraña el juego en sí mismo. Estos perfiles no son
coleccionismo de distintos ítems, entre otras.
únicos y cada desarrollador puede proponer nuevos perfiles, lo
Los componentes o elementos de juego dependen del tipo de importante es remarcar la existencia de varios perfiles para
juego y de la experiencia a generar. Algunos de los componentes generar distintas dinámicas de juego asociados a ellos. Los
más usuales son: los logros, los avatares, los emblemas o perfiles de jugadores de ejemplo mostrados por este autor son:
insignias, las misiones, las colecciones, el combate, el desbloqueo
• Killer: o ambicioso, muy motivado por ganar y ser el
de contenido, los regalos, las tablas de clasificación, los puntos,
primero en todas las competiciones en las que participa.
los niveles, los equipos, y los bienes virtuales.
Aunque un buen diseñador de gamificación debe pensar cuales • Archiver: o triunfador, jugador muy aventurero al que le
son las dinámicas que interesa provocar en los usuarios de su gusta descubrir nuevas cosas, superar los retos que se les va
sistema (sobre todo a largo plazo), son las mecánicas las que se marcando durante el juego y muy orgulloso de su estatus.
diseñan e implantan en el sistema ya que son las que consiguen • Socializer: o sociable es un persona muy social y necesita
despertar y avivar (a corto plazo) las motivaciones y actitudes en compartir con los demás sus logros y objetivos y para ello
los usuarios. Sin olvidar que al final hay que traducir las suele crear una red de contactos o amigos con los que se
mecánicas en componentes y en diseños concretos dentro de relaciona.
nuestro sistema. En los procesos de gamificación se le da mucha
importancia a lo que se llama el “ciclo de actividad”, Figura 1. El • Explorer: Característica del videojuego que mueve a la
juego no es un proceso lineal sino que el diseñador del juego tiene persona a realizar determinadas acciones y a persistir en ellas
que ir provocando en el jugador momentos de motivación, para su culminación.
incentivándole (1) en el juego y desafiándole (2) con un conjunto
de retos y puzles que el jugador tendrá que alcanzar, conquistando Emotional: o explorador, al que le gusta descubrir todo lo que le
(3) los objetivos planteados y por los que recibirá (4) una serie de es desconocido, le motiva en gran medida la capacidad de
recompensas que le volverán a motivar y a repetir el ciclo de autosuperación y le suelen divertir los retos que le ponen a prueba
actividad. Estos ciclos de actividad o comportamiento son la aunque sean muy difíciles.
esencia de un juego y un punto clave en el diseño jugable y También es interesante conocer la propuesta de Andrzej
gamificado. Marczewski, más centrada en la gamificación y en el estudio en
profundidad de los tipos de usuarios y las mecánicas y dinámicas
que podemos usar para motivarlos [42]. En su trabajo nos habla de
6 tipos de usuarios clasificándolos en función de si poseen fuertes
motivaciones intrínsecas, como son los triunfadores, sociables,
filántropos y los espíritus libres o si sus motivaciones son más
flojas, como serían los disruptores y los jugadores, Figura 2.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 330 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
A nivel de Narrativa Interactiva es importante mostrar los o Estructura con bifurcación: los puntos de interés se
hechos de la historia como hilo conductor entre los distintos retos pueden representar como un grafo dirigido donde la
y lugares donde se realiza la acción del juego para poder alcanzar consecución un punto de interés abre varios caminos
las metas y objetivos. Es necesario fijar el orden y la relación de alternativos. Un ejemplo es el “dominó” donde la
los distintos elementos que aparecen en la aventura y cómo se consecución de un punto de interés condiciona los
narrará la historia al jugador. posibles puntos siguientes, Figura 4.
o Estructura lineal: es una secuencia lineal de puntos de
interés con un orden establecido donde solo existe un
camino para completar la historia. Un ejemplo es la
“búsqueda del tesoro”, donde la consecución de un
punto de interés da acceso al siguiente, Figura 5.

Figura 2. Clasificación de los tipos de jugadores en un proceso

gamificado.
Es importante remarcar los tipos de exploración basados en
retos usando contextos aumentados que se adapten a las
experiencias propuestas, ver [41][40]. Se describe dichos tipos de
exploración en las figuras siguientes y hacen referencia a RJ (Reto
de Juego), AE (Actividad/Elementos), CA (Contexto
Aumentado):
• Estructura de Conjunto: puntos de interés que carecen de
relaciones explicitas entre sí, Figura 3. Destacamos:
Figura 4. Ejemplo de representación gráfica del recorrido de
estructura secuencial con bifurcación.

Figura3. Ejemplo de representación gráfica del recorrido de

estructura de conjunto.
o Palabras Cruzadas: es un conjunto de puntos de Figura 5. Ejemplo de representación gráfica del recorrido de
interés que contienen parte de una historia las cuales se estructura secuencial lineal.
han de combinar para construirla. No todas las 4. DESARROLLO DE APP YINCANA
combinaciones formaran una historia y el orden no es
relevante. BASADA EN EL CONTEXTO
A continuación presentamos distintos desarrollo de una
o Recolectando Mariposas: es un conjunto de puntos de aplicación típica geolocalizada como es una aplicación turística
interés donde cada uno de ellos contiene una historia. definida por un conjunto o lista de puntos a visitar en un espacio
No es necesario superarlos todos y el orden no es interior o exterior (desde ver una serie de monumentos a lo largo
relevante. de una ciudad o visitar una serie de obras de arte en un museo)
para mejorar la experiencia interactiva en base a las ideas
o Rompecabezas: se han de superar todos los puntos de
propuestas en los puntos anteriores basada en una yincana social
interés del conjunto para formar la historia. El orden no
educativa. Este ejemplo ha sido realizado realizado conjuntamente
es relevante.
por el grupo de investigación GEDES y SEJ-059 PROFESIOLAB
• Estructura Secuencial: puntos de interés con relaciones ambos de la Universidad de Granada. El juego está desarrollado
explicitas entre sí. En ella remarcamos: para un perfil de usuario adolescente y que hace uso de
tecnologías móviles como del teléfono y una tablet que usaran

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 331 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
para participar en el juego. Los objetivos principales a conseguir • Cartógrafo: lleva el dispositivo con el mapa y guía al grupo
con la aplicación son los siguientes: hacia los puntos donde se localizan las pruebas indicando el
recorrido que se está realizando.
• Fomentar valores como, responsabilidad, respeto e igualdad.
• Custodio: recoge las pruebas que ayudaran a resolver el
• Iniciar al alumnado el conocimiento del espacio cultural
enigma final si lo hubiese (anota en el dispositivo las pistas
desde la implementación de las nuevas tecnologías que se vayan consiguiendo).
concienciándolos de la importancia del conocimiento de
entorno donde se habita. • Fotógrafo: realiza las fotografías durante el recorrido (utiliza
un dispositivo tipo smartphone).
• Mejorar las habilidades y destrezas básicas y el desarrollo de
las cualidades motrices a través del juego. • Organizador: controla que todos cumplan con su rol y debe
ir asignando las tareas a desarrollar en cada momento. Vigila
• Mejorar la capacidad imaginativa en base a los retos y
que en cada punto utilice el dispositivo un compañero
narrativa interactiva del juego. distinto.
• Elaborar estrategias de forma individual y grupal para • Portador: se encarga de la Tablet o teléfono inteligente y
resolver las situaciones planteadas durante el desarrollo de debe entregarla a quien corresponda en cada momento, es la
las diferentes tareas. persona responsable del dispositivo.
• Desarrollar la creatividad y espontaneidad. • Redactor: redacta una crónica del recorrido (redactor de
• Aprender a trabajar en equipo para la consecución de los bitácora en formato digital utilizando un dispositivo móvil).
objetivos comunes que se les han presentado. Cada equipo estará formado por varios miembros de un perfil
• Motivar al alumnado usando el juego y el planteamiento de determinado, pudiéndose dar el caso en el que dos perfiles
obstáculos como medio para conseguir una actitud positiva, recaigan sobre la misma persona o hayan perfiles repetidos.
participativa y placentera. Además en cada equipo pueden darse varios dispositivos móviles
(teléfono o tablet). La organización es un aspecto clave y fomenta
El primer paso es crear la estructura de juego a aplicar, para ello el factor social, siendo una propuesta propia de cada grupo.
usaremos una estructura típica del juego de la Yincana donde
Además propondremos que cada usuario haga uso de un avatar.
aplicaremos mecánicas y elementos propios de videojuegos de
El avatar sirve para identificar al jugador del mundo real con el
tipo aventuras gráficas y RPG para mejorar la experiencia
jugador o elementos del mundo virtual, creando una asociación
interactiva. Siguiendo las ideas del apartado anterior. Una
bidireccional entre ambos. De esta manera proporcionamos
Yincana consiste en “un conjunto de pruebas de destreza o
mecanismos de identidad y de inmersión dentro de las dinámicas
ingenio que se realizan por equipos o de manera individual a lo
de juego. Otros de los aspectos a tener en cuenta es la narrativa
largo de un recorrido, normalmente al aire libre y con una
interactiva. Para ello haremos uso de mecanismos comúnmente
finalidad lúdica” y que se adapta al objetivo de crear experiencias
utilizados en aventuras gráficas y juegos de rol (RPG) que
geo localizadas. La experiencia de juego que se propone será
anteriormente hemos citado.
capaz de detectar la localización actual del jugador, de modo que
la información necesaria para superar los retos de la aventura será Como retos de juego planteamos unos retos (para crear mecánicas
presentada de forma contextual. Es decir, dependiendo de en qué y dinámicas y elementos de juego similares a las propuestas en el
lugar, posición, edificio, complejo, sala, etc. Se podrá acceder a apartado anterior) sencillos que se puede aplicar según el
unos datos/eventos u otros. Esto permitirá definir rutas guiadas contexto. Los objetivos de estos retos pueden ser puramente
por las diferentes localizaciones, que obedecerán a criterios lúdicos o con un objetivo de aprendizaje de elementos interesantes
preestablecidos para lograr un mejor aprovechamiento de la visita en la ruta turística basada en una yincana. Estos retos pueden ser:
por parte del usuario. De nuevo, el jugador percibirá este recorrido • Sopa de letras y crucigramas: muestra una sopa de letras o
como parte la narrativa en la que está inmerso y dentro de la crucigramas y cuadros de texto para introducir las respuestas
secuencia de retos que le propone el juego, bajo un proceso de encontradas relacionadas con la historia y el lugar donde se
gamificación de la visita. Finalmente, mediante el uso de la realiza la prueba o donde está situado el usuario.
realidad aumentada proponemos fusionar y complementar los
escenarios del juego con información real de los lugares visitados, • Trivial y Test: consiste en un conjunto de preguntas con
a los que se añadirá información sensible al contexto que permita distintas opciones de respuesta, siendo solo una de ellas la
enriquecer y comprender mejor lo que el usuario está viendo en correcta. En el caso de los Test se acepta varias preguntas
cada lugar. La aventura incluirá elementos generados de forma correctas válidas (opción múltiple).
automática y adaptados al jugador, de forma que se mantenga su
interés en el juego y se fomente la rejugabilidad, al cambiar • Adivinanzas: se presentan una serie de adivinanzas con su
dichos elementos entre dos instancias de la aventura (dos partidas) respectivo cuadro de texto para introducir la respuesta.
diferentes. • Validación mediante cámara: se usará la cámara de móvil
Antes de pasar a definir las dinámicas y mecánicas del juego es para analizar un elemento o tomar una foto y así desbloquear
interesante analizar los roles o perfiles que proponemos para los un reto o poder superarlo, como por ejemplo fotografiar la
distintos jugadores/usuarios para este tipo de juegos inscripción de una estatua de una plaza determinada.
caracterizados por la exploración y la visita de lugares. Respecto
• Búsqueda y ensamblaje de objetos: consistirá en encontrar
a la clasificación mostrada en el punto anterior estos serían sobre
distintos objetos siguiendo determinadas pistas para ir
todo del tipo “explorer” “archiver” y “socializer”. Estos son:

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 332 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
cambiándolos y usándolos en la resolución de retos
posteriores para avanzar en el juego.
Por otro lado usaremos niveles de completitud para los retos que
tenemos pendientes (barras de progresos, listas de tareas, etc.)
siguiendo algunos de los ejemplos de dinámicas y mecánicas
mostrados con anterioridad, de manera que estamos generando
una motivación extrínseca en el usuario que le genera un deseo de
completar las tareas pendientes.
A su vez por cada reto ofreceremos tres medallas según la forma
de completar el reto (grado de consecución). Por ejemplo
simplemente por hacerla correctamente una medalla de bronce,
por llegar a una puntuación media, una medalla de plata, y si su
consecución es del 100% una medalla de oro. Si todas las pruebas
llegan al 100% de completitud puede lograr la medalla de platino.
Esta mecánica consigue que el jugador se motive y repita una y
otra vez la prueba hasta conseguir las tres medallas. Además
tendremos un sistema recompensas de emblemas/puntos y de
clasificación. Los puntos (elementos de juego) son una forma
efectiva de realimentación, tanto para que el jugador conozca su
progresión dentro del juego como para mostrar los resultados a
otros jugadores.
Los emblemas o insignias son una versión más sólida de los
puntos. Los podemos ver como una representación visual de un Figura 6. Ubicación de las distintas pruebas en el mapa del
logro destacable dentro de nuestro sistema de gamificación. Son Campus de Cartuja
una forma de darle objetivos específicos a los jugadores y también Para la localización en exteriores se utilizará como tecnología de
un escaparate de las diferentes posibilidades que nos da el posicionamiento el sistema GPS. Además se ha incorporado un
sistema. sistema para aplicar corrección de errores y margen o radio de
Las tablas de clasificación permiten ver el nivel o la puntuación acción. Este radio de acción está parametrizado según las
alcanzada por otros jugadores así como comparar nuestros características de la Yincana. El objetivo de este componente es
progresos en el juego. Son una forma clara de motivar a los poder lanzar información al grupo de usuarios según vayan
jugadores a competir unos con otros. Para fomentar el factor llegando al punto de destino, así como ampliar la zona de acción
social incluimos un álbum de fotos donde los jugadores/usuarios en caso de que coincidan varios grupos dentro de un mismo reto.
comparten imágenes de su proceso de juego. Para la localización de interiores se utilizan marcadores con
códigos QR con el objetivo de fomentar la exploración y el
Con los elementos anteriormente citados fomentamos propiedades
seguimiento de pistas que con ellos se ofrecen. Para utilizar la
de la jugabilidad indispensables para una correcta experiencia
información temporal se hace uso del reloj del sistema con el
interactiva lúdica, como pueden ser la satisfacción personal
objetivo de establecer un tiempo límite para la realización de las
(puntuación obtenida), la inmersión (toma de fotos a elementos
pruebas.
del entorno), la efectividad (intentos y tiempo jugado/visitando los
elementos), la motivación (número de medallas conseguidas), Ejemplos de las distintas pruebas en exteriores pueden verse en
socialización (compartir fotos, retos en grupos/parejas) emoción las siguientes capturas, Figura 7.
(uso de tiempo en cada reto) o el aprendizaje (probar a superarse
así mismo).
El recorrido propuesto está localizado en el recorrido de la
Yincana y está localizado a lo largo del Campus de Cartuja de la
Universidad de Granada. Este recorrido está caracterizado por
tener puntos en el exterior e interior de algunos edificios, por lo
que se aplicarán técnicas de localización en exteriores e interiores.
Los jugadores/usuarios deben visitar edificios propios del campus
para superar el juego propuesto, Figura 6. El recorrido propuesto
es de tipo “búsqueda del tesoro” (estructura lineal), donde la
consecución de un punto de interés da acceso al siguiente.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 333 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela

Figura 9. Mapa de interior y localización de retos

Figura 7. Ejemplo de retos: sopa de letras, toma de foto,

cuestionario y adivinanzas
Para las pruebas que se realizan en interior se han diseñado
actividades de ensamblaje de objetos siguiendo la estructura de
los retos de la aventura gráfica y que se representan en una
estructura jerarquizada representada en la Figura 8. En ella los
jugadores deberán conseguir realizar la “poción gazpacho” a lo
largo del mapa interior del edificio, Figura 9. Para interactuar
entre elementos reales y virtuales se utilizan marcadores de
objetos y de funcionalidad tal y como se indica en [43]. Ejemplos
de capturas del juego en interior pueden verse en la Figura 10. En
ella se muestra el proceso de cultivar un tomate, la información
del elemento, la acción de recoger agua o amasar el pan necesario
para realizar la poción gazpacho.

Figura 10. Pruebas de interior usando realidad aumentada y

marcadores de funcionalidad
Para fomentar el factor social se hace uso de un álbum de fotos
donde los jugadores comparten sus experiencias durante el juego
y pueden realizar comentarios de las mismas, Figura 11.

Figura 8. Ejemplo de retos de interior: Poción Gazpacho

Figura 11. Álbum de imágenes y participación de los usuarios

El sistema ha sido desarrollado conforme a la arquitectura de
elementos propuestos en [43][44] como son: estructura de juego,
contenidos multimedia, contexto (temporal, posicional y social),
planificador de rutas, motor de juego de realidad aumentada y los

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 334 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
informes/estadísticas obtenidos. Además de la comunicación con Pregunta 3 40% 60%
un portal web donde compartir y visualizar los resultados. Pregunta 4 5% 5% 90%
Los elementos se han desarrollado usando UNITY 3D para crear Pregunta 5 100%
el sistema móvil sobre plataformas Android con las librerías
Pregunta 6 4% 11% 85%
Vuforia de Qualcom para la parte de la realidad aumentada.
Además se ha hecho uso de marcadores funcionales para
implementar las acciones del juego, así como completar los
mecanismos de localización interior tal y como hemos comentado 5. CONCLUSIONES Y TRABAJOS
en párrafos anteriores. FUTUROS
Para probar el sistema se diseñó una yincana turística con una A lo largo de este trabajo hemos presentado el diseño y uso de
participación total de 100 alumnos/as (estudiantes del grado de experiencias lúdicas, basadas en la jugabilidad y la gamificación,
Educación Infantil). Tras realizar y completar la yincana y los para mejorar la experiencia interactiva en la realización de
retos propuestos siguiendo la aventura interactiva, se realizó un distintas tareas geolocalizadas. El uso de estas experiencias ayuda
pequeño encuesta anónima de seis preguntas y cuya valoración va a mejorar la satisfacción, motivación y fidelización de los usuarios
desde 1 (más negativo, peor valorado) hasta 5 (lo más positivo, ante el conjunto de tareas a realizar en pos de un objetivo
mejor valorado) con el objetivo de recabar una primera impresión concreto.
o información inicial de la experiencia obtenida con este prototipo En el ejemplo presentado se han aplicado mecánicas y dinámicas
a través de su participación. El objetivo de esta encuesta era del juego, basados en aventuras gráficas y RPG interactuando con
obtener una idea inicial del estado del prototipo, y no se asemeja a el entorno según el contexto haciendo uso de dispositivos móviles
un cuestionario de evaluación de la satisfacción, que tal y como se y realidad aumentada. Para ello la Yincana propuesta hace uso de
puede leer en el siguiente apartado, es uno de los retos que perfiles de usuario, distintos retos, una historia y una forma de
tenemos presentes para realizar como trabajo futuro. Las narración propias de este tipo de juegos que dotan a la aplicación
preguntas a responder individualmente identifica la experiencia desarrollada mecanismos atractivos para los propios usuarios que
interactiva personal de haber superado el juego en grupo: mejoran las experiencia interactiva con los retos de juego que en
• Pregunta 1: ¿Con qué frecuencia has usado aplicaciones de ella se presentan para realizar las tareas.
juegos geolocalizados? Aunque el porcentaje de aceptación del prototipo es elevado, aún
se deben trabajar en la adaptación de mecánicas de juego
• Pregunta 2: ¿Los retos propuestos te han motivado para
específicas, así como mejoras en el factor social y adaptación de
completar el recorrido?
puntuaciones e insignias (tabla de rankings al perfil de usuario)
• Pregunta 3: ¿Son adecuadas las recompensas obtenidas para evitar situaciones de frustración y abandono de la tarea por
(puntuación, medallas…) para este tipo de aplicación? exceso de componentes lúdicos. Es por ello que es recomendable
analizar el impacto sobre los usuarios y conocer el equilibrio
• Pregunta 4: Indica el grado de colaboración y participación necesario en la utilización de estas técnicas para fomentar la
con tus compañeros para superar los retos y completar el experiencia adecuada según el caso de uso, proyecto propuesto o
recorrido: perfil de usuarios. Esto es uno de los puntos que marcan nuestro
• Pregunta 5: ¿Participarías en futuras pruebas de este tipo de trabajo futuro a través de una evaluación de la experiencia
aplicaciones? interactiva mucho más profunda y detallada con esta aplicación y
otras similares.
• Pregunta 6: ¿Cómo de interesante encuentras el uso de Además se está trabajando en mejorar la narrativa interactiva así
juegos geolocalizados en aplicaciones para visitas turísticas? como recoger las ideas de este trabajo para aplicarlas en mejorar
El resultado obtenido, Tabla 2, muestra como el diseño de la la experiencia de la experiencia de los turistas y visitantes de la
aplicación con mecanismos basados en experiencias lúdicas ciudad de Granada, y en particular por edificios emblemáticos
obtiene un alto grado de aprobación sobre las preferencias de donde los visitantes, que se verán impulsados por descubrir la
realizar las actividades de esta manera (con elementos de juego) y información que se oculta tras los retos del juego teniendo como
no sobre la manera tradicional (simple visita), en ella se destaca recompensa bonos de visita a museos y exposiciones propias que
ante todo el factor social y el poder compartir los resultados con el en algunos de los edificios son expuestos al público, así como
resto de usuarios, así como el participar de experiencias premios que la propia universidad ofrecerá a sus usuarios.
colectivas. Se destaca que los retos de juego han sido del agrado 6. AGRADECIMIENTOS
del jugador, así como la aventura que los incorpora dentro de una Este trabajo ha sido financiado por el Ministerio de Ciencia e
arquitectura de retos juego, destacando que las recompensas son
Innovación de España como parte del proyecto VIDECO
adecuadas y mostrándose muy abiertos en participar en nuevas
(TIN2011-26928) y el Proyecto de Excelencia de la Junta de
experiencias lúdicas usando este sistema para crear rutas de
Andalucía P11-TIC-7486 Videojuegos educativos para las aulas
interés cultural.
TIC: Metodología de desarrollo e implantación.
Tabla 2. Resultados del cuestionario de satisfacción de la
aplicación Yincana 7. REFERENCIAS
1 2 3 4 5 [1] Albert-László, B., & Réka , A. (1999). Emergence of Scaling
in Random Networks. Science, 286(5439), 509-512.
Pregunta 1 90% 10%
[2] Avouris, N., & Yiannoutsou, N. (2012). A review of mobile
Pregunta 2 3% 7% 90%
location-based games for learning across physical and virtual

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 335 |
 Desarrollo de experiencias lúdicas interactivas geolocalizadas
A. J. Soriano Marín | J. L. González Sánchez | F. L. Gutiérrez Vela
spaces. Journal of Universal Computer Science, 18(15), [9] Lliteras, A. B., Challiol, C., & Gordillo, S. E. (2012). Juegos
2120-2142. doi:10.3217/jucs-018-15-2120 Educativos Móviles Basados en Posicionamiento: Una Guía
[3] Azuma, R. T. (1997). A survey of Augmented Reality. para su Conceptualización. Argentine Symposium on
Software Engineering, (pág. 164).
[4] Bartle, R. A. (2004). Designing Virtual Worlds. New Riders.
[10] Marczewski, A. (2014). User Types 2.0 – The Hexad.
[5] Brown, E. (2010). Introduction to location-based mobile Obtenido de http://www.gamified.uk/user-types
learning. En Education in the wild: contextual and location-
based mobile learning in action (A report from the STELLAR [11] Soriano Marín, A. J. TFM: Videojuegos Educativos basados
Alpine Rendez-Vous workshop series (págs. 7-9). Learning en el Contexto con Realidad Aumentada. 2014.
Sciences Research Institute, University of Nottingham. [12] Soriano Marín, A. J. and González Sánchez, J. L. and
[6] Garrido, R., & García Alonso, A. (2007). Técnicas de Gutiérrez Vela, F.L. Realidad Aumentada en Videojuegos
Interacción para Sistemas de Realidad Aumentada. Educativos basados en el Contexto. In Actas del XV
Congreso Internacional de Actas del XV Congreso
[7] González Sánchez, J. L., Gutiérrez Vela, F., & Montero Internacional de (Puerto de la Cruz, Tenerife 2014),
Simarro, F. a.-Z. (2012). Playability: analysing user Asociación Interacción Persona-Ordenador (AIPO), 211-215.
experience in video games. Behaviour & Information
Technology, 31(10), 1033-1054. [13] Werbach, K., & Hunter, D. (2012). For the Win: How Game
Thinking Can Revolutionize Your Business.
[8] Kjeldskov, J., & Paay, J. (2007). Augmenting the City with
Fiction: Fictional Requirements for Mobile Guides. [14] Zichermann, G., & and Cunningham, C. (2011).
Proceedings of HCI in Mobile Guides, Mobile HCI 2007, Gamification by Design Implementing Game Mechanics in
(págs. 1-6). Web and Mobile Apps. O’Reilly.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 336 |
 La importancia de las emociones en el diseño de historias interactivas
J.R. López-Arcos | P. Paderewski Rodríguez | F. L. Gutiérrez Vela | N. Padilla-Zea | N. M. Fuentes García

La importancia de las emociones en el diseño de historias

interactivas
José Rafael López-Arcos Francisco Luis Gutiérrez Vela Natalia Padilla-Zea
GEDES, Universidad de Granada GEDES, Universidad de Granada GEDES, Universidad de Granada
[email protected] [email protected] [email protected]

Patricia Paderewski Rodríguez Noemí Marta Fuentes García

GEDES, Universidad de Granada GEDES, Universidad de Granada
[email protected] [email protected]

ABSTRACT Uno de los factores que otorga importancia a las emociones es que
Los videojuegos se benefician del poder de la narrativa para hacer éstas pueden actuar como motivadores esenciales. Es decir,
que el usuario se sienta inmerso en el mundo virtual que ofrecen. emociones como el afecto positivo o el placer pueden desarrollar
Los diferentes cambios de contexto dentro del abanico de una conducta motivacional dirigida hacia unos objetivos
mecánicas de juego pueden ser justificados mediante estas específicos. Estos objetivos pueden ser la propia diversión en el
historias. Esto mantiene la motivación a través de la inmersión y caso de un videojuego o el aprendizaje en el caso de un
favorece, por tanto, la enseñanza por medio de los videojuegos. El videojuego educativo.
vínculo necesario para que el receptor de la historia se involucre A nivel educativo, teorías como el condicionamiento operante [5]
con los personajes que aparecen en ella se consigue, a su vez, afirman que la experiencia emocional placentera puede ser
gracias a las emociones que se transmiten en la historia. El diseño considerada como un forma importante de refuerzo positivo en los
de las historias contenidas en videojuegos presenta una procesos de aprendizaje, incrementando la probabilidad de que se
complejidad alta debido a su carácter interactivo. En el presente repita la conducta que dio lugar a esa experiencia emocional.
trabajo, se expone la base de nuestra propuesta para diseñar y
evaluar estas historias interactivas e incluirlas en un videojuego, La narrativa, por su parte, juega un papel muy importante en el
prestando especial atención a la evaluación de las emociones diseño de la experiencia de usuario de un videojuego. Además, en
contenidas en ellas. el caso concreto de los videojuegos educativos, proporciona una
justificación motivacional a las actividades pedagógicas realizadas
Categories and Subject Descriptors por el usuario.
H.1.2 [Information Systems]: User/Machine Systems – Human
Para diseñar una experiencia narrativa, es imprescindible definir
factors.
las emociones contenidas en ésta. Las emociones en una historia
General Terms son imprescindibles para crear un vínculo entre el receptor de la
Measurement, Documentation, Performance, Design, historia y los personajes y situaciones descritos en ésta. Así lo
Experimentation, Human Factors. demuestran estudios como el presentado en [6], donde los autores
definen métodos de interacción con la narrativa basados en las
Keywords emociones. Esta relación, además, está basada en el interés que la
Interactive Storytelling, Educational Video Games, Evaluation, experiencia despierta en el invitado a lo largo de la duración de
Emotions, Interaction. ésta [7].
En anteriores trabajos hemos presentado [8][9] una aproximación
1. INTRODUCCIÓN para estructurar y analizar la historia que se incluye en los
Las emociones son una componente fundamental en el ser videojuegos educativos (VJE) como elemento que da sentido
humano, ayudándole a darle significado, valor y riqueza a las narrativo al proceso de juego. En ella, mediante un modelo
experiencias que viven [1]. En la actualidad, es reconocido, que conceptual y una serie de tareas, se proporcionan mecanismos
las emociones juegan un rol critico e imprescindible en todas las para diseñar la historia interactiva que va a formar parte del VJE.
relaciones con las tecnologías, desde el uso en videojuegos, la Esa historia, gracias a su flexibilidad y adaptabilidad, permite dar
navegación por sitios web, el uso de dispositivos móviles, entre una justificación narrativa a las diferentes tareas que el jugador
muchos otros [2] [3]. A nivel de investigación, existen numerosos realiza en el videojuego.
trabajos [4] donde se analiza qué aspectos de las emociones son
los importantes a la hora de realizar una evaluación de la El resto del presente trabajo se organiza del siguiente modo: el
usabilidad y de la experiencia del usuario y qué técnicas son las apartado 2 explica las bases de la propuesta en la que trabajamos
más adecuadas para realizar dicha evaluación. actualmente para diseñar historias interactivas. Para ello, en
primer lugar se exponen herramientas para evaluar la narrativa, su
Las técnicas de evaluación emocional son una forma importante estructura y su intensidad. En segundo lugar, se profundiza en la
de recopilar y medir información valiosa sobre aspectos evaluación de la narrativa desde el punto de vista del nivel de
cualitativos y cuantitativos de la experiencia de un usuario y van intensidad emocional. Finalmente, en el apartado 3 se exponen las
desde complejos cuestionarios hasta métodos más o menos conclusiones y el trabajo futuro.
invasivos.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 337 |
 La importancia de las emociones en el diseño de historias interactivas
J.R. López-Arcos | P. Paderewski Rodríguez | F. L. Gutiérrez Vela | N. Padilla-Zea | N. M. Fuentes García

2. Convertir la narrativa y su aspecto posibles instancias de una calidad pobre, los diseñadores podrán
tomar las medidas necesarias en una reestructuración de la historia
emocional en algo mesurable para que el jugador no pueda recibir estas malas historias.
Profundizamos en dicho análisis en los siguientes apartados.
2.1 Análisis de la historia
Los autores de la historia del videojuego y diseñadores de las 2.1.2 Análisis de la estructura narrativa de cada
mecánicas de juego deben definir los elementos de la historia instancia de historia
interactiva de tal forma que la experiencia de juego sea Con la estructura narrativa nos referimos a estructuras formales
satisfactoria a nivel narrativo. De este modo, el jugador recibirá tratadas en teorías clásicas sobre narrativa. Durante el diseño de la
una historia coherente y atractiva aunque la esté transformando historia, cada escena ha sido etiquetada con información referente
activamente por medio de sus acciones de forma interactiva. a esa estructura narrativa. Por tanto, como indicamos en [9], esa
Basándonos en nuestros trabajos previos [9] y para llevar a cabo
información puede ser extraída de cada instancia de historia para
este complejo diseño, proponemos las siguientes tareas: observar cómo está construida y conocer en qué grado cumple los
1. Definir las características de la experiencia que se cánones establecidos.
pretende diseñar. En [11] los autores proponen una serie de métricas y estadísticas
2. Elegir el tipo de historia correcta (género e importancia para abstraer y organizar esta información dada una historia.
de la narrativa en el juego). Algunas de ellas son: 1) contabilizar la ocurrencia de escenas
pertenecientes a cada etapa del Viaje del Héroe [12]; 2) calcular
3. Escribir un guión tradicional (Evolución Narrativa) qué porcentaje del tiempo que ocupa la historia pertenece a cada
como base para el diseño. uno de los actos de la estructura clásica narrativa de tres actos; 3)
4. Estructurar la historia en base a los modelos teóricos identificar eventos pertenecientes al núcleo de la historia y los
[8][9]. eventos satélite y contabilizarlos; 4) contabilizar el número de
personajes que correspondan a cada uno de los arquetipos
5. Realizar un guión esquemático de la historia. definidos por Vogler [12] y 5) calcular en qué porcentaje de cada
acto de la historia están presentes cada uno de esos personajes
6. Realizar un guión gráfico interactivo (o storyboard
arquetipo.
interactivo).
En nuestra propuesta, los diseñadores del videojuego pueden
7. Evaluar la estructura y la intensidad de la historia
diseñada, sus elementos y su efecto en la experiencia de obtener estas métricas y estadísticas automáticamente y comparar
juego. Para ello, es necesario tener en cuenta sus las instancias de la historia entre sí y con otras historias modelo.
posibles diferentes instancias. De este modo, se pueden detectar fácilmente posibles deficiencias
en la historia interactiva.
En el presente trabajo, nos centramos en la última de estas tareas.
Para llevarla a cabo, nos basamos en la estructura diseñada en la 2.1.3 Análisis de la intensidad de cada instancia de
tarea 4, que define las diferentes formas que una historia historia
interactiva puede adoptar dependiendo de cómo el jugador se Para poder medir la calidad de una experiencia de
comporte en ella. A éstas diferentes historias las llamamos entretenimiento, es necesario encontrar un elemento evaluable a lo
instancias [9]. Es necesario, por tanto, comprobar que cada una de largo de la duración de la experiencia.
ellas sea una historia coherente e interesante, y que proporcione
una experiencia de juego óptima. Para poder analizar y evaluar El interés despertado en el usuario invitado, en cada momento de
esta forma de narrativa, proponemos un nuevo conjunto de tareas la experiencia, puede ser evaluado y analizado. Gracias a ello es
posible generar “curvas de interés”. La Figura 1 muestra una
que podrán ser realizadas independientemente y de forma iterativa
típica curva de interés en una experiencia de entretenimiento
a lo largo del proceso de diseño:
óptima [7].
1. Generación de las posibles instancias de historia.
2. Análisis de la estructura narrativa de cada instancia de
historia.
3. Análisis de la intensidad de cada instancia de historia.
4. Evaluación de los elementos de la historia y de la
experiencia del jugador.
2.1.1 Generación de las posibles instancias de
historia
Al estructurar la historia de acuerdo al modelo conceptual
propuesto en [10], se genera un grafo que recoge todos los eventos Figura 1. Curva de interés típica.
que pueden ocurrir en la historia interactiva.
Estas curvas de interés son muy útiles para crear una experiencia
Obtener una selección de instancias interesantes de esa historia en de entretenimiento. Pueden compararse con otras curvas para
base al grafo construido, permite el análisis individual de cada una comprobar qué experiencia es más efectiva. Además, puede
de ellas. Por tanto, es posible detectar aquéllas instancias que compararse la curva de una experiencia real con la curva prevista
destaquen por su alta o baja calidad. En el caso de presentarse por el diseñador de la experiencia.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 338 |
 La importancia de las emociones en el diseño de historias interactivas
J.R. López-Arcos | P. Paderewski Rodríguez | F. L. Gutiérrez Vela | N. Padilla-Zea | N. M. Fuentes García
En el caso del método que proponemos, basamos el interés en la Respecto al primer enfoque, los autores pueden estimar en qué
intensidad emocional asociada a cada parte de la experiencia. El grado aparece cada emoción en cada parte de la historia que están
apartado 2.2 profundiza en el aspecto emocional y cómo medirlo construyendo. Al realizar la estructuración de la historia a la que
y analizarlo. se hace referencia en el apartado 2.1 y que se explica en [8], los
diseñadores pueden asignar esos grados de emoción a cada escena
2.1.4 Evaluación de los elementos de la historia y de o evento de la estructura de la historia. De este modo, de cada
la experiencia del jugador instancia de historia potencialmente generable se podrán obtener
Por último, proponemos que todos los elementos de la historia estos datos, analizarlos y compararlos para decidir si la historia
pueden ser evaluados por separado y dependiendo de la necesidad ofrece instancias buenas o con el carisma emocional deseado
del proceso de diseño. Es posible que el público al que va dirigido (Figura 2).
tenga características especiales o simplemente que los diseñadores
Sin embargo, cada jugador reacciona completamente diferente a
quieran asegurarse de que ciertas apuestas arriesgadas funcionan
correctamente. Para ello, proponemos la realización de una experiencia de juego. Además, en una historia interactiva, los
experiencias con usuarios mediante el uso de prototipos. eventos cambiarán de orden o el jugador puede estirar o acortar
deliberadamente el tiempo de la experiencia, perdiéndose en gran
A modo de ejemplo, en [13] presentamos una serie de medida el efecto emocional esperado por los autores. Para poder
experiencias para evaluar y así mejorar el diseño de los personajes estimar el impacto emocional real de una historia interactiva, es
de un videojuego educativo. Proponemos que estas experiencias necesario realizar experiencias con jugadores reales y observar los
en forma de pre-test, test y post-test pueden extenderse a la resultados.
evaluación de cualquiera de los elementos de la historia: partes
No obstante, medir las emociones de un jugador puede no ser una
concretas de la historia, intensidad emocional, personajes,
escenarios, objetos, diálogos o cualquier otro. tarea sencilla. Para este enfoque, proponemos dos técnicas
diferentes pero no excluyentes: a) solicitar a los jugadores de la
2.2 El uso de las emociones para medir el experiencia que rellenen un cuestionario sobre el impacto
emocional, y b) monitorizar las reacciones de los jugadores para
efecto de la historia en el jugador extraer datos sobre dicho impacto emocional.
En el apartado 2.1.3 hablamos de la importancia de realizar un
análisis de la intensidad de la experiencia de usuario en cada Respecto al uso de cuestionarios, proponemos hacerlo al finalizar
instancia de la historia. la experiencia de juego. El cuestionario a utilizar estará centrado
en los eventos más destacables de la experiencia o en los que sean
Debido a que, como se ha mencionado anteriormente, ese interés objeto de estudio. Además, debe ser un cuestionario adaptado a la
y la relación existente entre el jugador y la historia están muy experiencia particular y a lo jugadores involucrados. En las
ligados con el aspecto emocional, en la presente sección, experiencias descritas en nuestro trabajo previo [15], al ser los
proponemos enriquecer ese análisis mediante la medición de las sujetos del estudio niños de edad muy temprana, el cuestionario se
emociones concretas que participan en la experiencia. Aunque realizó de forma oral, y se usaron cartulinas con iconos que
existen innumerables aspectos y variantes a las reacciones representaban emociones fácilmente identificables por ellos.
emocionales que puede mostrar una persona, para simplificar,
proponemos regirnos por la lista de las seis emociones universales A diferencia de los niños, los participantes adultos pueden
humanas elaborada por Ekman [14]: alegría, aversión, ira, miedo, identificar distintos niveles para una misma emoción. Para ellos,
sorpresa y tristeza. proponemos usar un cuestionario impreso en el que para cada
escena o evento de la historia interactiva que sea objeto de
Dependiendo del tipo de historia y de los intereses de los autores, estudio, tengan que asignar un valor numérico a la intensidad de
será interesante centrarse en medir algunas de esas seis cada una de las emociones (Tabla 1).
emociones. Proponemos el análisis de curvas de intensidad
emocional específicas de las emociones seleccionadas. Sin
embargo, el problema reside en cómo es posible medir emociones. Tabla 1. Cuestionario sobre intensidad emocional de una
Para realizar una estimación del impacto emocional de la historia, parte concreta de la historia de una experiencia
proponemos dos enfoques: a) realizar una estimación por parte de
los autores, o b) aplicar técnicas de medición durante experiencias Nivel de intensidad de cada emoción
reales. Emociones 0 1 2 3 4
Alegría
Aversión
Ira
Miedo
Sorpresa
Tristeza

Figura 2. Valor de la intensidad de las seis diferentes De forma adicional, existe la posibilidad de monitorizar al jugador
emociones en siete eventos de la historia. e interpretar sus emociones de forma poco intrusiva. En [16], los
autores proponen un sistema con interacción emocional. En él,

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 339 |
 La importancia de las emociones en el diseño de historias interactivas
J.R. López-Arcos | P. Paderewski Rodríguez | F. L. Gutiérrez Vela | N. Padilla-Zea | N. M. Fuentes García
detectan las reacciones emocionales mediante el uso de una [6] Fred Charles, David Pizzi, Marc Cavazza, Thurid Vogt,
cámara y un software que detecta en la imagen del rostro del Elisabeth André. EmoEmma: Emotional Speech Input for
usuario signos de las seis emociones más la neutral. Para ello, Interactive Storytelling (Demo Paper) The Eighth
identifica puntos clave en el rostro del usuario y las variaciones en International Conference on Autonomous Agents and
la distancia entre ellos [17]. Este ejemplo es útil para comprender Multiagent Systems (AAMAS-Demos), Budapest, Hungary,
que es sencillo detectar las emociones que el jugador experimenta May 2009.
y comparar esos datos con los expresados por el propio jugador en [7] Schell, J. Understanding entertainment: story and gameplay
el cuestionario are one. Computers in Entertainment (CIE), 2005, 3(1), 6-6.
Estos datos sobre el impacto emocional de la experiencia de juego [8] Padilla-Zea, N., Gutiérrez, F. L., López-Arcos, J. R., Abad-
permiten estudiar este aspecto de la historia interactiva. Al Arranz, A., & Paderewski, P. Modeling storytelling to be
integrar este estudio dentro del proceso de desarrollo del used in educational video games. Computers in Human
videojuego se pueden usar esos datos para mejorar el producto y Behavior, 2014, 31, 461-474.
diseñar así una experiencia interactiva basada en una historia
compleja y satisfactoria para el jugador. [9] López-Arcos, J. R., Gutiérrez Vela, F. L., Padilla-Zea, N.,
Paderewski, P. A Method to Analyze Efficiency of the Story
3. CONCLUSIONES as a Motivational Element in Video Games. Proceedings of
En el presente trabajo se propone una serie de tareas que incluyen the European Conference on Games Based Learning, 2014,
la evaluación de la estructura de la historia y de su intensidad Vol. 2, p705.
emocional. En el aspecto emocional, se proponen una serie de [10] Padilla-Zea, N., Gutiérrez, F. L., López-Arcos, J. R., Abad-
técnicas para poder contabilizar estas emociones y así obtener Arranz, A., & Paderewski, P. Modeling storytelling to be
valores en base a unas métricas. Actualmente, estamos trabajando used in educational video games. Computers in Human
en experiencias para poner en práctica el uso del método. El Behavior, 2014, 31, 461-474.
objetivo es integrarlo en una metodología y, a raíz de esta, diseñar [11] Ip, B. Narrative structures in computer and video games: part
una herramienta automática para la construcción y la evaluación 2: emotions, structures, and archetypes. Games and Culture,
de la narrativa de un videojuego educativo. 2011, 6(3), 203-244
4. Agradecimientos [12] Vogler, C. The Writer's journey. Michael Wiese Productions,
Este trabajo está financiado por el Ministerio de Ciencia e 1998.
Innovación, España, como parte del Proyecto VIDECO [13] López-Arcos, J. R., Padilla-Zea, N., Paderewski, P.,
(TIN2011-26928), Red de Trabajo Iberoamericana de apoyo a la Gutiérrez, F. L., & Abad-Arranz, A. Designing stories for
enseñanza y aprendizaje de competencias profesionales mediante educational video games: A Player-Centered approach. In
entornos colaborativos y ubicuos (CYTED - 513RT0481) y el Proceedings of the 2014 Workshop on Interaction Design in
Proyecto de Excelencia P11-TIC-7486 financiado por la Junta de Educational Environments (p. 33). ACM. 2014.
Andalucía.
[14] Ekman, P. & Friesen, W. V. The repertoire of nonverbal
5. REFERENCES behavior: Categories, origins, usage, and encoding.
[1] A. Jacko, Julie; Sears, The Human-Computer Interaction Semiotica, 1969 1, 49–98.
Handbook, Fundamentals, Evolving Technologies and [15] Padilla-Zea, N., López-Arcos, J. R., Sánchez, J. L. G., Vela,
Emerging Application, Second Edi., 2008, vol. 20126252. F. L. G., & Abad-Arranz, A. A Method to Evaluate Emotions
CRC Press. in Educational Video Games for Children. Journal of
[2] Brave S. and Nass C. Emotion in Human-Computer Universal Computer Science, 19(8), 1066-1085. 2013.
Interaction.. in The Human-Computer Interaction Handbook. [16] Cerezo, E., Baldassarri, S., & Seron, F. Interactive agents for
Fundamentals, Evolving Technologies and Emerging multimodal emotional user interaction. In IADIS Multi
Applications., Second., vol. 29, no. 2, A. Sears and J. A. Conferences on Computer Science and Information Systems
Jacko, Eds. Standford University: Lawrence Erlbaum (pp. 35-42). 2007.
Associates, 2008, p. 1386.
[17] Cerezo, E., & Hupont, I. Emotional facial expression
[3] D. A. Norman, Emotional Design: Why We Love (or Hate) classification for multimodal user interfaces. In Articulated
Everyday Things, vol. 2006, no. 2. Basic Books, 2004, p. Motion and Deformable Objects (pp. 405-413). Springer
272. Berlin Heidelberg. 2006.
[4] M. Hassenzahl and N. Tractinsky, User experience - a
research agenda, Behav. Inf. Technol., vol. 25, no. 2, pp. 91–
97, Mar. 2006.
[5] Tarpy, Roger M. (2003). Aprendizaje: teoría e investigación
contemporáneas. McGraw Hill.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 340 |
 Agente virtual emocional para dispositivos móviles
S. Baldassarri | E. Cerezo

Agente virtual emocional para dispositivos móviles

Sandra Baldassarri Eva Cerezo
GIGA Affective Lab GIGA Affective Lab
Universidad de Zaragoza Universidad de Zaragoza
Zaragoza Zaragoza
+34976762357 +34976762356
[email protected] [email protected]

RESUMEN realizar por dos canales, el oral y el visual, y aunque considera

El rápido aumento de las capacidades computacionales de los emociones, no lo hace en todos los canales.
dispositivos móviles ha posibilitado el desarrollo de aplicaciones En este trabajo presentamos un agente virtual para dispositivos
que permiten la interacción multimodal con los mismos en tiempo móviles que permite la interacción con el usuario por los canales
real. Estos sistemas se suelen basar en el uso de agentes virtuales oral, escrito y visual, teniendo en cuenta la expresión de las
que facilitan la comunicación entre usuarios y servicios. Este emociones en todos los canales.
trabajo presenta una aplicación basada en agentes que permite la
interacción multimodal y emocional con los usuarios a través de 2. INTERACCIÓN CON EL AGENTE
los canales visual, oral y escrito. El sistema ha sido evaluado para VIRTUAL
comprobar percepción de los usuarios sobre la capacidad del En la Figura 1 se presenta la arquitectura del sistema desarrollado,
personaje en expresar las emociones. describiendo posteriormente en detalle cada uno de los módulos
que intervienen.
Categories and Subject Descriptors
H.5.2 [User Interfaces]: Graphical users interfaces (GUI), Input 2.1 Comunicación oral
devices and strategies, Interaction Styles, Evaluation El módulo de comunicación oral incorpora un Reconocedor de
Discurso, basado en la aplicación Google Voice Search, que se
General Terms encarga de captar el mensaje hablado del usuario a través del
Design, Human Factors. micrófono del dispositivo móvil, convertirlo a cadena de texto y
transmitirlo al módulo Gestor de Diálogo, y un Sintetizador de
Keywords Voz, basado en el sistema TTS Android, encargado de la
Personajes virtuales, interfaces emocionales, interacción táctil, conversión a voz de las respuestas en formato texto procedentes
dispositivos móviles del Gestor de Diálogo. Este sintetizador ha sido dotado de voces
emocionales con las que pronunciar el mensaje. La reproducción
1. INTRODUCCIÓN del discurso se lleva a cabo a través del altavoz del dispositivo.
La gran proliferación de aplicaciones móviles de hoy en día ha
dado lugar a que la interacción a través del teclado se utilice cada 2.1.1 Reconocedor del discurso
vez menos, dando paso a aplicaciones totalmente táctiles. En estos Entre las distintas posibilidades existentes para realizar el módulo
casos, la interacción generalmente se realiza a través de gestos Reconocedor el discurso del usuario en dispositivos móviles
con los dedos o a través de la voz. Sin embargo, debido al Android, dos opciones destacan sobre el resto: utilizar el sistema
continuo aumento tanto de la capacidad de procesado como de la de reconocimiento de voz PocketSphinx, específico para sistemas
memoria de los dispositivos móviles, empiezan a aparecer embebidos y dispositivos móviles o realizar peticiones de
sistemas basados en agentes virtuales desarrollados para ser reconocimiento de voz a una aplicación especializada instalada en
ejecutados sobre estos dispositivos. Una de las características más el dispositivo móvil. Una vez analizadas ambas opciones, se opta
importantes de los agentes virtuales es que deben ser capaces de por la segunda opción, para reducir los requerimientos de
interactuar con el usuario en tiempo real, y de forma multimodal, procesado del sistema y porque ofrece mejor calidad en el
teniendo en cuenta la interacción a través de la voz, la escritura y reconocimiento del discurso, lo que permite dotar de mayor
agentes animados. realismo a la interacción oral con el usuario, a pesar de que sea
En los últimos años, ya se ha estado trabajando en el uso de necesaria una conexión a internet.
agentes virtuales en aplicaciones para móviles. Una de las
primeras aplicaciones desarrolladas ha sido un presentador virtual
encargado de narrar los titulares de las noticias más importantes
del momento [1]. También se han utilizado como guía durante el
entrenamiento del usuario, ayudando a realizar correctamente
ejercicios de fitness [2], o como intérpretes, transmitiendo
mediante el lenguaje de signos los mensajes escritos por los
usuarios [3]. Existen también plataformas para la especificación
de agentes virtuales que soportan interacción en dispositivos
móviles, como Elckerlyc [4] que permite interactuar de forma
oral, textual y visual, pero sin considerar las emociones; o la
desarrollada para Android por [5] donde la interacción se puede

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 341 |
 Agente virtual emocional para dispositivos móviles
S. Baldassarri | E. Cerezo
del agente pueda tener matices emocionales. Se decide trabajar
con cinco emociones: alegría, enfado, sorpresa, tristeza y neutra.
Para ello, se procede a modificar los tres parámetros que el
sistema TTS de Android permite manipular: volumen, velocidad y
tono. Tras estudiar varios trabajos relacionados con la expresión
de emociones a través de la voz [6] [7] y realizar pruebas con 48
usuarios que se detallan en [8], los valores seleccionados para
cada voz emocional generada se presentan en la Tabla 1.
Tabla 1: Valores de volumen, velocidad y tono para cada
categoría de voz emocional
Voz Voz Voz Voz Voz
Trist Enfadad Neutr Alegr Sorprendid
e a a e a
Volumen 9.28 12.8 10.7 12.6 13.6
Velocida
0.8 1.6 1.0 1.3 1.2
d
Tono 1.3 1.2 1.7 2.1 2.3

2.2 Comunicación Escrita

Este módulo, paralelo al anterior, se encarga de gestionar la
interacción con el usuario a través del canal escrito. Dispone de un
Módulo de Lectura cuya función es trasladar los mensajes del
Área de Texto, escritos por el usuario a través del teclado del
dispositivo, al módulo Gestor de Diálogo. Además, dispone de un
Módulo de Escritura encargado de mostrar al usuario las
respuestas procedentes del Gestor de Diálogo a través de un
Panel Deslizable que se encuentra en la Interfaz Gráfica.
Con el objetivo de que el uso de la comunicación escrita del
sistema no implique pérdida de información con respecto a la la
comunicación oral, el sistema permite que agente exprese su
estado emocional a través del color de la fuente y de los bordes
del panel deslizable de los mensajes de texto. Los colores
Figura 1. Diseño del sistema
seleccionados para cada emoción son: Neutro: Azul, Alegría:
De entre las posibles aplicaciones de reconocimiento de voz Verde, Enfado: Rojo, Tristeza: Gris y Sorpresa: Amarillo [9] [10].
disponibles para Android: Voice Action Plus, Cyberon Voice
Commander, VoicePOD, Konele, Vlingo y Google Voice Search, 2.3 Gestor de Diálogo
se selecciona esta última ya que puede ser instalada de forma El Gestor de Diálogo se encarga de recibir los mensajes
gratuita, viene integrada en los dispositivos móviles Android más provenientes del usuario, interpretarlos y generar una respuesta
modernos, y es uno de los mejores reconocedores del habla acorde a los mismos. Para ello recibe la información obtenida
disponibles, siendo compatible con varios idiomas. tanto del Reconocedor de Discurso como del Módulo de
Lectura y envía las consiguientes respuestas al Sintetizador de
2.1.2 Sintetizador de voz Voz y al Módulo de Escritura. Para el desarrollo de este módulo
El módulo Sintetizador de Voz recibe las cadenas de texto que se estudiaron diferentes alternativas: Episteme Engine, uso de
proceden del Gestor de Diálogo con las respuestas para el usuario diccionario, de listas, de tablas hash, árboles, expresiones
y las reproduce a través del altavoz del dispositivo móvil. regulares, y el Programa AB. Una vez analizadas, se opta por el
Para lograr este objetivo es posible desarrollar un motor de Programa AB [11], ya que es de libre distribución y permite una
síntesis de voz propio que sea compatible con Android o hacer gestión del diálogo más compleja y realista. Esta gestión se lleva a
uso del motor de síntesis de discurso (Text To Speech o TTS) que cabo mediante ficheros AIML (Artificial Intelligence Markup
incorpora la plataforma Android. Se opta por esta última opción Language) con información de forma estructurada, lo que permite
que minimiza los requerimientos de procesado del sistema y que el desarrollador la comprenda y modifique fácilmente.
utiliza la aplicación de síntesis de voz seleccionada por defecto en
cada dispositivo móvil. Además, soporta múltiples idiomas y
2.4 Módulo motor
permite modificar ciertos parámetros del discurso como volumen, La función de este módulo consiste en gestionar las animaciones
tono y velocidad. Sin embargo, esta opción presenta el que incorpora cada uno de los personajes 3D utilizados en el
inconveniente de que no aporta información temporal ni para la sistema. Este módulo recibe del Gestor de Diálogo la
reproducción de los visemas, por lo que es imposible realizar información que se utiliza para establecer y transmitir a través de
sincronización labial. las animaciones reproducidas sobre el personaje el estado
emocional en que se encuentra el agente. Para la implementación
2.1.3 Expresión de emociones a través de la voz de este módulo se ha utilizado un Plugin de Unity 3D [12].
Para dotar de mayor realismo y naturalidad al proceso de
interacción oral con el agente, se considera necesario que la voz

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 342 |
 Agente virtual emocional para dispositivos móviles
S. Baldassarri | E. Cerezo

2.5 Interfaz Gráfica

La interfaz gráfica permite al usuario interactuar con el sistema a
través de la pantalla táctil del dispositivo y se encarga de iniciar
tanto el proceso de escucha del discurso del usuario como la
lectura de los mensajes de texto, gestionando a su vez el nivel de
volumen y los distintos modos de interacción del sistema. El
elemento principal de la interfaz gráfica es el agente virtual, cuya
representación gráfica se realiza a través de distintos personajes
tridimensionales (ver Figura 2).

Figura 3. Resultados obtenidos por cada una de las voces

emocionales.
Es importante reseñar que, en el caso de la voz emocional de
alegría, la segunda opción más votada es la de sorpresa, aunque
con un porcentaje muy inferior al obtenido por la opción alegre en
el caso de la voz emocional de sorpresa. Por su parte, sorprende
que la segunda opción más votada para la voz de enfado sea la
neutra, ya que no se habían detectado confusiones notables entre
ambas en las pruebas previas. Finalmente, la voz menos lograda
es la de sorpresa, siendo reconocida por menos del 50% de los
usuarios. Este hecho, unido a que aproximadamente un tercio de
Figura 2. Interfaz gráfica: modelo 3D de niño (izquierda) y los usuarios se decanten por el estado emocional de alegría al
modelo 3D de una mujer-robótica (derecha) escucharla, confirma las dificultades ya detectadas en pruebas
anteriores para generar voces emocionales de alegría y sorpresa
3. EVALUACION DE LA EXPRESIÓN que no conlleven confusión entre sí.
EMOCIONAL DEL AGENTE 3.2 Influencia del contenido semántico
En este apartado se presenta la evaluación de la expresividad Además de la calidad de las voces emocionales, se ha considerado
emocional del agente con el objetivo de determinar la calidad de interesante estudiar la influencia del contenido semántico de las
las voces generadas y su relevancia con respecto a otros dos frases reproducidas en la percepción emocional del usuario
aspectos del sistema que denotan emociones en el agente: el durante el discurso del agente. Con este fin, se ha llevado a cabo
contenido semántico del discurso y las animaciones del personaje. la segunda prueba seleccionando un conjunto de cinco frases con
En esta evaluación han participado 48 usuarios, siendo la mayoría las distintas connotaciones emocionales.
alumnos universitarios. La metodología seguida en todas las Inicialmente se mezclan las distintas voces emocionales
sesiones ha sido: realización de un Pre-test, para definir el perfil modeladas con las frases con connotación emocional
de los participantes, un Test compuesto por las Pruebas 1, 2 y 3, seleccionadas previamente. Para ello, se ordenan de forma
que se describen a continuación, y un Post-Test para determinar la aleatoria tanto el conjunto de las veinte voces emocionales como
opinión de los usuarios acerca de las pruebas realizadas. el conjunto de las veinte frases a reproducir, procediendo
posteriormente a la combinación de las voces y frases que se
3.1 Reconocimiento de las voces emocionales encuentran en la misma posición dentro de sus respectivos
La primera prueba se realiza con el fin de evaluar la capacidad de conjuntos, asegurando que, al menos una vez, la voz y frase
los usuarios para reconocer el estado emocional en que se correspondientes a un mismo estado emocional se reproduzcan
encuentra el agente únicamente a través de la voz emocional simultáneamente. Tras haber llevado a cabo la mezcla, se
escuchada. Para esta prueba, realizada a través de una encuesta de reproduce la combinación resultante a los usuarios e instándoles a
elección libre modificada, se toman las siguientes medidas: no se determinar qué estado emocional perciben en cada caso. Al igual
hace uso de imagen, se llevan a cabo cuatro repeticiones de cada que en la prueba anterior se toman las siguientes medidas: no se
reproducción, con orden aleatorio y se separan las reproducciones hace uso de imagen, se llevan a cabo cuatro repeticiones de cada
con 10 segundos de espera. reproducción, el orden es aleatorio y se separan las
En la Figura 3 se muestra el porcentaje de respuestas obtenido por reproducciones con 10 segundos de espera.
las distintas voces para cada categoría de voz emocional. Con un porcentaje medio de acierto del 88.89% en las
Como se puede apreciar, las voces emocionales neutra y triste reproducciones formadas por voces y frases emocionales acordes,
obtienen resultados notablemente satisfactorios, puesto que son se confirma que los usuarios encuestados perciben el estado
reconocidas por más del 80% y 70% de los usuarios encuestados, emocional de forma más sencilla y acertada en el caso de que
respectivamente. Las voces emocionales de alegría y enfado ambos factores emocionales correspondan a una misma emoción.
presentan resultados moderadamente aceptables, siendo El porcentaje de aciertos obtenido (ver Tabla 2) supera en todos
reconocidas por aproximadamente el 60% de los usuarios. los casos el 70%, alcanzando tasas cercanas al 98% de acierto en
los pares emocionales de enfado y neutro. En general, los
resultados son altamente satisfactorios menos en los pares

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 343 |
 Agente virtual emocional para dispositivos móviles
S. Baldassarri | E. Cerezo
emocionales sorprendido y alegre, cuyas voces son las menos para aumentar el realismo y la naturalidad en las voces generadas,
logradas. desarrollando mecanismos que permitan modificar los valores de
Tabla 2: Porcentaje de acierto obtenido por la conjunción de voz y los parámetros a lo largo de la reproducción del discurso, dando
contenido de la frase para una misma emoción lugar a distintas entonaciones dentro de una misma frase o
Porcentaje Porcentaje Porcentaje Porcentaje Porcentaje pudiendo enfatizar determinadas partes o palabras del discurso del
Aciertos del par Aciertos del par Aciertos del par Aciertos del par Aciertos del par
Sorprendido Neutro Enfadado Triste Alegre agente.
75.00% 97.91% 97.92% 94.79% 72.92% 5. AGRADECIMIENTOS
Las autoras agradecen el trabajo realizado por Jorge Barriendos
Sanz. Este trabajo ha sido parcialmente financiado por el gobierno
3.3 Relevancia de la imagen con respecto a la de España a través del contrato DGICYT TIN2011-24660 y por el
voz proyecto CYTED 512RT0461.
Finalmente, la prueba 3 consiste en estudiar la relevancia que
adquiere la imagen del agente con respecto a las voces 6. REFERENCIAS
emocionales modeladas a la hora de que el usuario reconozca un [1] Santi S., Koki U., Mitsuru I. 2003 Multimodal Presentation
estado emocional. Para esta prueba, se utiliza el torso de la mujer- Markup Language on Mobile Phones. In Proceedings of the
robótica como representación gráfica del agente de la Figura 2 Fourth International Workshop IVA, 226-230
(derecha) ya que las animaciones faciales y corporales que [2] Buttussi F., Chittaro L., Nadalutti D. 2006. Bringing mobile
incorpora para representar estados emocionales son de mayor guides and fitness activities together: a solution based on an
complejidad y realismo [13]. embodied virtual trainer. In Proceedings of Conference on
Inicialmente se mezclan las distintas voces emocionales Human-computer interaction with mobile devices and
modeladas con las animaciones de los estados emocionales. Para services, MobileHCI’06, 29-36
ello, se ordenan de forma aleatoria tanto el conjunto de las veinte [3] Buttussi F., Chittaro L., Coppo M. 2007. Using Web3D
voces emocionales como el conjunto de las veinte animaciones a technologies for visualization and search of signs in an
reproducir, asegurando que, al menos una vez, la voz y animación international sign language. In Proceedings of the
correspondientes a un mismo estado emocional se reproduzcan International Conference on 3D Web Technology, 61-70.
simultáneamente. Una vez realizada la mezcla, se reproduce la
combinación resultante y se insta a los usuarios a determinar qué [4] Klaassen R., Hendrix J., Reidsma D., Akker R., Dijk B.,
estado emocional perciben en cada caso. De este modo, a través Akker, H. 2013. Elckerlyc goes mobile-Enabling natural
de la comparación de los resultados obtenidos con los resultados interaction in mobile user interfaces. International journal on
advances in telecommunications, 6 (1 & 2), 45-56.
de la primera prueba, se pretende conocer la influencia de la
imagen del agente virtual en la percepción emocional del usuario. [5] Santos-Pérez M., González-Parada E., Cano-García J.M.
En este caso, se hace uso siempre de la frase neutra con el 2013. ECA-based Control Interface on Android for Home
objetivo de que el usuario no se vea influenciado por el contenido Automation System. IEEE International Conference on
de la frase. Consumer Electronics (ICEE 2013), 70-71.
Con un porcentaje medio de acierto del 92.5% en las [6] Francisco, V., Gervás, P., Hervás, R. 2005 Expresión de
reproducciones formadas por animaciones y voces emocionales emociones en la síntesis de voz en contextos narrativos.
acordes, es posible afirmar que los usuarios encuestados perciben Primer Simposio sobre Computación Ubicua e Inteligencia
el estado emocional de forma más sencilla y acertada en el caso de Ambiental (UCAmi'05),
que ambos factores emocionales correspondan a una misma [7] Baldassarri S., Cerezo E., Anaya D. 2009. Emotional speech
emoción (ver Tabla 3). synthesis in Spanish for natural interaction. In New Trends
Tabla 3: Porcentaje acierto obtenido por la conjunción animación on Human–Computer Interaction, 151-160. Springer.
y voz correspondientes a una misma emoción [8] Barriendos J. 2014, Desarrollo de un agente virtual
Porcentaje Porcentaje Porcentaje Porcentaje Porcentaje
Aciertos del par Aciertos del par Aciertos del par Aciertos del par Aciertos del par multimodal para dispositivos móviles, Proyecto Fin de
Sorprendido Neutro Enfadado Triste Alegre
Grado, Universidad de Zaragoza.
89.58% 93.75% 95.83% 100% 83.33%
[9] Kaya N., Epps H.H. 2004. Relationship between color and
emotion: a study of college students. College Student
A pesar de haber cosechado excelentes resultados en todos los Journal, Volumen 38, 396-405
casos, se vuelve a repetir la circunstancia de que los pares [10] Colores y Emociones,
emocionales peor evaluados sean sorprendido y alegre, cuyas http://elsindromedelahojaenblanco.wordpress.com/2012/11/1
voces emocionales son las menos logradas. 9/colores-y-emociones/ (último acceso 03-2015)
4. CONCLUSIONES [11] Program AB - Reference AIML 2.0 Interpreter,
En este artículo se ha presentado una aplicación basada en agentes https://code.google.com/p/program-ab/ (último acceso 03-
virtuales que permite mejorar la comunicación entre usuarios y 2015)
sistema móvil a través de la interacción multimodal y emocional [12] Unity Game Engine, http://unity3d.com/es (último acceso
por medio de agentes virtuales. El sistema ha sido evaluado a 03-2015)
través de encuestas y entrevistas a usuarios finales y, aunque los Baldassarri S., Cerezo E., Serón F.J. 2008. Maxine: a platform for
resultados son alentadores, en el futuro se pretende trabajar en embodied Animated Agents. Computers & Graphics vol. 32(4),
ampliar la funcionalidad del módulo de síntesis de voz del sistema 430-437.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 344 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

Second Mind: A System for Authoring Behaviors in Virtual

Worlds
Manish Mehta Andrea Corradini
Accenture Technology Labs Design School Kolding
San Jose, CA, USA Kolding, Denmark
[email protected] [email protected]

ABSTRACT dialogue where the player can engage in a compelling

In this paper, we present a design solution that enables novice conversation with NPCs that results in witty and dramatic
users to author the behaviors of non-playing characters in virtual storylines that fit the contex. In those games, a conversation is
worlds. We discuss the current authoring interface and the typically modelled using dialogue trees [1] where the player is
activities that the users need to conduct in order to generate presented choose from a limited list of interaction options in
behaviors. We also outline the system architecture, which we response to an NPC’s action. Each choice affects the course of the
called Second Mind, with focus on the behavior execution system game in a different way since each of them may result in a
and the interface layer that connects it to the external virtual different response from the NPC. Moreover, the possibility to
world. A series of user studies, show that our system can be choose between different options makes it necessary for the game
successfully used by novice users to produce reliable behaviors designer to customize the NPCs’ behaviors by modifying their
while also improving the efficiency of the authoring task. default scripts or creating entirely new ones.
Branching the overall storyline puts the player in charge of
Categories and Subject Descriptors learning about the game by engaging him in conversation with
[D.2.2 Design Tools and Techniques]: User interfaces; [H.1.2 NPCs. In this way, uncovering the storyline becomes part of an
User/Machine Systems]: Human information processing; [H.5.2 increased overall player experience [2]. Besides engaging
User Interfaces]: User-centered design. interactive narrative experiences [3] with unfolding storylines, it
is necessary to have the tools that make it possible to design and
General Terms create the appropriate behaviors. This is not just about making
Design, Experimentation, Human Factors. them funnier. It is also about creating more surprising and
unpredictable behaviors within the game constraints so that no
Keywords two interactions with a NPC turn out to be the same.
Authoring tools; behavior generation; virtual worlds; non-playing
characters; Second Life; Second Mind. Digital games have much to gain from adopting AI techniques to
author complex behaviors. This is however, a difficult task for it
1. INTRODUCTION requires a great deal of competences and resources. Usually, only
In order to increase player experience, the virtual characters that the game creators or people with specialized skills in both design
populate video games and virtual worlds should exhibit believable and programming can carry out such a task. As a result, authoring
behaviors. This does not apply only to the player characters, behavior can even become a bottleneck in the process of creating
which are the characters that are controlled by the player, and as the content displayed within digital worlds.
such are meant as more of an extension of the player. This also In this paper, we present Second Mind, a digital authoring tool
applies to any other character controlled by the game, usually solution that helps any players to populate virtual worlds with
through artificial intelligence: townsfolk, vendors, enemies, allies, NPCs which can exhibit believable interactive behaviors. Our
historical figures, bystanders, quest givers, etc. These characters solution is based on a previous study [4] carried out using paper
are referred to as non-player characters (NPCs) and are an integral prototypes to investigate how individuals without programming
part of the game. They exist to interact with the player characters, and design experience (who we refer to as “novice users”
other NPCs, and the environment in a way that reflects their own throughout this paper) carry out the behavior authoring process.
distinct personalities and dialog capabilities. The behaviors of That study revealed some major limitations that we accounted for
NPCs are usually scripted and are automatically triggered by in the development of Second Mind. In Second Mind, novice
certain actions, events, or dialogue. In that sense, player users can easily design the behavior of graphical characters that
characters play the game, while NPCs display some facet of the impersonate different tradesmen such as shopkeepers, museum
game and help to further the storyline. hosts, etc. in virtual worlds.
In early games, NPCs displayed only basic non-interactive Following this introduction, we have organized the rest of the
dialogue capabilities. Monologues realized with screens of text, paper as follows. In the next section, we discuss related work,
floating text, voiceovers, cut-scenes, text clouds and other non- which has influenced our current system. Thereafter, we provide
branching dialogue techniques make it possible for NPCs to an overview of Second Mind’s architecture and processing flow.
convey only an immediate impression of their personality in Eventually, we conclude with an outline of the work in process
reaction to or interact with the player. However, the lack of and our ideas for further improvement.
interactivity turns the player into a mere passive consumer of
content. More recently, advanced games allow for interactive

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 345 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

2. RELATED WORK 3. THE SECOND MIND SYSTEM

Believable characters play an important role in many interactive
stories and digital games [5] because they can contribute to the
3.1 Rationale and Context
In a previous study [4], we explored a design solution that would
overall narrative. Thus, the improvement and the control of their
enable novice users to successfully author AI behaviors. That
behaviors is propaedeutic to better game experience for players.
study provided, among others, the following key insights and
Common methods for authoring behaviors rely on heavy design directions for creating the authoring interface:
scripting, where a blend of dialogue scripts, hand-coded animation
scripts, motion-captured characters, and hard-coded behaviors is • The process to link behavior constraints to lower
used to exhibit a character's features. Usually, artists, designers level percepts must be as simple as possible; the
and programmers must work together to detail a character’ conditions themselves must be limited, clearly
attitude and to portray the appropriate character's movements understandable and should be expressed with an
given a set of constraints. However, scripting is labor intensive, it easy-to-use formalism
is rigid because it does not adapt to all variations induced by
interaction, and it requires programming and design skills. An
• The authoring task should be presented as a
alternative approach is to use artificial intelligent (AI) techniques sequential process and not rely on deep
to adapt character behaviors in response to the different contexts hierarchical structures
that are induced by different interaction sessions. Despite its own • Authors need continuous feedback on the authoring
inherent limitations, AI technology has been increasingly applied activity so that they can see the results of their ongoing
with success to digital game design and interactive narrative both process and thus promptly identify possible problems
to enhance interaction and to create novel gameplay scenarios [6].
AI techniques have been explored with diverse focus on character
believability and expressivity in terms of e.g. synchronization of
verbal and non-verbal modalities [7, 8], portrayal of emotion and
personality [9, 10].
The ease of the actual process of behavior-creation has been
explored by few researchers. The language ABL [11] is an early
effort in this direction. ABL is a Java-like programming language
that can be used to manually script behaviors [12, 13]. The
production costs are high and the use of ABL requires advanced
programming skills. Finite State Machines (FSMs) have also been
used in digital games for many years as an alternative to scripting.
However, FSMs can quickly become difficult to maintain and do
not scale well with increasing game complexity [14]. Various
FSMs modifications have been proposed to bypass part of these
problems [15, 16]. The use of AI techniques such as Layered
Statechart-based AI [17], Behavior Trees [18], automated
planning and scheduling strategies [19, 20, 21], has been explored Figure 1. Authoring a greeting behavior as a left-to-
as well. All these works require human intervention and a great right sequence of basic actions “talk (text)”, "salute"
deal of effort and skills to create and to debug the scripted and “talk (text)” on the timeline.
behaviors. A recent attempt at eliminating manual scripting in
favor of automatic script generation is presented in [22] where the
content creation for cyclic behaviors is facilitated using a tiered On the one hand, reducing the set of conditions takes away some
behavior architecture model. This model features a scheduling of the expressivity of the system, but on the other hand, it allows
algorithm to determine the objectives of virtual characters and to the authoring process to be kept simple. Conditions help the
specify the roles that satisfy these objectives dynamically during avatar identify when, what, and how a certain behavior must be
game play. The results of a user study that compares the creation performed.
of manually scripted behaviors with the behaviors generated by an We express conditions using what we refer to as “triggers”. We
implementation of their model are also presented. Three metrics, provide a set of triggers that can be used individually or combined
and notably behavior completion time, behavior completeness, resulting in more complex constraints. An example of a trigger is
and behavior correctness, are used as a measure for assessing "User Clicks on (button label)", which can be used when the
behavior reliability and designer efficiency. player wants the avatar to perform a behavior in response to a
Unlike most of the works presented previously, our solution button click (“button label" is a placeholder that must be provided
provides a graphical user interface with different constraint types. during the authoring process). Many other triggers are also
While a few graphical interfaces have also been put forward to provided such as "Avatar Close to (object name)", "Avatar
help people create scripts, these tools are aimed at story designers Touches (object name)", "Player Close to (object name)", "Player
and at people who want to learn to program. Our implementation Touches (object name)", "Behavior Ends (behavior name)",
targets any person who wishes to create behaviors with no "Pause after Behavior Ends (behavior name, time)”, "Run Event
prerequisites in terms of skills and experience. (event name)", and so on. Triggers can be composed using logical
operators. For instance, a "Trigger And (A, B)" can be used to
specify a behavior for the avatar to perform when both trigger A
and trigger B are specified.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 346 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

In order to present the behavior creation task as a sequential constraint to apply to the scene in the shopkeeper context could be
process, we applied the notion of timelines. Timelines indicate a "likeable" or "salesman skills".
left-to-right sequential sequence, where each event occurs in
chronological order. Various storyboard-authoring tools like Beside the contextual constrains, all scenes are associated by
Adobe Director and Apple's Garageband Timelines as well as AI- default with two additional general constraints, namely "overall
based interactive story authoring tools like Scribe [23], U-Create experience" and "avatar performance". The author uses a set of
[24] and Viper [25] employ timelines as an interaction design sliders to specify matching values on a 0 to 5 point scale for each
approach. We use timelines to represent the ordering of story of the constraints s/he wants to set. Moreover, the user can
segments in a left-to-right sequence while also allowing story associate semantic tags to the scene. Now, avatar behaviors can be
segments to be re-ordered on the timeline. In order to help the added within the scene.
user conceptualize the task concretely, the authoring activity Similarly to the scene creation process, the author can create a
along the timeline is presented in a mind-like shape (Figure 1). complex behavior from a list of simple elementary behaviors. To
Eventually, in order to comply with the need to provide a define a new behavior, the author needs to enter a name that
continuous feedback to the user, we connect our authoring uniquely identifies it, followed by the list of behaviors that make
environment to an online virtual world named Second Life [26] it up and by the specification of the constraints that characterize
where avatar’s behaviors can be executed and displayed the behavior as a whole.
graphically in 3D. In Second Life, users can create, design and
build their own clothes, buildings, artifacts, landscape, pets and
much more. We chose Second Life as platform to display 3D-
based user-generated content because it has a large user base, it
allows for easy connection to external controllers through an API,
and it provides a rich set of basic actions for the avatar in the form
of emotions and physical actions. Using our authoring interface
Second Mind, authors can create behaviors, execute them in
Second Life, and immediately see the effects of their authored
artifacts.

3.2 The Authoring Process

Figure 3. The player interaction with an avatar can be

initiated by clicking on the device carried by the avatar on
his left shoulder; clicking on the device causes a graphical
interface to pop up with various scenes to choose from

Using a set of sliders, the author can specify the values for the
constraints. For example, in order to have an "enthusiastic" and
"pushy" shopkeeper for a "greeting behavior", the user has to
select first the corresponding values for those two constraints that
give an idea of the avatar personality. These values influence the
way (e.g. speed, spatial extension, etc.) a certain behavior is
rendered. The user can then assign triggers to the behavior in
Figure 2. Initial steps for the creation of a scene. order to specify the context in which it should be carried out by
In order to use Second Mind, the user needs to create an account the avatar. The process is iterative because more complex
where to store, among other things, his/her Second Life login behaviors can be created as combination of both single ans
credentials. Once the account is created, the user is guided complex behaviors. During the creation of a specific behavior, the
through a tutorial, which explains the terminology used in the author can also make use of a constraint, which forces the avatar
authoring interface (like triggers, scenes, behaviors, actions, etc.). to initiates the interaction. Once a behavior is created, the user can
The user then can define one or more characters in Second Mind immediately play it out in Second Life and test its 3D graphical
and connect each of them to a different avatar in Second Life. representation.

At this stage, the user is ready to create a scene and its In order to interact with an avatar in Second Life, the player must
corresponding behaviors. A scene provides a social context such click on an attachment on the avatar's left shoulder. Upon
as a furniture shop where the avatar acts as a shopkeeper whose clicking, the player is presented with a GUI containing buttons
goal is to sell furniture items. The author creates a description of labeled with the names of the scenes that have been previously
the scene by entering some information such as the name and authored for the avatar (Figure 3). Each of these buttons
some details and then selecting one of the characters created in represents one "User Clicks on (button label)" trigger that is
earlier steps (Figure 2). The user can also associate a limited set of associated with the behaviors of a scene. To interact with the
constraints to the avatar in the given scene. For instance, a avatar, the player must simply click on a button.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 347 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

3.3 The System Architecture

Figure 5. Overview of Behavior Execution Engine

architecture e
The behavior execution engine can be correspondingly broken
down into three main modules:
a) Behavior Library: it is part of the Second Mind database
that holds behavior elements for a specific application domain. For
example, if the application domain were Second Mind in an
educational setting, the library would contain personalities
Figure 4. Overview of Second Mind overall architecture
corresponding to teachers, students, etc. If instead the application
With a basic understanding of the authoring activity, we can delve domain were a household robot, the library would contain
into the system architecture. As shown in Figure 4, Second Mind personality elements corresponding to cooking food, cleaning the
architecture has four core components and one application- house, etc.
specific component. The four core components are: b) Behavior Retrieval: it is responsible for loading all the
behaviors corresponding to the active scene from the behavior
• Second Mind Database: a MS-SQL server based
library. The active scene is the scene that the player selected when
database that stores scenes, players, behavior s/he was presented with all possible authored scenes associated
elements such as triggers, basic action, etc. with a certain avatar.
• Behavior Execution Engine: an execution engine c) Execution Engine: it receives the currently active domain
that loads the behavior elements and executes them independent sensors from the interface layer and checks the
in the external world. The behavior execution triggers for the behaviors corresponding to the active scene to see
engine connects to the external world through the if any of the behaviors were activated. It also takes the current
interface layer that is specific to the world that active behavior and sends it through the interface layer to the
Second Mind is connecting to. virtual world for the avatar to perform it.

• Graphical Interfaces: provides the interfaces for

3.3.2 Interface layer
creating user account, scenes and behaviors as 6.1.1 The Interface layer parses the domain dependent perceptual
described in the previous section. state into a set of domain independent sensors. The domain
dependent perceptual state from Second Life contains the
• Behavior Recommendation Engine: provides following items:
suggestions during the behavior-authoring task
a) Object Info: contains the information about various
In order to apply Second Mind to a given graphical world, one objects in the environment such as their current position, rotation,
application-specific component is required: type (whether an avatar or a passive object), region to which the
object belongs, etc.
• Interface Layer: The architecture of Second Mind is
domain-independent and can be connected to different b) User Button Click Info: contains information regarding
virtual world simulations via a custom interface layer. button clicks made by the player while interacting with the avatar.
The interface layer provides the ability to connect c) Event Info: contains the information regarding various
Second Mind to the simulation platform, such as Second events happening in Second Life such as the player moving
Life or other virtual worlds. The interface layer requires around in the environment, the player touching an object, etc.
the definition of percepts, actions, and other application- Objects in the environment can send various events to
specific elements along with the APIs to connect communicate with the avatar. If there are behaviors that have
Second Mind to the simulation. "Event happens (event name)" as triggers, avatar can react to
events with name "event name".
3.3.1 Behavior execution engine
The behavior execution engine (Figure 5) loads the behaviors The world specific information is parsed into a set of domain
corresponding to the currently active scene, identifies the independent sensors that correspond to various triggers. This
behavior(s) to carry out based on the current perceptual information is then sent to the execution engine, which checks the
information from the world, and sends the information about these sensors against the triggers corresponding to the current set of
behaviors to the graphical world for execution through the behaviors to identify if any new behavior(s) has become activated.
interface layer. The interface layer module also receives the action corresponding
to the active behavior and sends it to the virtual world. The
interface layer provides an abstraction layer and maps the action
name received from the execution engine to a virtual world
specific action name. For example, an action "laugh" in Second

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 348 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

Mind is mapped to actions that express "express_laugh", which is necessary to create and run a behavior along with the system
the specific name used by Second Life virtual world. Table 1 architecture and its four core components.
shows the mapping of a few action names in Second Mind with We evaluated the Second Mind approach to check whether it can
the corresponding names in the virtual world Second Life. be understood and easily used by non-programmers. In order to
Initially, in order to connect Second Mind to Second Life, we evaluate our approach, we invited 65 subjects to create a certain
modified an open source version of Second Mind clients. We soon set of behaviors for an avatar (e.g. act as a shopkeeper) on a
realized though that not all Second Mind users are willing to Second Life island using the authoring system. We divided the
install a modified version of Second Life. Therefore, we instead subjects into three broad categories: User Interface designers,
created scripts in Second Life's proprietary scripting language non-programmers, and programmers (with experience in Java).
named Linden Scripting Language (LSL). Those scripts make it We transcribed data from the subjects’ responses to interviews as
possible to send and to receive information through the Second well as from the notes taken while observing subject actions
Mind interface layer from/to Second Life for controlling the during the authoring process. To analyze this vast quantity of data
avatar with any genuine Second Life client. collected from the user study, we adopted a combination of
qualitative and quantitative analysis. Specifically, we employed a
Table 1. Action names in SM and corresponding virtual world Grounded Theory [28] approach to perform the qualitative
action names in Second Life analysis. A detailed report on the evaluation of the system is
outside the scope of this paper. However, for the sake of
Domain Independent Action Name Virtual World Action Name completeness, the evaluation results show that non-programmers
express_laugh were able to easily create behaviors using the scaffolding
Laugh provided by Second Mind. Moreover, no relevant statistical
express_shrug_emote
difference was noticed in the quantitative measures used to asses
Shrug authoring skills among the different subject categories. This
avatar_hold_bow
seems to indicate that designers, programmers, and non-
Bow programmers perform very similarly while authoring behaviors
with Second Mind.
The collection of scripts is packaged in a component called There is much work left to do in Second Mind, some of which is
Second Mind Controller and distributed through the Second Life research, some of which is in development. There are also
marketplace. Second Life marketplace is a social web portal from numerous open questions that form possible avenues of future
Second Life where all the items from various vendors are listed directions for this research. Authoring Support through
and available for purchase. Second Mind controller is listed on the demonstrating behaviors is something we are considering.
Second Life marketplace and is available for free. In order to have Learning from demonstration is also a point we would like to
their avatar controllable through Second Mind, players need to investigate. Human learning is often accelerated by observing a
download it from the Second Life marketplace. Purchasing items task being performed or attempted by someone else. These
from Second Life marketplace is a common activity performed by capabilities of the human brain are also evident in computer
Second Life users. Once users have bought the items from Second games where players go through a process of training and
Life marketplace, the Second Mind controller is downloaded imitating experienced players. These results have inspired
automatically onto their avatars, which they can then attach to researchers in artificial intelligence to study learning from
their avatar in Second Life. imitation techniques. By observing an expert's actions, new
behaviors can quickly be learnt that are likely to be useful since
3.3.3 Behavior Recommendation. they are already being used by the expert successfully.
During the behavior-authoring process, once the name of the
behavior is entered, the user is presented with a list of 5. REFERENCES
recommended behaviors that he or she can use as a starting point. [1] E. Adams, Fundamentals of game design, 2nd Edition, New
The behaviors are suggested based on similarity between the Riders, 2012.
behavior name specified for the currently authored behavior and
existing behaviors in the database. The semantic similarity is [2] B. Li, M. Thakkar, Y. Wang, and M. O. Riedl, “Data-driven
calculated based on a WordNet-based algorithm [27]. alibi storytelling for social believability”, Proceedings of the
Foundations of Digital Games Workshop on Social Behavior
4. CONCLUSION AND FUTURE WORK in Games, 2014.
We proposed a digital authoring tool that addresses the issues [3] H. Yu, and M. O. Riedl, “Personalized interactive narratives
raised by a user-study. To address the issue of having concrete via sequential recommendation of plot points”, IEEE
feedback on the authoring activity, we hooked up the authoring Transactions on Computational Intelligence and Artificial
interface with a concrete virtual world, namely Second Life, so Intelligence in Games, vol. 6(2), 2014.
that authors can play back, can view and can detect any possible
[4] M. Mehta, and A. Corradini, “An approach to behavior
problems with the created behaviors. In order to simplify the steps
authoring for non-playing characters in digital games”,
required to link behavior conditions to lower level percepts, we
Proceedings of the International Conference on Multimedia,
provided a base set of triggers that can be used individually or
Interaction, Design and Innovation (MIDI '14), Warsaw,
combined to create more complex triggers. In order to present the
Poland, 2014.
authoring task as a sequential one, instead of deep hierarchies, we
used a left to right sequential action creation approach represented [5] J. Rickel, S. Marsella, J. Gratch, R. Hill, D. Traum, and W.
on a timeline. Eventually, we outlined the authoring steps Swartout, "Toward a new generation of virtual humans for

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 349 |
 Second Mind: A System for Authoring Behaviors in Virtual Worlds
M. Mehta | A. Corradini

interactive experiences", IEEE Intelligent Systems, vol. 17, [18] E. Tomai, and R. Flores, “Adapting in-game agent behavior
2002. by observation of players with learning behavior trees”,
[6] M. O. Riedl, and A. Zook, “AI for game production”, Proceedings of the International Conference on the
Proceedings of the IEEE Conference on Computational Foundations of Digital Games, 2014.
Intelligence in Games, Niagara Falls, Canada, 2013. [19] A. Coman, and H. Munoz-Avila, “Plan-based character
[7] J. Cassell, "Embodied conversational agents: representation diversity”, Proceedings of the AAAI Conference on Artificial
and intelligence in user interfaces," AI Magazine, 22(3):67- Intelligence and Interactive Digital Entertainment
83, 2001. (AIIDE'12), 2012.

[8] A. Corradini, M. Mehta, N.-O. Bernsen, and M. Charfuelan, [20] R. Kadlec, C. Toth, M. Cerny, R. Bartak, and C. Brom,
“Animating an interactive conversational character for an “Planning is the game: action planning as a design tool and
educational game system”, Proceedings of the International game mechanism”, Proceedings of the AAAI Conference on
Conference on Intelligent User Interfaces, San Diego (CA), Artificial Intelligence and Interactive Digital Entertainment
USA, pp. 183-190, 2005. (AIIDE'12), 2012.

[9] J. Allbeck and N. Badler, "Representing and paramaterizing [21] J.P. Kelly, A. Botea, and S. Koenig, "Off-line planning with
behaviors", In: H. Prendinger and M. Ishizuka (eds): Life- hierarchical task networks in video games”, Proceedings of
Like Characters: Tools, Affective Functions and the AAAI Conference on Artificial Intelligence and
Applications, Springer, 2004. Interactive Digital Entertainment (AIIDE'08), 2008.

[10] R. S. Aylett, S. Louchart, J. Dias, A. Paiva, and M. Vala, [22] R. Zhao, and D. Szafron, “Using cyclic scheduling to
"FearNot! - An experiment in emergent narrative," generate believable behavior in games”, Proceedings of the
Proceeding of the International Conference on Intelligent AAAI Conference on Artificial Intelligence and Interactive
Virtual Agents (IVA ’05), Kos, Greece, 2005. Digital Entertainment (AIIDE’14), pp. 94-101, 2014.

[11] M. Mateas, and A. Stern, "A behavior language: joint action [23] B. Medler, and B. Magerko, "Scribe: A general tool for
and behavioral idioms", In: H. Prendinger and M. Ishizuka authoring interactive drama", Proceedings of the
(eds): Life-like Characters: Tools, Affective Functions and International Conference on Technologies for Interactive
Applications, Springer, 2004. Digital Storytelling and Entertainment, pp. 139-150, 2006.

[12] M. Mateas, and A. Stern, "Facade: an experiment in building [24] S. Sauer, K. Osswald, X. Wielemans, and M. Stifter, "U-
a fully-realized interactive drama", Game Developer's create: Creative authoring tools for edutainment
Conference: Game Design Track, San Jose (CA), USA, applications", Proceedings of the International Conference on
2003. Technologies for Interactive Digital Storytelling and
Entertainment, pp. 163-168, 2006.
[13] P.F. Gomes, and A. Jhala, "AI authoring for virtual
characters in conflict", Proceedings of the AAAI Conference [25] S. Agamanolis and V. M. Bove, "Viper: A Framework for
on Artificial Intelligence and Interactive Digital Responsive Television", IEEE MultiMedia, 10(1)88-98,
Entertainment (AIIDE'13), 2013. 2003.

[14] B. Schwab, AI Game Engine Programming, 2nd edition, [26] M. Rymaszewski, Second Life: The Official Guide. 2nd
Course Technology, 2009. edition, Sybex Inc, 2009.

[15] D. Isla, "Handling complexity in the Halo 2 AI", Proceedings [27] T. Simpson, and T. Dao, “WordNet-based semantic
of the Game Developers Conference, 2005. similarity measurement” (retrieved on February 18, 2015
from the following URL
[16] A. Champandard, “Behavior trees for next-gen AI”, http://www.codeproject.com/KB/string/semanticsimilarity
Proceedings of the Game Developers Conference, 2007. wordnet.aspx)
[17] C. Dragert, J. Kienzle, and C. Verbrugge, ”Statechart-based [28] A. Strauss, and J. Corbin, Basics of Qualitative Research:
game AI in practice”, Proceedings of the AAAI Conference Grounded Theory Procedures and Techniques. Sage, 199
on Artificial Intelligence and Interactive Digital
Entertainment (AIIDE'12), 2012.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 350 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet

Juegos Serios Tangibles con Objetos Reales como

Herramienta de Apoyo para Trabajar con Niños que
Requieren Necesidades Especiales
Iván Durango, Alicia Carrascosa, José A. Gallud and Víctor M. R. Penichet
Universidad de Castilla-La Mancha
Campus Universitario, Albacete - España
{ivan.durango,alicia.carrascosa}@yahoo.es,{jose.gallud,victor.penichet}@uclm.es
tratan de comprender como los niños usan los sistemas y
ABSTRACT productos.
Tras meses de trabajo con Mateo, los terapeutas del Centro de En esta perspectiva los juegos serios (del inglés "Serious Games"),
Desarrollo Infantil y Atención Temprana (CDIAT) quedan se presentan como una herramienta de gran utilidad para el
realmente sorprendidos al observar cómo usando el sistema aprendizaje en niños [3].
interactivo diseñado en el presente trabajo (Panel de Frutas
Interactivas o PFI), en sólo unas pocas sesiones, Mateo por fin Este tipo de juegos, permiten a los niños desarrollar habilidades de
consigue asociar de forma correcta cada una de las frutas con su una forma entretenida y motivadora, y tienen una gran relevancia
pictograma correspondiente. cuando estos niños presentan algún problema o discapacidad. De
este modo, y debido a los numerosos beneficios que ofrecen,
La aplicación interactiva que se presenta en este trabajo pretende expertos como [4], establecen el juego como medio fundamental
transformar un juego tradicional habitualmente usado por los para el desarrollo adecuado de los niños.
terapeutas de los CDIAT para establecer una asociación entre
En la actualidad, las Interfaces de Usuario Tangibles (TUI,
objetos reales (frutas) y su representación en forma de pictograma,
Tangible User Interface), promueven una forma de interacción
en una actividad tecnológica e interactiva donde se establecen
natural que aportan a los juegos serios un añadido cuando se
objetos reales como base de la interacción. De esta manera se
pretende trasladar la interacción que se realizada en un mundo
trabaja con niños que presentan una serie de necesidades especiales
puramente digital, a un entorno físico y tangible, y por
en uno de estos centros con ayuda de los profesionales en atención
consiguiente permite que se interactúe directamente con objetos
temprana, obteniendo así una serie de resultados que presentan los
reales. Con esto, se establece un ambiente más familiar para el
beneficios que aportan el uso de este tipo de juegos en el proceso
usuario, siendo particularmente apropiadas para el aprendizaje
de enseñanza-aprendizaje de los niños, como pueden ser, la mejora
desde diferentes puntos de vista (sistemas colaborativos, juegos
de la atención en la actividad o el aprendizaje más rápido de la serios, etc.).
asociación entre los pictogramas y el objeto real correspondiente.
Un punto muy importante a tener en cuenta dentro de las TUIs, es
el concepto de “espacio”, es decir, cómo será explotado el espacio
Categories and subject descriptors de interacción, aprovechando al máximo la habilidad innata de
Interaction for People with Disability - Interaction, Learning and actuar con objetos reales que establecen un símil metafórico entre
Teaching – Child-Computer Interaction - Natural Interaction. el sistema y el mundo real. Desde este punto de vista, las TUI
ofrecen una gran cantidad de beneficios como apoyo a la
Keywords enseñanza y sobre todo en aquellas personas con dificultades de
Early childhood Intervention, Special Education, Child-Computer aprendizaje o necesidades especiales [5].
Interaction, Serious Games, Tangible Computer Interface. Por ejemplo, estudios como el realizado por A. Carreras y N. Parés
[6], determinan mediante ejemplos que las interacciones desde una
1. Introducción perspectiva táctil fomentan la participación activa del alumno,
En la actualidad el estudio de la interacción de los niños con los siendo además, muy apropiadas para superar el miedo de usar
sistemas interactivos es un área en constante evolución y tecnologías para el aprendizaje, sobre todo cuando se trata de
crecimiento [1], esto es debido a su asociación directa con áreas en usuarios inexpertos o niños, fomentando así las habilidades
las que toma mucha importancia esta visión lúdica del mundo exploratorias y experimentales del usuario que usa este tipo de
como: la psicología, el aprendizaje y el juego. tecnología. Un ejemplo de aplicación que usa TUI dentro de este
campo es "Telestory" [7] (Figura 1).
Si se observa la sociedad actual, puede detectarse cómo los niños
se ven inmersos en un mundo tecnológico desde una edad
temprana, lo que crea una necesidad inmediata a la hora de diseñar
productos interactivos, ya que la interacción que tiene lugar entre
un computador y un niño (Child-Computer Interaction - CCI) dista
considerablemente de la interacción que mantiene un adulto
(Human-Computer Interaction - HCI), con lo que se genera una
nueva barrera de entendimiento para los investigadores [2], que

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 351 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
de las pautas que se establecen en el proceso de enseñanza-
aprendizaje con niños con necesidades especiales. Aunque en
nuestro caso, y tomando el sistema Co-Brain como ejemplo, es
complicado que los niños lo usen debido a la complejidad de las
imágenes y a que es necesario que nos niños cojan un dispositivo
para interactuar, aspecto que en ocasiones y debido a sus
problemas y/o dificultades es algo muy complejo para ellos.
De otro modo, el uso de pictogramas en el aprendizaje es habitual
en los CDIAT [13] y por consiguiente, estos son una herramienta
indispensable en muchas de las terapias.
Por ejemplo en [14], se presenta una aplicación para iPad en la que
niños con autismo pueden aprender a crear secuencias de acciones
usando como base los pictogramas que describen estas (Figura 2).
Figura 2. Telestory - Ejemplo de uso
(Autor de la imagen: Proyecto Telestory)
Esta aplicación trata de que niños con problemas en el lenguaje
puedan aprender un lenguaje de comunicación mediante
pictogramas digitales.
El objetivo principal del trabajo presentado en este artículo es la
adaptación de una actividad real, la cual se usa de forma habitual
en los centros de atención temprana, a un sistema interactivo que se
basa en una interacción tangible para hacer de ésta una actividad Figura 3: Secuencia de acciones necesarias para cepillarse los
más entretenida, motivadora y con la que obtener mejores dientes
resultados terapéuticos.
Este tipo de aprendizaje es de gran importancia dentro de los
CDIAT, ya que es una de las formas utilizadas por algunos de los
2. Trabajos relacionados niños para comunicarse y por tanto es usada como una herramienta
En los últimos años han sido muchos los intentos de diseñar, crear de apoyo en niños que no disponen de lenguaje oral.
y evaluar, juegos y actividades basadas en un soporte interactivo y
virtual [8] [9] [10], orientadas a niños con necesidades especiales. 3. Contexto del estudio
De hecho, existen multitud de aplicaciones, en su gran mayoría,
pensadas para tablet, smartphones y dispositivos de uso cotidiano 3.1. Centros de Atención Temprana
que tratan de fomentar temas educativos implícitos en un El sistema presentado en el presente artículo, se ha diseñado en el
videojuego (juegos serios) [11]. contexto de un centro de Desarrollo Infantil y Atención Temprana
Desde esta perspectiva, aunque existen gran cantidad de recursos (CDIAT).Un CDIAT ofrece servicios autónomos cuyo objetivo es
tecnológicos, como los que pueden encontrarse en iTunes, Google la atención a la población infantil de 0-6 años que presenta
Play o la Tienda de Windows, éstos en un porcentaje elevado se trastornos en su desarrollo o que tiene riesgo de padecerlos. Son
enfocan a niños con un desarrollo típico o que no requieran ningún centros con carácter interdisciplinar, donde profesionales de
tipo de ayuda especializada, de tal modo que los profesionales distintas áreas trabajan de forma conjunta. En estos centros se
encargados de intervenir con los niños con algún tipo de dificultad, otorga una mayor prioridad a niños de entre 0 a 3 años. Esto es así
encuentran un gran número de inconvenientes a la hora de buscar porque los expertos consideran que es en esa etapa cuando el
una aplicación que funcione con cada uno de los niños, pues no sistema nervioso está en plena maduración y presenta una
todas las aplicaciones tiene los mismos resultados terapéuticos, importante plasticidad, por lo que las posibilidades terapéuticas
cuando se usa de la misma forma, ya que dependerá de las muestran su mayor eficacia [4]. Es importante destacar que en los
características y necesidades del niño que la esté usando. CDIAT no sólo se atienden a niños con discapacidad o patologías
En juegos como Co-Brain Training y Azlgame [12], se utilizan sino que también se aborda el carácter preventivo de otras posibles
interfaces de usuario tangibles para ofrecer al usuario una forma de alteraciones, como podría suceder en casos de prematuridad, cuyos
interactuar más sencilla y amena. niños nacen antes de completar la semana 37 de gestación.
De este modo, se observa cómo se mejora la concentración del Cabe destacar la colaboración que se realiza entre los profesionales
usuario y sus habilidades cognitivas, ayudando además a que éste del CDIAT y el grupo de investigación autor del presente trabajo,
use el sistema de una forma amena y entretenida. En estos juegos creando así una unión de conocimientos tecnológicos y
se usa una interacción basada en gran parte en pictogramas, que terapéuticos/educativos que pueden observarse a lo largo del
mediante el uso de tecnologías como NFC (Near Field presente trabajo.
Communication) permiten interactuar con el sistema de una forma
natural.
3.2. Trabajo por objetivos en los CDIAT
La línea de trabajo que se plantea en los CDIAT está basada en las
Es importante tener en cuenta que aunque estos trabajos estén necesidades individuales que tiene el niño y no en el diagnóstico
orientados a entornos de rehabilitación cognitiva, pueden ser que presente. Este tipo de trabajo está avalado por numerosos
observados desde un punto de vista educativo, pues siguen muchas expertos como [15], [16], [17]. Por tanto, se considera importante

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 352 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
el planteamiento de trabajo realizado según unos determinados
objetivos, ya que ante dos niños con diagnósticos similares
existirán objetivos que se trabajarán de forma análoga y otros que
se trabajarán de forma diferente.
Por consiguiente y a modo de ejemplo, se pueden perseguir
objetivos diferentes al realizar una misma actividad con dos niños,
aunque su diagnóstico sea similar. De este modo, tal vez el objetivo
de uno sea aprenderse el nombre de las figuras geométricas y por el
contrario, el objetivo del otro sea que las figuras en su
representación de imagen sea asociada correctamente con su objeto
real, de tal forma que si se diferencian dichos objetivos: en uno se
trabajarían simplemente conceptos y en otro la abstracción de los
objetos para su uso en un sistema alternativo de comunicación.
Por tanto no se necesita una etiqueta que nos diga donde encasillar
el trabajo que se realizaría con esos niños, pudiendo además,
trabajar objetivos similares en niños con diagnósticos totalmente
diferentes. Por consiguiente, surge la siguiente cuestión: ¿Sería
posible enfocar el trabajo de un niño con síndrome de Down y de
otro con Trastorno del Espectro Autista según unos mismos
objetivos?
Según los expertos, la respuesta a este interrogante depende de las
características que presenten ambos niños, y por tanto se necesita
en primer orden conocer sus habilidades y sus intereses, y a partir
de ahí trazar su línea de intervención individual, donde se
establecerán una serie de objetivos que serán comunes y otros que
serán específicos de cada uno de los niños, permitiendo así la
utilización de la misma actividad para trabajar en una dirección
concreta aunque el objetivo final sea diferente. Cabe destacar que Figura 4: Juego de asociación tradicional
esta visión es contraria a estudios como [18], [19], [20], [21], El fin del juego es que el niño sea capaz de asociar el objeto real
donde todos los esfuerzos se centran en un solo tipo de con su representación en pictograma Este ejercicio es muy
discapacidad, como por ejemplo, el Trastorno del Espectro del importante y frecuente en los CDIAT, más si cabe cuando el niño
Autismo [17], dejando de lado los objetivos terapéuticos que necesita aprender un sistema alternativo de la comunicación basado
persigue la actividad. en pictogramas, donde esta asociación le ofrece al niño todas las
herramientas que necesitará en su aprendizaje.
3.3. Descripción de la actividad no tecnológica
Cabe destacar que los profesionales advierten de que para que los
sobre la que se trabajará niños lleguen a realizar esta asociación de forma adecuada se han
Para seleccionar y describir de una forma adecuada la actividad o
de realizar una serie de fases evolutivas donde se aumenta la
actividades sobre las que se llevará a cabo el experimento, se han
dificultad del juego y la abstracción de los objetos. Por tanto, en
llevado a cabo una serie de reuniones donde los profesionales del
una primera fase se trata de identificar cada uno de los objetos
centro nos indican aquella actividad que es relevante tanto en su
reales, actividad que realiza el terapeuta con el niño; en la segunda
trabajo diario como a un nivel terapéutico. Tras estas reuniones, se
fase el objetivo es asociar el objeto real con su imagen real
decide tomar como base una actividad que los niños realizan a
asociada. Una vez adquiridas las asociaciones pertinentes se pasa a
menudo y en la que se realiza un proceso de asociación entre un
una tercera fase donde se comienza a jugar con pictogramas. De
objeto real y su representación en un pictograma.
este modo, para que el niño consiga asociar un objeto real con su
El aprendizaje de esta asociación es fundamental cuando se trata de pictograma debe trabajar con cada uno de los siguientes materiales
niños con problemas en la comunicación y que requieren aprender (pasando por cada una de las tres fases descritas anteriormente):
un sistema alternativo de comunicación basado en pictogramas Objeto-Real - Imagen real – Pictograma.
para comunicarse, como por ejemplo [22]. En este proceso de
enseñanza-aprendizaje, los profesionales realizan un "juego" con el
niño donde se le muestra un objeto real, como por ejemplo una
4. EL Panel de Frutas Interactivo
fruta en forma de maqueta en miniatura, y posteriormente se le El Panel de Frutas Interactivo (PFI), es un juego serio tangible, en
muestra una serie de pictogramas con diferentes frutas (Figura 3). el que se puede realizar la actividad descrita en el punto anterior,
para así aprender a asociar un objeto real con su correspondiente
pictograma (con cada una de las fases necesarias en el proceso de
enseñanza aprendizaje).

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 353 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet

ARASAAC [23]. Asimismo, son los pictogramas usados de forma

habitual por los profesionales del CDIAT donde se realizarán las
evaluaciones, y se diseñan los aspectos terapéuticos del sistema.
Siguiendo algunos de los criterios detallados por Sweetser y Wyeth
[24] en su teoría de flujo propuesta, se tienen en cuenta aspectos
como el feedback, la inmersión del usuario en la actividad, el reto o
desafío y la concentración.
Con respecto al feedback que el niño obtiene desde la aplicación
tanto cuando selecciona la fruta correcta, donde recibirá un
feedback positivo como cuando la fruta seleccionada no es la que
se esperaba donde el feedback será menos positivo. Por tanto, al
seleccionar una fruta errónea su pictograma mostrará durante dos
segundos una cara triste (Figura 5).

Figura 5:Panel de Frutas Interactivas (PFI)

Como se observa en la Figura 4, la aplicación estará compuesta de
una parte puramente digital, es decir, un juego de escritorio, y un
panel de frutas fabricado en madera y papel plastificado, donde se
colocarán cada una de las frutas (reales) para que el niño pueda
usarlas como medio de interacción tangible con el juego digital. De
este modo, el medio de interacción con el sistema serían las frutas,
ya que al ser tocadas por el niño se interactuará con el juego
digital. Figura 6: Feedback No positivo
El objetivo de esta aplicación, es evolucionar las actividades que se Mientras que si el niño toca la fruta correcta, se mostrará una cara
realiza de forma habitual en los centros, para adaptarlas a un muy feliz y a continuación una representación de globos
mundo digital e interactivo con el fin de hacer un juego más acompañada de un sonido de ovación y aplausos por parte de niños
motivador, entretenido y que mejore la consecución de objetivos. (Figura 6).
El objetivo principal de la aplicación es motivar a los niños a jugar
para que adquieran la asociación entre pictogramas necesaria para
adoptar un sistema de comunicación alternativo. Además, y desde
una perspectiva cognitiva se desarrollan aspectos como la
concentración, la activación del sistema visual, la memoria y la
estimulación de la capacidad cognitiva para asociar objetos reales
con su representación en imagen.
Por otro lado, se pretende convertir una actividad que a priori
puede parecer aburrida en una actividad motivadora y divertida
para los niños.

4.1. Diseño del sistema

Antes de comenzar a diseñar la aplicación, se establecen una serie
de reuniones con los profesionales para determinar qué requisitos
se debe tener para que cubra las necesidades que presentan tanto
los niños como los profesionales con respecto al diseño del
sistema. De estas reuniones se determina que la aplicación a
desarrollar ha de ser un "juego" divertido y que motive a los niños.
De tal forma que se estimule la atención de los niños en la
actividad. Además, se establecen una serie de pautas a seguir con
respecto a las imágenes, pictogramas y la interfaz que debe
presentar, ya que en la mayoría de CDIAT, tanto españoles como Figura 7: Feedback Positivo
internacionales, se utilizan los pictogramas diseñados por
XVI Congreso Internacional de Interacción Persona-Ordenador | Games
| 354 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
Por otro lado un punto muy importante a tener en cuenta es la
concentración, ya que una de las mejoras que pretende aportar la
aplicación interactiva con respecto a la actividad no tecnológica
corresponde a que el niño mantenga mejor la atención y la
concentración en el desarrollo de la actividad y esto incida de
manera directa a la adquisición de conceptos. Para ello, se
establece una interfaz de usuario que evita las distracciones
innecesarias, de tal forma que el niño centre su atención en las
frutas del panel y en las imágenes y/o pictogramas que se le
muestran en la interfaz digital.
Con respecto a la inmersión del usuario en el juego, se trata de
lograr que el niño se vea envuelto en un mundo virtual donde se
interactúa con objetos reales. Para lograr esta inmersión se ha
añadido a la presentación en forma de imagen o pictograma de
cada una de las frutas una melodía característica, de tal modo que
ayude al niño a establecer un sonido particular con cada una de las
frutas. Desde el punto de vista del reto o desafío, y teniendo en
cuenta que cualquier juego tiene que proporcionar un reto, se
establece un sistema de puntuación donde al terminar la actividad Figura 9: Selección de la fase a trabajar
se le muestra al niño la puntuación obtenida teniendo en cuenta la 2. Configuración del modo de juego (Figura 9):
cantidad de frutas que ha acertado. Esta puntuación, se acompaña El terapeuta selecciona entre un modo de juego sencillo, donde las
de una melodía característica que sirve a la vez para animar al niño imágenes de las frutas aparecen siempre en el mismo orden, o un
a que vuelva a jugar para mejorar sus resultados y para que sea modo de juego avanzado, donde estas aparecen de forma aleatoria.
asociada esta melodía a la finalización de la actividad

Figura 8: Fin de el Juego

4.2. Funcionamiento
El funcionamiento del sistema para realizar la actividad es el
siguiente:
1. Selección de la fase a trabajar o tipo de juego (véase Figura 8):
Al iniciar la actividad el terapeuta ha de seleccionar el tipo de
juego que quiere usar para trabajar con el niño. En este contexto,
existen tres modos de juego diferentes, según en la etapa de Figura 10: Configuración del modo de juego
enseñanza aprendizaje que se encuentre el niño. Por tanto se debe 3. Selección de las frutas con las que se quiere jugar (Figura 10):
seleccionar si se quiere jugar sólo con imágenes de los objetos, si Por último y antes de comenzar a jugar el terapeuta seleccionará las
se pretende jugar con imágenes y pictogramas o por el contrario se frutas con las que desea que el niño interactúe. Esta selección hace
trabajará sólo con pictogramas. que la aplicación sea más flexible para el profesional, ya que en un
principio puede comenzar jugando con un menor número de frutas
para posteriormente aumentar el número de éstas según las
necesidades del niño.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 355 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet

Figura 12: Pulsera que hace capaz la interacción entre el

usuario y el sistema
Es importante destacar que debido el bajo amperaje que presenta la
electrónica usada (Arduino), se presenta una forma de interactuar
completamente segura e inocua para el usuario.
Figura 11: Selección de las Frutas que aparecerán en el juego
En nuestro caso en particular, tal como se muestra en la Figura 11,
4. Comienza el Juego: cuando el niño tenga puesta la pulsera en su muñeca y pulse el
Una vez seleccionada la configuración del juego el niño comenzará "plátano", cierra el circuito establecido para esa fruta en concreto
a jugar. y se genera un pulso de teclado que corresponde con la tecla "a".
5. Finalización del Juego De este modo la aplicación interactiva, recibe la pulsación de la
El juego finaliza cuando transcurren cuatro minutos desde el tecla "a" y comprueba si el pulso recibido corresponde a la tecla
comienzo de la partida. El tiempo es establecido teniendo en cuenta asociada al pictograma que se encuentra en pantalla, generando así
la opinión de los profesionales, ya que es el tiempo que un feedback positivo si la asociación es correcta.
aproximadamente dedican a cada una de las actividades dentro de
sus programaciones, consiguiendo así introducir la actividad como 4.4. Arquitectura del sistema
una más en la rutina diaria de los niños. El Panel de Frutas Digital se compone de dos partes bien
diferenciadas: una parte lógica que corresponde a un juego de
4.3. Mecanismos de Interacción escritorio, y una parte tangible, donde se encuentra el panel de
El sistema interactivo, se basa en la conductividad eléctrica que frutas con los objetos reales.
presenta tanto el cuerpo humano como los objetos seleccionados La parte lógica de la aplicación ha sido desarrollada con Scratch
como base de interacción, es decir, las frutas. Estos elementos [25]. De este modo, se desarrolla una interfaz de usuario fácil de
poseen un gran porcentaje de agua y por tanto muestran una gran transformar por los terapeutas, otorgando así una gran flexibilidad
conductividad eléctrica. Éste aspecto es importante, ya que debe a la aplicación y dando la posibilidad de que el terapeuta use para
tomarse como base para comprender como funciona la interacción cada uno de los niños aquellos fondos y sonidos que más les
que hace el niño con las frutas para cerrar un circuito eléctrico y motiven [26]. Por otro lado, se separa la interfaz de usuario de la
generar así un pulso que es enviado al computador y que será lógica de la aplicación, siendo ésta desarrollada en JSON
diferente según la fruta que toque el niño. El pulso enviado, es el (JavaScript Object Notation) ya que es el lenguaje que usa Scratch
mismo que cuando se pulsa una tecla u otra del teclado, con lo que a nivel interno. La parte tangible del Panel de Frutas, ha sido
se puede diferenciar un pulso de otro de manera sencilla. construido con madera, pintura, papel y plástico. Internamente, se
Por tanto, la clave reside en establecer como parte positiva cada usa una placa Arduino Duo [27], para proveerlo de inteligencia. A
una de las frutas expuestas, que se unen al circuito con una serie de continuación se muestra de forma general un esquema de la
clics metálicos, y como parte negativa o de masa, el propio cuerpo arquitectura interna que presenta el panel (Figura 12):
del niños, con lo que se hace necesario que este se ponga una
pulsera fabricada en velcro y papel de estaño (Figura 11), de este
modo cuando el niño toca la fruta, se consigue conductividad entre
el usuario y la fruta cerrando de este modo el circuito eléctrico
necesario para enviar el pulso con la tecla correspondiente.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 356 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
•

5.3. Dispositivos utilizados

Los dispositivos utilizados han sido los siguientes:
• Portátil Asus U33JC (512GB de memoria interna, 4GB de
RAM, procesador Intel i3 1.8GHZ),
• Arduino Due.
• 1 cable de conexión (micro-USB macho a USB macho).
• 6 cables y 6 clics metálicos aislados.
• 5 frutas (1 plátano, 1 fresa, 1 pera, 1 manzana y 1 naranja).
• 2 piezas de madera contrachapada.

5.4. Procedimientos
Para realizar los experimentos se han dividido las evaluaciones en
tres fases: Test de juego tradicional, test con el PFI y post-test con
el PFI.
En la primera fase o Test de juego tradicional, el terapeuta juega
con el niño usando el juego tradicional (sin tecnología), con esto se
pretende establecer un punto de partida para determinar de qué
punto se parte, con respecto a las asociaciones que el niño es capaz
Figura 13: Esquema de la aplicación
de hacer, y poder así comparar este inicio con la fase final de este
Observando el esquema anterior, puede comprenderse cómo la experimento.
placa Arduino es la encargada de comunicar a la aplicación de
Tras terminar de jugar el profesional anota la cantidad de fallos y
escritorio la fruta que ha sido tocada por el niño (proyecto similar
aciertos que comete el niño y a continuación responde a una serie
[28]).
de preguntas:
• Define la motivación y concentración del niño al realizar
5. Evaluación la actividad.
En el presente apartado se presenta la evaluación realizada para
• Al realizar la actividad, ¿El niño quiere volver a jugar?
comprobar la reacción de los niños al usar el Panel de Frutas
Interactivo. • ¿Se ha divertido el niño realizando la actividad?
En la segunda fase de las evaluaciones, el terapeuta usa el panel de
5.1. Participantes frutas interactivo para trabajar con el niño. Tras esto, el terapeuta
Los participantes seleccionados para jugar con el sistema anota la cantidad de errores y aciertos y responde al siguiente
interactivo, han sido 10 niños de entre 3 y 6 años de edad que cuestionario:
necesitan adquirir un sistema de comunicación alternativo basado
en pictogramas, con lo que tienen objetivos terapéuticos comunes. • ¿Ha mejorado la motivación y concentración del niño al
realizar la actividad con respecto al juego tradicional?
Estos niños presentan diagnósticos variados como: trastorno del • Al realizar la actividad, ¿El niño quiere volver a jugar?
espectro autista, trastornos del lenguaje, dificultades de atención, • ¿Se ha divertido el niño realizando la actividad?
síndrome de Down y otros.
• ¿Se nota alguna diferencia entre usar o no usar el panel
5.2. Objetivos que se tratan conseguir a nivel de frutas interactivo?
terapéutico La tercera fase, se realiza a la semana siguiente de realizar la fase
El objetivo principal que se trata de conseguir en este ejercicio anteriormente descrita. En esta fase se vuelve a jugar al panel de
recae en la adquisición de la asociación entre un pictograma con su frutas interactivos y después el terapeuta responde a las siguientes
objeto real de forma rápida y fluida. Además, de este objetivo preguntas:
pueden detectarse otra serie de su subobjetivos asociados como: • ¿El niño recuerda el juego?
La asociación de vocabulario con su pictograma para • Con respecto al número de fallos y aciertos, ¿Se ha
mejorado el número de aciertos con respecto a la prueba
• La asociación de vocabulario con su pictograma para la
anterior?
adquisición de un sistema alternativo de la comunicación.
• ¿Cómo es el feedback que se percibe de los niños?
• La mejora de la atención de los niños con respecto a su
espacio. Es importante tener en cuenta que en cada una de las evaluaciones
• El entrenamiento y la mejora de la memoria visual. que se realizan con el panel de frutas, al finalizar éstas se guarda
• La adquisición de vocabulario asociado a los objetos, como: su una captura de la interfaz donde se muestra el número de errores y
nombre, su color, su forma, etc. aciertos del usuario, estableciendo la fecha y hora en la que se
almacenó. De este modo, el terapeuta puede recuperar estos
resultados de forma fácil (Figura 13)

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 357 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
disminución del 51% en el número de errores en sólo dos sesiones.
De tal forma que se observa una progresión donde cada vez
aumenta más el número de aciertos y disminuye el número de
errores. Con el niño que no ha logrado una mejora, cabe destacar
que aunque el número de aciertos y errores siguen siendo similares
a los que se consiguieron en el test con el Juego tradicional (5
aciertos y 9 errores), sí que se ha logrado que el niño esté más
concentrado en el juego y no se levante constantemente de la silla,
cosa que ocurría en la primera evaluación pero no sucede cuando
se usa el PFI.
Los profesionales han manifestado un grado alto de satisfacción
con el PFI, ya que en el 80% de los casos se ha obtenido una buena
valoración en el Smileyometer, en concreto se han obtenido: 8
"Realmente Bueno/Fantástico", 1 "Bueno" y 1 "No muy bueno" en
la segunda fase de evaluaciones descrita en el punto 5.4, y para la
tercera fase, se obtienen las siguientes valoraciones: 8 "Realmente
Bueno/Fantástico" y 2 "Bueno".
Figura 14: Número de aciertos y errores Además, en los comentarios que los profesionales han realizado
Además, se debe destacar que los niños juegan sentados y tras las evaluaciones, se detecta que los niños presentan un mayor
manteniendo una postura cómoda, relajada y a una distancia grado de atención al usar el sistema interactivo que cuando realizan
apropiada para facilitar la visualización tanto de la pantalla como el la actividad tradicional, lo que facilita que los niños asimilen de
panel de frutas de una forma apropiada, ayudando además a centrar una forma más rápida la asociación entre los objetos y los
toda su atención en el juego. pictogramas.
En la misma línea, se observa que al cambiar las frutas por otro
5.5. Método tipo de objetos, como medios de transporte, los niños presentan un
Para llevar a cabo la evaluación se debe tener en cuenta que, según aprendizaje en la mecánica de trabajo que ayuda en el aprendizaje
los expertos, es muy difícil realizar una comparativa entre los de nuevas asociaciones y conceptos. Desde la perspectiva del
resultados de los niños, ya que cada uno de ellos es un mundo. Si profesional, se detecta un alto grado de satisfacción a la hora de
bien, el análisis se centra en comparar como aprende un mismo trabajar con otros objetos debido a la facilidad que presenta el
niño usando el panel de frutas y sin usarlo. sistema a ser modificado y adaptado según las necesidades del
De este modo, se utiliza un test donde el terapeuta realiza una niño.
prueba a los niños para comprobar si son capaces de asociar la Por otro lado, se detecta que los profesionales tienen problemas
fruta real con su pictograma correspondiente. con los niños al inicio del juego, ya que la mayoría se niega a
Tras esto, se comprueba cómo es el aprendizaje del niño cuando el ponerse la pulsera. Esto ha sido solventado añadiendo un velcro en
terapeuta usa otros objetos diferentes y otros pictogramas como por la parte visible de la pulsera, el cual hace de base para que los
ejemplo: medios de transporte. niños que son reacios a ponerse la pulsera realicen un pequeño
Una vez concluido el test, el terapeuta escribirá unas breves líneas dibujo en un papel y a continuación este es acoplado mediante el
ofreciendo su visión desde el punto de vista de la intervención y el velcro a la pulsera. De este modo, los niños personalizan la pulsera
logro de los objetivos establecidos en esta. y el 100% de las ocasiones comienzan a jugar sin presentar mayor
problema.
Para concluir realizará un test Smileyometer (Ver Figura 14). En
este test el terapeuta podrá elegir el grado de satisfacción con la Por otro lado al realizar las evaluaciones, los terapeutas nos
aplicación y con los resultados obtenidos, representándola proponen que use el Panel de Frutas una niña autista. Esta niña
mediante una de las siguientes emociones: aunque ya es capaz de hacer correctamente asociaciones entre los
objetos reales y los pictogramas debido a que usa un sistema
alternativo de la comunicación denominado PECS [22], presenta
dificultades en la alimentación, ya que tiene intolerancia con
algunas texturas, entre ellas las frutas.
Horrible / No muy bueno / Bueno / Realmente bueno / Fantástico Por tanto, en este caso el objetivo terapéutico no es la asociación
Figura 15: Test Smileyometer entre el objeto real y el pictograma si no que se basa en que la niña
sea capaz de interactuar con las frutas.
Este test permite que los profesionales se basen en sus propias
emociones al finalizar la actividad con los niños, y teniendo en Nuestra sorpresa fue que la niña comenzó a jugar con el panel de
cuenta su sencillez, hace que estos lo realicen de una forma rápida forma instantánea y debido a su concentración en éste, se olvidó de
y divertida. su intolerancia a las texturas y jugó sin problemas. Al acabar el
juego, la niña se acercó a su libreta de pictogramas y cogió el
5.6. Resultados pictograma que representa una tablet para comunicarle al terapeuta
Una vez llevadas a cabo las evaluaciones se han obtenido los que quería volver a jugar.
siguientes resultados: de los 10 niños, 9 han logrado mejorar la
asociación entre las frutas y los pictogramas, consiguiendo una
mejora aproximada del 40% en el número de aciertos y una
XVI Congreso Internacional de Interacción Persona-Ordenador | Games
| 358 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet

6. Beneficios de la aplicación [3] A. Caird-Daley, «Training decision making using serious

El PFI ha de establecerse como una herramienta de apoyo a los games: Requirements analysis for decision making training.,»
profesionales que les facilita su trabajo diario y ayuda a los niños a de Human Factors Integration Defence Technology Centre ,
adquirir, a través del juego, una serie de habilidades requeridas 2007..
para lograr tener un sistema de comunicación con el que
[4] (GAT), Federación Estatal de Asociaciones de Profesionales
comunicarse con su entorno diario. De este modo, el sistema
de Temprana, Libro Blanco de la Atención Temprana -
interactivo presentado ha sido diseñado por profesionales y para
55/2005, Madrid: Real Patronato sobre
profesionales de los CDIAT, esto conlleva a que el sistema permita
Discapacidad.(Ministerio de Trabajo y Asuntos Sociales),
no sólo trabajar y aprender la asociación entre las cinco frutas
2011.
seleccionadas y sus pictogramas, si no que debido a su sencillez a
la hora de modificar los objetos, las imágenes y los pictogramas los [5] P. Marshall, «Do tangible interfaces enhance,» TEI’07, 2007.
profesionales podrían usar el sistema para el aprendizaje de otros
conceptos como hortalizas, medios de transporte, animales, etc. En [6] A. Carreras y N. Parés, «Diseño de una instalación interactiva
esta misma línea, y tomando como base la forma de trabajar basada destinada a enseñar conceptos abstractos,» AIPO, 2007.
en objetivos y en las características de los niños, se establece una
[7] Seth Hunter, Jeevan Kalanithi, David Merrill, «Make a Riddle
herramienta de apoyo que puede ser transformada para lograr otros
and TeleStory: designing children's applications for the
objetivos diferentes, como por ejemplo el aprendizaje de los
siftables platform,» Proceedings of the 9th International
colores, donde el objeto real con el que interactuarían los niños
Conference on Interaction Design and Children (IDC '10), nº
podría ser simplemente plastilina (con un alto contenido en agua)
DOI=10.1145/1810543.1810572
de distintos colores, y los pictogramas representarían objetos de
http://doi.acm.org/10.1145/1810543.1810572, pp. 206-209,
distinto color. Desde otro punto de vista, se presenta un sistema
2010.
con un coste económico reducido y al alcance de todos los centros,
ya que los objetos con los que se interactúa, en nuestro caso frutas, [8] S. Freitas, «Learning in immersive worlds. A review of game-
tienen un coste reducido y la tecnología y el material usado no based learning.,» JISC e-learning programme , 2006.
supera los 40 euros (sin tener en cuenta el ordenador, ya que podría
usarse con cualquier ordenador disponible en el centro). [9] Backlund, P., Engström, H., Johannesson, M., & Lebram, M.,
«Games for traffic education: an experimental study of a
game-based driving simulator.,» Simulation and Gaming,, vol.
7. Conclusiones 41, pp. 145-169, 2010.
En este artículo proponemos PFI (Panel de Frutas Interactivo). Un
juego interactivo digital basado en interfaces de usuario tangibles y [10] R. Blunt, «Does game-based learning work? Results from
donde los niños juegan tocando directamente los objetos reales. three recent studies.,» eLearn Magazine. Education and
Este sistema se ejecuta en cualquier dispositivo con sistema Technology in Perspective, 2009.
operativo Windows o Linux. En este juego se presenta una forma
de interacción natural y fácil de aprender donde para comenzar a [11] C. González-González y F. Blanco-Izquierdo, «Designing
jugar sólo se requiere que el niño se ponga una pulsera y comience social videogames for educational uses,» Elsevier Ltd, nº
a interactuar directamente con las frutas del panel. 0360-1315/$, 2011.
De este modo el proceso de enseñanza-aprendizaje es motivador [12] E. de la Guía, M. D. Lozano and V. R. Penichet, “Cognitive
tanto para el niño como para el profesional, ya que ambos se ven Rehabilitation Based on Collaborative and Tangible Computer
inmersos en el juego. La evaluación por parte de los profesionales Games,” Pervasive Computing Technologies for Healthcare
ha sido muy positiva, y los niños han disfrutado y aprendido (PervasiveHealth), 2013 7th International Conference on, pp.
jugando. 389 - 392, May 2013.
Como trabajo futuro, se realizará un acercamiento del sistema a los [13] Abigail Durrant, Jonathan Hook, Roisin McNaney, Keir
padres, donde se pretende llevar a cabo una evaluación del Williams, Thomas Smith,Mathew Kipling, Tony Stockman1,
aprendizaje donde los niños se encuentren en un ambiente familiar Patrick Olivier, “Design to Support Interpersonal
y relajado. Tratando así de que tanto padres como profesionales Communication in the Special Educational Needs Classroom,”
dispongan de este tipo de sistemas para trabajar con los niños de Proceeding IDC '13 Proceedings of the 12th International
forma conjunta y en una misma dirección. Conference on Interaction Design and Children, pp. 46-55,
2013.
8. REFERENCias [14] Ceymi Doenyasa, Emre Şimdi, Ezgi Çagla Özca, Zehra
[1] J. C. Read, J. P. Hourcade, P. Markopoulos, A. N. Antle and Çataltepe, “Autism and tablet computers in Turkey: Teaching
N. Parés, “Child Computer Interaction,” in CHI 2008, April 5 picture sequencing skills via a web-based iPad application,”
– April 10; ACM 978-1-60558-012-8/08/04., Florence, Italy, International Journal of Child-Computer Interaction, July
2008. 2013.
[2] P. Markopoulos, J. Read, J. Hoysniemi y S. MacFarlane, [15] Altema, «ATELMA (ASOCIACIÓN TRASTORNO
«Child computer interaction: advances in methodological ESPECIFICO DEL LENGUAJE DE MADRID),» [En línea].
research,» Cogn Tech Work, Springer-Verlag London, vol. 10, Available: http://www.atelma.es/. [Último acceso: 2
nº DOI 10.1007/s10111-007-0065-0, pp. 79-81, 2007. Septiembre 2014].

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 359 |
 Juegos Serios Tangibles con Objetos Reales como Herramienta de Apoyo para Trabajar con Niños que Requieren Necesidades
Especiales
I. Durango | A. Carrascosa | J. A. Gallud | V. M. R. Penichet
[16] A. E. S. d. Down, «DOWN,» [En línea]. Available: [En línea]. Available: http://www.pecs-spain.com/.
http://www.sindromedown.net/. [Último acceso: 3 Septiembre
2014]. [23] Portal Aragonés de la Comunicación Aumentativa y
Alternativa, Febrero 2015. [En línea]. Available:
[17] A. A. d. Psiquiatría, Guía de consulta de los criterios catedu.es/arasaac/.
diagnósticos del DSM-V, 1000 Wilson Boulevard, Arlington,
VA 22209-3901: American Psychiatric Publishing, 2014. [24] P. &. W. P. Sweetser, «GameFlow: a model for evaluating
player enjoyment in games.,» ACM Computers in
[18] Benton, Laura; Jhonson, Hilary; Ashwing, Emma; Brosnan, Entertainment, vol. 3, p. 1–24, 2005.
Mark; Grawemeyer, Beate, «Developing IDEAS: Supporting
Children with Autism within a Participatory Design Team,» [25] MIT - Massachusetts Institute of Technology, Febrero 2015.
CHI, Mayo 2012. [En línea]. Available: https://scratch.mit.edu/.

[19] J. P. Hourcade, N. E. Bullock-Rest y T. E. Hansen, [26] Grupo Lifelong Kindergarten del Laboratorio de Medios del
«Multitouch Tablet Applications and Activities to Enhance the MIT, Febrero 2015. [En línea]. Available:
Social Skills of Children with Autism Spectrum Disorders,» https://scratch.mit.edu/.
Personal and Ubiquitous Computing, nº DOI: [27] CC, Arduino, Febrero 2015. [En línea]. Available:
10.1007/s00779-011-0383-3. http://arduino.cc/en/Main/ArduinoBoardDue.
[20] Alqahtani, Amal; Jaafar, Nouf; Alfadda, Nourah, «Interactive [28] J. LLC, «Makey Makey,» 2012. [En línea]. Available:
Speech Based Games for Autistic Children with Asperger http://www.makeymakey.com/. [Último acceso: 16 Marzo
Syndrome,» IEEE, nº 978-1-4673-0098-8/11, 2011. 2015].
[21] J. P. Hourcade, S. R. Williams, E. A. Miller, K. E. Huebner y
L. J. Liang, «Evaluation of Tablet Apps to Encourage Social
Interaction in Children with Autism Spectrum Disorders,» de
CHI 2013: Changing Perspectives, Paris, France, 2013.
[22] Pyramid Educational Consultants, Inc (PECS), Marzo 2015.

XVI Congreso Internacional de Interacción Persona-Ordenador | Games

| 360 |
USABILITY AND USER EXPERIENCE (I)
 ECUSI: una herramienta que apoya la Evaluación Colaborativa de la Usabilidad de Sistemas Interactivos
A. Solano | J. C. Cerón | C. A. Collazos | H. M. Fardoun | J. L. Arciniegas

ECUSI: una herramienta que apoya la Evaluación

Colaborativa de la Usabilidad de Sistemas Interactivos
Andrés Solano Juan Camilo Cerón César A. Collazos
Universidad Autónoma de Occidente Universidad del Cauca Universidad del Cauca
Cll 25# 115-85, vía Cali-Jamundí. Carrera 3 #3N-100, Sector Tulcán. Carrera 3 #3N-100, Sector Tulcán.
(57)+2+3188000, ext. 11378 (57)+2+8209800, ext. 2133 (57)+2+8209800, ext. 2133
[email protected] [email protected] [email protected]

Habib M. Fardoun José Luis Arciniegas

King Abdulaziz University Universidad del Cauca
Abdullah Sulayman, Jeddah 22254 Carrera 3 #3N-100, Sector Tulcán.
(966)+5+45668512 (57)+2+8209800, ext. 2114
[email protected] [email protected]

RESUMEN satisfacción cuando lo usan, lo miran, lo sostienen, lo abren o

La usabilidad es una característica de calidad fundamental para el cierran” [1]. La UX abarca diferentes facetas relacionadas a la
éxito de un sistema interactivo. Corresponde a un concepto que calidad de un producto software como [2]: usabilidad,
incluye una serie de métricas y métodos con el objetivo de obtener accesibilidad, emotividad, multiculturalidad, jugabilidad, entre
sistemas fáciles de usar y aprender. Los métodos de evaluación de otras. Actualmente, el término UX está siendo ampliamente
usabilidad son bastante diversos, por tal razón, la elección de un utilizado, tanto así que estándares como la ISO relacionada a
método o combinación de métodos para evaluar la usabilidad de calidad de uso y SQuaRE (Systems and Software Quality
un sistema interactivo no resulta sencilla. Por otro lado, la Requirements and Evaluation) [3], ya se refieren a esta
literatura ofrece descripciones a un alto nivel sobre el proceso de terminología. Así, la herramienta software presentada en este
evaluación de usabilidad. En ese sentido, este artículo presenta la artículo se enfoca exclusivamente en la faceta usabilidad de la
herramienta software ECUSI – Evaluación Colaborativa de la UX, siguiendo el enfoque de una usabilidad formativa [4].
Usabilidad de Sistemas Interactivos, la cual pretende apoyar la “La usabilidad es un atributo de calidad del software” [5] que
ejecución de las actividades que conforman un conjunto de conlleva una serie de métricas y métodos con el objetivo de
métodos de evaluación de usabilidad. ECUSI pretende ser útil obtener sistemas fáciles de usar y de aprender, además influye
para evaluadores de usabilidad de sistemas software interactivos. directamente en el éxito de cualquier nueva aplicación o sistema
interactivo [6]. La usabilidad reduce los errores ocasionados por
Categorías y Descriptores Temáticos los usuarios y lleva a que estos realicen las tareas de manera más
H.1.2 [Modelos y Principios]: Sistemas Usuario/Maquina – eficiente y efectiva, aumentando así su satisfacción y mejorando
factores humanos. su experiencia global con la aplicación o sistema con el cual
interactúan [4].
Términos generales
Factores Humanos. Ahora bien, una variedad de investigadores han ejecutado varios
métodos que intentan evaluar el grado de satisfacción de la
Palabras claves usabilidad de diferentes sistemas interactivos, sin embargo, en
Usabilidad, métodos de evaluación, evaluación colaborativa de la dichos trabajos la información detallada del proceso, como:
usabilidad. entregables, requerimientos, roles, entre otra, no es lo
suficientemente bien definida. Así, en [7] se propone la
1. INTRODUCCIÓN especificación colaborativa de un conjunto de métodos de
Los sistemas interactivos están creciendo en popularidad, y evaluación (elaborada siguiendo los principios definidos en la
actualmente con las innovaciones tecnológicas, ocupan un lugar Ingeniería de Colaboración [8]), la cual provee una secuencia de
de importancia en la sociedad; esto incrementa el potencial de actividades bien definidas, especificación de entregables,
dichos sistemas para que los usuarios puedan entretenerse, descripción de los diferentes participantes del proceso de
informarse, comunicarse o usarlos en diferentes áreas de evaluación y especificación del proceso de comunicación entre los
aplicación. Lo anterior implica un constante desafío, como es participantes. Esto con el fin de ofrecer documentación acerca de
mejorar la calidad de los sistemas interactivos. cómo ejecutar evaluaciones colaborativas de usabilidad de
En el entorno actual, en el que los sistemas interactivos están sistemas interactivos.
dirigidos a un público cada vez más amplio, a usuarios cada vez Por otro lado, la evaluación de la usabilidad de un sistema
menos expertos en el manejo de los mismos, la Experiencia de interactivo es una de las etapas más importantes dentro del diseño
Usuario (UX, por sus siglas en inglés User eXperience) [1] es un centrado en el usuario [9]. Los Métodos de Evaluación de
aspecto fundamental para el éxito de dichos sistemas. La UX se Usabilidad (MEU), que permiten medir la aplicación de éste
refiere a “cómo se sienten las personas acerca de un producto y su atributo en cierto sistema y bajo ciertos factores, son bastante

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 362 |
diversos. Su realización depende de variables tales como: costos, soporte a procesos colaborativos. Adicionalmente, son escasas las
disponibilidad de tiempo, recursos humanos, entre otros. Así, la guías existentes acerca de qué métodos son apropiados en qué
elección de un método o combinación de métodos para evaluar la circunstancias o escenarios, por lo cual la selección de los MEU
usabilidad de un sistema interactivo no resulta sencilla [10]. De se basa en la experiencia personal.
esta manera, en [7] han sido propuestas una serie de
combinaciones de MEU (según escenarios específicos) para 3. COMBINACIONES DE MEU
evaluar sistemas interactivos que corresponden a diferentes áreas Los MEU se agrupan generalmente en [9]: métodos de inspección
de aplicación. Dichas combinaciones de MEU son presentadas y métodos de prueba, los cuales pueden ser usados iterativamente
más adelante en la sección 3. y en etapas distintas del desarrollo de un sistema. Los MEU tienen
fortalezas y debilidades y están enfocados a avaluar aspectos
Con base en lo anterior, han sido especificados una serie de específicos de usabilidad, por lo que expertos en el área
requerimientos con el objetivo de implementar una herramienta recomiendan combinarlos en una evaluación para
software que brinde soporte a la especificación colaborativa de un complementarlos entre sí [11]. Así, es posible definir los
conjunto de MEU, así como también a la ejecución de los problemas de usabilidad en una primera etapa (con expertos), para
mismos. La herramienta denominada ECUSI (Evaluación luego evaluar empíricamente la influencia de esos problemas (con
Colaborativa de la Usabilidad de Sistemas Interactivos), pretende usuarios representativos). Con base en lo anterior, ECUSI soporta
apoyar la ejecución de las actividades que conforman un conjunto las siguientes combinaciones de MEU [12].
de MEU, esto haciendo uso de documentos compartidos en
Google Docs que permiten la participación de diferentes personas 3.1 Evaluación global: alta detección de
involucradas en el proceso de evaluación.
problemas
Mediante este artículo se pretende dar a conocer la herramienta a Esta combinación está enfocada en el análisis completo de un
practicantes y/o evaluadores de usabilidad, la cual fue concebida sistema, incluye los métodos: evaluación heurística, interacción
con el objetivo de que les oriente durante el proceso de evaluación constructiva y entrevistas (ver Figura 1). Esta combinación se
de usabilidad de sistemas interactivos que pertenecen a diferentes estima que funciona correctamente cuando es requerido un
áreas de aplicación, tales como: televisión digital interactiva, web análisis de tipo global, mediante el cual será identificado un buen
transaccional, aplicaciones móviles, redes sociales, entre otras. número de problemas de usabilidad, tanto por parte de los
evaluadores como del análisis de la interacción de los usuarios
La sección 2 presenta una serie de trabajos relacionados. La representativos. Las entrevistas, como método de interrogación
sección 3 describe un conjunto de combinaciones de MEU complementario, van a permitir obtener información
disponibles en la herramienta software. La sección 4 describe la adicional/complementaria acerca de la percepción de los usuarios
herramienta software propuesta, luego, la sección 5 presenta respecto al sistema evaluado, con lo cual también sería posible
información relacionada al proceso de desarrollo de la misma. confirmar problemas críticos identificados por los dos métodos
Finalmente, la sección 6 presenta algunas conclusiones y trabajos antes realizados.
futuros.

2. TRABAJOS RELACIONADOS
Existen herramientas que son capaces de sugerir una serie de
métodos a utilizar en el proceso de evaluación de usabilidad, tal es
el caso de Usability Planner [10]. Esta herramienta apoya el
proceso de selección de los MEU más apropiados para ejecutar en
función de las características de un proyecto y restricciones
organizacionales. En dicha herramienta las reglas empleadas se
derivan de las normas ISO y son complementadas con reglas
según la experiencia del autor. Usability planner sugiere un
conjunto de MEU a utilizar según una serie de reglas, sin
embargo, no ofrece soporte para ejecutar colaborativamente las
actividades que conforman los métodos sugeridos.
Por otro lado, expertos en usabilidad recomiendan aplicar
métodos de prueba una vez se han aplicado métodos de
inspección, esto con el fin de obtener información más adecuada Figura 1. Evaluación global - alta detección de problemas.
respecto a la usabilidad de un sistema. Existe una gran variedad de 3.2 Evaluación específica: reducción de
servicios software para la evaluación de usabilidad, tales como los
que se ofrecen en los sitios web: Usertesting.com, Userzoom.com, tiempo
Loop11.com, entre otros. Sin embargo, en una buena cantidad de Esta combinación está enfocada a evaluar ciertos escenarios o
estos servicios los MEU pueden aplicarse de forma independiente. funcionalidades de un sistema, incluye los métodos: evaluación
Es así como ECUSI supera estas desventajas y ofrece la heurística, método del conductor y cuestionarios (ver Figura 2).
posibilidad de ejecutar MEU de forma independiente o Esta combinación, resulta ser de utilidad para evaluar
combinaciones de estos, combinaciones que están predefinidas o funcionalidades específicas debido a que la información obtenida
pueden ser creadas por el usuario. mediante el método del conductor permite detectar problemas en
aquellos puntos donde el usuario solicita ayuda/información al
En general, y teniendo en cuenta el estado actual de la literatura, evaluador (conductor). Esto es, en los puntos donde hay
son escasas las herramientas software propuestas para la comunicación entre el usuario y conductor es muy probable que
evaluación de la usabilidad de sistemas software interactivos que haya necesidades de información en el sistema.
contemplen la combinación de MEU, y que además, brinden

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 363 |
 4.1 Uso del sistema
ECUSI ofrece las siguientes funcionalidades generales:
• Consulta de la especificación colaborativa de un
conjunto de métodos de evaluación de usabilidad (de
inspección y prueba).
• Creación de proyectos de evaluación los cuales están
conformados por la combinación de varios MEU.
• Edición de documentos compartidos para el desarrollo
de las actividades (colaborativas y no colaborativas) que
conforman los MEU seleccionados en un proyecto de
evaluación.
4.1.1 Consulta de la especificación colaborativa de
Figura 2. Evaluación específica – reducción de tiempo. MEU
3.3 Evaluación enfocada a tareas específicas: ECUSI brinda soporte a la ejecución de los métodos: evaluación
heurística, recorrido cognitivo, experimentos formales,
sin restricciones de tiempo interacción constructiva, método del conductor, entrevistas y
Esta combinación tiene como objetivo analizar tareas concretas de cuestionarios. Además, provee la especificación colaborativa de
un sistema interactivo, incluye los métodos: recorrido cognitivo, dichos métodos mediante los elementos [8]: Modelo de
experimentos formales y cuestionarios (ver Figura 3). En esta Facilitación del Proceso (MFP) y Agenda detallada. El MFP
combinación los tres métodos aportan sus importantes presenta el flujo del proceso de manera gráfica mediante una
características, pero son los experimentos formales los que notación propuesta para el modelado de procesos colaborativos
marcan la diferencia respecto a la evaluación específica. Los [13], mientras que la agenda detallada presenta de manera
experimentos formales permiten realizar un eficiente análisis de extendida información de las actividades que forman parte del
las tareas de interés. Estos tienen un buen nivel de objetividad y se proceso diseñado, tal como: nombre de la actividad, entregables,
complementan de forma adecuada con los cuestionarios (pre-test y patrón de colaboración, proceso de comunicación (en caso de que
post-test), que también presentan buena objetividad y permiten la actividad sea colaborativa), participantes, entre otra. La
obtener información cuantitativa. Así, al ejecutar los 3 métodos especificación colaborativa de los MEU está dividida en 3 etapas:
que conforman esta combinación, se estima que la información planeación, ejecución y análisis de resultados. Así, cada método
obtenida acerca de la usabilidad de las tareas sería completamente de evaluación tiene asociados 3 MFP y 3 agendas detalladas,
objetiva. Sin embargo, esta combinación conviene utilizarla correspondientes a las etapas mencionadas.
cuando la disponibilidad de tiempo es alta.
4.1.2 Creación de proyectos de evaluación
En ECUSI, los usuarios registrados en el sistema pueden crear
proyectos de evaluación de usabilidad. Cuando el usuario crea un
proyecto asume el rol de evaluador supervisor (creador del
proyecto) y debe ingresar información básica como es: título y
descripción del proyecto. También, debe seleccionar el tipo de
sistema a evaluar (web, nativo, videojuego u otro) y el dispositivo
hardware que lo soporta (móvil, tableta, televisor, computador
personal, consola de videojuegos u otro). Adicionalmente, un
proyecto puede estar conformado por uno o más MEU. Para ello,
el usuario puede seleccionar de forma manual el o los métodos a
ejecutar, o incluso, puede seleccionar alguna de las combinaciones
de MEU predefinidas (ver sección 3). Finalmente, el usuario debe
seleccionar la lista de evaluadores que colaborarán en el proceso.
Los evaluadores invitados reciben un correo electrónico mediante
el cual pueden confirmar su participación en el proyecto.
Figura 3. Evaluación enfocada a tareas específicas - sin
restricciones de tiempo. El usuario puede gestionar (consultar, editar y eliminar) los
proyectos creados. El Gestor de proyectos (opción disponible en
4. ECUSI - Evaluación Colaborativa de la el menú principal) permite al usuario observar la fecha de
Usabilidad de Sistemas Interactivos creación del proyecto, fecha de la última actualización, estado
ECUSI es una aplicación web que brinda soporte a la ejecución de (activo o terminado) de un proyecto, incluso puede observar el
las actividades que conforman un conjunto de métodos de porcentaje de cumplimiento de las actividades que conforman los
evaluación de usabilidad. Esto haciendo uso de documentos MEU del proyecto. También es posible consultar aquellos
compartidos en Google Docs que permiten la participación de proyectos en los cuales ha sido invitado como evaluador.
diferentes personas–a menudo distribuidas geográficamente– Una vez el proyecto se encuentra en ejecución, es posible que el
involucradas en el proceso de evaluación. ECUSI fue concebida evaluador supervisor seleccione las actividades cumplidas. De
con el objetivo de ofrecer a los practicantes de la usabilidad una esta manera, el porcentaje de cumplimiento del proyecto es
herramienta que los oriente durante el proceso de evaluación de actualizado automáticamente. Adicionalmente, ECUSI presenta la
usabilidad de diferentes sistemas interactivos. lista de evaluadores que participan en el proyecto de evaluación

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 364 |
de tal forma que es posible consultar su perfil y establecer versión final) que permita efectuar pruebas con usuarios reales. Se
contacto mediante correo electrónico. Es importante mencionar sugiere que el sistema a evaluar tenga cierto grado de avance o
que en los métodos de prueba ECUSI ofrece la posibilidad de funcionalidad, para que los expertos puedan evaluarlo de una
compartir material multimedia (grabaciones de las pruebas) a los mejor manera y puedan obtenerse resultados más completos.
evaluadores mediante un enlace creado en Dropbox, por ejemplo.
Esto con el fin de que el material sea observado, y posteriormente, Mediante ECUSI puede contarse con la colaboración de
los evaluadores identifiquen los respectivos problemas de evaluadores que se encuentran distribuidos geográficamente, con
usabilidad. lo cual los gastos (de movilidad, por ejemplo) se disminuyen en
gran medida. Por otro lado, se tiene la ventaja de que los
4.1.3 Edición de documentos compartidos practicantes de la usabilidad pueden consultar la especificación
Luego de que el evaluador ha creado un proyecto de evaluación, colaborativa de un conjunto de MEU, ya que en la literatura la
automáticamente son creados 3 documentos compartidos para descripción de estos se tiene a un alto nivel. Por último, se puede
cada método incluido en el proyecto. Cada documento compartido acceder a ECUSI mediante Internet, por lo cual es accesible desde
corresponde a una plantilla que contiene información sobre las cualquier lugar y desde cualquier dispositivo que disponga de
actividades a realizar. Adicionalmente, por cada etapa de un conexión a la red.
método de evaluación (planeación, ejecución y análisis de Las combinaciones de MEU predefinidas en ECUSI permitirían
resultados) son presentados como recursos adicionales imágenes cubrir los puntos más críticos para obtener la medida de la
de los MFP y agendas detallas. usabilidad con un nivel de precisión aceptable. Primero, existe,
Adicional a las funcionalidades anteriores, ECUSI ofrece una por lo menos, un método de inspección y un método de prueba.
serie de recursos (documentos) que pueden ser de utilidad para los En segundo lugar, incluyen evaluaciones que realizan análisis
usuarios (evaluadores), tales como: formato de acuerdo de cuantitativos y cualitativos, objetivos y subjetivos, y de
confidencialidad, formato de documento guía para una evaluación evaluación global y evaluación específica del sistema. Por esto, es
heurística, formato de documento guía para el usuario en los posible decir que se están cubriendo todos los factores necesarios
experimentos formales, entre otros. para evaluar, en buena medida, la usabilidad de un sistema
software interactivo.
Por último, al finalizar un proyecto de evaluación, se ofrece al
evaluador la posibilidad de incluir realimentación acerca de la 5. PROCESO DE DESARROLLO
combinación de MEU utilizada. El evaluador puede calificar La herramienta ECUSI ha sido diseñada siguiendo el enfoque de
(mediante un número de puntos) la combinación utilizada, Diseño Centrado en el Usuario [14]. Desde el inicio del proceso
además, puede incluir información acerca de aspectos positivos y de desarrollo fueron incluidos practicantes de la usabilidad y
negativos según la experiencia con los métodos seleccionados. evaluadores con el fin de conseguir que el producto sea agradable
Esto con el fin de que los evaluadores conozcan dicha para dicho público objetivo.
información y decidan qué combinación de MEU resulta
apropiada, según experiencias de otros. En primer lugar, se realizó un análisis preliminar de requisitos que
debía tener la herramienta software. Este análisis fue realizado
4.2 Ventajas con posibles usuarios finales de la herramienta, es decir, con un
ECUSI trata de integrar tecnologías conocidas para los conjunto de personas que realizan frecuentemente diferentes
evaluadores, por tal razón hace uso de los documentos métodos de evaluación de usabilidad (de inspección y prueba). De
compartidos en Google Docs como herramienta ofimática este análisis de requisitos surgieron las funcionalidades básicas de
colaborativa, los cuales resultan familiares y de uso constante para ECUSI y la información que esta debía incluir.
los evaluadores. En segundo lugar, fueron diseñados prototipos en papel para
ECUSI es lo suficientemente flexible para permitir la inclusión de mostrar a los usuarios una primera idea de cómo evolucionaría la
un mayor número de MEU. La primera versión de ECUSI incluye interfaz y cómo sería presentada la información en ella. Una vez
un conjunto de métodos de inspección y prueba, sin embargo, lo obtenida la realimentación de los usuarios sobre los prototipos,
esperado es incluir la especificación colaborativa de más métodos una serie de requisitos fueron ajustados y los prototipos
con el fin de ampliar el abanico de opciones para la evaluación de actualizados.
sistemas interactivos. De igual forma, ECUSI permite la Posteriormente, fueron planeadas y priorizadas las
definición de nuevas combinaciones de MEU que obedecen a funcionalidades que tendría la herramienta, y de esta manera,
otros escenarios, factores u objetivos de evaluación. hacer iteraciones incrementales con entregas constantes a los
Como ya se ha mencionado, ECUSI almacena información interesados para su respectiva verificación y validación. Durante
(especificación colaborativa) de un conjunto de MEU, tal como: este proceso fueron realizadas evaluaciones de usabilidad
actividades, entregables, patrones de colaboración, proceso de informales en las instalaciones de la Universidad del Cauca
comunicación, participantes, entre otra. Así pues, ECUSI podrá (Colombia), específicamente fue aplicado el método del conductor
ser utilizada tanto por practicantes de la usabilidad como por con el fin de descubrir las necesidades de información de los
evaluadores de sistemas interactivos que deseen llevar a cabo un usuarios en el sistema.
estudio desde un enfoque de usabilidad formativa [4]. Luego de seis iteraciones, la herramienta fue liberada en versión
La herramienta ECUSI puede ser utilizada en, prácticamente, beta, la cual fue evaluada con expertos y usuarios potenciales
cualquier etapa del ciclo de desarrollo de un sistema interactivo. mediante el uso de la combinación específica (ver sección 3.2).
De igual forma, las combinaciones de MEU pueden ser utilizadas Esta combinación incluye los MEU: evaluación heurística,
tanto en etapas tempranas de diseño como en etapas intermedias, métodos del conductor y cuestionarios, tal como se presenta en la
aunque probablemente se adaptan mejor en etapas tempranas, siguiente sección.
cuando se tenga un prototipo funcional (no necesariamente una

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 365 |
5.1 Evaluación de usabilidad Debido al uso del framework Django, ECUSI cuenta con una
La evaluación heurística fue realizada por un conjunto de 3 arquitectura MTV (Model Template View), similar a la usual
evaluadores que inspeccionaron el diseño de la interfaz de ECUSI arquitectura MVC (Model View Controller). La herramienta corre
con base en una serie de principios específicos para aplicaciones en el servidor web Nginx y utiliza MySQL para almacenar los
Web, definidos en [15]. Este método permitió identificar una serie datos persistentes.
de problemas mayores y menores en el sistema. Si bien, no se
trató de problemas que colocaban en riesgo el funcionamiento del
6. CONCLUSIONES Y TRABAJO FUTURO
sistema, sí atentaban contra la facilidad de uso, y por lo tanto, con ECUSI pretende ser una herramienta de uso frecuente para la
el buen aprovechamiento que este pudiese tener. En general, el evaluación colaborativa de la usabilidad de sistemas software
nivel de criticidad de los problemas fue bajo, una poca cantidad de interactivos que pertenecen a diferentes áreas de aplicación, tales
como: televisión digital interactiva, web transaccional, redes
problemas (6 de 24) fueron calificados, en promedio, con notas
mayores a 6 (en una escala de 0 a 8), y 18 de los 24 problemas sociales, videojuegos, entre otros.
detectados fueron calificados con notas inferiores a 6. El alto Considerando que en trabajos relacionados la información
número de problemas de usabilidad identificados tuvo su causa en detallada del proceso de evaluación de usabilidad no es lo
que el sistema no estaba terminado por completo, con lo cual la suficientemente bien definida, la herramienta ECUSI presenta la
versión beta de ECUSI presentó varios aspectos posibles de especificación colaborativa de un conjunto de MEU. Dicha
mejorar. especificación provee una secuencia de actividades bien definidas,
El método del conductor fue realizado por un conjunto de 3 especificación de entregables, descripción de los diferentes
participantes del proceso de evaluación y especificación del
evaluadores y 8 usuarios potenciales adecuados al perfil de
usuario definido. Previa firma del acuerdo de confidencialidad, los proceso de comunicación entre los participantes de la evaluación.
usuarios realizaron las tareas indicadas por el conductor sobre el En ese sentido, se estima que el aprendizaje, tanto de ECUSI
sistema (con escenario de uso preestablecido). Los sucesos como de los MEU soportados en ella, no llevaría una cantidad
ocurridos durante la prueba fueron grabados (previa autorización significativa de tiempo ya que se ofrece información detallada al
de los usuarios) y distribuidos entre los evaluadores para su usuario (evaluador) para que lleve a cabo un determinado proceso.
análisis. La ejecución de este método permitió identificar 15 Por otro lado, se tiene escasa documentación (guías o
problemas de usabilidad en las funcionalidades estudiadas, entre lineamientos) acerca de cómo ejecutar evaluaciones colaborativas
los cuales fueron confirmados los 6 problemas más críticos de usabilidad de sistemas interactivos. Con base en lo anterior,
identificados en la evaluación heurística. surgió la idea de desarrollar la herramienta ECUSI, la cual se
Los cuestionarios fueron realizados luego de llevar a cabo el espera sea utilizada por practicantes de la usabilidad o personas
método del conductor, por lo cual el número de usuarios que responsables de estructurar el proceso de evaluación de la
diligenciaron los cuestionarios después (post-test) de dicho usabilidad de diferentes sistemas interactivos. La finalidad
método fue 8, una cifra aceptable para obtener conclusiones sobre perseguida es brindar una herramienta software a la comunidad
la percepción subjetiva de los usuarios. Los cuestionarios para facilitar el desarrollo de una cultura de evaluación de la
permitieron obtener resultados alentadores respecto a la usabilidad de software.
satisfacción subjetiva de los usuarios con el sistema. Algunos Como principales actividades futuras se destacan el refinamiento
promedios superaron la nota 4 (en una escala de 1 a 5), por lo que de la interfaz gráfica de la herramienta. Adicionalmente, conviene
se puede decir que, en general, los usuarios estuvieron conformes incluir en ECUSI una segunda opción de idioma (inglés) con el
con la consistencia y control sobre la aplicación, sin embargo, fin de fomentar su proyección internacional. También, para una
consideraron que el diseño de esta puede mejorarse para aumentar segunda versión de ECUSI conviene incluir requerimientos
significativamente la productividad y satisfacción al usarla. relacionados a la automatización de actividades que conforman la
Una vez identificados los problemas de usabilidad por el método evaluación heurística (tales como: la calificación de severidad y
de inspección (evaluación heurística), y confirmados mediante frecuencia, generación de rankings de criticidad y severidad, entre
métodos de prueba (conductor y cuestionarios), fueron sugeridos otras) e inclusión de mecanismos que promuevan la comunicación
entre evaluadores. Finalmente, animamos a la comunidad
una serie de ajustes a la herramienta software. Luego de
implementar los ajustes respectivos, se liberó una versión estable científica a utilizar la herramienta software con el fin de obtener
de ECUSI la cual está disponible en la URL: realimentación acerca de las combinaciones de MEU que utilicen
http://www.ecussi.com/. en los proyectos de evaluación.
En la herramienta software propuesta la faceta usabilidad ha sido
5.2 Aspectos técnicos de la aplicación considerada como eje central, por lo que convendría ampliar el
ECUSI hace uso de los credenciales de Google para realizar la alcance de la herramienta pasando de “evaluación colaborativa de
autenticación, de esta manera, con un correo de Gmail y su la usabilidad” a “evaluación colaborativa de la experiencia de
respectiva contraseña se puede acceder a las funcionalidades de la usuario”. Así, para un trabajo posterior, sería adecuado incluir (o
herramienta. También implementa la API de Google Drive la cual combinar) elementos de otras facetas, como por ejemplo:
permite gestionar los documentos colaborativos al momento de emotividad, multiculturalidad, entre otras, a la especificación
crear proyectos de evaluación. colaborativa de los MEU.
El back-end (lado del servidor) de la aplicación fue desarrollado
con en el lenguaje de programación interpretado Python, 7. AGRADECIMIENTOS
implementando el framework para desarrollo web Django. Para el Este trabajo ha sido apoyado por el grupo de Investigación y
front-end (lado del cliente) se usó el lenguaje JavaScript y la Desarrollo en Ingeniería de Software (IDIS) de la Universidad del
librería jQuery junto con el framework Twitter Bootstrap para Cauca (Colombia) y el Grupo de Investigación en Telemática e
permitir que la herramienta contará con diseño adaptativo. Informática Aplicada (GITI) de la Universidad Autónoma de
Occidente (Colombia). Ha sido parcialmente financiado por el

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 366 |
proyecto UsabiliTV: “Framework para la evaluación de la [8] G. Kolfschoten, G.-J. D. Vreede, "The Collaboration
usabilidad de aplicaciones en entornos de Televisión Digital Engineering Approach for Designing Collaboration
Interactiva” (COLCIENCIAS, código 110352128462), el Processe," in International Conference on Groupware:
proyecto “Propuesta metodológica para la evaluación Design, Implementation and Use, 2007.
colaborativa de la usabilidad de sistemas software interactivos” [9] R. Otaiza, C. Rusu, S. Roncagliolo, "Evaluating the usability
(Dirección de Investigación, Universidad Autónoma de of transactional Web Sites," in Third International
Occidente). Además, ha sido financiado por la “Convocatoria Conference on Advances in Computer-Human Interactions
para el apoyo a proyectos con Asia 2014” de COLCIENCIAS. (ACHI'10), Saint Maarten, 2010, pp. 32-37.
8. REFERENCIAS [10] X. Ferré, N. Bevan, "Usability planner: a tool to support the
[1] H. Sharp, Y. Rogers, J. Preece, Interaction Design Beyond process of selecting usability methods," in Human-Computer
Human - Computer Interaction, 2 ed.: Wiley, John & Sons, Interaction–INTERACT 2011, ed: Springer, 2011, pp. 652-
Incorporated, 2007. 655.
[2] L. Masip, M. Oliva, T. Granollers, "User experience [11] X. Ferré, "Marco de integración de la usabilidad en el
specification through quality attributes," in Human- proceso de desarrollo software," Tesis Doctoral, Lenguajes y
Computer Interaction–INTERACT 2011, ed: Springer, 2011, Sistemas Informáticos e Ingeniería del Software, Universidad
pp. 656-660. Politécnica de Madrid, Madrid, 2005.
[3] ISO, "International Software Quality Standard, ISO/IEC [12] A. Solano, "Metodología para la evaluación colaborativa de
25010," in Systems and software engineering - Systems and la usabilidad de sistemas software interactivos," Tesis
software Quality Requirements and Evaluation (SQuaRE) - doctoral, Departamento de Sistemas, Universidad del Cauca,
Systems and software quality models, ed, 2011. Popayán, 2015.
[4] J. Nielsen, Usability engineering: Morgan Kaufmann, 1993. [13] A. Solano, T. Granollers, C. A. Collazos, C. Rusu,
"Proposing formal notation for modeling collaborative
[5] ISO, "International Organization for Standardization,
processes extending HAMSTERS notation," in World
Ergonomic requirements for office work with visual display
Conference on Information Systems and Technologies
terminals, ISO 9241," ed, 1998.
(WorldCIST’14), Madeira, Portugal, 2014, pp. 257-266.
[6] T. Granollers, "MPIu+a una metodología que integra la
[14] C. Abras, D. Maloney-Krichmar, J. Preece, "User-centered
ingeniería del software, la interacción persona-ordenador y la
design," Bainbridge, W. Encyclopedia of Human-Computer
accesibilidad en el contexto de equipos de desarrollo
Interaction. Thousand Oaks: Sage Publications, vol. 37, pp.
multidisciplinares," Tesis Doctoral, Departamento de
445-456, 2004.
Sistemas Informáticos, Universidad de Lleida, 2007.
[15] J. Nielsen. (1995, 12/11/2014). Ten Usability Heuristics.
[7] A. Solano, "Metodología para la evaluación colaborativa de
Available:
la usabilidad de sistemas software interactivos," Tesis
http://www.useit.com/papers/heuristic/heuristic_list.html
doctoral, Facultad de Ingeniería Electrónica y
Telecomunicaciones, Universidad del Cauca, Popayán, 2014.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 367 |
 Towards an Integration of Usability and Security for User Authentication
P. C. Realpe | C. A. Collazos | J. Hurtado | A. Granollers

Towards an Integration of Usability and Security for User

Authentication
Paulo C. Realpe, Cesar A. Collazos, Antoni Granollers
Julio Hurtado University of Lleida
University of Cauca GRIHO Research Group
IDIS Research Group Lleida, Spain
Popayan, Colombia [email protected]
{prealpe, ccollazo, ahurtado}
@unicauca.edu.co

ABSTRACT challenges, for instance how conflicts between security and

Computer security is one of the more important tasks currently in usability could be solved. The main question is how a balance
the digital world. However, to the best of our knowledge, there is suitable between security and usability in software applications
little research for designing effective interfaces in the context of could be achieved [5].
security management systems. Security problems for these Usability principles (also called criteria or heuristics) represent an
systems include vulnerabilities due to hard to use systems and instruction manual for a HCI (Human-Computer Interface)
poor user interfaces due to security constraints. Nowadays, designer. Although researches have been partially applying
finding a good trade-off between security and usability is a principles to security systems, some of these criteria have
challenge, mainly for user authentication services. This paper problems to be adapted to the overall methodology, especially in
presents a systematic review about usability principles, evaluation the design and development suitable of user authentication
methods and development processes for security systems. services [6]. Besides, usability evaluation methods are not entirely
Moreover, a research approach to integrate usability and security suitable for evaluating security-based systems, because they do
for user authentication systems is proposed. not identify many of the negative effects on the usability [1].
Also, Zurko and Simon [7], Whitten and Tygar [8] and Chiasson
Categories and Subject Descriptors and Biddle [3] have
I.3 [Security service]: Authentication; J.2.1 [Interaction design]:
shown that cognitive aspects definitely influence the usability of
Interaction design process and methods−User centered design
security mechanisms in the user authentication methods.
General Terms Researchers argue that security concepts used in these
Security, Usability mechanisms are not easily understood by many users.
Authentication is the process for establishing whether someone is
Keywords who declares to be. In computer networks, authentication is
Usable security, HCISec, authentication, principles, evaluation, frequently implemented using passwords. There are six factors of
design process user authentication that might be employed to increase the
security level: something you have (smart card), something you
1. INTRODUCTION know (passwords or PIN), something you are (fingerprint),
Computer security is the area in computer science in charge of the something you recognize (graphical passwords), something you
confidentiality and integrity of the systems and data. In order to do (keystroke) and where you are (GPS) [5].
manage the security problems, the human aspects require a special
attention at development time [1]. Security has been an important The HCI community has been gradually developing research work
factor in many interactive systems, whereas, usability has been a in usable security guidelines for software such as design
driver required in these kind of systems. However, there is a belief principles for security [9], and guidelines for designing and
that security is only related with technical aspects and it could be evaluating usable secure software [10]. However, to the best of
designed to user interface. On the one hand, only technical aspects our knowledge, there are no suitable usability principles to
are normally considered for designing the security requirements provide evaluation methods that take into account security and
[2]. Also, the design of usable interfaces require that security usability for developing a process or technique from the system
issues be considered because humans are prone to mistakes. design to implementation and the application for user
authentication services [6].
Usable security has focused in designing interfaces for achieving
a trade-off between usability and security [3]. This paper presents a systematic review about usability principles,
Usable security, according to Jøsang and Patton [4], deals with evaluation methods and development processes for security
managing security information in the user interfaces. Security and systems. Moreover, a research approach to integrate usability and
security for user authentication systems is proposed.
usability can vary according to usage, including users profiles
(who are the users), tasks, hardware (including network 2. HCI SECURITY
equipment), software, and physical or organizations environments
Johnston et al. [11] define HCI Security (HCISec) as: “the part of
[2]. However, there is little research in usable security in
a user interface which is responsible for establishing the common
particular, about the relationship between security and usability.
ground between a user and the security features of a system.
For designing secure and usable systems, these present critical

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 368 |
 Towards an Integration of Usability and Security for User Authentication
P. C. Realpe | C. A. Collazos | J. Hurtado | A. Granollers
HCISec is human computer interaction applied in the area of The criteria used by Yee [10] for admitting an essential principle
computer security”. They claim that the aim of HCISec is to is that it should be a valid and non-trivial concern. Each principle
improve in the user interface in order to achieve reliability and is valid by showing how a violation of the principle would lead to
robustness. This leads to the system becoming more secure, robust a security breach. In the statement of these ten principles, the term
and reliable. However, security features sometimes are perceived "actor" is used to mean “user”. The term “authority” only refers to
to make a system more difficult to use. HCISec addresses this the capability to take a particular action.
issue using a set of principles for achieving a trade-o_ between Garfinkel [18] proposes how a trade-off between security and
security and usability. usability could be improved implementing specific functionality
HCISec has little associated research, particularly oriented to in current operating systems and applications. To carry out this
authentication methods [6]. Despite security methods are aim, Garfinkel developed a set of design principles for building
understood, implemented and used, human factors must be taking and evaluating security systems using the Yee's principles [10].
into account in their design. According to CompTIA (Computing Although end-users are the principal actors in the usable security
Technology Industry Association), human error is the main cause field, the interfaces for security are important too since the
of security breaks and it accounts the 80% of security problems in consequences of usability problems can be vulnerable to attacks.
the organizations [12]. Whitten and Tyggar [8] presented the first Chiasson et al. [19] developed an initial set of ten design
work about HCISec where they evaluated usability of an principles for security management interfaces. For their usability
authentication method for implementing privacy in an e-mail study using a two-password based authentication methods, they
application, called PGP (Pretty Good Privacy). proposed two additional principles based in the Whitten and
Johnston et al. [11] propose seven principles for HCISec: convey Tygar's principles [8].
features, visibility of system status, learnability, aesthetic and Major risks are caused by poor usability. It is necessary to
minimalist design, errors, satisfaction and trust. These criteria are consider vulnerability analysis and risk assessment in order to
based on Nielsen's heuristics [13] and have been modified and properly manage current and emerging risks. Jøsang et al. [20]
synthesized in order to address the essentials aspects in a security examine a set of eight usable security principles, they propose
environment. The HCISec principles can be used by software how can be incorporated into the risk management process, and
engineers for allowing that usability is developed into a security discuss the benefits for applying these principles and process to
user interaction. The principles can also be used for evaluating the current and future security solutions.
interfaces of new security products [11].
Herzog and Shahmehri [21] present concrete and verified
Katsabas et al. [14] presented ten HCISec principles based in guidelines for enhancing usability and security that delegates
Nielsen [13] and Johnston et al. [11] for supporting the inclusion security decisions to users and captures these user decisions as a
of security features within applications. The HCISec community security policy. They hypothesized that existing tools for runtime
has considered the challenge for designing usable security from set-up of security policies are not sufficient. As this proved true,
two different viewpoints: design principles and user-centered they apply usability engineering with user studies to advance the
security. Zurko and Simon [7] define user centered security as: state-of-the-art.
“security models, mechanisms, systems, and software that have
usability as a primary motivation or goal”. An analysis of user interfaces related to security has shown a need
in for generating usable interfaces that help users to make
3. USABLE SECURITY PRINCIPLES decisions. Ibrahim et al. [22] proposed a further set of HCISec
Please In order to design and evaluate user interfaces, a set of usability principles addressing the interface design of security
design rules and principles are required. Usability rules and alerts to the end-users; these principles take into account explicit
principles give the direction to the designer to produce usable and useful information for allowing a timely response and a
systems [15]. Among these principles we have: Shneiderman's consistent presentation of information.
eight golden rules [16], Nielsen's ten heuristics [13] and Gestalt The works presented in the literature have focused on identifying
principles of visual perception, where these principles help us to usability issues of security and propose guidelines and
identify which is figure and which is ground [17]. recommendations to address them. Nurse et al. [23] present an
The work of Whitten and Tygar [8] is a pioneer work in usable overview of the major developments in cybersecurity usability
security field for an application of public key infrastructure (PKI). and HCISec, which provide guidance and recommendations for
This application was evaluated as a good user interface, but the highly usable cybersecurity systems. One of the main
results shown that was not suitable usable to provide effective contributions of this paper is to consolidate a number of existing
security for most of the users. Design strategies for developing design guidelines and offer an initial core list for future work.
usable security need take into account these properties.
4. USABLE SECURITY DESIGN PROCESS
Saltzer and Schroder [9], define the principle of psychological
acceptability as: “it is essential that the human interface be AND EVALUATION
designed for ease of use, so that users routinely and automatically Place Braz et al. [5] assert the need to develop a framework that
apply the protection mechanisms correctly. Also, to the extent that provides evaluation methods which take into account security and
the user's mental image of his protection goals matches the usability for user authentication mechanisms. Braz et al. [24]
mechanisms he must use, mistakes will be minimized”. According introduce a new cognitive model that aims to model the tourist
to the above, they present eight practical principles for the task when using a user authentication system. This method can to
architecture-level software design, regardless of platforms or assist security designers to specify, design, inspect, and evaluate
languages used. Software developers, use these design principles the security as well as usability aspects of user authentication
as a benchmark for building a more secure software [6]. mechanisms. This model integrates usable security concerns

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 369 |
 Towards an Integration of Usability and Security for User Authentication
P. C. Realpe | C. A. Collazos | J. Hurtado | A. Granollers
earlier into the requirements and design phase of the development analysis improves their understanding of security, and increases
lifecycle. their motivation to comply with policies.
Mapayi et al. [25] develop a model for evaluating a functional Markotten [34] proposed the concept of user-centered security
usable authentication method using artificial neural networks. engineering to bridge the gap between security and usability. This
Jaferian et al. [26] propose and evaluate a new set of heuristics for method has been pursued for the development and implementation
evaluating ITSM (Information Technology Security Management) of a security tool. This model can be a good starting point for
tools. The focus of these heuristics is oriented to and problems developing a complete model for designing and implementing
that hinder the use of tools in those ITSM activities that are usability in security software. Yeratziotis et al. [35] present a
distributed over time and space, involve collaboration between framework within the context of on-line social networks that are
different stakeholders, and require knowledge to deal with the particular to the health domain. The framework consists of three
complexity. components: a phase process, a validation tool and a usability
Braz [5] proposes a trade-off between security and usability evaluation.
frameworks. She establishes a new usable security protocol using Faily [36] presents a framework called IRIS (Integrating
an inspection method called USS (Usable Security Symmetry), to Requirements and Information Security) for specifying usable and
address the usable security issues in the context of user secure systems. The framework considers the system design
authentication methods. These methods allow the design of process using three viewpoints: usability, security and
authentication systems which are secure and usable. When trying requirements. This work shows that IRIS integrates techniques
to relate more closely to web-based authentication, the framework and tools for their supporting the design of usable and secure
given by Straub and Baier [27] is very useful. They present a systems. Finally, Parveen et al. [37] propose a flow diagram for
framework for evaluating PKI-based applications including 15 secure and usable requirements specification process which
evaluation categories with accompanying example questions. identifies functional and non-functional requirements which
Mihajlov et al. [28] present a quantifying approach for evaluating includes threats, vulnerabilities and risks.
security and usability that can be used in authentication
mechanisms and is based in Renaud's work [29].The purpose of
5. ANALYSIS OF APPROACHES
this approach is to guide the evaluation process of authentication 5.1 Research Method
mechanisms in a given environment by balancing usability and The research method is based in the systematic review proposed
security and defining quantifiable quality criteria. Palmer [30] by Kitchenham et al. [38]. The systematic review starts
establishes over 200 evaluation criteria for aiding to take decision establishing some research questions according to a proposed
on the selection of the most appropriate mechanism for enabling approach. The main research questions were:
computer systems to identify individuals using APIM (Automated
Personal Identification Mechanisms). Q1: What usability principles are applied to security systems,
mainly for user authentication?
The use of design tools suitable for a specific software
development process, is a complex task that requires training and Q2: Which methodological approaches are used for evaluating the
experience. Design tools selected for different requirements can usability in security and in user authentications systems?
conflict with each other, reducing their effectiveness and usability Q3: What processes aid to design secure systems and user
and causing security vulnerabilities. Hausawi and Allen [31] authentication services based on principles and methods of
proposed principles to improve the selection of the appropriate evaluation according to usable security requirements?
design tools for security, usability, and usable security in order for
the design process to meet the requirements' needs. Moreover, the For the search process, on-line databases such as IEEE, Springer,
principles help to identify trade-offs in the selection of design ACM, Science Direct, Scopus and Web of Knowledge were used.
tools by dis covering conflicts and helping to overcome In order to filter the search, keywords according to the research
contradictions in design decisions. questions were used. The keywords used for the systematic review
were:
Current approaches to security fail to implement user centered
designs, resulting in systems hard to use and ineffective security 1. “usable security” AND “design process”
mechanisms. A way for achieving a trade-off between security 2. “usable security principles” AND “authentication principles”
and usability, can include a description of how design decisions 3. “usable security evaluation” AND “usability authentication
can lead to find vulnerabilities while users use the system. Collet evaluation”
[32] presents requirements and guidelines for designing usable 4. "authentication design process” AND “security design
and secure systems, and examines how these principles can be process”
incorporated into the software development process. 5. “user-centered security” AND “user-centered authentication”
Also, the search process is limited to the years from 1995 to 2015.
AEGIS (Appropriate and Effective Guidance for Information The papers found were filtered according to their title, keywords
Security) presented by Flechais et al. [33] was initially designed and an analysis of the abstract, introduction and conclusion. Table
to help developers to identify security challenges early in the 1 presents the on-line databases, amount of papers found and
development process and supplying systematic methods. The relevant papers according to the above criteria.
main aim of AEGIS is to provide better support for the
development of secure systems. This method includes asset
identification, risk and threat analysis and context of use, and its
application to case studies. An additional benefit of the method is
that the involvement of stakeholders in the high level security

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 370 |
 Towards an Integration of Usability and Security for User Authentication
P. C. Realpe | C. A. Collazos | J. Hurtado | A. Granollers
Table 1: On-line Databases According to the works presented in Table 4, there are some
process design methods considering usability and usability
Database Papers Found Relevant Papers
concerns. However, the lack of suitable usable security principles
IEEE 18 3
and evaluation methods taking into account to user, are not
Springer 7 3
considered in these design processes mainly for user
ACM 14 4 authentication. An important feature of these design processes is
Science Direct 11 2 that they are focused on some phase of the development process.
Scopus 13 10 Therefore, software developers will need to combine independent
Web of Knowledge 10 3 process models making more complex the design tasks.
Total 73 25
Based on the systematic review, usable security principles are not
By using the systematic review method, about 73 papers were sufficient to solve usability problems in the design of user
found and 25 of them which could answer the research questions authentication services. The lack of suitable usability principles
were selected. for designing user authentication systems, imply that the usability
evaluation methods are not suitable because they do not take into
5.2 Results and Discussion account the user. Finally, the lack of usability principles and
According to an analysis of the relevant papers obtained in the evaluation methods for de signing user authentication systems, is
systematic review, the answers to the research questions are a disadvantage for developing a process or technique that extends
presented below: from design to implementation and applications.
Q1: What usability principles are applied to security systems, According to the above, the following research question is
mainly for user authentication? proposed: What processes aid the design of users
authentication systems through a user-centered approach
Table 2 presents a comparison of some usable security principles
based on usability principles and evaluation methods for
for the works presented in section 3. An important characteristic is
security systems?
that the principles proposed by the authors are similar and some of
them with applications development [6]. However, these 6. CONCLUSIONS AND FUTURE WORK
principles are not enough to solve problems of usability in the In this paper we have carried out a systematic literature review on
design of secure systems, especially for the design of user usable security, mainly about user authentication systems. Many
authentication services. Moreover, an agreement about a useful users are not able to perceive the security issues correctly,
criteria for usable security design has not been established. generating a security threat due to misunderstanding and avoid
Q2: Which methodological approaches are used for evaluating tactics to protect the system. The usability principles presented
the usability in security and in user authentications systems? above are very similar to each other. In this context, criteria based
An important challenge in usable security is to achieve a user- on existing usability principles to find the usability problems in
centered development process that integrates usability and software security are required. Although there are different
security concerns, and taking into account usable security methods for evaluating the usability of security systems, these
principles. According to Table 3 we can see that although there methods are not user-centered due to the lack of suitable usability
are works about usability evaluation methods for security, lack of principles. Finally, we have presented some processes for
usability principles suitable for security systems mainly to user incorporating usability into secure software design including
authentication, this makes that the evaluation methods are not integration and validation. Future work will be oriented to
complete because most of them, not take into account the user. studying and analyzing the usable security principles for the
The inability of users to evaluate the security properties, design and development of user authentication services.
complicates the experimental design. Besides, users are not able 7. ACKNOWLEDGMENTS
to describe the security problems they have experienced [39].
Paulo C. Realpe thanks to Colciencias for the scholarship. The
Q3: What processes aid to design secure systems and user authors would like to thank Professor Eduardo B. Fernández for
authentication services based on principles and methods of his comments and suggestions.
evaluation according to usable security requirements?

Table 2: Comparison of some usable security principles

Work [8] [9] [10] [11] [18] [19] [20] [23] [21] [22] [16]
Principle
Aesthetic and minimalist design -    -  -    
Design Interfaces based on user's mental model  -   -      
Mental and physical load must be tolerable - - - - - -  - - - 
Accommodate all type of users - - - - - - -  - - -
Trust and satisfaction  - -  - - -  -  
Users must have sufficient knowledge to make decisions - - -  -  - - - - -
Consistent meaningful vocabulary and terminology - -    - -  -  
Appropriate boundaries - -  - -  - - - - -
Make security functionality visible and accessible - - - - - - -  -  -
Security tools are not solutions - - - - - - -  - - -

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 371 |
 Towards an Integration of Usability and Security for User Authentication
P. C. Realpe | C. A. Collazos | J. Hurtado | A. Granollers

Table 3: Comparison of criteria for usable security evaluation

Work [24] [25] [26] [27] [28] [30] [35]
Criteria
User-centered evaluation  - -  - - -
Evaluation for user authentication -  - -  - -
Uses usable security principles  -  - -  
Evaluate usability and security  -  -   -

Table 4: Comparison of criteria for design process

Work [32] [31] [24] [33] [34] [5] [35] [36] [37]
Criteria
User-centered design process  - - -  - -  -
Process applying to systems development -  -  - -   
Used to user authentication - -  - -  - - -

[10] K.-P. Yee, “User interaction design for secure systems,” in

8. REFERENCES 4th International Conference on Information and
[1] M. Mihajlov, S. Josimovski, and B. Blazic, “A conceptual
Communications Security, 2002.
framework for evaluating usable security in authentication
mechanisms usability perspectives,” in 5th International [11] J. Johnston, J. Eloff, and L. Labuschagne, “Security and
Conference on Network and System Security (NSS), 2011, pp. human computer interfaces,” Computers & Security, vol. 22,
332-336. no. 8, pp. 675-684, December 2003.
[2] C. Braz, A. Seffah, and D. Raihi, “Designing a trade-off [12] C. Braz and J.-M. Robert, “Security and usability: the case of
between usability and security: A metrics based-model,” in the user authentication methods,” in 18th International
Human-Computer Interaction - INTERACT, vol. 4663. Conference of the Association Francophone d'Interaction
Springer Berlin Heidelberg, September 2007, pp. 114-126. Homme-Machine. New York: ACM, 2006, pp. 199-203.
[3] S. Chiasson, R. Biddle, and A. Somayaji, “Even experts [13] J. Nielsen. (2003) Introduction to usability. Nielsen Norman
deserve usable security: Design guidelines for security Group. [Online]. Available: http://www.nngroup.com/
management systems,” in Workshop on Usable IT Security [14] D. Katsabas, S. Furnell, and P. Dowland, “Using human
Management (SOUPS '07). ACM, 2007. computer interaction principles to promote usable security,”
[4] A. Jøsang and M. A. Patton, “User interface requirements for in 5th International Network Conference, 2005.
authentication of communication," in 4th Australasian user [15] F. Mazumder and U. Das, “Usability guidelines for usable
interface conference on User interfaces, vol. 18. ACM, 2003, user interface,” International Journal of Research in
pp. 75-80. Engineering and Technology, vol. 3, no. 9, pp. 79-82,
[5] C. Braz, A. Seffah, and P. Poirier, “Designing usable, yet September 2014.
secure user authentication services: A user authentication [16] B. Shneiderman, Designing the User Interface: Strategies for
protocol,” in 5th International Conference on Applied Effective Human-Computer Interaction, J. Dubau and A.
Human Factors and Ergonomics, vol. 20. AHFE, July 2014, Willner, Eds. Addison-Wesley Publishing Company, 1998.
pp. 155-165.
[17] S. Bradley. (2014) Design principles: Visual perception and
[6] A. Pedersen, “Usability of authentication in applications the principles of gestalt. [Online]. Available:
web. A literature review,” University of Copenhagen, Tech. http://www.smashingmagazine.com
Rep., 2010.
[18] S. Garfinkel, “Design principles and patterns for computer
[7] E. Zurko and R. Simon, “User-centered security,” in VCLA systems that are simultaneously secure and usable,” Ph.D.
Conference on New Security Paradigms Workshops, 1996. dissertation, Massachusetts Institute of Technology, 2005.
[8] A. Whitten and J. Tygar, “Usability of security: A case [19] S. Chiasson, R. Biddle, and P. Oorschot, “A second look at
study,” in School of Computer Science EECS Carnegie the usability of click-based graphical passwords,” in 3rd
Mellon, 1999, pp. 1-39. symposium on Usable privacy and security. New York:
[9] J. Saltzer and M. Schroeder, “The protection of information ACM, 2007, pp. 1-12.
in computer systems,” in Security and Privacy on the [20] A. Jøsang, B. AlFayyadh, and T. Grandison, “Security
Internet. ACM, 2000. usability principles for vulnerability analysis and risk
assessment,” in 23rd Annual Computer Security Applications
Conference. IEEE, December 2007, pp. 269-278.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 372 |
[21] A. Herzog and N. Shahmehri, “Usable set-up of runtime [29] K. Renaud, “Quantifying the quality of web authentication
security policies,” in International Symposium on Human mechanisms a usability perspective,” Journal of Web
Aspects of Information Security and Assurance, vol. 15, no. Engineering, vol. 3, no. 2, pp. 95-123, October 2003.
5. Emerald Group, 2007, pp. 394-407. [30] [30] A. Palmer, “Criteria to evaluate automated personal
[22] Ibrahim, S. Furnell, M. Papadaki, and N. Clarke, “Assessing identification mechanisms,” Journal of computers &
the usability of personal internet security tools,” in 8th security, vol. 27, no. 7-8, pp. 260-284, December 2008.
European Conference on Information Warfare and Security, [31] Y. Hausawi and W. Allen, “Usability and security trade-off:
2009. A design guideline,” Journal of computers & security, vol.
[23] J. Nurse, S. Creese, M. Goldsmith, and K. Lamberts, 4663, pp. 114-126, September 2014.
“Guidelines for usable cybersecurity: Past and present,” in [32] L. Collett, “Designing usable security for effective security,”
Third International Workshop on Cyberspace Safety and Duke University, Pittsburgh, Tech. Rep., July 2003.
Security (CSS). IEEE, 2011, pp. 21-26.
[33] I. Flechais, A. Sasse, and S. Hailes, “Bringing security home:
[24] C. Braz, P. Porrier, and A. Seffah, “Designing usable, yet a process for developing secure and usable systems,” in New
secure user authentication service: The cognitive dimension,” Security Paradigms. New York: ACM, 2003, pp. 49-57.
in Web-based international research network for tourism
professionals. E-Review of Tourism Research, 2010. [34] D. Markotten, “User-centered security engineering,” in 4th
EurOpen/USENIX, 2002.
[25] T. Mapayi, O. Olaniyan, N. Isamotu, and O. Moses,
“Evaluating usability factors in different authentication [35] A. Yeratziotis, D. Greunen, and D. Pottas, “A framework for
methods using artificial neural network,” African Journal of evaluating usable security: The case of online health social
Computing & ICT, vol. 6, no. 1, pp. 69-78, March 2013. networks,” in 6th International Symposium on Human
Aspects of Information Security & Assurance, 2012.
[26] P. Jaferian, K. Hawkey, A. Sotirakopoulos, M. Rojas, and K.
Beznosov, “Heuristics for evaluating it security management [36] S. Faily, “A framework for usable and secure system
tools,” in Symposium on Usable Privacy and Security design,” Ph.D. dissertation, University of Oxford, 2011.
(SOUPS). New York: ACM, 2011, pp. 1633-1638. [37] N. Parveen, R. Beg, and M. Khan, “Integrating security and
[27] T. Straub and H. Baier, “A framework for evaluating the usability at requirement specification process,” International
usability and the utility of PKI-enabled applications,” in 1st Journal of Computer Trends and Technology, vol. 10, no. 5,
European PKI Workshop Research & Applications, vol. pp. 236-240, April 2014.
3093. Springer Berlin Heidelberg, 2004, pp. 112-125. [38] B. Kitchenham and S. Charters, “Guidelines for performing
[28] M. Mihajlov, B. Blazic, and S. Josimovski, “Quantifying systematic literature reviews in software engineering,”
usability and security in authentication,” in 35th IEEE University of Durham, Tech. Rep., 2007.
Annual Computer Software and Applications Conference, [39] S. Garfinkel and H. Lipford, Usable Security: History,
2011. Themes, and Challenges, E. Berntino, Ed. Morgan &
Claypool, 2014, vol. 1.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 373 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes

Evaluando la usabilidad de aplicaciones groupware

mediante un método dirigido por modelos para el análisis
de la interacción del usuario
Rafael Duque Medina Alicia Nieto-Reyes
Department of Mathematics, Department of Mathematics,
Statistics and Computer Science Statistics and Computer Science
University of Cantabria University of Cantabria
Avenida de Los Castros S/N Avenida de Los Castros S/N
39005 Santander (Spain) 39005 Santander (Spain)
[email protected] [email protected]

ABSTRACT Keywords
La evaluación de la usabilidad de un sistema interactivo implica Computer-supported cooperative work; Software; Usability.
analizar el grado en el cual el sistema puede ser utilizado con
efectividad, eficiencia y satisfacción en un contexto de uso 1. INTRODUCCIÓN
específico. Este tipo de evaluaciones habitualmente tienen que Uno de los puntos clave en los métodos de desarrollo del software
enfrentarse con la dificultad de determinar la usabilidad de centrados en el usuario es la realización de frecuentes
aplicaciones groupware que permiten a un grupo de usuarios evaluaciones que verifiquen la usabilidad del sistema en
interactuar entre ellos en cualquier momento y lugar mediante construcción. La Organización Internacional para la
dispositivos móviles. Las evaluaciones de usabilidad realizadas en Estandarización [83] define la usabilidad como “la medida en la
laboratorios pueden estar lejos de reproducir esta realidad donde que un producto se puede usar por determinados usuarios para
usuarios geográficamente distribuidos interactúan entre sí conseguir objetivos específicos con efectividad, eficiencia y
constantemente. Sin embargo, las evaluaciones de usabilidad satisfacción en un contexto de uso especificado”. Por tanto, la
pueden aprovechar las ventajas de un soporte computacional que evaluación de la usabilidad de un sistema interactivo implicará
observe el comportamiento de los usuarios en contextos reales de llevar a cabo un enfoque metodológico para identificar problemas
uso (trabajo, hogar, lugares públicos, etc.) para calcular métricas con el sistema para ser usado con efectividad, eficiencia y
que aporten información acerca de la efectividad y eficiencia de la satisfacción. En la literatura se pueden encontrar un importante
interacción (tasa de errores, velocidad de ejecución, etc.). Para número de métodos de evaluación (heurísticas, inspecciones, test
ello, este artículo describe un entorno que permite a los de laboratorio, etc.) [84], entre los que pueden seleccionarse
evaluadores especificar modelos que representen las interacciones aquellos que mejor encajen con factores como el tiempo
de los usuarios que deben ser estudiadas y un conjunto de disponible, la criticidad del sistema o los costes asumibles [85].
métricas que deben ser calculadas para analizar la usabilidad del
Las evaluaciones de usabilidad llevadas a cabo con usuarios
sistema. Posteriormente, el entorno genera automáticamente el
finales en contextos reales de uso pueden implicar la necesidad de
soporte computacional para ejecutar el proceso de evaluación de
analizar actividades llevadas a cabo por grupos de usuarios que
la usabilidad que ha sido modelado. Finalmente, el entorno genera
interactúan entre sí en cualquier momento y lugar mediante
un informe con los resultados de la evaluación y aplica un modelo
dispositivos móviles. Por tanto, una evaluación de la usabilidad
estadístico para encontrar la relación entre los valores de estas
mediante un experimento controlado en laboratorio podría estar
métricas y la satisfacción del usuario con el sistema, la cual es
lejos de reproducir este contexto real de uso de la aplicación bajo
obtenida mediante un cuestionario. El artículo incluye un caso de
evaluación. Las herramientas software pueden contribuir a
estudio donde el entorno ha sido puesto en acción para evaluar la
efectuar evaluaciones en estos contextos reales de uso mediante
usabilidad de una aplicación groupware para dispositivos móviles
mecanismos que observen las interacciones de los usuarios y
que permite la realización de apuestas deportivas.
generen métricas con información sobre el grado en el cual el
Categories and Subject Descriptors sistema es utilizado con efectividad y eficiencia (velocidad de
H.5.2 [Information Interfaces and Presentation]: User ejecución, tasa de errores, etc.). Sin embargo, algunos factores
Interfaces - Evaluation/methodology, User-centered design deben tenerse en cuenta a la hora de construir este tipo de
herramientas para que lleven a cabo el proceso de evaluación de
H.5.3 [Information Interfaces and Presentation]: Group and forma efectiva. Primero, es preciso que estas herramientas
Organization Interfaces - Computer-supported cooperative work permitan a los evaluadores definir las métricas más apropiadas
para el sistema en evaluación y el correspondiente contexto de
General Terms uso. Segundo, debe reducirse el esfuerzo extra de programación, y
Measurement, Design, Human Factors, Verification. por tanto el coste asociado, que implica desarrollar un soporte
computacional para observar la interacción del usuario y asignar
valores a las métricas definidas. Tercero, estas herramientas deben
ayudar a los evaluadores a determinar qué valores deben adquirir

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 374 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes
las diferentes métricas para considerarlas indicativas de un buen que pueden utilizarse en estas evaluaciones. Seffah et al. [89]
nivel de usabilidad en el sistema. Por ejemplo, cuál es la cantidad definen un modelo llamado Quality in Use Integrated
de tiempo que puede considerarse como aceptable para llevar a Measurement (QUIM) que incluye 127 métricas para evaluar los
cabo una determinada tarea con el sistema. Cuarto, debe este tipo de atributos y conceptos relacionados con la usabilidad
analizarse la relación entro los valores de estas métricas, del sistema. QUIM trata de unificar en un único modelo los
calculadas automáticamente, y la satisfacción del usuario que principales modelos conceptuales y estándares de la literatura.
puede conocerse mediante otros métodos de evaluación como un Aunque los autores hacen un esfuerzo para establecer un modelo
cuestionario. Así, el evaluador puede tener conciencia de qué consolidado, este tipo de trabajos no considera la posibilidad de
aspectos evaluados por una métrica de usabilidad (operatividad, disponer de un mecanismo flexible por el cual los evaluadores
facilidad de aprendizaje, etc.) tiene mayor impacto en la puedan diseñar sus propias métricas ni proporcionan unas pautas
satisfacción del usuario. Quinto, los métodos de evaluación metodológicas para seleccionar aquellas métricas que mejor se
ejecutados por este tipo de herramientas deben considerar las adapten a un sistema específico ni a un contexto de uso
características específicas del sistema [86]. Por ejemplo, las determinado. Para ello, Suárez et al. [90] establecen una
aplicaciones groupware deben permitir al usuario no sólo estar al taxonomía de sistemas y proponen criterios específicos para
tanto de los efectos de sus propias interacciones con el sistema evaluar la usabilidad de cada sistema, según sus funcionalidades y
sino también conocer los efectos de las interacciones efectuadas características. Esta clasificación incluye el groupware como un
por otros usuarios. Por tanto, la evaluación debe considerar tipo específico de sistema. Entre las particularidades del
criterios específicos como la efectividad de los mecanismos de groupware, cabe resaltar que permite no sólo la comunicación
awareness para que el usuario se sienta parte de un colectivo. entre usuarios sino la creación colaborativa de artefactos en
espacios de trabajo compartido. Las aplicaciones groupware
Para abordar estos retos, este artículo propone un entorno que soportan interacciones cuyos efectos pueden ser percibidos por
automatiza la evaluación de la usabilidad de aplicaciones otros usuarios [91]. Por tanto, la evaluación debe considerar
groupware en contextos reales de uso. Este entorno incluye criterios específicos como la efectividad de los mecanismos de
mecanismos que permiten a desarrolladores y evaluadores definir, awareness de la aplicación y el soporte para generar interacciones
de forma flexible, las métricas que mejor se ajusten a la aplicación sociales. Mattsson [92] propone algunas métricas específicas que
y a su contexto de uso. Esto se consigue mediante un enfoque evalúan el grado en el cual la aplicación permite efectuar de forma
dirigido por modelos que permite a evaluadores modelar las
intuitiva interacciones sociales. Sin embargo, esta propuesta
interacciones a analizar y las métricas a calcular con ayuda de un tampoco proporciona un enfoque flexible para que el evaluador
lenguaje visual. Además, este lenguaje permite clasificar cada defina sus propias métricas.
interacción de acuerdo a su propósito (comunicarse con otros
usuarios, manipular elementos compartidos, etc.). Posteriormente, El análisis de los valores que toman las métricas de usabilidad
el entorno crea automáticamente el código fuente del soporte puede efectuarse mediante técnicas de aprendizaje automático que
computacional que lleva a cabo la evaluación para reducir así el indiquen el grado de influencia de ciertos atributos
esfuerzo de programación. Finalmente, el entorno aplica un (navegabilidad, comprensibilidad, etc.) en la usabilidad del
modelo estadístico para determinar el valor que debe tomar cada sistema [93]. Este análisis puede procesar información generada
métrica con objeto de maximizar la satisfacción del usuario con la por distintos métodos de evaluación y usar técnicas de minería de
aplicación. datos [94] para generar informes sobre los problemas detectados.
De acuerdo a Nielsen & Levy [95], en la mayoría de los casos
2. TRABAJOS RELACIONADOS existe una correlación entre la satisfacción subjetivas del usuario y
Los métodos de desarrollo centrados en el usuario no han sido aquellas métricas que objetivamente cuantifican aspectos sobre la
ajenos a las tendencias de los últimos años para automatizar la calidad de la interfaz de usuario. Sin embargo, los evaluadores
especificación y construcción del software. Este es el caso de la deben identificar cuáles son esas métricas que están relacionadas
propuesta de Molina & Toval [87] que permite a desarrolladores con la satisfacción del usuario.
especificar funcionalidades relacionadas con la usabilidad del
sistema y generar automáticamente el correspondiente código 3. UN ENTORNO PARA ANALIZAR LA
fuente mediante un enfoque dirigido por modelos. Sin embargo, INTERACCIÓN DEL USUARIO
estos métodos dirigidos por modelos han sido utilizados La Figura 1 muestra los principales pasos seguidos por el entorno
generalmente para construir el sistema interactivo y no han sido para ejecutar el proceso de evaluación de la usabilidad. Estos
aprovechados para generar soporte computacional que permita pasos son los siguientes:
evaluar su usabilidad. Esta situación hace que el evaluador tenga a
su disposición herramientas como USherlock [88] que evalúa una 1. El evaluador define un modelo de observación con las
serie de propiedades predefinidas de la interfaz de usuario como el acciones soportadas por la aplicación y que deben ser
contraste de colores o el tamaño de los widgets para identificar analizadas. El evaluador también define un modelo con
problemas de usabilidad, pero no permite a los evaluadores las métricas a calcular y se genera automáticamente el
configurar el proceso de evaluación. código fuente del soporte computacional que ejecutará
la evaluación. El proceso de evaluación es puesto en
Además, la evaluación de la usabilidad no tiene por qué limitarse marcha para asignar valores a las métricas y se pregunta
a considerar una serie de propiedades de la interfaz de usuario al usuario cuál es su satisfacción con la aplicación.
sino que un análisis de las interacciones generadas por el usuario 2. El entorno aplica un modelo estadístico para encontrar
en un contexto de uso real puede enriquecer esta evaluación. las relaciones entre los valores de las métricas
Madan & Dubey [84] efectúan una revisión de los principales calculadas automáticamente y la satisfacción del
conceptos y atributos (facilidad de aprendizaje, flexibilidad, etc.) usuario.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 375 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes
3. El entorno genera un informe con los resultados del considerada como comunicativa cuando existe un intercambio de
proceso de evaluación. Este informe incluye los valores mensajes entre usuarios. Las acciones protocolarias no implican
que toman las distintas métricas y un listado de un intercambio de mensajes entre los participantes, pero ayuda a
recomendaciones. coordinar o armonizar el proceso colaborativo, por ejemplo,
votando una propuesta. Las acciones del espacio de contenidos
Las siguientes subsecciones describen el soporte que proporciona son calificadas como instrumentales o cognitivas. Las acciones
este entorno para llevar a cabo cada uno de estos tres pasos. instrumentales modifican un elemento de un espacio compartido.
Sin embargo, las acciones cognitivas usan un elemento
compartido sin modificar su estado, por ejemplo, para hacer una
copia de él.
Finalmente, los evaluadores asignan un nombre a cada acción en
el cuarto nivel de la jerarquía. Además, los evaluadores describen
cómo esas acciones contribuyen a resolver determinadas tareas. El
concepto de tarea es considerado como una parte del proceso
colaborativo que produce artefactos o resultado de interés. Para
ello, cada acción puede relacionarse con una tarea con alguna de
estas asociaciones:
• Asociación start: Esta acción se ejecuta por el usuario al
comienzo de una nueva tarea.
• Asociación end: La acción implica la finalización de
una tarea.
Figura 1. Ciclo de vida de evaluación de la usabilidad. • Asociación solve: La acción contribuye a resolver una
3.1 Modelando acciones y métricas tarea, pero no la inicia ni la finaliza.
El entorno proporciona un Lenguaje Específico de Dominio La Figura 2 (izquierda) ilustra el uso de esta herramienta con un
(Domain-Specific Language, DSL) para representar las acciones fragmento de un modelo de observación. Este modelo incluye
soportadas por la aplicación bajo evaluación. De acuerdo a Hilbert alguna de las acciones soportadas por una aplicación groupware
& Redmiles [96], estas acciones deben ser capturadas porque que soporta el diseño de diagramas de flujo. Se puede observar
suponen una fuente de información fructífera para conocer el como el elemento raíz específica el nombre de la aplicación
comportamiento de los usuarios. El modelo de observación (colEdit en la Figura 2, izquierda). El segundo nivel incluye las
incluye mecanismos para clasificar el propósito de cada acción, la herramientas integradas en la aplicación. En este caso la
herramienta concreta de la aplicación que la soporta y cómo aplicación incluye un chat estructurado y un editor colaborativo
contribuye a resolver tareas de mayor granularidad. de diagramas. El tercer nivel incluye iconos que definen el tipo de
Posteriormente, el entorno transforma estos modelos de acciones soportadas por cada herramienta. Las acciones
observación para generar código fuente del soporte computacional soportadas por el chat son clasificadas como comunicativas. Las
que procese las acciones de los usuarios y calcule las métricas acciones soportadas por el editor son clasificadas como
definidas [97]. instrumentales. El cuarto nivel del modelo incluye un conjunto de
Este DSL ha sido representado utilizando un lenguaje visual que identificadores para las acciones soportadas por estas
permite a los evaluadores modelar un diagrama jerárquico cuyo herramientas.
elemento raíz define la aplicación groupware. El segundo nivel de El evaluador introduce asociaciones con información descriptiva
la jerarquía está compuesto de las herramientas integradas en la de cómo las acciones contribuyen a llevar a cabo la tarea de
aplicación. Los evaluadores clasifican las acciones según su edición de diagramas de flujo. Así, cada vez que el usuario abre
propósito en el tercer nivel de la jerarquía. De acuerdo al modelo una nueva ventana, comienza la tarea de edición. Posteriormente
dual de Barron [98], los procesos colaborativos incluyen un el usuario ejecuta una serie de acciones (insert y delete) para
espacio relacional en el cual los usuarios efectúan acciones para construir el diagrama. Finalmente, el usuario cierra la ventana
interactuar socialmente con otros usuarios y un espacio de (acción exit) y la tarea de edición finaliza.
contenidos en el cual los usuarios ejecutan acciones para resolver
tareas. El modelo de observación clasifica las acciones del espacio
relacional como comunicativas o protocolarias. Una acción es

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 376 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes

Figura 2. Fragmentos de los modelos de observación (izquierda) y de métricas (derecha).

Tras definir el modelo de observación, los evaluadores pueden por el entorno. Así, podemos distinguir entre métricas objetivas,
usar un lenguaje visual para definir un modelo con un conjunto de aquellas calculadas automáticamente por el entorno, y subjetivas,
métricas que analicen la interacción del usuario. Para definir aquellas especificadas por el usuario. El objetivo es determinar
métricas de las acciones efectuadas por los usuarios, primero el qué métricas objetivas deben ser considerados de alta prioridad en
evaluador puede especificar de tres modos las características de las nuevas iteraciones del ciclo de vida del software por estar
las acciones que deben ser analizadas: (i) son todas las acciones relacionadas con la satisfacción del usuario.
correspondiente a un tipo concreto (instrumental, cognitiva,
comunicativa, protocolaria), (ii) es una acción específica (insert, Por lo tanto, el propósito principal de esta sección es encontrar
exit, etc.), (iii) son todas las acciones soportadas por una una función de las métricas objetivas que explique la(s)
métricas(s) subjetiva(s), como respuesta. Esto podría hacerse
herramienta (editor, chat, etc.). Posteriormente, el evaluador
especifica si deben contabilizarse las ocurrencias de las acciones sencillamente mediante la aplicación de un modelo de regresión
modeladas o el tiempo empleado en su ejecución. Finalmente, el múltiple.
evaluador puede definir nuevas métricas que son calculadas 3.2.1 Modelos de regresión múltiple
aplicando operaciones aritméticas y estadísticas a métricas La regresión múltiple es una metodología estadística que modela
calculadas previamente. La Figura 2 (derecha) muestra un la relación entre dos o más métricas objetivas y una o más
fragmento de este modelo en el cual el evaluador define unas métricas subjetivas. El modelo de regresión múltiple más común
métricas elementales (NActions) para contabilizar el número de es la Regresión Lineal Múltiple (RLM), cuando sólo hay una
acciones ejecutadas y para conocer el tiempo empleado en ellas métrica subjetiva, y la regresión lineal múltiple multivariante, o
(NTime). Finalmente, el evaluador define una nueva métrica (ver simplemente regresión lineal multivariante, cuando hay diversas
Actions_Density en Figura 2, derecha) con el número de acciones métricas subjetivas. Para facilitar la explicación, nos
ejecutadas por unidad de tiempo. concentramos en el caso básico de una sola métrica subjetiva.
3.2 Relacionando métricas de usabilidad Así, dadas X1, X2, ..., Xp métricas objetivas, con p mayor o igual
objetivas y subjetivas que dos, y Y una métrica subjetiva, el modelo de RLM busca 𝛽𝛽0,
𝛽𝛽1, … , 𝛽𝛽p tal que 𝑌𝑌 = 𝛽𝛽0 + 𝛽𝛽1 X1 + ⋯ + 𝛽𝛽p Xp+𝜀𝜀, donde 𝜀𝜀 es una
Una vez que se han calculado los valores de las métricas, el
variable aleatoria de media cero y varianza finita independiente de
entorno analiza esta información para identificar qué métricas,
X1, X2, ..., Xp, conocida como ruido. La Ecuación 1 muestra la
calculadas automáticamente (tasa de errores, tiempo de
aprendizaje, etc.), tienen un mayor impacto en la satisfacción expresión que relaciona la estimación de la métrica subjetiva 𝑌𝑌�
subjetiva del usuario, la cual no ha sido obtenida automáticamente con las métricas objetivas (X1, X2,..., Xp,). 𝛽𝛽̂0 sería el término
constante de la ecuación (véase la Ecuación 1).

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 377 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes
𝑌𝑌� = 𝛽𝛽̂ 0 + 𝛽𝛽̂ 1 X1 + ⋯ + 𝛽𝛽̂ p Xp. tiempo que tenemos en cuenta la naturaleza categórica de la
métrica subjetiva.
Ecuación 1. Relación entre las métricas subjetivas y objetivas.
Para estimar los límites de cada categoría utilizamos el siguiente
Para medir la calidad del ajuste de los datos al modelo estadístico
procedimiento fácil de implementar, aunque otros procedimientos
se utiliza el coeficiente de determinación, calculado como
más complejos pueden utilizarse también, por ejemplo, utilizando
∑𝑛𝑛𝑖𝑖=1 (𝑌𝑌�𝑖𝑖 − 𝑌𝑌�)2 ⁄(𝑌𝑌𝑖𝑖 − 𝑌𝑌�)2, donde 𝑌𝑌� denota la media muestral y n la estimación de máxima verosimilitud. Dada una métrica
el tamaño muestral, y toma valores entre 0 y 1; valores cercanos a subjetiva categórica con 𝐽𝐽 categorías, denotamos por 𝑉𝑉𝑗𝑗 ≔
1 indicando un buen ajuste al modelo. Cuando el coeficiente de
�𝑌𝑌�𝑖𝑖 �𝑌𝑌𝑖𝑖 = 𝑗𝑗�, 𝑗𝑗 = 1, … , 𝐽𝐽, al conjunto de valores de la variable
determinación no está cerca del valor 1, se debe utilizar un
modelo más complejo que el lineal. continua 𝑌𝑌� (ver Ecuación 1) que corresponden al valor 𝑗𝑗 de la
métrica subjetiva.
Dadas X1, X2, ..., Xp, una posibilidad es simplificar el modelo de
RLM mediante un subconjunto de estas variables. Esto se realiza Por todo ello, denotamos 𝑙𝑙𝑙𝑙𝑙𝑙𝑗𝑗 ≔ 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚�𝑚𝑚𝑚𝑚𝑚𝑚𝑉𝑉𝑗𝑗−1 , 𝑚𝑚𝑚𝑚𝑚𝑚𝑉𝑉𝑗𝑗 �. Por lo
mediante el cálculo del coeficiente de determinación de cada una tanto, cualquier valor menor que 𝑙𝑙𝑙𝑙𝑙𝑙2 se redondea a 1, cualquier
de todas las posibles combinaciones de variables X1, X2, ..., Xp. valor mayor que 𝑙𝑙𝑙𝑙𝑙𝑙𝐽𝐽 se redondea a J y cualquier valor
Por lo tanto, se selecciona la combinación que contiene menos perteneciente a �𝑙𝑙𝑙𝑙𝑙𝑙𝑗𝑗 , 𝑙𝑙𝑙𝑙𝑙𝑙𝑗𝑗+1 � se redondea a 𝑗𝑗 para 𝑗𝑗 = 2, … , 𝐽𝐽 −
variables dentro de aquellas que tienen un alto coeficiente de 1.
determinación.
Sin embargo, en el caso en el que la métrica subjetiva consiste en 3.3 Informe
categorías ordenadas, variable ordinal, los modelos de regresión El entorno proporciona un informe con los resultados del proceso
múltiple utilizados son los modelos logit ordenado y probit de evaluación de la usabilidad. Este informe incluye, en primer
ordenado. Cuando la variable, subjetiva, categórica ordenada tiene lugar, el conjunto de métricas con sus valores y con una
J categorías, cualquiera de estos modelos consiste en un conjunto descripción de sus objetivos. Estas métricas se clasifican como de
de J-1 ecuaciones. Un ejemplo de categoría ordenada se utiliza a alta prioridad cuando según el modelo estadístico tienen una
menudo en los cuestionarios que preguntan a los usuarios si se influencia directa en la satisfacción del usuario y de baja prioridad
sienten satisfechos con el sistema y deben responder: (i) muy en cuando no es así. El entorno calcula los valores óptimos de las
desacuerdo, (ii) en desacuerdo, (iii) ni de acuerdo ni en métricas de alta prioridad como aquellos que maximizan la
desacuerdo, (iv) de acuerdo o (v) totalmente de acuerdo. Podemos satisfacción del usuario según el modelo estadístico.
corresponder estas respuestas con los valores 1, 2, 3, 4 y 5, Posteriormente, el informe muestra la distribución porcentual de
respectivamente. El motivo de aplicar el logit ordenado o el probit usuarios según su grado de satisfacción con la aplicación. Así,
ordenado, y no una regresión lineal, es que los números asociados cuando existe un bajo porcentaje de usuarios satisfechos con la
a las categorías ordenadas no significan mucho ya que las aplicación, el entorno relaciona este dato con la necesidad de
distancias entre los diferentes valores de la medición no son mejorar los atributos de usabilidad evaluados por las métricas de
necesariamente equivalentes a las distancias entre los valores a los alta prioridad. Para lograr estas mejoras, el entorno sugiere al
que las hemos hecho corresponder. Es decir, en el ejemplo evaluador desarrollar mecanismos que mejoren los aspectos
anterior puede haber una distancia mayor entre las personas que analizados por estas métricas de alta prioridad. Para que el
están muy en desacuerdo y aquellas que están en desacuerdo sobre evaluador sea consciente del tipo de mecanismos que debe
un asunto en particular que entre las que están en desacuerdo y las desarrollar, el entorno procesa los modelos de observación y de
que ni están de acuerdo ni en desacuerdo sobre esa materia; métricas y muestra qué aspectos son evaluados por las distintas
mientras que la distancia entre 1 y 2 es la misma que entre 2 y 3. métricas (facilidad para usar con rapidez una herramienta, las
interacciones necesarias para resolver una tarea, etc.). Por último,
3.2.2 Propuesta para la medición subjetiva el informe también incluye los resultados generados por el modelo
categórica ordenada estadístico (gráficas, ecuaciones, etc.).
Por lo tanto, la ventaja de RLM es que modela los datos utilizando
solamente una ecuación, lo que simplifica la interpretabilidad del 4. CASO DE ESTUDIO
modelo; mientras que el modelo logit ordenado, o probit Esta sección describe un caso de estudio en el cual el entorno ha
ordenado, requiere de múltiples ecuaciones. Si el modelo de RLM sido utilizado para llevar a cabo una evaluación de la usabilidad
se aplica a un conjunto de datos donde la medición subjetiva es de una aplicación groupware móvil que soporta la realización
categórica, la variable 𝑌𝑌� resultante es continua, no categórica. La colaborativa de apuestas deportivas. La Figura 3 (izquierda)
solución directa sería estimar 𝑌𝑌 redondeando 𝑌𝑌� a cero posiciones muestra la interfaz de usuario principal de esta aplicación. La
decimales, pero entonces tendríamos la problemática expuesta aplicación gestiona una lista de eventos deportivos y un usuario
anteriormente que las distancias entre los diferentes valores de la puede seleccionar uno de ellos para efectuar una apuesta.
medición no son necesariamente equivalentes a las distancias Posteriormente, el usuario solicita la colaboración de un grupo de
entre los valores a las que les hemos hecho corresponder. De esta usuarios para pronosticar cual será el resultado del evento
forma, nuestra propuesta consiste en estimar los límites adecuados deportivo y discutir la cantidad de dinero a apostar. Para este
para hacer el redondeo. Es decir, el número 2.4943 redondea propósito, la aplicación incluye un panel de votaciones (ver Figura
naturalmente al número entero 2, pero si los límites estimados son 3, centro) que permite al usuario proponer un pronóstico y una
por ejemplo (1,5195, 2,4918) para 2 y (2,4918, 3,5062) para 3, cantidad de dinero a apostar. Los otros usuarios del grupo utilizan
tendríamos que 2,4943 redondearía a 3. Así, cuando la medición este panel para votar la aprobación o el rechazo de esta propuesta.
subjetiva es categórica, logramos el beneficio de ambos modelos, Cuando la mayoría de los miembros del grupo aceptan la
logit / probit y RLM, al obtener una sola ecuación al mismo propuesta, el resultado se introduce automáticamente en un
espacio con las apuestas deportivas del usuario. Los usuarios

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 378 |
 Evaluando la usabilidad de aplicaciones groupware mediante un método dirigido por modelos para el análisis de la interacción del
usuario
R. Duque Medina | A. Nieto-Reyes
también pueden utilizar un chat estructurado (ver Figura 3, Tras definir el modelo de observación, el evaluador utilizó el
derecha) para intercambiar ideas sobre la mejor forma de generar correspondiente lenguaje visual para modelar las siguientes cuatro
las apuestas. Todas estas funcionalidades están explicadas métricas [99]:
mediante un tutorial interactivo incluido en la aplicación.
• Tiempo de aprendizaje: Es una métrica que cuantifica cuánto
Un evaluador utilizó el entorno propuesto para evaluar la tiempo necesitan los usuarios para aprender a usar la
usabilidad de esta aplicación. Esta evaluación se llevó a cabo aplicación. La aplicación incluye un tutorial que explica las
mediante un conjunto de métricas que analizan la interacción de funcionalidades de la aplicación. El evaluador estableció esta
21 grupos de 4 usuarios, formados aleatoriamente, quienes métrica que cuenta los segundos empleados por los usuarios
generaron colaborativamente 5 apuestas deportivas. Las siguientes en este tutorial y en la exploración de la aplicación antes de
subsecciones describen el proceso de evaluación que se siguió. empezar a realizar apuestas.
• Velocidad de ejecución: Esta métrica cuenta el tiempo que
4.1 Definiendo los modelos tarda el usuario en llevar a cabo las tareas soportadas por la
El modelo de observación estuvo compuesto por un elemento raíz aplicación. Para ello, el evaluador estableció esta métrica que
que identifica la aplicación para realizar apuestas. El segundo contabiliza el número de segundos empleados por cada grupo
nivel de la jerarquía integra los siguientes seis herramientas o en la tarea de generar cinco apuestas.
espacios de la aplicación (ver Figura 3, izquierda): (i) el espacio • Tasa de error por el usuario: Esta métrica cuantifica el
para seleccionar un nuevo evento deportivo, (ii) la lista de número de acciones que realizan los usuarios y que no siguen
apuestas del usuario, (iii) el panel de votación, (iv) el chat el orden cronológico modelado en el modelo de observación
estructurado, (v) el espacio para crear nuevos grupos y (vi) el para establecer una apuesta (por ejemplo, un usuario intenta
tutorial. El tercer nivel de la jerarquía clasifica del siguiente modo introducir una apuesta deportiva antes de hacer una propuesta
las acciones de cada herramienta. Las acciones soportadas por el que cuente con el apoyo de los demás usuarios del grupo).
chat estructurado se clasifican como comunicativas, ya que • Recuerdo en el tiempo: Esta métrica cuantifica el qué medida
permite a los usuarios intercambiar mensajes. Las acciones los usuarios pueden recordar cómo funciona la aplicación.
soportadas por la herramienta de votación se clasifican como Para este propósito, el evaluador reutilizó la métrica tiempo
protocolarias, ya que se utilizan para llegar a un acuerdo acerca de de aprendizaje en un nuevo caso de estudio que se llevó a
las apuestas. Cuando los usuarios logran un acuerdo sobre cómo cabo una semana después de este caso de estudio inicial. Por
realizar una apuesta, la aplicación realiza automáticamente una lo tanto, el evaluador pudo analizar en qué medida el usuario
acción instrumental para generar esta apuesta. Las acciones recuerda el funcionamiento de la aplicación ya que no tiene
soportadas por las otras herramientas se clasifican como que volver a emplear su tiempo en el tutorial para recordar el
cognitivas; no interactúan con ningún elemento compartido y funcionamiento de la aplicación.
tienen otros propósitos como mostrar las funcionalidades de la
aplicación con un tutorial. Por último, el evaluador asigna La satisfacción del usuario es una métrica subjetiva, que no se
identificadores a las distintas acciones en el último nivel de la calcula de forma automática por el entorno. Por esta razón, los
jerarquía y describe la secuencia de acciones que debe ejecutar el usuarios contestaron un cuestionario que pedía cuantificar su
usuario para generar apuestas. satisfacción por medio de una escala Likert de cinco puntos.

Figura 3. Interfaz de usuario principal de la aplicación (izquierda), panel de votación (centro) y chat estructurado (derecha).
4.2 El modelo estadístico respecto a la satisfacción subjetiva del usuario. De izquierda a
Tenemos un conjunto de 84 usuarios, organizados en 21 grupos, a derecha y de arriba a abajo, están representados el tiempo de
los que se le han tomado cinco métricas. Cada una de las métricas aprendizaje, la velocidad de ejecución, la tasa de error por usuario
se registró por usuario, con la excepción de la velocidad de y la retención en el tiempo, en comparación con la satisfacción.
ejecución que se registró por grupo. La Figura 4 muestra las
cuatro métricas de usabilidad generadas por el entorno con

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 379 |
 Por lo tanto, un modelo de RLM se aplica al conjunto de datos
representado en la Figura 4 utilizando las variables tiempo de
aprendizaje y velocidad de ejecución, frente a la satisfacción. La
�
Ecuación 2 incluye la expresión obtenida donde 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆ó𝑛𝑛
denota la satisfacción estimada no redondeada, Tiempo representa
el tiempo de aprendizaje y Velocidad representa la velocidad de
ejecución. Por lo tanto, esta ecuación nos proporciona un valor
estimado de la satisfacción, dados el tiempo de aprendizaje y la
velocidad de ejecución. La satisfacción sólo toma valores en el
conjunto {1, 2, 3, 4, 5}, mientras que la satisfacción estimada no
redondeada puede tomar valores decimales. Por lo tanto, después
de aplicar la Ecuación 2 redondeamos adecuadamente la
�
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆ó𝑛𝑛 para obtener la satisfacción estimada.
�
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆ó𝑛𝑛 = 1.8559 ∗ 10−3 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 − 2.2625
∗ 10−3 𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉𝑉 + 4.4592
Ecuación 2. Relación de la satisfacción de los usuarios con el
tiempo de aprendizaje y la velocidad de ejecución.
Figura 4. Gráfica de las métricas objetivas con respecto a la Utilizando el conjunto de datos, representado en la Figura 4,
métrica subjetiva. dibujamos en la Figura 6 la satisfacción verdadera frente a la
Como la satisfacción es una métrica subjetiva categórica, nuestro satisfacción estimada. La gráfica de la izquierda de la Figura 6
objetivo es utilizar el procedimiento propuesto en la Sección 3.2 representa la satisfacción verdadera frente a la satisfacción no
consistente en aplicar un modelo de RLM al conjunto de datos redondeada; es decir, representa la satisfacción como se da en el
anterior y luego encontrar los límites adecuados para redondear la conjunto de datos, representado en la Figura 4, con respecto al
variable continua propuesta por el modelo de RLM. resultado de la aplicación de la Ecuación 2 al tiempo de
Para ello calculamos el coeficiente de determinación de todos los aprendizaje y la velocidad de ejecución dadas en el conjunto de
subconjuntos variables objetivas, seleccionado aquel que mejor datos. Mientras tanto, la gráfica de la derecha de la Figura 6
explica la satisfacción al tiempo que ofrece un modelo simple. representa la satisfacción verdadera frente a la satisfacción
Los resultados del análisis de todos los subconjuntos de variables redondeada de forma natural; es decir, representa, como antes, la
se proporcionan en la Figura 5. Esta figura representa las posibles satisfacción como se da en el conjunto de datos pero, aquí, con
combinaciones de las variables objetivas, y el término constante, respecto al resultado de redondear, a los números naturales, la
con respecto al coeficiente de determinación (r2). Podemos aplicación de la Ecuación 2 al tiempo de aprendizaje y a la
observar que cuatro de las combinaciones nos proporcionan un velocidad de dadas en los datos.
coeficiente de determinación de 0,93 (una vez redondeado a dos Debido a que los límites apropiados en el redondeo no están
decimales). De mayor a menor coeficiente de determinación, estas siendo utilizados, hay tres errores de clasificación, que son
combinaciones se componen de una combinación de las cuatro visibles en Figura 6 de color azul, rojo y verde. Se corresponden
métricas, dos combinaciones de tres métricas y una combinación con el valor 2 de la satisfacción, medida en el conjunto de datos, y
de dos métricas. Seleccionamos la combinación de dos métricas, �
los valores 2.500571, 2.506287 y 1.477177 de 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆ó𝑛𝑛 .
ya que nos proporciona el modelo más simple que tiene un Después de realizar el redondeo natural, estos valores
coeficiente de determinación más alto. Si redondeáramos a cuatro corresponden respectivamente a 3, 3, y 1, en lugar de a 2, el valor
decimales, coeficiente de determinación de esta combinación es que toma la medición de la satisfacción en el conjunto de datos.
0.9266. Las dos métricas son el tiempo de aprendizaje y la
velocidad de ejecución.

Figura 6. Satisfacción vs. satisfacción estimada, antes de ser

redondeada de forma natural (izquierda) y después (derecha).
Como se propone en la Sección 3.2, el siguiente paso es encontrar
los límites adecuados para hacer el redondeo. Se obtiene
(𝑙𝑙𝑙𝑙𝑙𝑙2 , 𝑙𝑙𝑙𝑙𝑙𝑙3 , 𝑙𝑙𝑙𝑙𝑙𝑙4 , 𝑙𝑙𝑙𝑙𝑙𝑙5 ) =
(1.336973, 2.529228, 3.516011, 4.505122). Como los tres
Figura 5. Análisis de todos los subconjuntos de variables en la errores de clasificación anteriores pertenecen a (𝑙𝑙𝑙𝑙𝑙𝑙2 , 𝑙𝑙𝑙𝑙𝑙𝑙3 ), no
RLM. hay ninguna clasificación errónea cuando se utilizan los límites
apropiados en el redondeo.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 380 |
 sentirse más satisfechos, ya que podría facilitarles una mejora en
la velocidad de ejecución. Por otra parte, el informe muestra los
resultados del modelo estadístico con las gráficas y la ecuación
para predecir la satisfacción de los usuarios a partir de los valores
de las métricas de alta prioridad.

Figura 7. Velocidad contra Tiempo donde la satisfacción del

usuario está coloreada en la gama de negro (1 en la escala de
Likert) a azul claro (5 en la escala de Likert), pasando por el
rojo (2), verde (3) y azul marino (4).
Del modelo dado por la RLM (Ecuación 2) se deriva que para
estimar la satisfacción del usuario es suficiente con tener el
tiempo de aprendizaje y la velocidad de ejecución. De hecho, la
satisfacción es directamente proporcional al tiempo de aprendizaje
e inversamente proporcional a la velocidad de ejecución. Esto se
puede observar en la Figura 7, donde se representa la velocidad
Figura 8. Fragmento del informe.
frente al tiempo. En esta figura los datos se han coloreado de
acuerdo con el valor de la satisfacción, desde el negro (1 en la 5. CONCLUSIONES
escala Likert) al azul claro (5 en la escala Likert), pasando por el Las herramientas software pueden ser de utilidad para llevar a
rojo (2), verde (3) y azul marino (4). Por lo tanto, los 5 grupos cabo procesos de evaluación de la usabilidad en contextos reales
pueden ser fácilmente identificables. Se puede deducir que, como de uso como aquellos en los cuales los usuarios de aplicaciones
muestra la Ecuación 2, una disminución de la velocidad y / o un groupware interactúan entre sí en cualquier momento y lugar
aumento en el tiempo produce datos que suben al siguiente punto mediante dispositivos móviles. Estas herramientas software
en la escala. En particular, para los datos con escala Likert 1 se pueden observar automáticamente las interacciones de los
observa que un aumento en el tiempo suficiente para subir a usuarios y generar un conjunto de métricas con información sobre
escala Likert 2. Análogamente, para los tres datos con escala la usabilidad de la aplicación.
Likert 4 y menor velocidad, basta un aumento en el tiempo para
actualizar a escala de Likert 5. Este artículo ha descrito un entono que, siguiendo un método
dirigido por modelos, permite construir soporte computacional
4.3 Generando informes que ejecuta automáticamente procesos de evaluación de las
La generación de un informe fue el último paso en este proceso de usabilidad previamente modelados. El entorno permite a los
evaluación. Este informe mostró al evaluador que las métricas de evaluadores configurar modelos con las interacciones que deben
alta prioridad son el tiempo de aprendizaje y la velocidad de observarse y con las métricas a calcular. De este modo el proceso
ejecución porque tienen una alta influencia en la satisfacción del de evaluación puede adaptarse a las características del sistema y
usuario. Sin embargo, el recuerdo en el tiempo y el ratio de del contexto de uso. Finalmente, el entorno genera el código
errores son métricas de baja prioridad. El informe incluye fuente del soporte computacional que lleva a efecto este proceso
información sobre el propósito de estas métricas. de evaluación de la usabilidad y así el esfuerzo de programación
La Figura 8 muestra un extracto de este informe, el cual incluye se ve reducido.
los valores que toman las dos métricas de alta prioridad cuando se El proceso de evaluación también analiza las relaciones entre los
maximiza la satisfacción del usuario en las tareas de evaluación. valores de las métricas y la satisfacción del usuario con la
Además, el informe muestra el porcentaje de usuarios satisfechos aplicación. Este análisis se realiza con un modelo estadístico que
con la aplicación. Así, el evaluador conoce el grado de genera una ecuación mediante la cual se puede conocer qué
satisfacción de los usuarios y qué aspectos deben mejorarse en métricas tienen una mayor correlación con la satisfacción del
futuras iteraciones del ciclo de vida de desarrollo del software. usuario y qué valores deben adoptar las métricas para optimizar
Por último, el informe sugiere a los evaluadores reducir el tiempo esta satisfacción. Como resultado del proceso de evaluación, el
necesario para realizar apuestas deportivas con la aplicación y entorno genera un informe con los valores que toma cada métrica
mejorar el tutorial que explica las funcionalidades del sistema. y un conjunto de recomendaciones para mejorar las debilidades de
Aunque los usuarios empleen más tiempo con este tutorial, suelen la aplicación. Estas debilidades se identifican gracias a que los
evaluadores incluyen el propósito de cada métrica en los

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 381 |
correspondientes modelos y, por tanto, es posible acceder al factor solving. Master of Science Thesis in Intelligent System
que causa problemas en la usabilidad de la aplicación. Design, University of Gothenburg, 2011.
El entorno ha sido puesto en práctica para evaluar la usabilidad de [11] Lee, Y. and Kozar, K.A. 2012. Understanding of website
una aplicación groupware móvil que soporta la realización usability: Specifying and measuring constructs and their
colaborativa de apuestas deportivas. Esta evaluación ha relationships. Decision Support Systems, 52, 2, 450-463
proporcionado un informe que muestra cómo los usuarios más [12] González M.P, Granollers, T., Lorés, J. 2008. Enhancing
satisfechos son aquellos que finalizan sus tareas en un corto Usability Testing through Datamining Techniques: A Novel
periodo de tiempo. Por tanto, la velocidad de ejecución debe ser Approach to Detecting Usability Problem Patterns for a
un atributo de usabilidad prioritario por su impacto en la Context of Use. Journal Information and Software
satisfacción del usuario. Esta evaluación muestra también que Technology, 50, 6, 547-568.
aquellos usuarios más satisfechos son aquellos que emplearon
mayor tiempo en comprender el sistema con un tutorial. Por tanto, [13] Nielsen, J. and Levy, J. 1994. Measuring usability:
parece que la aplicación necesita un periodo de entrenamiento preference vs. performance. Commun. ACM 37, 4, 66-75.
DOI=10.1145/175276.175282
para ser utilizada con un buen nivel de rapidez y crear un
sentimiento de satisfacción en los usuarios. Así, los [14] Hilbert D.M. and Redmiles, D.F. 2000. Extracting usability
desarrolladores y evaluadores son conscientes de que deben poner information from user interface events. ACM Comput. Surv.
su atención en esos factores durante el proceso de desarrollo. 32, 4, 384-421. DOI=10.1145/371578.371593
http://doi.acm.org/10.1145/371578.371593.
En el futuro se pretende trabajar en la evaluación de la
expresividad del DSL, la posibilidad de utilizar otros dominios de [15] Duque, R., Bravo, C., and Ortega, M. 2011. A model-based
aplicación e integrar los resultados del informe en el DSL para framework to automate the analysis of users' activity in
permitir refinar rápidamente los modelos con nuevas métricas. collaborative systems, Journal of Network and Computer
Applications, 34, 4, 1200-1209.
6. AGRADECIMIENTOS [16] Barron, B. 2003. When Smart Groups Fail. Journal of the
Este trabajo está financiado por el Ministerio de Economía y Learning Sciences, 12(3), 307-359.
Competitividad, proyecto MTM2014-55262-P, y por el Ministerio
de Ciencia y Tecnología, proyecto MTM2011-28657-C02-02. [17] Nielsen, J. 1993. Usability Engineering. Academic Press,
Boston.
7. REFERENCIAS
[1] ISO 9241–11, Ergonomic Requirements for Office Work
with Visual Display Terminals (VDTs) – Part 11: Guidance
on Usability, 1998.
[2] Dubey S.K. and Rana, A. 2012 Usability evaluation
Methods: A Literature Review. International Journal of
Engineering Science and Technology 4, 2, 590-599.
[3] Shneiderman B., and Plaisant, C. 2010. Designing the User
Interface: Strategies for Effective Human-Computer
Interaction. Fifth Edition. Addison-Wesley.
[4] Melody Y. Ivory and Marti A Hearst. 2001. The state of the
art in automating usability evaluation of user interfaces.
ACM Comput. Surv. 33, 4, 470-516.
[5] Molina, F. and Toval, A. 2009. Integrating usability
requirements that can be evaluated in design time into Model
Driven Engineering of Web Information Systems, Advances
in Engineering Software, 40, 12, 1306-1317.
[6] Cassino, R., Tucci, M., Vitiello, G., and Francese, R. 2015.
Empirical validation of an automatic usability evaluation
method, Journal of Visual Languages & Computing, 28, 1-
22.
[7] Seffah, A., Donyaee, M., Kline, R.B., and Padda, H.K. 2006.
Usability measurement and measurements: A consolidated
model. Software Quality Control 14, 2, 159-178.
[8] Suárez, M.C., Martínez, A.B., Alvarez, D. and Alva, M.E.
2013. Sirius: A heuristic-based framework for measuring
web usability adapted to the type of website. J. Syst. Softw.
86, 649-663.
[9] Martínez, A. 2003. Método y modelo para el apoyo
computacional a la evaluación en CSCL. Universidad de
Valladolid.
[10] Mattsson, S. Measures of collaboration in CSCW: Usability
and collective measures in remote and co-located problem-
XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 382 |
 Supporting Users Experience in a 3D eCommerce Environment
D. Contreras | M. Salamó | A. Yañez

Supporting Users Experience in a 3D eCommerce

Environment
D.Contreras M. Salamó, I. Rodríguez, A. A. Yañez
Facultad de Ingeniería y Arquitectura, Puig Facultad de Ingeniería y Arquitectura,
Universidad Arturo Prat, Chile Dept. Matemàtica Aplicada i Anàlisi, Universidad Arturo Prat, Chile
Dept. Matemàtica Aplicada i Anàlisi, Universitat de Barcelona, Spain. [email protected]
Universitat de Barcelona, Spain. {maria, inma,
[email protected] anna}@maia.ub.es

ABSTRACT First of all, when users arrive to our proposed 3D virtual store
3D eCommerce spaces may often be highly populated spaces, they find a registration panel, which allow them to provide their
with a huge number of users, areas and selling products. current preferences, e.g. camera price or manufacturer. Second, an
Therefore, their design should address different strategies to guide interactive map helps users to be aware of the whole buying space
and assist users to locate and to be aware of the most interesting and thereby find places and/or other users based on given
areas, products or users in there. In this paper, we present an preferences. And finally, during the buying process, they interact
approach to enhance and facilitate user buying experience in a with a conversational recommendation panel which helps them to
large 3D eCommerce space. Specifically, we propose a reach their desired product (i.e., using a conversational
collaborative recommendation framework which incorporates recommender) and, at the same time, to review two type of
three types of interactive objects –the User Registration Object suggestions provided by another recommender: (1) suggestions
(URO), the Interactive Map Object (IMO), and the about similar products that have been previously purchased by
Recommendation Object (RO)– to guide users along the whole other users; and (2) suggestions about affine people currently in
buying experience. the virtual store. The first type of suggestions is based on a HOR
(History and Opinions based Recommender) and the second one
Categories and Subject Descriptors is based on the similarity with other users’ preferences specified
I.3.6 [Methodology and Techniques]: Interaction techniques; to the CCR (Collaborative and Conversational Recommender)
H.5.2 [User Interfaces]: Interaction styles during the session.

General Terms 2. FRAMEWORK OVERVIEW

Theory. This section presents an overview of our collaborative
recommendation framework. As shown in Figure 2, it is divided
Keywords in three layers.
3D Virtual Worlds, Recommender Systems, Scalability. In the top of Figure 1, the 3D Space Client is an immersive 3D
1. INTRODUCTION virtual space where users interact with the following elements: the
User Registration Object (URO), the Interactive Map Object
3D Virtual Environments are collaborative 3D interfaces which
(IMO), the Recommendation Objects (RO) to acquire a desired
allow multiple users, represented as avatars, interact each other
product, and available Rooms in the space. In particular, a
and with the environment to synchronously develop either fun
Recommendation Object contains two differentiated areas, one for
(e.g., gaming) or serious (e.g., learning, shopping) activities.
searching products and another for receiving suggestions about
Depending on the domain, these activities are formed either by a
similar products and affine users. For each type of interactive
controlled set of users, for example a number of learners, or an
object (URO, IMO, RO), this layer triggers user registration,
unknown high number of potential users such as customers in a
interactive map, and recommendation object events respectively.
shop. Specifically, 3D eCommerce spaces may often be highly
populated spaces, with a huge number of users, areas and selling
products, and therefore their design should address different
strategies to guide and assist users to locate the most interesting
areas, products, or users in there.
In a previous work, we presented a framework that integrated a
recommender 2 in a 3D eCommerce space [1, 2]. The previous
work focused on demonstrating the collaboration aspect, whereas
this work addresses the scalability of the framework. To do so, we
enrich our framework to consider a high number of users
interacting in a large 3D eCommerce environment. In this context,
this paper contributes with new interaction objects to enhance and
facilitate users buying experience.

2
Recommender systems are well-known tools which provide the users a
personalized assistance when buying a product [3,5].

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 383 |
 Supporting Users Experience in a 3D eCommerce Environment
D. Contreras | M. Salamó | A. Yañez

Figure 1. Framework overview.

To facilitate a scalable number of users connected to the system, request when users interact within their searching area in their
the 3D store is organized in different spaces (Room, Hallway). RO.
Rooms have a limited number of Recomendation Objects and In the middle of Figure 1, the 3D Space Server is the
users interacting with them. When a room is full, new users can be communication layer responsible for the connection through the
located in another room taking into account their preferences. In Communication Management Module between previous layers,
case all rooms were full, we can dynamically generate new 3D interface, and recommenders. The Communication Management
content (rooms and ROs) using the Virtual World Grammar Module maps user events triggered in the 3D Space Client to user
concept proposed in [6]. Note that a group of rooms is usually information or recommendation actions and, in reverse,
focused in a particular product domain (e.g., smartphone). Then, recommendation actions or user information to user events (i.e.,
hallways would be groups of rooms that allocate different types of Recommendation Events, Suggestions Events, and Information
products. For example, the ‘hallway of Smartphones’ or the Events in 3D Space Client). Furthermore, it is responsible for the
‘hallway of PCs’. management of the 3D content objects (i.e., the 3D Content
In the bottom of Figure 1, the Recommenders Space layer hosts Management Module), and the users preferences (i.e., the Users
the User Models Information Module and the Recommenders Management Module). First, the 3D Content Management
Management Module. The former is responsible for obtaining the Module is in charge of updating the visual components in the 3D
user buying preferences about recommended products from User Space Client, the URO, the IMO, and the RO. Second, the Users
Models, U i , and general information about users (i.e., personal Management Module interchanges the user information, user
buying preferences and personal information, between the 3D
information as name, age, gender among others). The latter
Space Client and the Recommenders Space layers.
contains two types of recommendation algorithms, the HOR and
the CCR. Both are conversational recommenders, which are 3. INTERACTION OBJECTS
content-based recommendation [4]. The History and Opinions
In this section we detail the different interaction objects proposed
Based Recommender (HOR) algorithm receives a
for a 3D eCommerce store with a scalable number of participants.
recommendation request when users interact with an IMO or the
They allow users to get information and to be aware of other users
suggestions area in a RO. The Collaborative Conversational
and products locations, by helping them in their shopping
Recommender (CCR) algorithm receives a recommendation

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 384 |
decision, and by providing them with relevant information about however our framework allows the scalability and growth of the
similar products and users with similar preferences. space.
The interface layer in the collaborative recommendation Each 3D panel (RO) consists of several interactive elements (see
framework incorporates three types of interactive objects: the Figure 5), where a) are visual affordances representing the
URO, the IMO, and the RO. These objects guide users in each features of the current recommended product, b) displays one (
phase of their buying experience: the arriving (URO, IMO) and <> ) or two ( + , − ) icons that are used for performing
the buying phase (RO). Next, we describe each of them in the
domain of selling smartphone.
The User Registration Object (URO) is represented in the 3D
eCommerce store by a terminal machine as shown in Figure 2.
This terminal machine implements the metaphor of individual
communication between customers and bank systems, this time
mediating customer-recommender system communication.
Icons in the terminal machine represent two main preferences
needed by the recommender (e.g., mobile phone operating system
and price, in the Smartphone domain). Both of them visually
represented by the corresponding semantic icons, operating
system logo and dollars in the case of price. When the user
introduces her preferences, they are highlighted to provide visual
feedback, then she confirms them.

Figure 3. Screenshot of the interactive map.

critiques 3, with the value of the feature on top of it, c) is an image

of the current recommended product, d) are buttons for the
collaborative actions or for finishing the recommendation process
when the target is reached by pressing the Buy it button, e) shows
suggestions about people with similar preferences that may be
Figure 2. Screenshot of the user registration object.
useful for collaborating, and f) displays suggestions about bought
products by other users with similar preferences (i.e.,
The Interactive Map Object (IMO) is represented by a 3D panel recommendation based on off line history information).
which shows all rooms in the eCommerce space and users that are
interacting with Recommendation Objects (RO) (see Figure 3).
Note that the recommendation framework allows users to buy
products interacting either individually or collaboratively (i.e.,
with a group of users).
This map displays eCommerce store locations and related
information, where a) are rooms of the 3D e-commerce space, b)
are users into the rooms which may be green circle (i.e., users
interacting individually with a RO) and a red circle (i.e., two or
more users interacting collaboratively with a RO), c) are buttons
that allow users ask for suggestions about rooms where they
should go to find the users or products that matches their
preferences, d) is the map legend. Users can require information
Figure 4. Screenshot of 3D recommendation panels.
about another user by pressing the Affine Users button. The
information of other users is displayed through a text message and
it shows the similarity of preferences among them. Therefore, the user interacts with the RO: to critique one feature
The Recommendation Object (RO) is represented through 3D of the product, for example “I want a less expensive camera
panels into each room as shown in Figure 4. The 3D eCommerce
3
space now have six rooms with three panels for each room, A critique is a directional preference over a feature of the product to
determine users’ constraints and preferences.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 385 |
price”; to select a product suggested automatically by the generates an event that is revised by the recommender. Moreover,
recommender; and to select to collaborate with another user. All the critique action is first used to update the user model, U , as it
these actions are translated to recommendations events that arrive contains the preferences of the user, including the initial
at the Recommenders Management Module in the Recommenders preferences defined in the URO, and it is also used for
Space layer. This module uses two recommendation algorithms recommending another product. The process finishes when the
(i.e., CCR and HOR) for selecting the next product user either find (and buy) the desired product or give up the
recommendations to the user. When the Recommenders recommendation process.
Management Module returns the recommendation actions the RO
is updated to visualise the new recommended product. 5. CONCLUSION
Note that the visualisation of both IMO and RO is automatically In this paper we present a 3D recommendation framework which
updated from information coming from the 3D Space Server and provide eCommerce customers a collaborative interface to buy
Recommenders Space layers. products with the help of recommenders. It is composed by three
layers: the 3D interface, the recommendation and a
communication layer which allows the connection between the
other two. In this paper we focus on the former and advocate for
high populated 3D eCommerce Environments designed to
facilitate users awareness of locations, products and other users
which are also buying in the virtual store.
To do so, we have designed a new framework that enables
scalability and we have concentrated on this paper on the front-
end. That is, on the interaction mechanisms to enhance and
facilitate users buying experience. These mechanisms are
facilitated by several interface elements such as a registration
(preferences) panel, an interactive map and a recommendation
panel. As on-going work we are developing the framework
backend and then we plan to evaluate the usability of the proposal
with real users.

6. ACKNOWLEDGMENTS
D. Contreras is supported by a doctoral fellowship "Becas Chile"
from Chile Government. This research has also received support
from the project TIN2012-38603-C02, 2014SGR-623, and
Figure 5. Screenshot of the recommendation object. TIN2012-38876-C02-02 from the Spanish Ministry of Science
and Innovation.
4. USER INTERACTION FLOW 7. REFERENCES
This section describes the interaction flow that users may follow [1] D. Contreras, M Salamó, I. Rodrguez, and A. Puig.
during their buying experience. First, users arrive at the virtual Integrating a Conversational Recommender System within a
eCommerce space and provide some initial preferences. Next, 3D Collaborative Environment. In Proceedings of the XV
they locate a room to go or a user for starting collaboration. Conference of Spanish Association for Artificial Intelligence,
Finally, they start searching their desired product. Note that this pages 89–98, 2013.
flow is particularly useful for the users that have never been
before in the virtual eCommerce space and they need some [2] D. Contreras, M Salamó, I. Rodrguez, and A. Puig. An
guidance. However, users have freedom of choice and they may approach to improve user experience with conversational
follow a different course of actions. recommenders through a 3D virtual environment. In
Proceedings of the XV International Conference on Human
In detail, when users just arrive to the 3D e-commerce space (i.e., Computer Interaction, pages 403–406, 2014.
the 3D Space Client) they may register their initial preferences
about the desired product using the User Registration Object [3] J. Konstan and J. Riedl. Recommender systems: from
(URO). This is the first object they will find at the entrance of the algorithms to user experience. User Modeling and User-
eCommerce space. Note that users are allowed to come back to Adapted Interaction, 22(1-2):101–123, 2012.
any URO at any time and change their initial preferences. [4] P. Lops, M. Gemmis, and G. Semeraro. Content-based
Apart from the URO, at the hall users will see an Interactive Map recommender systems: State of the art and trends. In
Object (IMO) where they can be aware of the distribution of the Recommender Systems Handbook, pages 73–105. Springer,
space and the location of both rooms and other users. Once users 2011.
have selected a room in the IMO, they can use the teleporting [5] P. Resnick and H. Varian. Recommender systems.
mechanism or just walk to the room. There, they start a Communication of ACM, 40(3):56-58, 1997.
conversational recommendation session (i.e., individual or
[6] T. Trescak, M. Esteva, and I. Rodriguez. Vixee an
collaborative recommendations).
innovative communication infrastructure for virtual
A typical session with a conversational recommender consists of a institutions. In The 10th Int. Conference on Autonomous
series of recommend-review-revise-update cycles, where both the Agents and Multiagent Systems - Vol. 3, pages 1131–1132,
user and the recommender actively participate. Basically, the 2011.
recommender suggests a product and the users review the
recommended product by doing a critique action. This action

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 386 |
 La delgada línea roja entre la usabilidad y la experiencia de usuario
Y. Puerta Cruz | C. A. Collazos | T. Granollers

La delgada línea roja entre la usabilidad y la experiencia

de usuario

Msc. Yuliana Puerta Cruz Phd. Cesar A. Collazos Phd. Toni Granollers
Universidad del Cauca Universidad del Cauca Universidad de Lleida
Popayán-Colombia Popayán -Colombia Lleida- España
Corporación Universitaria [email protected]
Rafael Núñez
[email protected]
Cartagena-Colombia
[email protected]

RESUMEN permanece como un concepto ambiguo y puede ser difícil de

entender la naturaleza de la misma [2]. Diferentes autores
Este artículo busca debatir acerca de dos términos muy proponen similitudes en sus definiciones que dejan pensar que al
habituales en el campo del diseño de sistemas interactivos: la alcanzar Usabilidad se logra alcanzar cierto nivel de UX [3],
Usabilidad y la Experiencia de Usuario (UX). La necesidad considerando algunos a la Usabilidad como una de las diferentes
surge de observar (de forma no empírica) que tanto la industria facetas de la UX.
del software, la del diseño de interfaces e incluso en la
academia, utilizan dichos términos sin tener total claridad sobre N. Bevan [4] conceptualiza la UX a partir de la comparación
qué se refiere cada uno. Son incluso percibidos como atributos entre diferentes métodos de evaluación de Usabilidad, para lo
intercambiables “según la moda” del momento. Con la intención cual define una serie de diferencias a partir de la categorización
de profundizar empíricamente sobre estos conceptos, se estudia de métricas de UX y de Usabilidad. En [5] se estudian los
la literatura existente y se realiza una encuesta para tener efectos de medir la Usabilidad a nivel de interfaz de usuario y a
información de primera mano procedente de diferentes actores nivel del sistema, sugieren ampliar el concepto de Usabilidad a
relacionados con el Diseño Centrado en el Usuario (DCU) y la nivel del sistema para incluir la facilidad de aprendizaje, la
Interacción Persona-Ordenador (IPO) en general. La principal accesibilidad y la seguridad, contribuyendo a la experiencia
motivación en la cual reside nuestro interés científico está en general del usuario.
verificar si las aproximaciones DCU actuales son adecuadas Durante el desarrollo de este artículo se han revisado diferentes
tanto para el desarrollo de sistemas usables como para sistemas aproximaciones conceptuales y se ha elaborado el estado del arte
con alto grado de UX. de algunos trabajos que han intentado conceptualizar y/o
Categorías y Descriptores Temáticos establecer similitudes o diferencias entre estos conceptos.
También se ha diseñado y aplicado un instrumento de encuesta
H.5.2 [Interfaces de Usuario]: Metodologías/Evaluación, entre la comunidad de expertos académicos y profesionales con
Estilos de Interacción, Diseño Centrado en el Usuario. el fin de revisar como son entendidos y aplicados estos
Términos generales conceptos en el diseño de productos interactivos.

Diseño, Confiabilidad, Factores Humanos. Este artículo está organizado de la siguiente forma: en la sección
2 se realiza una revisión del estado del arte de los conceptos de
Palabras claves Usabilidad y UX, así como aquellos que consideran cercanía
Experiencia de Usuario, Usabilidad, Diseño Centrado en el entre estos conceptos. En la sección 3 se presenta el instrumento
Usuario, Interacción Humano Computador. utilizado para recolectar información en la comunidad de
expertos respecto a las diferencias y/o similitudes entre UX y
1. INTRODUCCIÓN Usabilidad. En la sección 4 se realiza en detalle el análisis de
resultados de la encuesta. Finalmente, se presentan las
Desde hace bastante tiempo, las personas pertenecientes a
conclusiones y trabajos futuros.
comunidad relacionada con la IPO vienen utilizando los
términos “Experiencia de Usuario” y “Usabilidad” de forma casi
sinónima. Lo cierto es que a la hora de definir ambos conceptos
existe una confusión generalizada entre aquellas personas que, 2. EXPERIENCIA DE USUARIO Y
incluso siendo conocedores del tema, no llegan a profundizar del USABILIDAD: EL DILEMA
todo e incluso tratan de enfrentar ambos conceptos [1]. Por esta Tomaremos las siguientes definiciones cómo punto de partida:
razón, partiendo de las definiciones más o menos aceptadas, en
• La Usabilidad se define como: “La medida en que un
este artículo se intenta indagar en las diferencias conceptuales
sistema, producto o servicio puede ser usado por usuarios
entre Usabilidad y UX para explicar algunos detalles que los
específicos para alcanzar metas con efectividad eficiencia y
distinguen, así como aquellos que los relacionan.
satisfacción en un contexto de uso específico” [6] y [7]. En
La UX es un tópico discutido frecuentemente en la ingeniería de [8] agregan que se refiere a “la facilidad de uso, sin
software en los últimos años, pero a pesar de los esfuerzos no se experimentar frustraciones ”
ha logrado un consenso de lo que significa exactamente y • Mientras que la UX se define como: “Las percepciones y o

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 387 |
 La delgada línea roja entre la usabilidad y la experiencia de usuario
Y. Puerta Cruz | C. A. Collazos | T. Granollers
respuestas que resultan del uso o anticipado uso de un A pesar del esfuerzo por encontrar diferencias y o similitudes
producto, sistema o servicio” [6] y [7]. En [9] afirman que entre estos conceptos, se observa que todavía no hay un
no es solo un buen diseño o interfaces de lujo, si no de crear consenso bien establecido y frecuentemente se llega a pensar
experiencias positivas a través del uso del producto. que, cuando se trata la Usabilidad y la UX, en realidad, se está
Estas definiciones resultan ser distintas, pero encierran cierto hablando de lo mismo. Así pues, con la intención de aportar
grado de ambigüedad que, al momento de utilizarlas de forma nuevo conocimiento al respecto creemos necesario disponer de
práctica, generan dudas en relación a lo que realmente abarca datos empíricos que nos aporten algo de luz con la que alumbrar
cada una de ellas y sobre cuales son los puntos que las la discusión abierta. Para ello, hemos diseñado un formato de
diferencian. Actualmente este es tema de discusión entre encuesta para aplicar entre los conocedores de estas áreas y
diferentes autores e investigadores del área de IPO. Como se desde el cual se pretende aclarar esta ambigüedad a partir de las
mencionó anteriormente en este trabajo se pretende abordar esa concepciones de los expertos. Los resultados obtenidos nos
cercanía, para determinar si a partir de las aproximaciones de permitirán despejar dudas y orientar el curso de nuestra
DCU se logra alcanzar buenos niveles de UX y no tan solo de investigación futura (ubicada en la construcción de metodologías
Usabilidad. orientadas al desarrollo de sistemas interactivos maximizando su
UX).
En [4] y [5] se afirma que la UX y la Usabilidad pueden ser
medidos durante o después del uso de un producto o servicio y 3. ENCUESTA ELABORADA
ponen a consideración la similitud entre la característica de Con el objetivo de lograr identificar esas similitudes y/o
percepciones y/o respuestas de la UX y la satisfacción que diferencias se elabora una encuesta dirigida a expertos en IPO
abarca el concepto de Usabilidad. Además, también en [5] se tanto en el ámbito empresarial como académico a nivel mundial.
deja ver que el disfrute o diversión es un aspecto de la UX que La encuesta intenta analizar las diferencias y/o similitudes
también contribuye significativamente a alcanzar satisfacción de existentes entre la Usabilidad y la UX, a partir de las opiniones
un producto, característica de la Usabilidad. Y para ello la UX de profesionales académicos y empresariales de estas áreas,
debe satisfacer las metas pragmáticas y hedónicas definidas en considerando sus diferentes puntos de vista y opiniones al
[10]. respecto, con el fin de consensar su definición y definir la línea
Las investigaciones en la comunidad académica han producido que separa estos conceptos.
diferentes Modelos y Frameworks para la evaluación de UX, De acuerdo a lo anterior, la hipótesis planteada en el desarrollo
pero en la práctica la industria adopta el término de UX para el de esta encuesta esta centrada en verificar si existen diferencias
desarrollo de productos basándose en los métodos de la considerables entre los conceptos de UX y Usabilidad. En la
Usabilidad tradicional [11]. De acuerdo a esto, se hace necesario encuesta se formularon preguntas abiertas y cerradas, a partir de
el desarrollo de métodos prácticos de evaluación de UX para ser las variables definidas, las cuales son: áreas de estudio de
usados en la industria y, para ello, es necesario tener claridad del preferencia (Usabilidad o UX), rango de diferencia entre UX y
concepto de UX e intentar adaptar otros aspectos de IPO a Usabilidad (medida como ninguna, mínimas, considerables,
aspectos particulares de la evaluación de UX [12]. totalmente diferentes), relación entre alcance de Usabilidad y
UX(evaluada como si, no, no sabe), nivel de importancia de las
Desde [13] dejan ver que un uso común o definición de UX aún
facetas que componen la UX (de 1-10 según grado de
no es definido claramente. Tanto la industria como la academia
importancia) y metodologías de diseño (esta variable se
coinciden en afirmar que la definición de UX incluye más que la
considero para identificar si se utilizan y cuales se usan).
definición previa de Usabilidad. En [12] también se busca
establecer y definir el alcance y concepto de la UX, para lo cual Considerando una población limitada (alrededor de 600 personas
aplican una encuesta entre académicos y profesionales del área que participan activamente en asociaciones como AIPO 4,
de UX, teniendo en cuenta el creciente interés en el tema y que UXPA 5, UX Alliance 6así como miembros de grupos de
no ha sido fácil establecer un concepto y un alcance común del investigación dedicado a trabajar en temas de IPO con
significado de la UX. conocimientos suficientes y trayectoria de investigación en estas
áreas) se ha estimado un error muestral de 5% (confianza del
Actualmente se utilizan diferentes métodos para la evaluación 95%), lo cual nos permite manejar una muestra adecuada
de Usabilidad y UX, pero el mayor grado de subjetividad que cercana a los 40 participantes. Al contar con 45 participantes en
encierra la UX no permite que sea evaluada de igual forma que la muestra, los resultados presentados son estadísticamente
la Usabilidad, es necesario conocer cómo se siente el usuario en representativos.
la interacción con el producto [14]. La UX llega a cubrir en su
totalidad los aspectos de interacción de un producto o servicio, a Su distribución se hizo a través de la herramienta gratuita de
partir de brindar altos niveles de calidad. Debe ayudar a la Google para crear formularios Google Form y fue distribuida
persona a acceder al producto con facilidad, para ello el entre los conocedores. Se obtuvo una muestra de 44 encuestas
producto debe ser simple, placentero, elegante, además de respondidas, cabe resaltar que su aplicación llego a ser
incluir aspectos culturales [15]. complicada porque a pesar que el contacto a través de la Internet
fue rápido, los tiempos de espera en las respuestas fueron lentos,
A partir de una revisión de diferentes trabajos, en [16] se logra sin embargo se logró con éxito su aplicación.
definir una serie de aspectos de la UX que van más allá de
aspectos emocionales, entre los que se destacan: la
accesibilidad, la propia Usabilidad, la jugabilidad, la plasticidad,
4
la utilidad o, entre otros, los aspectos contextuales. En [17] Asociación Interacción Persona Ordenador (aipo.es)
también dejan ver la importancia de incluir otros conceptos que 5
UserExperienceProfessionalsAssociation (https://uxpa.org)
van más allá de la Usabilidad, más relacionados con el contexto 6
de uso [7]. UX Alliance (www.uxalliance.com)

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 388 |
 La delgada línea roja entre la usabilidad y la experiencia de usuario
Y. Puerta Cruz | C. A. Collazos | T. Granollers

3.1 Análisis General de la Encuesta En relación a las opiniones respecto a las diferencias entre UX y
La tendencia de los encuestados estuvo marcada en trabajar Usabilidad, tanto en el área Académica como la Empresarial son
sobre ambos conceptos con aceptación de un 93,2% de los parejas: uno y otro afirman una tendencia que las diferencias
encuestados, este comportamiento estuvo determinado así tanto entre ambos conceptos son considerables, lo que deja ver que,
para los académicos como para los profesionales. para ellos, son dos áreas distintas; con características en común,
pero distintas.
En relación a las diferencias entre UX y Usabilidad hubo un
59,1% afirmó que existían diferencias considerables entre estos Las respuestas dejan ver que el listado de facetas expuestas es de
aspectos, aunque un 22,7% de los encuestados indicó que importancia cuando se quiere alcanzar UX, algunas en mayor
existían diferencias mínimas. Entre estos datos cabe destacar proporción que otras. Entre las facetas más destacadas se
que, para ninguno de los encuestados son exactamente iguales. encuentran la Usabilidad y el estado emocional en mayor
A pesar que un pequeño porcentaje de los encuestados indican proporción tanto para los académicos como los del área
que no existe ninguna diferencia entre estos conceptos, la mayor empresarial, sin olvidar la importancia de otras facetas como la
tendencia esta concentrada en afirmar que existen diferencias utilidad, la confianza o el contexto. Tanto los académicos como
considerables, lo que permite ver que la línea que separa estos quienes trabajan en la industria del diseño de sistemas
conceptos realmente es delgada y que realmente existe interactivos coincidieron en que la plasticidad es el área de
ambigüedad, pero no llegan a ser iguales. menos importancia.
Respecto a la relación entre el alcance de Usabilidad NO asegura En cuanto a las facetas propuestas para definir la UX, los
el alcance de buenos niveles de UX, un 45,5% de los académicos consideraron que el Estado Emocional era la faceta
encuestados indica que alcanzar Usabilidad en los productos NO más relevante, seguido de la Usabilidad y en menor importancia
asegura buenos niveles de UX, mientras que un 38,6% piensa la Plasticidad. De acuerdo a esto dejan ver que el alcance de
que SI se logra. La tendencia de estas opiniones es que al no ser buenos niveles de UX va más allá del concepto de Usabilidad,
el único aspecto que debe considerarse al evaluar la UX, la siendo necesarias otras facetas.
Usabilidad permite alcanzar cierto nivel UX, pero no en los De forma general los encuestados del campo empresarial
niveles deseados, porque es necesario considerar otros aspectos también consideraron que estas facetas, algunas en mayor
relevantes. Al realizar pruebas de de T-student se logra observar medida y otras en menor eran de real importancia al momento de
que el “P-value” es 0.685 > 0.25, se concluye que la Usabilidad alcanzar buenos niveles de UX. Las facetas de más importancia
no asegura el alcance de buenos niveles de UX. para ellos eran, la Usabilidad, la Utilidad y los aspectos
En relación al nivel de importancia de las facetas de UX Contextuales. Siendo las áreas de menor importancia la
definidas por [10] y puestas a consideración en este estudio, Jugabilidad y la Plasticidad
fueron tenidas en cuenta unas más que otras en el momento de
alcanzar buenos niveles de UX. Un 80% de los encuestados
5. CONCLUSIONES Y TRABAJO FUTURO
considera que la Usabilidad corresponde a una faceta La revisión del estado del arte deja ver que existe una gran
relacionadas con la UX, cuando se busca alcanzar buenos cercanía entre los conceptos de Usabilidad y UX, coincidiendo
niveles de UX en el diseño de un sistema interactivos. Un 64% en varios trabajos que la definición de los estándares ISO [7] al
considera que el Estado Emocional y la Utilidad son las otras indicar que una de las facetas de la Usabilidad es Satisfacción se
facetas de mayor importancia. La faceta con menos elección es alcanza en diferentes facetas de la UX. Por otro lado esta
la Plasticidad, pero también considerada importante con un cercanía se corrobora al considerar la Usabilidad como una
faceta de la UX.
25,7% de escogencia. Esto deja ver que la Usabilidad es
considerada como una faceta de importancia cuando se pretende En términos generales, se define la Usabilidad como una faceta
alcanzar buenos niveles de UX, sin embargo no es la única importante para la UX, no siendo la única que permite alcanzar
faceta que debe considerarse. buenos niveles. Se consideran como importantes otras facetas
En la verificación de las metodología de diseño y evaluación de como el estado emocional, la utilidad, el contexto, utilidad,
productos de software con buenos niveles de Usabilidad y/o UX, jugabilidad, accesibilidad, entre otras.
se pudo observar que la mayoría destacaban técnicas para la La línea que separa estos conceptos realmente está
evaluación de productos, principalmente de evaluación de especialmente manifestada con los aspectos emocionales que
Usabilidad, pero el uso de metodologías fue casi nulo. definen la UX. Al momento de considerar la Usabilidad como
3.2 Comparación de Resultados Sector una faceta de UX, se podría afirmar que al lograr Usabilidad se
logra un buen nivel de UX, pero no el nivel suficiente, porque es
Profesional y Académico. necesario considerar el resto de aspectos emocionales.
La tendencia de los encuestados tanto del campo Académico
como Empresarial es a estudiar ambas disciplinas, solo una Por otro lado, el hecho de realizar un sondeo sobre las
pequeña muestra de ambos campos coincidió en estudiar metodologías de diseño de Usabilidad y de UX, dejó ver entre
exclusivamente la UX. Y un académico indico estudiar los encuestados que actualmente utilizan mayormente técnicas
exclusivamente la Usabilidad. Las opiniones de los académicos de evaluación de estos conceptos frente al uso de metodologías
tuvieron mayor tendencia a que la Usabilidad no asegura que aseguren el alcance desde el diseño.
alcanzar buenos niveles de UX, mientras que las opiniones del
La revisión conceptual de Usabilidad y UX ha permitido
sector empresarial estuvieron divididas en esta cuestión. Sin
verificar la necesidad de considerar las diferentes facetas de la
embargo, y a pesar de estas opiniones, ambos estuvieron
UX en el desarrollo de productos y no solo desde la evaluación,
sectores coincidieron en asegurar que la Usabilidad SI es un
sino desde el diseño mismo. En este sentido, se orientara el
aspecto a considerar al momento de alcanzar buenos niveles de
desarrollo de esta investigación, hacia el diseño e
UX.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 389 |
 La delgada línea roja entre la usabilidad y la experiencia de usuario
Y. Puerta Cruz | C. A. Collazos | T. Granollers
implementación de una propuesta metodológica para el [5] Bevan, N. (2008, June). Classifying and selecting UX and
desarrollo de sistemas interactivos basada en el diseño centrado Usability measures. COST294-MAUSE Workshop:
en el usuario que integre facetas de UX. Meaningful Measures: Valid Useful User Experience
Measurement.
Las conclusiones mencionadas nos hacen emerger otras
preguntas importantes para ser valoradas en un análisis [6] Bevan, N. (2008). UX, Usability and ISO Standards.
posterior. Las nuevas preguntas son: London: W3 9RG.

• ¿podemos considerar que teniendo presentes los [7] ISO. (2009). ISO FDIS 9241-210. : ISO.
aspectos emocionales de las personas mientras utilizan [8] Krug, S. (2005). Don't Make me Think. Second Edition .
un sistema interactivo, es suficiente para garantizar un New Riders Publishing , California USA.
elevado grado de UX?, [9] Marc Hassenzahl (2013): HYPERLINK
• si coincidimos en que las metodologías de DCU son "http://www.interaction-
las más adecuadas para desarrollar sistemas design.org/encyclopedia/user_experience_and_experience_
interactivos altamente usables, ¿podemos decir que, design.html" User Experience and Experience Design . In:
reformulando los procesos DCU mediante la inclusión Soegaard, Mads and Dam, Rikke Friis (eds.). "The
de los mecanismos de diseño y evaluación emocional, Encyclopedia of Human-Computer Interaction, 2nd Ed.".
estos se transforma en procesos de Diseño Centrado en Aarhus, Denmark: The Interaction Design Foundation.
la UX? [10] Cockton, G. (2008). In:Maturing Usability. Quality in
Software Interaction and Value. Springer , .
Cuestiones que abordaremos para seguir avanzando en nuestra
línea de investigación. [11] Väänänen, K. V., & Roto, V. (2008 , June 18). Towards
Practical User Experience Evaluations Methods .
2. AGRADECIMIENTOS VUUM2008 , 19-22.
Este trabajo ha sido apoyado por el grupo de Investigación y [12] Law, E. L., Roto, V., Hassenzahl, M., Vermeeren, A., &
Desarrollo en Ingeniería de Software (IDIS) de la Universidad Kort, J. (2009). Understanding, Scoping and Defining User
del Cauca (Colombia) y el Grupo de investigación en Experience: A Survey Approach. CHI2009 , 719-722.
Interacción Persona-Ordenador e Integración de Datos (GRIHO)
[13] Hellman, M., & Rönkkö, K. (2008, June). Is User
de la Universidad de Lleida (España). Se agradece además a Experience Supported effectively in existing software
todos los especialistas en HCI, académicos y profesionales que development processes? VUUM2008 , 32-37.
colaboraron en el diligenciamiento de la encuesta.
[14] Obrist, M., Roto, V., & Väänänen, K. V. (2009). User
3. REFERENCIAS Experience Evaluation- Do you know which method use?
[1] White, Erin, "Usability and UX for libraries" (2014). VCU CHI 2009 , 2763-2766.
Libraries Faculty and Staff Presentations. Paper 6. [15] Jeong, B., & Chen, M. (2015). The New Understanding of
http://scholarscompass.vcu.edu/libraries_present/6 User Centered Design through the Extension of User
[2] Hellweger S., Wang X. What is User Experience Really: Definition and Goal . IPO Korea 2015, 26-30.
Towards a UX Conceptual Framework. Proceedings of the [16] Masip, L. A. (2013). User experience methodology for the
International MultiConference of Engineers and Computer design and evaluation of interactive systems. Lleida:
Scientists 2015 Vol II,March 18 - 20, 2015, Hong Kong. Universidad de Lleida.
[3] Majrashi K., Hamilton M and Uitdenbogerd A. Multiple [17] Bargas, J. A., & Hornbaek, K. (2011, Mayo 7-12). Old
User Interfaces And Crossplatform User Experience: Winw in New Bottles or Novel Challenges? A critical
Theoretical Foundations. SEA – 2015 pp. 43–57, 2015. © Analysis of Empirical Studies of User Experience. . CHI
CS & IT-CSCP 2015 DOI : 10.5121/csit.2015.50205 2011 , 2689-2698.
[4] Bevan, N. (2009). What is the difference between the
purpose of usability and user experience evaluation
methods? London: W3 93G.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (I)
| 390 |
USABILITY AND USER EXPERIENCE (II)
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis

Effect of Snippets on User Experience in Web Search

Mari-Carmen Marcos Ferran Gavin Ioannis Arapakis
Universitat Pompeu Fabra Softonic Barcelona, Spain Yahoo Labs Barcelona, Spain
Barcelona, Spain [email protected] [email protected]
[email protected]
ABSTRACT This work aims to understand better how the enrichment of
snippets with multimedia elements, recommendations, and geo-
In recent years, the search engine results pages (SERP’s) have location information affects user behaviour during interactions
been augmented with new markup elements that introduce with SERP’s. More specifically, we investigate the following
seamlessly additional semantic information. Examples of such three dimensions: (i) noticeability, (ii) interest, and (iii)
elements are the aggregated results disseminated by vertical conversion. As noticeability we regard the ability of rich
portals, and the enriched snippets that display meta-information snippets to compete for user attention against other types of
from the landing pages. In this paper, we investigate the gaze elements in a SERP. Interest is the overall engagement and
behavior of web users who inter- act with SERP’s that contain involvement with a given resource, whereas conversion is how
plain and rich snippets, and observe the impact of both types of this attention translates to click-throughs and, eventually,
snippets on the web search experience. For our study, we monetisation. Our experimental approach involves the use of
consider a wide range of snippet types, such as multimedia eye tracking technology and click logging, for capturing the
elements (Google Images, Google Videos), recommendation effects of rich snippets shown on SERP’s. This is a high-value
snippets (Author, Google Plus, Reviews, Google Shopping research area, yet poorly understood. We further believe that this
Product), and geo-location snippets (Google Places). We work can impact future web search interfaces and interaction
conduct two controlled user studies that employ eye tracking techniques for studying UX, as well as complement the common
and mouse tracking, and analyse the search interactions of 213 batch evaluations in information retrieval.
participants, focusing on three factors: noticeability, interest, and
conversion. Our findings indicate that ranking remains the most 2. RELATED WORK
critical factor in relevance perception, although in certain cases Until recently, the results of a SERP were presented using the
the richness of snippets can capture user attention. same layout, and the relative ranking position of a result was the
most determining factor that users considered [14]. This has
Categories and Subject Descriptors been demonstrated by previous studies [8] that have altered the
H.5.m [Information interfaces and presentation (e.g., HCI)]: original rankings of the results in SERP’s and observed that
Miscellaneous users would still assign more importance to top-ranked results.
In the same work [8], the authors introduced additional
General Terms information to the snippets and found that the performance for
Measurement, Design, Experimentation, Human Factors informational tasks improved significantly. The reason for that
was that users clicked on results while being certain about the
Keywords relevance of the corresponding landing pages, hence the
importance of snippets.
Search engines, rich snippets, eye tracking, user experience
A critical step in designing and building efficient search services
1. INTRODUCTION is understanding how people interact with SERP’s. In promoting
Information access is an area of research that, among other such knowledge, past work has proposed techniques ranging
aspects, deals with the improvement of SERP’s, both in terms of from direct observation and user surveys to log analysis, and
the presentation as well as the relevance of the retrieved results. most recently eye-tracking studies. Eye tracking is a promising
technique for the study of user behaviour in web search, since it
In the last decades, companies with online presence developed a can register accurately short- term changes in gaze activity that
strong dependence on tracking metrics provided by search are not measurable by other means. Furthermore, It provides an
engines. As a result, many companies turn to specialists for account of users’ unconscious behaviours and cognitive
optimising their web content for search engines and improving processing that are needed for interpreting their actions, as well
their ranking, a process known as Search Engine Optimisation as useful for mapping the user experience [6]. Some of the first
(SEO). At the same time, search engines constantly upgrade works that studied user interactions with SERP’s with the help
their algorithms and interfaces, seeking to facilitate better of eye tracking hardware, were published in the last decade [12,
services in terms of performance, content relevance, and user 13]. Since then, there has been a surge of research [2, 4, 3,
experience. One such example are the semantic markup
elements (multimedia items) introduced in the SERP’s, which 16, 26, 9, 5, 27, 19, 24, 7, 18, 10] that has employed eye
show aggregated results from vertical portals. If we consider the tracking and mouse tracking to analyse different aspects of the
case of Google, we can identify several other types of similar SERP’s.
elements sharing the space with the typical results, composed by
title, snippet (defined in [1] as “query-biased search result 2.1 Aggregated Search
summaries”), and URL. In addition to web search, the major commercial search portals
offer access to specialised search services (e.g., news, local
business, online products) or verticals of different in- formation

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 392 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
sources (e.g., images, videos, books). Aggregated search identified as an important factor that warrants further
attempts to achieve diversity by fusing results from different investigation.
verticals into one SERP, and complement the standard web
results. Arguello and Capra [3] employed several aggregated Chen et al. [7] studied click behaviour in aggregated SERP’s
verticals like images, news, shopping and video, and evaluated and proposed a novel federated click model, which accounts for
whether these verticals can influence user interactions with other the fact that user attention is attracted by vertical results. The
components in the SERP. The results from a large-scale study provided evidence which indicates that the visual attention
crowdsourcing study revealed that the level of influence may received by the vertical results can increase the chances that
depend on the vertical, its surrogate representation, where it is other nearby web results are also examined, and that click
displayed, and how it is distinguished from other components in behaviour on vertical results can lead to more clues of search
the SERP. In [2, 4], the authors examined the aggregated search relevance due to their presentation style.
coherence, i.e., the extent to which results from different sources 2.3 Rich Snippets & Social Annotation
focus on similar senses of an ambiguous or underspecified
Several works have also examined the interaction between rich
query. Both studies provided evidence that users are more likely
snippets and social annotations. A study by Muralidharan,
to interact with the web results when the vertical results are
Gyongyi, and Chi [18] demonstrated that placing a social
more consistent with the users’ intended query-sense, e.g., a user
annotation at the bottom of the snippet block reduces the
searching for information about the planet “Saturn” is more
probability of users fixating on it. The same study also revealed
likely to interact with the web results if the blended images
that positioning the social annotation at the top of the snippet
contain pictures of the planet versus the car.
block can mitigate this issue. The authors attribute this
2.2 Gaze and Cursor Behaviour in Web behaviour to the “inattentional blindness effect” [17], which
leads people to notice mainly what they are actively looking for.
Search In this study, the tasks were con- ducted using mock-ups and
Several studies have investigated the gaze and click behaviour in expert searchers, which places the reported effects in an artificial
aggregated search. Liu et al. [16] collected eye tracking data, environment rather than a real-life setting.
click-through data, and users’ feedback on their examinations of
SERP’s. The findings of this study reveal than a large Fernquist and Chi [10] investigated the impact of enriched
proportion (45.8%) of the results fixated by users were not elements with social annotation in SERP’s. More specifically,
recognised as being “read”. In addition, the authors of [16] the authors conducted an eye-tracking study using a
found that before users actually read the results, there is often a retrospective think-aloud protocol, based on the design
skimming step during which they briefly scan the results without guidelines discussed in [18]. The authors blended the users’
reading them, and perform judgments according to different personal organic search results with regular, live search results,
signals. using a personalised ranking function. Their findings reveal that
users employ annotations for local and shopping query types
Wang et al. [26] and Diaz et al. [9] found that different result more often than fact-finding and news-related query types.
appearances can create different biases on gaze behaviour for Additionally, the authors discovered that users make use of
both vertical and other results on SERP’s. A number of studies annotations only when the task context and situation fit the need
[5, 27] also showed that multimedia components in SERP’s and to look for social resources, such as subjective queries like
result attractiveness may influence users’ gaze and click-through shopping, products, and restaurant reviews.
behaviour. Furthermore, Navalpakkam et al. [19] conducted a
controlled study where they varied the presence and relevance of Despite the above research efforts, there is still little
a rich informational panel placed to the right of organic search understanding on what constitutes a well-designed, attractive,
results. and engaging aggregated SERP. This paper provides some
initial insights by examining the design of aggregated results
The authors discovered that the information panels contain- ing and their effect on user gaze and click-through behaviour, in the
information relevant to the user’s task can attract more attention context of web search. In what follows, we provide a description
and facilitate longer mouse cursor hovers. of our experimental method and discuss the main findings.
Sushmita et al. [24] investigated the factors affecting users’
click-through behaviour on aggregated search interfaces. Their
3. METHODOLOGY
study led to several findings. Foremost, it revealed that the To demonstrate the impact of enriched snippets on user search
experience we carried out two controlled experiments, thereof
position of search results was only significant for the blended
referred to as Study 1 and Study 2.
interface. Secondly, participants’ click-through behaviour on
videos was different compared to other sources. Finally,
capturing a task’s orientation towards particular sources was

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 393 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis

(a) SERP with plain snippets. (b) SERP with rich snippet (#2).
Figure 1: SERP example showing the original plain results (a) and the results with the enriched snippet (b).
Both studies asked a number of participants to complete a series 6th, or 7th position. While retaining the original look and feel of
of search tasks using a commercial search engine, and shared a the retrieved SERP’s, we edited the HTML code of the pages so
similar objective: investigate to what extent the richness of the that they either contained only one rich snippet or no rich
snippets can affect user behavior in web search. Study 2 snippets; any additional, unwanted rich snippets were modified
followed Study 1, and aimed at validating the preliminary to appear as plain snippets instead.
findings discovered in Study 1.
Given that the retrieved SERP’s showed the original top ten
3.1 Study 1 results obtained from Google search, we considered them as
Study 1 consisted of an offline and an online part. The offline topically relevant to the associated search query. This
part was conducted in a laboratory setting, where the assumption was further validated by a manual examination. We
experimental procedure was followed and eye tracking data also controlled that none of the snippet text (plain or rich)
were collected. The online part repeated the same experimental contained directly the answer to the query. The reason for that is
design in an online setting, and collected mouse tracking data. because we wanted to study click behaviour (con- version),
therefore displaying the answer in a snippet would bias to click
3.1.1 Design on it, or avoid it if its not the correct one
The study used a mixed design with three independent variables:
Our dependent variables were noticeability, interesting- ness,
snippet richness (with two levels: “plain”, “rich”), snippet
and conversion, all aspects of the search experience that can be
position (with two levels: “top-ranked”, “bottom- ranked”),
potentially affected by the richness of the snip- pets. To study
and snippet type (with five levels: “Author”, “Google Plus”,
the effects of our experimental manipulation on our dependent
“Google Places”, “Multimedia”, “Review”). To control for
variables, we obtained several metrics of gaze behaviour and a
snippet richness, we prepared two versions of every SERP
metric of search task performance [6], as shown bellow:
shown in the study: one version containing a rich snippet
among the organic results (at top-ranked position • Noticeability: Time to First Fixation, Fixations Before
#2 or #3, or bottom-ranked position #6 or #7) and an- other • Interest: Total Fixation Duration, Fixation Count,
version without the rich snippet, showing only plain web results. Visit Duration
More specifically, for each task introduced in the study (Section • Conversion: Click count
3.1.3), predefined search queries were sub- mitted to Google
Search until obtaining SERP’s that contained at least one of the
types of rich snippets shown in Table 1, in the 2nd, 3rd position,

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 394 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
3.1.2 Apparatus To analyse gaze behaviour, we used a Tobii 1750 eye tracker,
The relationship between attention and eye movements has been integrated into a 17” TFT monitor with a 1280×1024 resolution.
investigated extensively in the past [11, 20, 21, 22]. When we When activated, the eye tracker illuminates the user with two
read, examine a scene, or search for an object, we continuously infrared projections that generate reflection pat- terns on the
make eye movements called saccades. Saccades are rapid corneas of the eyes. A video camera gathers these reflection
movements that occur when we change focus, and can reach patterns along with the position of the user and, through digital
velocities as high as 500◦ per second. When the visual gaze is image processing; the pupil locations are extracted at a rate of
maintained on a single location for several milliseconds we have 50 Hz. The pupil positions are then mapped to gaze locations on
a fixation. The importance of gaze in the assessment of attention the screen. For the gaze behaviour analysis we used the eye
focus lies in the fact that, although looking might appear to be a metrics listed in Table 2, which were extracted automatically
process that is under voluntary control, conscious and deliberate using the Tobii Studio Statistics application. The metrics were
control of fixation happens infrequently. As with other calculated based on defined Areas of Interest (AOIs) and data
components of voluntary performance, looking is controlled by selection time intervals. We defined as our AOI(s) the results
a general intention, and consciousness plays a minor role in the shown in the SERP. In addition to using eye tracking in the
execution of the intended sequence of fixations [15]. offline part of Study 1, we also used the software CrazyEgg to
log participants’ mouse tracking data for the online part of the
study.
Table 1: Types of rich snippets and search tasks used in Study 1
Rich snippet Search task
1. Find a hotel near Sants Station
Google Places
2. Find a destination management company in Barcelona
3. Find the Fitur events program
Google Plus
4. Find the definition of “revenue management
5. Find statistics of the use of Facebook in companies of the
Author touristic sector
6. Find what actions is taking the Ushuaia Hotel in Facebook
7. Find in what Facebook Places does consist on
Multimedia
8. Find a webpage with reviews of the Ushuaia Hotel
9. Find a webpage with reviews of the Pach´a Hotel
Review
10. Find a good blogs directory about tourism and travel

Table 2: Eye metrics used to analyse gaze behaviour top-ranked (#2 or #3) or a low- ranked position (#6 or #7), and
1 Time to First Fixation: Time taken (in seconds) before a one showing only plain snippets.
participant fixates on an AOI for the first time. 3.1.4 Participants
2 Fixations Before: Number of times a participant fixates on A total of 60 participants (female=38, male=22), between the
the media before fixating on an AOI for the first time. ages of 18 to 58 and free from any obvious physical or sensory
impairment, were recruited through a campus- wide ad. All
3 Total Fixation Duration: Sum of the duration for all
participants were frequent users of web search engines. Besides
fixations within an AOI.
the 60 participants who were involved in the offline study that
4 Fixation Count: Number of times a participant fixates on collected eye tracking data, another 110 participants were
an AOI. involved in the online study that collected mouse tracking data.
5 Visit Duration: Duration of each individual visit within an The participants were mainly locals (i.e., Catalan, Spanish) and
AOI. had graduated from, were currently studying, or working at the
at Pompeu Fabra University, Barcelona. Finally, participants
were all proficient with the English and Spanish languages.
3.1.3 Search Tasks 3.1.5 Procedure
A total of ten tasks (shown in Table 1) were performed by every For every search task, participants were presented with an initial
participant, two for each type of rich snippet. In the context of search query and the retrieved SERP for this search query. They
these search tasks, the participants assumed the role of an expert were instructed to examine the SERP as they would normally
in social media for the tourism sector and were invited to do, although they were not able to issue additional search
participate in a panel for a particular city. Examples of the tasks queries. As long as the answer did not appear in the snippet
involved finding a hotel near the venue, retrieving the description of the retrieved results, participants were encouraged
conference program, and other. For each search task, two to click on any of the result links and examine the landing pages,
SERP’s were available: one that contained a rich snippet at a prior to continuing to the next task. Two tracks were designed in
order to show the SERP’s in an alternating manner (Figure 2).

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 395 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
There- fore, for every type or rich snippet participants were When comparing top-ranked vs. bottom-ranked snippets, we
shown a SERP with a rich snippet (at a top- or low-ranked posi- identify several statistically significant differences across our
tion) and a SERP containing only plain snippets. Selecting a eye metrics, and for both rich and plain snippets, and different
particular result as the one the participants felt would an- swer snippet types (Author, Google Plus, Google Places, Multimedia,
the search task question, would conclude the task. To enrich the Review). More specifically, for the metric Time to First
sample for the conversion measure, 110 partici- pants performed Fixation, the Mann-Whitney test reveals that top- ranked plain
the same search tasks online, while we were logging their mouse snippets (M dn = 2.88) received their first fixation significantly
tracking data. faster than bottom-ranked plain snippets (M dn = 15.71), U =
1289.50, z = −10.03, p = .000, r =−.48. Similarly, the top-ranked
3.1.6 Results rich snippets (M dn = 3.25) received their first fixation
To choose an appropriate statistical test, we first examine the significantly faster than bottom- ranked plain snippets (M dn =
distribution of our data using the Anderson-Darling and Cramer- 15.30), U = 1135.50, z =−9.71, p = .000, r = −.46. For the metric
von Mises tests. These tests are known to perform better Fixations Be- fore, the Mann-Whitney test reveals that top-
compared to the Kolomorov-Smirnov test [23, 25], al- though in ranked plain snippets (M dn = 9.00) had significantly fewer
large samples they tend to be significant even for scores that are fixations on other items before being noticed than bottom-
marginally different from a normal distribution; we thus ranked plain snippets (M dn = 51.50), U = 1405.50, z = −9.79,
interpret them in conjunction Q-Q plots, while also accounting p =.1, r = −.46. The same highly significant effect is observed
for the skew and kurtosis values. Since in all cases we observe a when comparing top-ranked rich snippets (M dn = 9.00) against
non-normal distribution in the absolute differences of the bottom-ranked rich snippets (M dn = 46.00), U = 1112.50, z =
estimated error, we opt for the Mann- Whitney test and report −9.77, p = .000, r = −.46. Additionally, when examining the
our results at an α level of .05. metric Total Fixation Duration, the Mann-Whitney test reveals
Finally, to take an appropriate control of Type I errors in that the top-ranked plain snip- pets (M dn = 2.08) received
multiple pair-wise comparisons we apply the Bonferroni longer fixations than bottom- ranked plain snippets (M dn =
correction. 0.81), U = 3644.00, z =−5.08, p = .000, r = −.24. Likewise,
participants fixated for longer times on the top-ranked rich
snippets (M dn = 2.08) than bottom-ranked rich snippets (M dn
= .90), U = 3364.50, z = −4.65, p = .000, r = −.22. For the
Fixation

(a)Only plain snippets .(b) With rich snippet (#7).

Figure 3: Heatmaps showing allocation of user attention in
web search, on two SERP’s.
U= 3364.50, z = −4.65, p = .000, r = −.22. For the Fixation
Count metric the Mann-Whitney test reveals that the top-ranked
plain snippets (M dn = 10.00) received significantly more
fixations than bottom-ranked plain snippets (M dn = 4.00), U =
3422.50, z = −5.56, p = .000, r = −.26.
The same effect, is observed when comparing top-ranked rich
snippets (M dn = 10.00) against bottom-ranked rich snippets (M
dn = 5.00), although it has a smaller effect size, U = 3550.00, z
= −4.23, p = .000, r = −.20. Finally, for the metric Visit
Duration, participants spent significantly more time examining
the top-ranked plain snip- pets (M dn = .81) than the bottom-
ranked plain snippets (M dn = .59), U = 4975.50, z = −2.28, p =
.022, r = −.10.
Likewise, participants spent significantly more time examining
the top-ranked rich snippets (M dn = .74) against bottom-ranked
rich snippets (M dn = .53), U = 4035.00, z =
Figure 2: Experimental protocol. −3.12, p = .002, r = −.14. These findings provide further
evidence that the results shown in top-ranked positions at- tract

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 396 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
significantly more attention, and for longer time, com- pared to snippet typologies, the number of clicks is similar; a statistically
the results shown in bottom-ranked positions. But does this also significant difference is only found for the Google Plus
hold for plain vs. rich snippets that are dis- played in the same recommendation snippet (p = .04). For this type of rich snippet,
position in a SERP? Does the richness or type of snippet the recommended result received eight clicks, while the plain
introduce any effect? result received only two. However, we cannot conclude that
To answer the above questions, we perform a comparison Google Plus recommendations per se had a strong influence on
between rich and plain snippets that have the same relative the participants’ click behaviour. We need to account for the
position in the layout of the SERP, across all eye metrics. We context as well as the person that recommends the result, as dis-
observe that rich snippets generally tend to attract more cussed in [18]. In this case, the picture we used as a
attention. For example, rich snippets receive their first fixation recommender is a public and acknowledged person. We
faster, have fewer fixations on other elements prior to being speculate that the results could have been different if the
noticed, and receive more and longer fixations than plain recommender was someone less popular or unknown to the wide
snippets. However, none of these differences appear to be public. Finally, the rich snippet with the least clicks is the
statistical significant, and even less when comparing snippets in Multimedia snippet, most likely because people seeking
top-ranked positions (Figure 3). Most likely, when a snippet information do not take so much into consideration multimedia
appears in a top-ranked position, this by itself introduces a bias results. An- other possible interpretation is that any deviation
which makes users consider it as topically relevant [1]; thus the from the traditional, textual presentation of he results may lead
absence of significant effects in top-ranked snippets. to “banner blindness”, and consequently result in less attention
to the enriched snippet. No significant differences in
Next, we repeat our analysis for each snippet type separately. As performance are found when we compare enriched results with
before, we observe differences between the rich and plain plain snippets.
snippets that span across the different types of snippets, like
Google Places, Author, Multimedia, etc., al- though most of 3.2 Study 2
them appear not to be statistically significant. As side-findings, The second study is a follow-up to Study 1. It was per- formed
we report that participants fixated faster on the top-ranked rich to investigate users’ behaviour while interacting with a different
Multimedia snippets (M dn = 1.87) than the top-ranked plain set of snippets (Figure 4) and also allowed us to replicate the
Multimedia snippets (M dn = 2.49), U = 218.00, z = −2.36, p = preliminary findings of the first study. More specifically, we
.018, r = −.24. More- introduce the Google Shopping snippet and remove the Google
Plus (given recent findings reported in [18]) and Multimedia
over, the bottom-ranked rich Multimedia snippets (M dn =.45)
snippet types. The reason for that was the increasing popularity
were observed for less time than the bottom-ranked plain
and visibility of Google’s Shop- ping Product snippet. In
Multimedia snippets (M dn = .71), U = 104.50, z =−1.99, p =
.045, r = −.20. addition, we now compare only top-ranked rich vs. plain
snippets, and exclude from our analysis bottom-ranked results,
in the light of the findings provided by Study 1.
Table 3: Types of rich snippets and search tasks used in
Study 2
Rich snippet Search task
Author 1. iPad 3 Vs. iPad 2 comparison
Google Shopping 2. Buy iPad 3
Google Places 3. Apple store Madrid
Review 4. Best moment to sell an iPhone

3.2.1 Design
The study used an independent measures design with two
independent variables: snippet richness (with two levels:
“plain”, “rich”), and snippet type (with five levels: “Author”,
“Google Places”, “Google Shopping”, “Review”). To control for
snippet richness, we prepared two versions of every SERP
shown in the study: one version containing a rich snippet among
the organic results (at position #2) and another ver- sion without
the rich snippet, showing only plain web re- sults. More
specifically, for each task introduced in the study (see Section
Figure 4: Visual representation of the four rich snip- pet 3.2.3), predefined search queries were submitted to Google
types used in Study 2. Search until obtaining SERP’s that contained at least one of the
When analysing the click-through data of the 110 participants types of rich snippets shown in Table 3 in the 2nd or 3rd
who performed the online part of Study 1, we observe that the position. While retaining the original look and feel of the
top-ranked snippets of type Review, Author, and Google Plus retrieved SERP’s, we edited the HTML code of the pages so that
(all recommendation snippets), receive the highest percentage of they either contained only one rich snippet or no rich snippets;
clicks: 32%, 21%, and 18% respectively. In 4 out of 5 rich

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 397 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
any additional, unwanted rich snippets were modified to appear asked to click on the result that they felt would answer the
as plain snippets instead. question presented in the search task. After per- forming the four
Given that the retrieved SERP’s showed the original top ten tasks, the users in the group that saw the enriched SERP’s were
results obtained from Google search, we considered them as shown the rich snippet highlighted and were asked to answer the
topically relevant to the associated search query. This following questions: (i) Did you notice the highlighted result?,
assumption was further validated by a manual examination. (ii) If you noticed it, did you click on it?, (iii) What was the
Similarly to the previous study, we controlled that none of the reason behind (not) clicking on it?, and (iv) Did the image
snippet text (plain or rich) contained directly the answer to the (pictures, maps, stars) influence your click decision?
query. Our dependent variables were again noticeability, 3.2.6 Results
interestingness, and conversion, all aspects of the search To analyse the user behaviour (noticeability, interesting- ness,
experience that can be potentially affected by the richness of the and conversion,) we use the Mann-Whitney test on all eye
snippets. To study the effect of our experimental manipulation metrics. Our analysis does not reveal any statistically significant
on the dependent variable, we obtained the same five metrics of differences between plain and rich snippets, for any of the
gaze behaviour and one metric of search task performance SERP’s and snippet types (Author, Google Places, Google
discussed in Section 3.1.1. Shopping, Review). In addition, we apply the Chi-Square test of
3.2.2 Apparatus Association on the click data obtained by the eye tracking
The study used the setup discussed in Section 3.1.2. software. A significant association between the snippet richness
(plain, rich) and the number of participants who clicked on the
3.2.3 Search Tasks enriched results is observed for the task that involved the Google
Four search tasks were used in the study, as shown in Table 3. Places snippet, (χ2(1) = 4.00, p = .045). The reason could be that
Each search task asked the participants to retrieve relevant the map and address shown in the rich snippet were perceived as
information to answer a question, e.g., find information on how helpful and topically relevant, and the participants felt inclined
to arrive to the AppStore located in Madrid. For each search to select it.
task, participants were presented with an initial search query and Furthermore, we apply the Chi-Square Goodness-Of-Fit test on
the retrieved SERP for this search query. Participants were the questionnaire data, and more specifically to questions (i),
instructed to examine the SERP as they would normally do, (ii), (iii), and (iv) presented in Section 3.2.5. The test reveals a
although they were not able to issue additional search queries. consistent effect across all snippet types. More specifically, the
As long as the answer did not appear in the snippet description, Chi-Square Goodness-Of-Fit test indicates that significantly
the participants were encouraged to click on any of the results more participants reported having noticed the rich snippet of
and examine the web pages prior continuing to the next task. type Author (χ2(1) = 6.00, p = .014), Google Shopping (χ2(1) =
Selecting a particular result as the one that answered the search 8.16, p = .004), Google Places (χ2(1) = 8.16, p = .004), and
task question would conclude the task. Review (χ2(1) = 6.00, p =.014). Finally, significantly more
Half of the participants were shown SERP’s that contained only participants reported that the presence of the pictures, images, or
plain snippets and the other half SERP’s that included a rich stars in rich snippets did not influence their decision of clicking
snippet at a top- ranked position. the result, that those who reported that it did. Again, this effect
is consistent across all snippet types, i.e., Author (χ2(2) = 15.75,
3.2.4 Participants p =.000), Google Shopping (χ2(2) = 19.00, p = .000), Google
A total of 43 participants (female=26, male=17), between the Places (χ2(2) = 13.00, p = .002), and Review (χ2(2) = 7.00, p =
ages of 18 to 45 and free from any obvious physical or sensory .030).
impairment, were recruited through a campus- wide ad. All
participants were frequent users of web search engines. The 4. DISCUSSION & CONCLUSIONS
participants were mainly locals (i.e., Catalan, Spanish) and had In this paper we presented a two-part study of the gaze and
graduated from, were currently studying, or working at the at mouse behaviour of web users, while interacting with SERP’s.
Pompeu Fabra University, Barcelona. Finally, participants were Considering our User Experience (UX) context, our goal was to
all proficient with the English and Spanish languages. investigate whether the richness of snip- pets can affect user
behaviour and introduce a bias to the subjective perception of
3.2.5 Procedure relevance, i.e., which results people perceive as more useful or
Two groups were created using an independent measures design. relevant in a SERP.
Each group performed four search tasks. The search tasks were
Foremost, the findings of our first study indicate that the relative
part of a larger scenario that instructed the participants as
position of a result in a SERP remains the most influential factor
follows: “You want to buy the new iPad 3, but before you do so,
of click behaviour, although snippet rich- ness appears to
you need to know how much it costs, what are the technical pros
become a more important variable, especially when examining
and cons in comparison with iPad 2, how to arrive to the
bottom-ranked results. Our analysis did not indicate an effect of
AppStore in a particular city, and the best moment to sell your
snippet richness on the gaze behaviour for top-ranked positions:
iPad 2”. Four SERP’s were selected and were enriched with four
none of the eye metrics we examined was found to be
types of rich snippets: Product, Author, Places, and Reviews.
statistically significantly different for rich snippets, followed by
For this study, the rich snippets and its respective plain snippet
similar findings in regards to the recorded clicks. Nevertheless,
were always placed at a top position (#2).
snippet richness is a factor that must be taken into account when
For each group, half of users would see the SERP’s with the considering bottom-ranked results, since they are noticed much
plain results and the other half would see the SERP’s that earlier than the plain snippets and for longer periods of time.
contained the rich snippet. As in Study 1, the participants were Furthermore, Multimedia snippets were the most noticeable

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 398 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
element across the evaluated types of snippets in this study. Conference on information &#38; knowledge management,
Also, we demonstrated that the social content in a snippet has a CIKM ’13, pages 1989–1998, New York, NY, USA, 2013.
growing importance, as indicated by the higher click count it ACM.
achieved, compared to the equivalent plain results of the same [5] J. Bar-Ilan, K. Keenoy, M. Levene, and E. Yaari.
ranking. Presentation bias is significant in determining user
The second study compared rich and plain snippets dis- played preference for search results—a user study. Journal of the
at top positions. Our analysis did not provide any evidence that American Society for Information Science and Technology,
rich snippets attract attention faster or for longer periods of time 60(1):135–149, 2009.
than their corresponding plain snip- pets. Additionally, the [6] A. Bojko. Eye Tracking the User Experience: A Practical
number of clicks did not differ be- tween the rich and plain Guide to Research. Rosenfeld Media, 2013.
snippets, apart from the Google Places snippet. The reported
answers in the questionnaire suggested that the participants were [7] D. Chen, W. Chen, H. Wang, Z. Chen, and Q.
aware of the presence of these enriched results, but their Yang.Beyond ten blue links: Enabling user click modeling
decision of clicking was not conditioned by their presence in the in federated web search. In Proceedings of the Fifth ACM
SERP. International Conference on Web Search and Data Mining,
WSDM ’12, pages 463–472, New York, NY, USA, 2012.
In conclusion, our study provides further evidence, which ACM.
confirms the importance of ranking in relevance judgments, and
also indicates that snippet richness is not as influential as one [8] E. Cutrell and Z. Guan. What are you looking for?: An eye-
would originally anticipate. However, this may change in the tracking study of information usage in web search. In
future, but more work is needed to understand how rich snippets Proceedings of the SIGCHI Conference on Human Factors
should be presented, and in which position in a SERP they are in Computing Systems, CHI ’07, pages 407–416, New
most effective. We consider that this work can impact future York, NY, USA, 2007. ACM.
web search interfaces and interaction techniques for studying [9] F. Diaz, R. White, G. Buscher, and D. Liebling. Robust
UX, as a complement to the common batch evaluation that models of mouse movement on dynamic web search results
dominates the IR field. Eye-tracking is a suitable method for pages. In Proceedings of the 22Nd ACM International
such research, as it accurately captures a user’s initial (low- Conference on Conference on Information &#38;
level) attention, much more accurately than other methods, such Knowledge Management, CIKM ’13, pages 1451–1460,
as clicks logs for example. New York, NY, USA, 2013. ACM.
Finally, we acknowledge that the results discussed in this paper [10] J. Fernquist and E. H. Chi. Perception and understanding of
are preliminary, and further testing is warranted with additional social annotations in web search. In Proceedings of the
snippet types and search tasks. We leave for future work search 22Nd International Conference on World Wide Web,
tasks where the scenario is ambiguous: will rich snippets play a WWW ’13, pages 403–412, Republic and Canton of
more important role when the user cannot provide a definitive Geneva, Switzerland, 2013. International World Wide Web
search phrase? Nevertheless, we feel that we have raised a main Conferences Steering Committee.
issue for a topic that is largely unexplored and it is of
[11] M. H. Fischer. An investigation of attention allocation
importance to research conducted in human-computer
during sequential eye movement tasks. The Quarterly
interaction (HCI), information retrieval (IR), as well as SEO
Journal of Experimental Psychology Section A: Human
practitioners.
Experimental Psychology, 52(3), 1999.
5. REFERENCES [12] S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T.
[1] M. Ageev, D. Lagun, and E. Agichtein. Towards task-based White. Evaluating implicit measures to improve web
snippet evaluation: preliminary results and challenges. In search. ACM Trans. Inf. Syst., 23(2):147–168, Apr. 2005.
C. L. A. Clarke, L. Freund, M. D. Smucker, and E. Yilmaz,
editors, Proceedings of the SIGIR 2013 Workshop on [13] L. A. Granka, T. Joachims, and G. Gay. Eye-tracking
Modeling User Behavior for Information Retrieval analysis of user behavior in www search. In Proceedings of
Evaluation (MUBE 2013), pages 1–2, 2013. the 27th Annual International ACM SIGIR Conference on
Research and Development in Information Retrieval,
[2] J. Arguello and R. Capra. The effect of aggregated search SIGIR ’04, pages 478–479, New York, NY, USA, 2004.
coherence on search behavior. In Proceedings of the 21st ACM.
ACM International Conference on Information and
Knowledge Management, CIKM ’12, pages 1293–1302, [14] G. Hotchkiss. Eye tracking on universal and personalized
New York, NY, USA, 2012. ACM. search, September 2007.

[3] J. Arguello and R. Capra. The effects of vertical rank and [15] D. Kahneman. Attention and Effort (Experimental
border on aggregated search coherence and search Psychology). Prentice Hall, 1973.
behavior. In Proceedings of the 23rd ACM International [16] Y. Liu, C. Wang, K. Zhou, J. Nie, M. Zhang, and S.
Conference on Conference on Information and Knowledge Ma.From skimming to reading: A two-stage examination
Management, CIKM ’14, pages 539–548, New York, NY, model for web search. In Proceedings of the 23rd ACM
USA, 2014. ACM. International Conference on Conference on Information
[4] J. Arguello, R. Capra, and W.-C. Wu. Factors affecting and Knowledge Management, CIKM ’14, pages 849–858,
aggregated search coherence and search behavior. In New York, NY, USA, 2014. ACM.
Proceedings of the 22nd ACM international conference on

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 399 |
 Effect of Snippets on User Experience in Web Search
M. C. Marcos | F. Gavin | I. Arapakis
[17] A. Mack and I. Rock. Inattentional blindness. The MIT Quarterly Journal of Experimental Psychology, 38(3):475–
Press, Cambridge, MA., 1998. 491, August 1986.
[18] A. Muralidharan, Z. Gyongyi, and E. Chi. Social [23] M. A. Stephens. EDF statistics for goodness of fit and some
annotations in web search. In Proceedings of the SIGCHI comparisons. Journal of the American Statistical
Conference on Human Factors in Computing Systems, CHI Association, 69(347):730–737, 1974.
’12, pages 1085–1094, New York, NY, USA, 2012. ACM. [24] S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors
[19] V. Navalpakkam, L. Jentzsch, R. Sayres, S. Ravi,A. affecting click-through behavior in aggregated search
Ahmed, and A. Smola. Measurement and modeling of eye- interfaces. In Proceedings of the 19th ACM International
mouse behavior in the presence of nonlinear page layouts. Conference on Information and Knowledge Management,
In Proceedings of the 22Nd International Conference on CIKM ’10, pages 519–528, New York, NY, USA, 2010.
World Wide Web, WWW ’13, pages 953–964, Republic ACM.
and Canton of Geneva, Switzerland, 2013. International [25] H. C. Thode. Testing for normality. CRC Press, 2002.
World Wide Web Conferences Steering Committee.
[26] C. Wang, Y. Liu, M. Zhang, S. Ma, M. Zheng, J. Qian, and
[20] K. Rayner. Eye movements and visual cognition: Scene K. Zhang. Incorporating vertical results into search click
perception and reading. In K. Rayner, editor, Springer models. In Proceedings of the 36th International ACM
series in neuropsychology, pages 46–65. Springer-Verlag, SIGIR Conference on Research and Development in
1992. Information Retrieval, SIGIR ’13, pages 503–512, New
[21] R. W. Remington. Attention and saccadic eye York, NY, USA, 2013. ACM.
movements.Journal of Experimental Psychology: Human [27] Y. Yue, R. Patel, and H. Roehrig. Beyond position bias:
Perception and Performance, 6(4):726–744, 1980. Examining result attractiveness as a source of presentation
[22] M. Shepherd, J. M. Findlay, and R. J. Hockey. The bias in clickthrough data. In Proceedings of the 19th
relationship between eye movements and spatial attention. International Conference on World Wide Web, WWW ’10,
pages 1011–1018, New York, NY, USA, 2010. ACM.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 400 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella

Evaluación de la eficiencia de uso de las versiones de

escritorio y tableta de una aplicación
Juan P. Moreno Antonio Peñalver Federico Botella
Statistics, Mathematics and Center of Operations Research Center of Operations Research
Informatics Department University Institute University Institute
Miguel Hernandez University of Elche Miguel Hernandez University of Elche Miguel Hernandez University of Elche
Elche, Spain Elche, Spain Elche, Spain
[email protected] [email protected] [email protected]

ABSTRACT cuales, tradicionalmente, se han empleado ordenadores de

Ya es un hecho que el proceso de diseño y creación de productos sobremesa. De esta forma, un usuario debe plantearse para qué
software se está orientando cada vez más a los dispositivos tipo de tareas merece la pena emplear una tableta, con las ventajas
móviles. Si se habla del entorno web, las grandes compañías que supone usar un dispositivo de este tipo (movilidad, ligereza,
emplean ya más esfuerzo de desarrollo en las versiones móviles versatilidad, duración de batería…); frente a emplear un laptop
de sus sitios webs corporativos y de venta que en sus versiones de (con sus cómodos teclados y grandes tamaños de pantalla (por
escritorio. Y lo mismo ocurre con otros productos software. Las encima de 11 pulgadas [1]), o un ordenador de sobremesa
aplicaciones móviles, que pueden usarse cómodamente en móviles tradicional, con pantallas ya por encima de 27 pulgadas [2].
y tabletas para tareas como banca electrónica, redes sociales, Parece lógico asumir que un smartphone no es lo más adecuado
reserva de instalaciones, reproducción de contenidos digitales… para realizar tareas complejas de forma eficiente. Este tipo de
etc. están desbancando progresivamente a los programas clásicos dispositivos son adecuados para tareas simples, tales como
que se solían emplear en ordenadores de sobremesa. Llegados a consultas de información, introducción de pequeñas cantidades de
este punto, cabe preguntarse qué ocurre cuando el usuario debe información, captura de imágenes, etc. Pero si consideramos las
realizar operaciones de cierta complejidad con el software, como tabletas, ¿puede un usuario llegar a ser tan eficiente como lo sería
gestionar una pequeña o mediana empresa. ¿Realmente es usable con un ordenador en la realización de tareas complejas, como
y eficiente una aplicación móvil de gestión para una empresa? En puede ser la gestión integral de una empresa? O en cambio,
este trabajo realizamos un estudio con el fin de medir la eficiencia ¿existirá algún tipo de tarea que sea difícil de realizar con una
y la usabilidad percibida de una aplicación de escritorio, tableta? Y si no es posible ser tan productivo con una tableta
enfrentando los resultados a los obtenidos en un estudio de esa como con un ordenador, ¿cuánto más productivo se es con un
misma aplicación en su versión de tableta. De esta forma, se ha ordenador que con una tableta?
obtenido un ratio de eficiencia de una aplicación de escritorio con
En este artículo se pretende probar la hipótesis de que un usuario
respecto a su respectiva versión para tableta, que puede resultar
es más eficiente realizando tareas de cierta complejidad con un
relevante para el desarrollo de futuras aplicaciones tanto de ordenador que con una tableta, así como que la usabilidad
escritorio como móviles.
percibida por éste es mayor cuando trabaja con el ordenador que
Categories and Subject Descriptors cuando lo hace con el dispositivo móvil. Para ello se ha diseñado
un experimento destinado a mediar la usabilidad de una aplicación
H.5.2 [Information Interfaces and Presentation (e.g., HCI)]:
de gestión adaptada tanto para versión escritorio como para la
User Interfaces – Evaluation/methodology, Input devices and
tableta. Dicho experimento se ha llevado a cabo con ocho usuarios
strategies (e.g., mouse, touchscreen), Screen design (e.g., text,
que han realizado tareas complejas con ambos dispositivos.
graphics, color).
Además se recogieron datos cualitativos acerca del uso de la
General Terms aplicación y la usabilidad percibida por estos usuarios. Este
Measurement, Experimentation, Human Factors estudio revela la importancia de un buen diseño adaptativo para
los distintos dispositivos y tamaños de pantalla, y cuantifica el
Keywords incremento de tiempo que conlleva realizar una tarea compleja
Mobile usability, screen size, SUS, mobile efficiency, mobile con una tableta en relación con el tiempo necesario para realizarla
effectiveness con un ordenador o laptop.
El resto del artículo se organiza de la siguiente forma: la sección 2
1. INTRODUCCIÓN presenta un resumen del estado del arte. En la sección 3 se
Hoy en día podemos encontrar en el mercado una gran variedad describe la aplicación empleada en el estudio. La sección 4
de dispositivos móviles con distintas resoluciones y tamaños de presenta una descripción detallada del experimento, así como la
pantalla. Desde los smartphones, cuyo tamaño de pantalla oscila metodología empleada. En la siguiente sección se detallan y
entre las 3 y las 6 pulgadas (incluso las 7); a las tabletas, que analizan los resultados obtenidos en el experimento y, finalmente,
presentan unos tamaños de pantalla que van desde las 7 pulgadas en la sección 6 se exponen las conclusiones obtenidas y las líneas
hasta pantallas de más de 10 pulgadas. Además, las tabletas de de trabajo futuro.
última generación, presentan cada vez interfaces más eficientes,
incluyendo teclados en pantalla cada vez más intuitivos y 2. TRABAJOS RELACIONADOS
versátiles, lo que convierte a estos dispositivos en serias Son muchos los estudios centrados en analizar la usabilidad de las
alternativas para poder realizar con ellos tareas complejas, para las tabletas en distintos ámbitos de uso: evaluar el rendimiento de

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 401 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
niños que emplean estos dispositivos en sus tareas escolares [3]; o Photo Solution Pro es una aplicación de gestión para estudios de
la usabilidad de distintos tipos de tabletas cuando son usadas por fotografía. Esta aplicación permite llevar el control global de una
alumnos universitarios con algún tipo de discapacidad [4]; o bien empresa de este tipo, permitiendo al usuario gestionar todo su
estudiar la influencia del uso de tabletas como una herramienta de flujo de trabajo: citas, presupuestos, sesiones, modelos,
ayuda para que personas mayores se familiaricen con las nuevas facturación, gastos, marketing, beneficios, tesorería etc.
tecnologías [5]. En [6] se estudia la influencia de factores La aplicación está dividida en seis grandes áreas a las que se
externos en la percepción de usabilidad de usuarios cuando éstos puede acceder desde su pantalla de inicio:
trabajan con dispositivos móviles, teniendo en cuenta aspectos del
movimiento físico, combinado con cualquiera de las necesidades • TPV: esta sección permite gestionar la caja diaria,
de navegación en el espacio físico, o la división de la atención permitiendo introducir ventas por caja, crear tickets de venta,
mientras se emplean dispositivos móviles. Todos ellos hacen imprimirlos, editarlos, eliminarlos, imprimir listados de caja,
hincapié en la importancia de un buen diseño de los menús, etc.
pantallas y áreas de navegación cuando se trabaja en pantallas • Agenda: desde esta sección se gestiona la agenda del
reducidas..En [7] se realiza un estudio que demuestra que los fotógrafo. Permite introducir citas rápidas, citas basadas en
usuarios sufren un déficit en su capacidad de recordar la trabajos en curso, editar las citas, mostrar la agenda en
información que han leído cuando lo han hecho en pantallas distintas vistas (día, semana y mes), listar citas o tareas
pequeñas en relación con lo que recuerdan cuando leen en pendientes, etc.
pantallas de grandes dimensiones. Cuando los usuarios tienen que
realizar tareas complejas que implican la escritura de texto en • Clientes: este apartado permite listar clientes, buscarlos a
dispositivos móviles, los estudios demuestran que el rendimiento través de distintos filtros, introducir nuevos clientes,
de los usuarios se reduce considerablemente. Si el usuario necesita editarlos, eliminarlos, etc.
escribir algo en una aplicación móvil, los problemas de usabilidad • Contabilidad: esta sección permite gestionar los
se acentúan, ya que se hace necesario mostrar en pantalla un presupuestos, albaranes y facturas de la empresa. Así como
teclado virtual, de manera que la cantidad de información que se generar multitud de informes de ingresos, gastos, beneficios,
puede mostrar en pantalla se hace todavía menor [8]. El teclado gráficas de distintos tipos, etc.
virtual (imprescindible para escribir en tabletas) reduce el área
visual con información para el usuario, lo que aumenta • CRM: apartado para gestionar los aspectos de la relación con
considerablemente la dificultad de la tarea de escribir pulsando los clientes, filtros de clientes según sus compras, fechas de
con los dedos (o con un dispositivo señalador) sobre una pequeña nacimiento, preferencias… Una vez obtenidos los listados de
pantalla. Adaptar la interfaz de una aplicación tanto a las clientes, se les puede enviar e-mails, newsletters, SMS, etc.
necesidades del usuario como al tamaño de la pantalla es crucial • General: gestión de productos, empleados, proveedores y
para mejorar el rendimiento y la eficacia del usuario [9]. archivo digital. Desde esta sección también se acceder a la
personalización y preferencias de la aplicación.
En relación a estudios comparativos entre dispositivos de
escritorio frente a dispositivos móviles, se han escrito trabajos La aplicación ha sido diseñada de manera que se pueda acceder a
como [10]., que proporciona una visión general de los estudios de ella desde tres tipos de dispositivos: ordenadores de escritorio o
usabilidad de aplicaciones móviles existentes, proponiendo un laptops, tabletas y dispositivos móviles. Para cada una de estas
marco genérico para la realización de pruebas de usabilidad de versiones de la aplicación se han empleado técnicas de diseño
aplicaciones móviles. En cuanto al diseño adaptativo o distintas. Es decir, se ha realizado un diseño adaptativo de la
“responsive”, los usuarios se sienten más satisfechos y, en aplicación, siguiendo las directrices de diseño adecuadas para
consecuencia, presentan un mayor rendimiento cuando usan cada versión de la aplicación en cuanto a tamaños de texto,
aplicaciones cuyos interfaces presentan un diseño adaptativo [11], tamaños de los botones, distancia entre los elementos,
en lugar de aplicaciones cuyo interfaz es rígido y normalmente resoluciones de pantalla, etc.
adaptado a versiones de sobremesa o a pantallas de mayor El diseño de cada una de las pantallas de la aplicación ha sido
tamaño. Otros trabajos analizan la satisfacción del usuario y sus adaptado a todos los dispositivos desde los cuales se puede
preferencias en relación al uso de tabletas en comparación con acceder a ella: Escritorio, iPad y iPhone. Para el interfaz, se
ordenadores portátiles en la realización de tareas comunes [12]. diseñaron cientos de presentaciones con distintos tamaños para
Por último, también existen estudios que enfocan las evaluaciones adaptarlas de la mejor manera posible a los distintos tamaños y
de usabilidad en función de quién las realiza,, basado en la resoluciones de pantalla disponibles. Los tamaños de letra,
perspectiva del usuario (que se expresa por la percepción del botones, paneles, etc. fueron diseñados expresamente para cada
usuario de la aplicación) y la perspectiva del especialista en versión de la app, así como los diferentes menús y pantallas.
interacción persona – ordenador [13].

3. DESCRIPCIÓN DE LA APLICACIÓN
En este trabajo se ha evaluado la usabilidad de una aplicación para
la gestión de estudios de fotografía desarrollada en tres versiones:
escritorio, tableta y smartphone. Se trata de una versión de prueba
de la aplicación Photo Solution Pro 7, de la cual se han empleado
ciertos módulos para el diseño de las pruebas con usuarios.

7
http://www.photosolutionpro.com

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 402 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
pantalla de 15,6’’. Del mismo modo, el estudio pretende analizar
la eficacia de los usuarios al realizar las tareas, así como la
usabilidad percibida con estos dos dispositivos de tan diferentes
características.
Para ello se desarrolló en primer lugar un plan de pruebas donde
se definieron una serie de ítems que fueron tenidos en cuenta
durante la realización de todo el experimento. Los puntos
incluidos en este plan de pruebas fueron los siguientes: objetivos y
preguntas de investigación, características de los usuarios, diseño
de las pruebas, definición de las tareas, equipo y entorno de
pruebas, métricas de evaluación, resultados y conclusiones.

Figura 1. Captura de la versión Desktop.

4.1 Objetivos
El objetivo principal de este estudio ha sido medir la caída del
rendimiento de los usuarios al realizar varias tareas de cierta
dificultad con un dispositivo de reducidas dimensiones como un
iPad, en comparación de la realización de esas mismas tareas con
un dispositivo de mayores dimensiones, como es un ordenador de
escritorio; todo ello con la misma aplicación en ambos
dispositivos. Para ello se definieron previamente las siguientes
cuestiones a investigar:
a) El rendimiento en la ejecución de una serie de tareas en un
ordenador de sobremesa es mayor que en la ejecución de esas
mismas tareas en una tableta.
b) La usabilidad percibida por el usuario será distinta
dependiendo de con qué dispositivo realice la tareas,
ordenador o tableta.
c) El número de problemas de usabilidad detectados por el
usuario es menor cuando el usuario realiza las tareas en un
ordenador de sobremesa que cuando las realiza en una
tableta.
Para dar respuesta científica a estas cuestiones se realizaron
pruebas con usuarios reales, con el objetivo de experimentar la
influencia de la interacción de un mismo usuario con dos
versiones de una misma aplicación diseñada para distintos
dispositivos (en este caso Escritorio e iPad).

4.2 Participantes
Para este experimento se han seleccionado ocho usuarios que se
ofrecieron voluntariamente para realizar el estudio de usabilidad
de la aplicación. Antes de la realización de las pruebas se le
facilitó a los usuarios un cuestionario pre-test [14], que debía ser
respondido en unos dos minutos aproximadamente. Estas
Figura 2. Captura de la versión iPad con los objetos cuestiones tenían una doble utilidad: 1) Hacer que los usuarios se
adaptados al tamaño de la pantalla. sientan más cómodos rompiendo el hielo antes de comenzar las
pruebas y 2) Conseguir la información necesaria para
4. METODOLOGÍA clasificarlos. Así se obtuvieron los siguientes datos demográficos
En el presente estudio se ha realizado una evaluación de y de experiencia en el uso de dispositivos móviles.
usabilidad de una aplicación de gestión diseñada para dispositivos
de escritorio y para tabletas, con diferentes diseños para cada uno, Se realizaron las pruebas a cinco hombres y tres mujeres. Las
mediante sesiones guiadas con usuarios reales que fueron edades de los usuarios reclutados para las pruebas de la aplicación
registradas a través de distintas cámaras y analizadas con están comprendidas entre los 24 y los 35 años (Media = 31,50; SD
posterioridad. El objetivo de este experimento fue comprobar = 4,38). Cada uno de los usuarios es propietario de un ordenador
cuánto tiempo lleva a un usuario realizar ciertas tareas complejas y de una tablet (aunque no todos ellos tienen un iPad como los
en un ordenador de escritorio, para después comprobar cuánto que se han utilizado en nuestro laboratorio). En general, todos los
tiempo le lleva realizar esas mismas tareas en una tableta, es decir usuarios hacen un uso intensivo de sus dispositivos móviles, ya
comprobar la eficiencia de los usuarios al realizar estas tareas en sean smartphones o tabletas, durante la semana (Media = 20,38
un dispositivo móvil con pantalla de 7,9’’ en comparación con horas / semana; SD = 14,60); sin embargo, no todos los usuarios
realizarlas en un ordenador de sobremesa con teclado y una tienen demasiada experiencia en el uso de aplicaciones móviles en
sus dispositivos móviles, dejando al margen las habituales
XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 403 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
aplicaciones de mensajería instantánea, redes sociales, mini- grabada en vídeo con el fin analizar las imágenes para detectar
juegos, etc. En cuanto al trabajo con ordenadores de escritorio, en problemas de usabilidad de la aplicación. Durante los siguientes 2
general los usuarios también realizan un uso intensivo, con una minutos, se le realizó al usuario el cuestionario con las preguntas
media semanal de 18,13 horas (SD = 18,46). Uno de los usuarios pre-test. A continuación, se emplearon los siguientes 3 minutos en
declara usar programas de gestión en ordenador de sobremesa pedir al usuario que navegara por los distintos menús de la
muy frecuentemente, cuatro usuarios declaran hacerlo de manera aplicación y explicara en voz alta qué sensaciones le transmitía la
frecuente, mientras que los otros tres usuarios dicen no usar aplicación y que tratara de definir cuál pensaba que era el
programas de gestión en ordenadores. Sin embargo, tres de los propósito final de la aplicación. Ninguno de los usuarios
ocho usuarios declaran emplear otro tipo de software en sus seleccionado conocía esta aplicación antes de comenzar con las
ordenadores de manera muy frecuente, otros tres usuarios de pruebas con la tableta, por lo que era importante una primera fase
manera frecuente, un usuario afirma hacerlo de vez en cuando, de familiarización con la app y recoger sus impresiones. Tras la
mientras que sólo un usuario dice usar el ordenador de manera etapa de familiarización, los usuarios comenzaron con la
nada frecuente. realización de las tareas, se les dejó trabajar libremente, sin
proporcionar a los participantes ningún tipo de feedback acerca
Ninguno de los usuarios había trabajado previamente con la del éxito o fracaso de sus progresos. Sólo en el caso de que un
aplicación puesta a prueba en este estudio. Como todos los participante tuvo problemas graves en una tarea, el moderador la
usuarios se encuentran en el mismo rango de edad, se puede detuvo e invitó al usuario a continuar con la siguiente tarea. Tras
afirmar que el efecto de la usabilidad percibida en función de la la realización de las tareas, se llevó a cabo una fase de indagación
edad de los participantes ha sido mínima [15]. de 5 minutos aproximadamente en la que cada usuario
4.3 Procedimiento cumplimentó un cuestionario del tipo SUS (System Usability
Scale [19]), con el fin de que quedara registrada la usabilidad
Las pruebas fueron realizadas por ocho usuarios, que realizaron
general de la aplicación que cada participante había percibido.
las tareas en ambos dispositivos. Ocho usuarios es un número
aceptable de participantes para obtener resultados válidos en 4.3.1 Equipo físico para las pruebas
estudios de usabilidad [16]. Cada uno de los 8 participantes En nuestro laboratorio empleamos el siguiente equipo: un iPad
realizó en primer lugar las tareas en orden secuencial en una Mini de 64 Gb. Pantalla Retina, Multi-Touch retroiluminada por
tableta iPad. Los resultados de las pruebas con la tableta fueron LED de 7,9'' (en diagonal). Un ordenador Acer Aspire 5741G, con
publicados en nuestro trabajo previo “How efficient can be a user pantalla de 15,6" HD (1366x768) con tecnología LED. Para
with a tablet versus a smartphone?” [17]. Unos meses más tarde, grabar las pruebas se empleó una camára USB IPEVO Ziggi HD
los mismos ocho participantes realizaron las mismas tareas en el Document Cámara, con 5 Megapixel de resolución con micrófono
mismo orden secuencial con un ordenador de sobremesa. De esta incorporado. Esta cámara grabó los movimientos de las manos y
forma, el tiempo transcurrido entre las pruebas con la tableta y las la voz del usuario mientras realizaba las pruebas con la tableta.
pruebas con el ordenador hace que el efecto aprendizaje no sea Para capturar la cara y movimientos gestuales de los usuarios se
tenido en cuenta en los resultados. empleo una cámara Canon EOS 5D Mark II. 21.1 MegaPixel, con
Se evaluaron las dos versiones de la aplicación (versión iPad y la que se grabaron las pruebas en Full HD (1080, 30 fps). Para el
versión escritorio) en un diseño de pruebas basado en el modelo almacenamiento y edición de las grabaciones se empleó un iMac
‘Within-Subject Design’, definido por Jeffrey Rubin y Dana 2,93 GHz Intel Core Duo. 8 GB RAM DDR 3 1067 MHz..
Chisnell [18]. El modelo ‘Within-Subject Design’ es un sistema
4.3.2 Tareas realizadas
de evaluación basado en que un mismo grupo reducido de
A continuación se describen las cinco tareas secuenciales que los
usuarios realice todas las tareas a evaluar.
participantes realizaron en ambos dispositivos:
Se instaló la aplicación Filemaker GO 13 en un iPad Mini y en un • Tarea 1. Dar de alta un nuevo cliente. El usuario debe
ordenador Acer Aspire para poder conectar de esta forma con el introducir un nuevo cliente en la aplicación, completando los
servidor de Photo Solution Pro y poder ejecutar la aplicación a siguientes datos: nombre, apellidos, empresa, cargo, CIF,
probar sobre los dos aparatos. Cuando los usuarios llegaron al fecha de nacimiento, teléfono móvil, teléfono fijo, e-mail,
laboratorio se encontraron con ambos dispositivos preparados con web, dirección, C.P., ciudad, provincia, país y aceptación de
las dos versiones de Photo Solution Pro en perfectas condiciones publicidad. El tiempo estimado de esta tarea es de 180
para comenzar con las pruebas. segundos para iPad y 150 para Escritorio.
Para la duración de las pruebas, se acordó previamente que la • Tarea 2. Dar de alta un producto. El usuario debe introducir
duración total estimada de todas las tareas a realizar por un mismo un nuevo producto en la aplicación. Los datos del producto
participante, incluidas las tareas con los dispositivos y la son: artículo, fabricante, número de referencia, categoría,
realización de todos los cuestionarios, debería ser de 30 minutos ubicación, peso, foto (que el usuario debe tomar con la
como máximo con cada uno de los dispositivos. cámara de la tableta, o bien insertar desde el disco duro del
Todos los usuarios realizaron las mismas pruebas en el mismo ordenador), precio coste, margen, precio venta sin IVA,
laboratorio y bajo las mismas condiciones. Antes de que los precio venta con IVA, existencias y stock mínimo. Esta tarea
usuarios comenzaran a realizar sus tareas, se preparó una etapa de tiene un tiempo estimado de 160 segundos para iPad y 140
bienvenida de unos 5 minutos aproximadamente. En esta etapa, el segundos para Escritorio.
moderador leía un guion escrito en el cual daba la bienvenida al • Tarea 3. Dar una cita a un cliente. El usuario debe asignar
usuario y le explicaba con exactitud al usuario el motivo y una cita al cliente dado de alta en la tarea 1. Los datos de la
objetivos de las pruebas. Además, durante esta fase se invitó a cita son los siguientes: fecha, hora desde, hora hasta, lugar,
cada participante a firmar un formulario de consentimiento de fase de trabajo, tipo de trabajo, cliente y notas. La duración
grabación, donde se le informaba de que su prueba iba a ser

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 404 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
estimada de esta tarea es de 140 segundos para iPad y 120 finalizaron todas sus tareas. Sólo un participante, trabajando con
segundos para Escritorio. la tableta, no consiguió terminar una de las tareas. En este caso el
• Tarea 4. Rellenar el presupuesto de la sesión de fotos. En moderador simplemente invitó al participante a continuar con la
esta tarea el usuario debe confeccionar un presupuesto para siguiente tarea.
una sesión de fotos añadiendo un par productos al 4.4.2 Eficiencia en cada dispositivo
presupuesto una entrega a cuenta por parte del cliente. Los Mediante el análisis de las grabaciones de las pruebas, se calculó
datos a cumplimentar son: artículo 1, artículo 2, fecha de el tiempo exacto en segundos que cada participante necesitó para
entrega a cuenta, importe y forma de pago. El tiempo la realización de cada una de las cinco tareas, primero con la
estimado para esta tarea es de 200 segundos para iPad y 80 tableta y después con el ordenador. Todas las tareas se
para Escritorio. completaron con éxito, excepto una de ellas. Para esta tarea, se
• Tarea 5. Enviar un newsletter a un cliente. En la última tarea, hizo necesario estimar el tiempo de la tarea que el usuario no
el usuario debe enviar un newsletter publicitario a un cliente. completó adecuadamente, con el fin de calcular la eficiencia de
Los datos del newsletter son cliente, foto (que el usuario forma precisa.
debe tomar con la cámara del dispositivo móvil), titular y
cuerpo. Esta tarea tiene una duración estimada de 240 4.4.3 Usabilidad percibida de la aplicación
segundos para iPad y 150 segundos para Escritorio. Para calcular la usabilidad percibida por cada uno de los
participantes, se empleo un cuestionario SUS (System Usability
En la Figura 3 se puede observar a un usuario introduciendo un Scale). Tras completar todas las tareas con ambos dispositivos, se
nuevo producto en la aplicación usando el iPad, mientras se le pasó a cada participante dicho cuestionario. El moderador
graban sus manos realizando la tarea con una cámara situada realizó las preguntas del cuestionario a cada participante y les
sobre ellas. Al mismo tiempo se captura y se graba la pantalla indicó que respondieran lo más rápidamente posible a las
mediante el software Reflector en un iMac para registrar con más preguntas, sin pensar demasiado, ya que está es considerada la
detalle todos los movimientos y pulsaciones del usuario. forma idónea de realizar el cuestionario. Los participantes
puntuaron los 10 items del cuestionario en una escala del 1
(Totalmente en desacuerdo) al 5 (Totalmente de acuerdo). El
motivo de seleccionar el cuestionario SUS como herramienta para
evaluar la usabilidad percibida fue la gran aceptación de este
cuestionario por la comunidad científica que trabaja en temas de
usabilidad, y por ser un cuestionario libre, fácil de cumplimentar y
de evaluar.

5. RESULTADOS

Figura 3. Un usuario realizando la prueba en nuestro 5.1. Efectividad en tareas

laboratorio, en el momento de introducir un nuevo cliente en Todos los participantes en el experimento realizaron las cinco
la aplicación. tareas propuestas tanto en una tableta iPad Mini como en un
ordenador de sobremesa Acer Aspire. Entre las pruebas con el
4.4 Medidas iPad y las pruebas con la versión de escritorio pasaron meses de
diferencia. Ningún usuario había empleado la aplicación antes de
4.4.1 Efectividad en cada dispositivo las pruebas, ni volvió a usarla en el periodo que transcurrió entre
Antes de que las realizaran los usuarios, todas las tareas fueron ambas pruebas, así que el efecto de aprendizaje de la aplicación,
realizadas por el equipo de desarrollo de la aplicación en el que pudiera haber existido entre pruebas, queda descartado al ser
laboratorio de pruebas, con el fin de estimar el tiempo requerido las pruebas distantes en el tiempo. El interfaz de cada una de las
para realizar cada una de las cinco tareas en ambos dispositivos: versiones de la aplicación (iPad y Escritorio) no fue el mismo;
Tabla 1. Tiempos estimados para completar cada tarea (en dependiendo del dispositivo y del tamaño de su pantalla, cada
segundos) usando la tableta (columna t) y el ordenador versión dispone de distintas presentaciones, tamaños de botones y
(columna d). textos, menús, etc.
En la Tabla 2, podemos ver los resultados de la ejecución de todas
las tareas por parte de los ocho participantes. En ella puede
observarse que todos los participantes completaron el 100% de
cada una de las tareas, excepto el Participante 5, quien completó
la Tarea 2 al 45% realizándola con el ordenador de sobremesa.
Todas las actividades de los participantes fueron grabadas en
Todos los tiempos mostrados en la Tabla 2 son tiempos reales
vídeo mientras éstos completaban las tareas en los dos
requeridos por los usuarios para completar sus tareas. El tiempo
dispositivos. De esta manera se pudo comprobar, a posteriori, si el
indicado del Participante 5 corresponde a la anteriormente
usuario había finalizado completamente cada una de las tareas y
mencionada tarea que quedó pendiente de completar, así que fue
calcular el tiempo empleado. Además, la grabación permite
necesario calcular el tiempo estimado que el usuario hubiera
obtener datos cualitativos acerca de las impresiones de los
necesitado para completar la tarea inconclusa, para de esta forma
usuarios mientras interactuaban con la aplicación. El moderador
poder comparar con el resto de tiempos y completar el estudio.
no proporcionó a los participantes ningún feedback acerca del
éxito o del fracaso del usuario en la realización de las tareas. Se
permitió a todos los participantes trabajar libremente hasta que

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 405 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
5.1.1 Medidas de rendimiento de las tareas En la Tabla 3, se puede observar la diferencia entre el tiempo
empleado por cada usuario para la ejecución de cada tarea en
comparación con los tiempos estimados mostrados en la Tabla 1.

Tabla 2. Tiempo (segundos) y porcentaje de finalización de tareas para cada usuario en diferentes dispositivos (t=tablet;
d=desktop).

Se puede comprobar que, trabajando con la tableta, los usuarios versión de escritorio, pero fue también una de las que menos
necesitaron menos tiempo del estimado en 24 de las 40 tareas. tiempo les llevó realizar con la tableta, este hecho sugiere que el
Mientras que trabajando con el ordenador, fueron 19 de las 40 diseño de la pantalla de introducción de los datos del producto
tareas, en las que los usuarios terminaron antes del tiempo debe ser reconsiderado en la versión de escritorio. La Tarea 4,
estimado. Cabe destacar que el Participante 3 necesitó un tiempo una tarea compleja que precisa una búsqueda previa de
significativamente mayor para realizar sus tareas, sobre todo con información que el usuario ha generado en las Tareas 2 y 3
la tableta. El motivo de esta mayor diferencia entre el tiempo (debe crear un presupuesto asociado a una cita dada de alta en la
estimado y el real fue que el Participante 3 no tenía demasiada Tarea 3, con algunos productos, uno de los cuales lo debe haber
experiencia en el uso de este tipo de dispositivos, unido al hecho creado el usuario en la Tarea 2), resultó ser una tarea sencilla
de que el entorno de pruebas y la grabación realizada, le ponían para los participante con el iPad, ya que en general terminaron
nervioso. Esto contrasta, por ejemplo, con el Participante 8, que en menos tiempo de lo esperado. De nuevo los resultados fueron
fue uno de los usuarios que menos tiempo necesitó para realizar algo peores para esta tarea con la versión escritorio, debido
todas las tareas tanto con el iPad como con el ordenador. principalmente a un diseño y colocación de botones mejorable
en esta versión de la aplicación. Por último, para la Tarea 5, al
Tabla 3. Diferencia de tiempos (en segundos) entre el tiempo igual que la Tarea 3, se obtuvieron unos tiempos cercanos a los
real necesario y el estimado para cada tarea. estimados.

5.2. Eficiencia en la ejecución de tareas

Todos los participantes llevaron a cabo las cinco tareas del
experimento tanto en tableta como en ordenador de sobremesa.
Mediante el análisis de las grabaciones se calculó la diferencia
de tiempo necesitado por el usuario para realizar cada tarea en el
iPad con respecto al tiempo que había necesitado para realizar la
misma tarea con el ordenador. En la Tabla 4 se puede observar
el porcentaje de tiempo que cada participante necesitó para
completar todas las tareas en la tableta con respecto al
necesitado para completarlas con ayuda del ordenador (los datos
de la Tabla aparecen en %).
Para la Tarea 1, los ocho participantes se ajustaron bastante al Tabla 4. Porcentaje de tiempo que cada participante necesitó
tiempo estimado trabajando con la tableta. Sin embargo, con el para completar todas las tareas en la tableta con respecto al
ordenador se obtuvieron tiempos en general por encima de lo ordenador
estimado. Esto es así porque estimamos que el hecho de escribir
los datos del cliente en el teclado del ordenador reduciría
considerablemente el tiempo necesario para realizar la tarea, sin
embargo las pruebas demostraron que esta diferencia no fue tan
acentuada como esperábamos. El Participante 5 no logró
terminar la Tarea 2 con éxito trabajando con el ordenador,
debido a que no consiguió agregar la imagen a la ficha del
producto y no pudo finalizar. No fue el único usuario que tuvo
problemas con el proceso de inserción de fotografía para el
producto, lo que indica que debe ser mejorado este aspecto de la
aplicación. Un aspecto interesante de la Tarea 2 es que resultó
ser una de las más complejas de realizar para los usuarios en su

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 406 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
Se puede apreciar que, en general, un usuario necesitará de
media alrededor de un 17,50% más de tiempo para realizar una
tarea en la tableta de lo que necesitaría para realizarla con un
ordenador de sobremesa. Los resultados obtenidos parecen
indicar que la diferencia de tiempo necesario por los usuarios
para realizar una tarea con el iPad con respecto al necesitado
para realizarla en un ordenador de sobremesa tiende a
incrementarse conforme van transcurriendo las tareas, siendo
similar en las primeras tareas y más pronunciada en posteriores
tareas. Esto parece indicar que cuando el usuario no conoce una
aplicación su rendimiento es semejante en ambos dispositivos,
pero conforme va conociendo la aplicación con la que trabaja, se
va volviendo más productivo con el ordenador que con el Como indica la tabla, la media de problemas encontrados por los
dispositivo móvil. Se puede observar también que cuanto más usuarios con la tableta fue de 6,25 (SD = 3,92), mientras que los
compleja es la tarea, más se acentúa la diferencia de participantes encontraron una media de 5,50 (SD = 2,56)
productividad, como se aprecia en la Tarea 4. Esta tarea es la problemas de usabilidad para la versión escritorio de la
más compleja de las realizadas por los participantes, ya que es aplicación. Es interesante comprobar que la media de errores
necesaria la búsqueda de información de citas y de productos encontrados en ambos dispositivos es semejante, cuando la
(información que proviene de otras tareas); y es en esta Tarea 4 lógica parecía indicar que la media de errores encontrados iba a
donde el incremento del tiempo necesario para realizarla con ser mucho mayor con la tableta, al ser un dispositivo menos
tableta con respecto al necesario para realizarla con el ordenador utilizado tradicionalmente para tareas de gestión.
es más alto, un 49,13%. Otro factor importante que queda Los problemas de usabilidad detectados fueron de diversa
demostrado es que el hecho de tener que escribir texto en la índole. A modo de resumen, la Tabla 6 muestra los errores que
tarea influye notablemente, algo que la lógica parecía indicar a se repitieron más una vez en el transcurso de las pruebas:
priori dada la naturaleza de los teclados de ambos dispositivos.
Si nos fijamos en la Tarea 3, volvemos a tener tiempos Tabla 6. Algunos problemas de usabilidad detectados.
parecidos, ya que se trata de una tarea en la que apenas se ha de
escribir, se actúa sobre ella con botones, listas desplegables,
ruletas de fecha, etc. Sin embargo, la Tarea 5 es de las que más
texto requiere ser escrito, ya que se trata de enviar un texto
publicitario, por lo que el tiempo necesario para completar la
tarea en el iPad con respecto al ordenador es de nuevo elevado,
un 33,49%. Por último, cabe destacar los resultados del
Participante 8, que obtuvo mejores tiempos con el iPad que con
el ordenador de sobremesa. El motivo de este resultado es que
este participante es usuario habitual de dispositivos Apple, tanto
de iMac como de iPad y iPhone, siendo usuario experto en el
manejo de los dispositivos móviles de este fabricante. Esto,
unido a su falta de costumbre a trabajar con ordenadores que
funcionen bajo el sistema operativo Windows, fue lo que
explicaría tiempos mejores con el iPad que con el ordenador. Cabe destacar en este apartado que los tres primeros errores en
cuanto al número de veces que se repiten, vienen derivados de
5.3 Problemas de usabilidad detectados las herramientas que ofrece la plataforma Filemaker (el motor
Todos los participantes en el experimento fueron grabados sobre el cual está montada la aplicación). Los calendarios
mientras realizaban sus tareas con ambos dispositivos. Todos los desplegables para introducir fechas, el interfaz de introducción
problemas de usabilidad detectados por los usuarios (directa o de imágenes en el sistema, así como los botones de introducción
indirectamente) fueron anotados mientras se examinaban las de registros que ofrece dicha plataforma parecen confundir al
grabaciones después de que los usuarios completaran sus usuario en lugar de ser elementos de ayuda. Por este motivo, este
pruebas. En la Tabla 5, se pueden apreciar el número de equipo recomienda evitar dichos elementos en aplicaciones tanto
problemas de usabilidad detectados por los participantes en móviles como de escritorio, si se trabaja con la plataforma
ambas versiones de la aplicación. Filemaker como base para su desarrollo. En esta ocasión
estamos hablando de problemas con la versión escritorio, pero
Tabla 5. Problemas de usabilidad detectados por los estos problemas también se dieron en nuestro anterior trabajo, en
usuarios en función del dispositivo. el que analizábamos las versiones de tableta y smartphone de la
aplicación, por ejemplo con las ruletas de introducción de fechas
y con las ruletas de introducción de horas.
5.4 Usabilidad percibida
Tras realizar todas las pruebas en ambos dispositivos, cada uno
de los usuarios contestó a un cuestionario SUS, cuya finalidad
fue definir la usabilidad percibida de la aplicación en su versión
de Escritorio. Este formulario arrojó una usabilidad percibida de
89,38 puntos (medidos sobre 100) con una SD = 3,47. Los

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 407 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
resultados del cuestionario SUS para cada uno de los usuario, por lo que se recomienda evitar su uso, sustituyéndolos
participantes pueden verse en la Tabla 6. Es interesante destacar por elementos de programación propia. En cuanto a la usabilidad
que en nuestro anterior estudio, en el que comparábamos las percibida por los usuarios, el cuestionario SUS ha arrojado unos
versiones de tableta y smartphone de la aplicación [17], el resultados de 89,38 puntos sobre 100, siendo esta puntuación
resultado obtenido en el cuestionario SUS fue de una usabilidad 5,94 puntos mayor de la obtenida en un anterior estudio en el
percibida de 83,44 puntos son una SD de 11,18 puntos. Se puede que solamente evaluábamos las versiones móviles de la
observar, que la usabilidad percibida por los usuarios al trabajar aplicación, de lo que se deduce que el uso de un dispositivo
con la versión de escritorio de la aplicación ha aumentado en como un ordenador de sobremesa incrementa la usabilidad
5,94 puntos con respecto a la usabilidad percibida cuando sólo percibida por el usuario.
trabajaron con la versión móvil de la aplicación.
Como trabajos futuros, se plantea la posibilidad de analizar las
Tabla 7. Resultados del cuestionario SUS. mismas preguntas de investigación de este estudio con un mayor
número de usuarios y tareas, con el fin de confirmar la robustez
de los resultados obtenidos en el presente estudio. También sería
interesante realizar las pruebas de usabilidad con usuarios
representativos del sector al que va dirigido la aplicación, con la
finalidad de comprobar si profesionales del sector de la
fotografía obtendrían mejores resultados.

7. AGRADECIMIENTOS
Nos gustaría dar las gracias a todos los participantes del
experimento, así como a la empresa Imagic-Group Creativos S.L
por cedernos su equipo e instalaciones.

6. CONCLUSIONES Y TRABAJO 8. REFERENCIAS

FUTURO [1] Bangor, A., Kortum, P. T., Miller, J. T. 2008. An empirical
En este trabajo se ha podido comprobar que el uso de evaluation of the system usability scale. Intl. Journal of
ordenadores de sobremesa puede mejorar la eficacia de los Human–Computer Interaction, 24, 6, 574-594.
usuarios cuando tienen que realizar tareas de cierta complejidad, [2] Bevan, N. 2009. International standards for usability should
si comparamos con la eficacia que se pueden obtener al realizar be more widely used. Journal of Usability Studies, 4,3, 106-
estas mismas tareas en un dispositivo de menores dimensiones y 113
con teclado en pantalla, como una tableta; necesitando de media
[3] Wyeth, P., McEwan, M., Roe, P., MacColl, I. 2011.
alrededor de un 17% más de tiempo para realizar una tarea en la
Expressive interactions: tablet usability for young mobile
tableta que en el ordenador.
learners. Proceedings of the 23rd Australian Computer-
En las pruebas que se presentan en este experimento, todos los Human Interaction Conference.
usuarios fueron capaces de realizar todas las tareas con éxito con [4] Millán, A., Antonio Pérez, A. 2014. Usabilidad de tablets
la tableta, y sólo un usuario no fue capaz de completar una de las para el acceso a campus virtuales universitarios por
tareas al 100% en la versión escritorio de la aplicación). Así que, alumnos con discapacidad.
de forma general, se podría decir que tareas complejas pueden
ser realizadas en un dispositivo de las características de una [5] Barnarda, Y., Bradleyb, M., Hodgsona, F., Lloydc, A.
tableta, aunque las mismas tareas realizadas en un ordenador de 2013. Learning to use new technologies by older adults:
sobremesa, con pantalla de mayores dimensiones y teclado Perceived difficulties, experimentation behaviour and
tradicional, serán llevadas a cabo de una manera más eficiente. usability.
[6] Kjeldskov, J., Stage, J. 2003.New techniques for usability
Queda demostrado con este estudio que, normalmente, a un
evaluation of mobile systems.
usuario le llevará más tiempo realizar una tarea en una tableta
del tiempo que le llevará realizar esa misma tarea en una [7] Sanchez, C. A., Branaghan, R. J. 2011. Turning to learn:
ordenador de sobremesa. Este tiempo se puede incrementar Screen orientation and reasoning with small devices.
considerablemente si el usuario ha de escribir algún tipo de texto Computers in Human Behavior, 27, 2, 793-797.
o tiene que buscar algún tipo de información en el dispositivo. [8] Dell’Amico, M., Díaz, J. C. D., Iori, M., & Montanari, R.
Por otro lado el número de problemas de usabilidad detectados 2009. The single-finger keyboard layout problem.
es ligeramente mayor cuando los usuarios trabajan con la tableta Computers & Operations Research, 36,11, 3002–3012.
con respecto a los problemas detectados cuando lo hacen con el [9] Schmitz, B., Zwick, C. 2006. Designing for small screens.
ordenador de sobremesa, detectándose una media de 0,75 AVA Publishing.
problemas más (ver Tabla 5). Siendo esta diferencia menor de la [10] Zhang, D., Adipat, B. 2005. Challenges, methodologies,
esperada a priori por el equipo de investigación. Además se ha and issues in the usability testing of mobile applications.
podido constatar que algunos de los elementos de interfaz
nativos de la plataforma Filemaker, sobre la cual está montada la [11] Findlater, L., McGrenere, J. 2008. Impact of screen size on
aplicación, son una fuente de problemas en lugar de una ayuda. performance, awareness, and user satisfaction with adaptive
Elementos como los calendarios desplegables, los botones de graphical user interfaces. In Proceedings of the SIGCHI
introducción de registros, o el sistema de introducción de Conference on Human Factors in Computing Systems CHI
imágenes en campos contenedores tienden a confundir al ’08, 1247-1256.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 408 |
 Evaluación de la eficiencia de uso de las versiones de escritorio y tableta de una aplicación
J. P. Moreno | A. Peñalver | F. Botella
[12] Ozoka, A., Bensonb, D., Chakrabortyb, J., Norciob, A. [16] Nielsen, J. 2000. Why you only need to test with 5 users.
2008. A Comparative Study Between Tablet and Laptop Retrieved December 14, 2009 from
PCs: User Satisfaction and Preferences. http://www.useit.com/alertbox/20000319.html
[13] Rangel de Queiroz, E., De Sousa, D. 2009. A [17] F. Botella, Juan P. Moreno, A. Peñalver. 2014. How
multidimensional approach for the evaluation of mobile efficient can be a user with a tablet versus a smartphone?
application user interfaces. XV Congreso Internacional "Interacción 2014".
[14] Krug, Steve. 2010. Rocket surgery made easy. The do-it- [18] Rubin, J., Chisnell D.2008.Handbook of Usability Testing.
yourself guide to finding and fixing usability problems. [19] Brooke, J. 1996. SUS: A ‘quick and dirty’ usability scale.
[15] Bangor, A., Kortum, P. and Miller, J. 2008. An empirical Usability evaluation in Industry (pp. 189 – 194).
evaluation of the system usability scale. International
Journal of Human-Computer Interaction.
[20]

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 409 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos

Lenguaje Visual de Consulta sobre Grafos de Datos: Un

enfoque desde el Diseño Centrado en el Usuario

María Constanza Pabón César A. Collazos

Ph.D. student at Universidad del Valle - Universidad del Cauca
Lecturer at Pontificia Universidad Javeriana-Cali Popayán, Colombia
Cali, Colombia [email protected]
[email protected]

RESUMEN herramientas que les facilite explorar esos datos y proponer

Ofrecer a los usuarios finales, expertos en un dominio, la consultas ad hoc (no predefinidas, no conocidas con anticipación).
posibilidad de explorar y consultar datos almacenados en los Adicionalmente, la información muchas veces está dispersa en
sistemas de información con los que trabajan puede generar diversas fuentes de datos que tienen representaciones y tipos de
grandes beneficios para las organizaciones y la sociedad. Con datos heterogéneos [73]. Un ejemplo en el dominio médico es la
frecuencia, formular preguntas ad hoc usando un lenguaje de información clínica de los pacientes. Ofrecer a estos usuarios, la
consulta resulta una tarea compleja para un usuario final. Aun, en posibilidad de explorar y consultar datos almacenados en los
aquellos casos en los que se usan lenguajes visuales e interfaces sistemas de información con los que trabajan podría generar
gráficas. Generalmente las interfaces gráficas de consulta usan beneficios para las organizaciones y la sociedad.
conceptos y elementos de los lenguajes que son poco intuitivos Nuestro propósito es ofrecer a los usuarios finales un mecanismo
para el usuario final. En este artículo se presenta, desde la que les facilite el acceso a los datos. Para ello se desarrolla
perspectiva del Diseño Centrado en el Usuario (DCU), el proceso GraphTQL, un lenguaje visual de consulta que le permite a los
de diseño de GraphTQL, un lenguaje visual de consulta sobre usuarios finales formular consultas ad hoc. El caso de aplicación
grafos de datos. El objetivo de GraphTQL es facilitarles a los particular es el dominio médico. Con este propósito, se sigue un
usuarios finales la formulación de consultas de mediana proceso de diseño centrado en el usuario (DCU) [75] en el que se
complejidad. El caso de aplicación de este trabajo es el dominio ha involucrado un conjunto de profesionales de la salud, con el fin
médico, en particular las consultas sobre datos clínicos. Se de que nuestra propuesta se adapte a sus necesidades y a las tareas
presentan los resultados de las entrevistas y pruebas de usabilidad que ellos realizan.
con profesionales de la salud que, desde las primeras etapas del
proyecto, se llevaron a cabo como parte de este enfoque de Este artículo describe cómo se ha llevado a cabo el proceso de
diseño. Y se describe cómo estos resultados, y el enfoque de diseño centrado en el usuario (DCU) [75] y cómo este proceso ha
diseño, influenciaron y enriquecieron las decisiones de diseño de enriquecido el diseño del lenguaje, siendo un factor determinante
GraphTQL. en las decisiones que se han tomado.
El resto del artículo está organizado en cuatro secciones. En la
Categories and Subject Descriptors Sección 2 se describe las principales decisiones de diseño del
H.5.2 [Information Interfaces and Presentation]: User
lenguaje visual y el lenguaje propuesto. La Sección 3 presenta los
Interfaces; H.2.3 [Database management]: Languages.
experimentos realizados como parte del proceso de diseño
General Terms centrado en el usuario y cómo han impactado el diseño del
Human Factors, Languages. lenguaje de consulta. La Sección 4 incluye los trabajos
relacionados con sistemas visuales de consulta sobre grafos.
Palabras Clave Finalmente, en la Sección 5 se presentan algunas conclusiones y
Lenguajes visuales de consulta, Lenguajes de consulta sobre se proponen líneas de trabajo futuro.
grafos, Diseño centrado en el usuario.
2. GRAPHTQL
1. INTRODUCCIÓN En esta sección se presentan algunas decisiones de diseño del
Las organizaciones recopilan grandes cantidades de datos en sus lenguaje y una descripción breve del mismo.
sistemas de información. Sin embargo, generalmente los usuarios
expertos en el dominio de las aplicaciones no cuentan con

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 410 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos

Figura 1. Interfaz de GraphTQL

posibilita definir operaciones de consulta que se aplican de

2.1 El modelo de datos manera uniforme sobre objetos y atributos, ya que ambos se
El modelo de datos que subyace a GraphTQL se caracteriza por: representan en el grafo de manera uniforme, por medio de nodos.
a) Permitir representar la organización de los datos en el nivel Con base en estas características se escogió GDM [77] como
conceptual. b) Tener una representación gráfica simple. c) Hacer modelo subyacente al lenguaje visual de consulta.
evidente la relación entre los datos para evitar, por ejemplo, el uso
explícito de conceptos como el join, necesario en las consultas En GDM [77] los nodos del grafo esquema (Figura 1-(a))
sobre el modelo relacional. d) Facilitar el mapeo con otros representan clases y los nodos del grafo instancia (Figura 2), las
modelos de datos de manera que a través de la vista de nivel instancias de esas clases. Los arcos representan atributos en
conceptual se pueda acceder a los datos de diversas fuentes de ambos, esquema e instancia. GDM ofrece tres tipos de nodos,
datos. nodos clase objeto (los rectángulos grandes), nodos de valor
básico (los óvalos) y nodos de valor compuesto. Estos últimos
Teniendo en cuenta que los modelos de datos basados en grafos permiten representar relaciones n-arias (los rectángulos pequeños)
tienen las características mencionadas, permiten modelar de o agrupar atributos (los círculos pequeños). Las Figura 1-(a) y 2
manera natural los datos [66], y que representarlos en un diagrama muestran un ejemplo de esquema e instancia de una base de datos
facilita su entendimiento por parte de los usuarios, GraphTQL está con información clínica de pacientes. Estos incluyen los datos
basado en grafos. demográficos del paciente y los datos clínicos como descripción
de la enfermedad actual, diagnósticos, ordenes médicas, y
El modelo que subyace a GraphTQL distingue entre el grafo
resultados de exámenes diagnósticos, entre otros.
esquema y el grafo instancia. El grafo esquema se usa para
proveer una representación de los datos en el nivel conceptual, lo 2.2 Descripción del Lenguaje
cual favorece la interacción con los usuarios finales. Como se mencionó antes, GraphTQL usa el grafo esquema como
Adicionalmente, esta separación (esquema e instancia) evita la base para la interacción con el usuario. Por tanto, la interfaz
complejidad que se añade al modelo cuando la metadata del despliega la representación diagramática del grafo esquema
esquema agrega nodos y arcos a los datos de la instancia, como (Figura 38-(a)), que a su vez ofrece una vista genérica de todos los
sucede en RDF [80]. El modelo también se caracteriza por tener datos disponibles para consulta. Los datos se seleccionan por la
nodos y arcos simples (sin atributos). Los atributos de los objetos aplicación sucesiva de operaciones que transforman las vistas de
se representan con nodos de tipo valor básico. Esta característica los grafos esquema e instancia. La metáfora que se aplica es la

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 411 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
reducción progresiva del grafo esquema y de la instancia los pacientes que han tenido un diagnóstico de “Diabetes”. A
respectiva, hasta llegar al subconjunto de datos que el usuario diferencia de Filter, “todos los datos” incluyen otros
necesita recuperar. Un ejemplo de ello se muestra en la secuencia diagnósticos recibidos, además de “Diabetes”. El esquema no
de Figura 1-(a), 1-(f), y 1-(g). cambia, se recuperan los pacientes que cumplen la condición
y los nodos que están conectados a esos pacientes a través de
Los parámetros de entrada de los operadores se definen por un camino simple del grafo esquema. Para aplicar este
manipulación directa del grafo esquema, y cada vez que el usuario operador basta marcar una clase de interés (en el ejemplo,
elije una operación, recibe retroalimentación, ya que se despliega paciente) y la condición que deben cumplir las instancias de
el grafo esquema modificado, sin ejecutar la operación. Cuando el esa clase.
usuario ha elegido la secuencia de operaciones para su consulta, la
• Put two objects closer: esta operación reemplaza un camino que
ejecuta.
conecta dos nodos por otro camino que crea una relación entre
2.2.1 Operadores de Transformación de Grafos ellos. En este caso no hay interés en los datos del camino
Las operaciones que transforman los grafos son: reemplazado, por lo tanto no hace parte del grafo respuesta. Por
ejemplo, en el resultado del ejemplo de Filter se quiere
reemplazar el camino que conecta la identificación del paciente
con la descripción de los diagnósticos que ha recibido. Para ello,
se marcan los nodos y el camino a reemplazar (Figura 1-(f)). El
resultado en el esquema se muestra en la Figura 1-(g).
2.2.2 Mecanismo de Interacción
GraphTQL tiene una interfaz simple que consta de cinco marcos
(Figura 1-(a) a (e)): (a) Vista Principal, donde se despliega el
grafo esquema. (b) Vista de referencia del grafo esquema, para
facilitar la ubicación y el desplazamiento en el grafo. (c)
Operadores. (d) Funciones de control, que incluyen limpiar las
marcas, ayuda, retroceder un paso, volver al inicio, zoom, y
ejecutar la consulta. (e) Historia de los operadores aplicados
durante la formulación de una consulta.
Para seleccionar las entradas de cada operación se tienen 3
notaciones visuales: nodos de valor básico con una condición de
filtro, clase de interés, y otros nodos y arcos involucrados en la
operación. En el diagrama del grafo, los nodos con condición
Figura 2. Grafo instancia con información clínica de pacientes. aparecen en color rojo, con la condición escrita al lado de la
etiqueta del nodo; la clase de interés aparece en color verde; y los
otros nodos y arcos marcados, en color rojo (Figura 1-a)). El
• Select a portion: permite seleccionar una porción del grafo. Por usuario especifica los nodos con condición y la clase de interés
ejemplo, si se requiere la fecha de nacimiento, la raza, el haciendo click derecho sobre el nodo, los otros nodos se marcan
nombre y la identificación de los pacientes, el usuario marca haciendo click sobre ellos. Cada vez que se marca un nodo, la
estos nodos y los arcos entre ellos y selecciona esa porción del interfaz se encarga de encontrar y marcar el camino que lo
grafo. La operación no busca la coincidencia exacta de un conecta con los nodos previamente marcados. Cuando hay más de
patrón (no es pattern matching), puesto que recupera todos los un camino, se realiza un diálogo de clarificación que presenta
nodos y los arcos que corresponden con los tipos marcados en el todos los caminos posibles para que el usuario elija uno o varios
esquema. Por tanto, para el caso de ejemplo, se recuperan de ellos.
pacientes con datos incompletos.
• GraphTQL ofrece dos tipos de filtro: En el diagrama del grafo, los óvalos representan los nodos de
- Filter: Permite seleccionar los objetos de cierto tipo, la clase valor básico, de esta manera el usuario puede distinguir fácilmente
de interés, que cumplen una condición específica, y algunos sobre cuales nodos especificar las condiciones de los filtros.
datos conectados a ellos. Por ejemplo, si interesa la Con el fin de simplificar la interfaz, los operadores de filtro
identificación, el nombre y la fecha de nacimiento de los requieren que el usuario marque solamente una clase de interés y
pacientes que han recibido el diagnóstico de “Diabetes”, el las condiciones que ésta debe cumplir; además, en el caso
usuario marca la clase de interés, Paciente, la condición que particular de Filter, otros nodos a seleccionar. Sin embargo, la
deben cumplir las instancias de esa clase, ha recibido un sencillez de esta representación genera dos problemas: el primero,
diagnóstico de “Diabetes”, y otros datos que se requieren que limita la expresividad de los filtros y el segundo, que cuando
(nombre y fecha de nacimiento) (Figura 1-(a)). Estos últimos se necesita especificar más de una condición, la representación
son opcionales y aparecen en el resultado, si están disponibles. gráfica es ambigua y por tanto, se hace necesario especificar si se
El esquema del resultado contiene los nodos que se habían trata de una conjunción o de una disyunción de las condiciones.
marcado (Figura 1-(f)), y la instancia los nodos Con el fin de proveer mayor expresividad en los filtros, sin
correspondientes con el esquema y que cumplen la condición agregar conceptos o notación gráfica que el usuario deba aprender
(están conectados al diagnóstico con descripción “Diabetes”). a usar, el lenguaje realiza un diálogo de clarificación del filtro. La
- Filter + additional data: Permite seleccionar todos los datos interfaz identifica los nodos de bifurcación, es decir aquellos
relacionados con un objeto que cumple una condición. Por donde se bifurcan los caminos que conectan la clase de interés con
ejemplo, seleccionar todos los datos de la historia clínica de las clases de condición. El diálogo de desambiguación consta de

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 412 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
una serie de preguntas que se plantean al usuario para que la ubicación (coordenadas x e y) del nodo en la ventana principal.
especifique para cada nodo de bifurcación, si se aplica una Cuando no se proveen las coordenadas se aplica el algoritmo
conjunción o una disyunción a las condiciones que se desprenden kkLayout [79] para calcular la organización del grafo en la
de él. ventana. La opción de aportar las coordenadas de los nodos se
implementó porque, a pesar de que kkLayout genera una buena
2.2.3 Características del diseño del lenguaje distribución de los nodos en este tipo de grafos, persisten algunos
GraphTQL tiene como características distintivas que el usuario: cruces de arcos o agrupaciones de nodos que pueden generar
(a) Tiene una vista de nivel conceptual de los datos disponibles, y confusión en los usuarios.
de los que ha seleccionado en cada paso de la formulación de la
consulta. (b) No requiere explorar el esquema de los datos y no La interfaz permite que los objetos puedan tener asociada una
requiere construir patrones de consulta desde cero. (c) No necesita imagen, que se pinta en el nodo.
definir cuando hay patrones exigidos y patrones opcionales. La La interfaz captura la secuencia de operaciones que el usuario usa
herramienta calcula como patrones exigidos aquellos que en la formulación de la consulta y genera en XML10 un árbol que
conectan la clase de interés con los atributos que tienen una representa esta secuencia de operaciones con los parámetros
condición. Y como patrones opcionales los que conectan los correspondientes. Este árbol se envía al motor de ejecución para
nodos que no hacen parte del patrón exigido. (d) No necesita que realizar la consulta. El manejo de los datos en el motor se realiza
definir variables, variables de salida, ni distinguir entre variables y con Neo4j 11.
valores literales. (e) Opera solamente sobre grafos. En lenguajes
como SPARQL solamente el patrón básico de búsqueda opera El código fuente del prototipo de GraphTQL está disponible para
sobre el grafo, las otras operaciones se aplican sobre conjuntos de ser descargado12.
mapeos. Esto hace que la semántica de estas operaciones, cuando
hay datos incompletos, pueda resultar engorrosa aun para 3. PROCESO DE DISEÑO
programadores. (e) Los filtros no son locales, se filtra la clase de Para el desarrollo de GraphTQL se siguió un proceso de Diseño
interés por la conexión con nodos de otras clases que cumplen Centrado en el Usuario estructurado en cuatro fases (Tabla 6). La
cierta condición, lo cual ofrece mayor expresividad que las metodología para la realización de la entrevista preliminar y las
herramientas de exploración de datos. (f) El diálogo de evaluaciones siguió las recomendaciones dadas por Rubin [82] y
clarificación de filtros da la posibilidad de combinar conjunción o Nielsen [81].
disyunción en diferentes puntos del grafo, sin necesidad de que el
usuario cree un árbol con la expresión lógica. 3.1 Entrevista Preliminar
En el primer experimento, la entrevista preliminar, participaron
Adicionalmente, el diseño de la interfaz incluye las características cuatro médicos especialistas. La entrevista se desarrolló en
básicas que Nielsen [81] propone para una interfaz desarrollada formato libre, sin cuestionario guía, y el moderador guio la
con principios de usabilidad: (a) Tiene un diseño simple, que conversación para responder a las siguientes preguntas: ¿Cuáles
contiene solamente la información que el usuario necesita. El datos se registran en una historia clínica?, ¿Durante la atención a
grafo se despliega de forma estética, atendiendo a parámetros de un paciente qué información puede necesitar?, ¿Sería importante
simetría y evitando el cruce de arcos y el solapamiento de nodos. relacionar los datos de un paciente y los de las historias clínicas de
(b) Usa el lenguaje del usuario en el modelo conceptual, y los sus familiares?, y ¿En cuáles procesos (i.e. investigación,
nombres de las operaciones se cambiaron teniendo en cuenta las enseñanza) puede ser útil una herramienta de consulta de datos
opiniones que los usuarios aportaron durante las pruebas de clínicos? Cada entrevista tuvo una duración aproximada de una
usabilidad realizadas. (c) Minimiza la carga de memoria del hora.
usuario desplegando elementos de diálogo que lo guían. (d) Es
consistente, por ejemplo, en los efectos de los comandos y en la A partir de la información recolectada se concluyó que las
ubicación de la información. (e) El usuario recibe consultas ad hoc sobre la historia clínica de los pacientes son
retroalimentación durante la formulación de la consulta. (f) necesarias en los siguientes escenarios: evaluación y vigilancia
Ofrece, en todo momento, salidas marcadas según las epidemiológica, consulta externa y citas de control,
convenciones del sistema operativo. (g) Ofrece un historial de la administración de los servicios de salud, investigación de casos
interacción, el usuario puede deshacer una o varias operaciones o clínicos, seguimiento de la evolución de los pacientes, procesos de
volver al estado inicial. (h) Provee ayuda a través del botón de enseñanza-aprendizaje, y planeación de programas de prevención.
ayuda y por los tool tips que se despliegan cuando el mouse pasa Dentro de estos escenarios los médicos dieron 31 ejemplos de
por los botones. Y los mensajes de error ofrecen información que consultas, entre ellas:
guiar al usuario en la solución del problema. • En consulta externa y citas de control: Revisar los conceptos
dados por médicos de otra especialidad al paciente que está en
2.3 Detalles de Implementación consulta.
El prototipo de la interfaz se desarrolló en Java. El despliegue y • En administración de los servicios de salud: Obtener estadísticas
manipulación del grafo se realiza usando Jung8. Los comandos de diagnósticos o de resultados de exámenes. Por ejemplo,
que por defecto provee Jung para manipular el grafo se cambiaron ¿Cuántos hemogramas se hicieron en el mes?, ¿Cuáles son las
para que no se requirieran combinaciones de teclas y clicks en la patologías más frecuentes?
interacción del lenguaje.
La interfaz toma como entrada un grafo esquema especificado en
10
GraphML9, que además de los datos de cada nodo, puede incluir http://www.w3.org/TR/REC-xml/
11
http://neo4j.com/
8 12
http://jung.sourceforge.net http://labsistemas.javerianacali.edu.co:8000/~mcpabon/GraphT
9
http://graphml.graphdrawing.org/ QL-interface.zip
XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 413 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
• En investigación de casos clínicos y procesos de enseñanza- • En planeación de programas de prevención y seguimiento de la
aprendizaje: Encontrar grupos de pacientes que cumplen un evolución de los pacientes: Revisar las historias clínicas de
perfil. Por ejemplo, datos de los pacientes de género femenino recién nacidos cuyas madres tuvieron diagnóstico de
que estuvieron hospitalizados en la sala de pediatría general hipertensión durante el embarazo.
durante los años 2009 y 2010 con diagnóstico de enfermedad
por reflujo gastro-esofágico.
Tabla 1. Pruebas de usabilidad realizadas durante el desarrollo del proyecto.
Prueba Objetivo Momento de aplicación
Entrevista Conocer cuál información se registra en una historia clínica, cuál información Antes de iniciar el diseño del
preliminar necesita consultar un médico cuando está atendiendo un paciente, y en que lenguaje
situaciones es útil una herramienta para formular consultas sobre datos clínicos
Prueba Validar si la representación de los datos en grafos y las operaciones de Punto intermedio en el diseño, antes
exploratoria transformación de grafos son entendibles para el usuario de iniciar la implementación
1a. prueba de Evaluación, sobre dos opciones de la interfaz, de un experto en diseño de Punto intermedio en la
evaluación interfaces y un grupo de usuarios de diverso perfil implementación de la interfaz
2a. prueba de Evaluar la facilidad de uso del lenguaje visual de consulta Terminada la primera etapa de
evaluación implementación de la interfaz
Prueba Evaluar la facilidad de uso del lenguaje visual de consulta y compararlo con Terminada la primera etapa de
comparativa una interfaz gráfica para formulación de consultas en SPARQL implementación de la interfaz
Las consultas de ejemplo dadas por los médicos se clasificaron en interesante” y “Es un diagrama que permite encontrar y amarrar la
tres tipos: Consultas que recuperan datos de la historia clínica de relación entre los datos”.
un paciente, consultas para seleccionar grupos de pacientes que
cumplen un perfil, y consultas que incluyen operaciones de En cuanto a la actividad de la parte 4, el 91,6% de los casos
agregación. El desarrollo actual del lenguaje visual de consulta se fueron identificados correctamente, y en un 16,6% el moderador
centra en los dos primeros tipos, las operaciones de agregación se intervino para hacer notar alguna parte del grafo. Los participantes
incluirán en una etapa posterior. comentaron que las operaciones permiten realizar consultas
pertinentes para el ejercicio clínico y la investigación. Todos los
3.2 Prueba Exploratoria participantes coincidieron en que los íconos y los textos
Con la prueba exploratoria se trató de corroborar si el uso de un propuestos son claros.
grafo como modelo de datos, las operaciones propuestas para el Como comentarios adicionales, los participantes afirmaron que la
lenguaje, los iconos de las operaciones y los textos explicativos primera impresión al ver los grafos es que esta representación
son entendibles. Para esta prueba el perfil de usuario fue más (tipo red) es compleja, pero que cuando empezaron a usarlos y
amplio, en esta oportunidad participaron tres profesionales de la responder las preguntas notaron que era fácil encontrar la
salud, entre ellos un epidemiólogo y un enfermero. información y las relaciones entre los datos.
La prueba se hizo con prototipos de papel y se dividió en cuatro De la prueba exploratoria derivamos las siguientes conclusiones:
partes: 1. Presentación de la noción de grafo y su uso como Para los participantes fue fácil interpretar los grafos esquema e
modelo de representación de datos. 2. Presentación del grafo instancia y las operaciones del lenguaje transformación de grafos.
esquema como una representación general de la organización de Para ellos fueron claros los íconos y textos propuestos para la
los datos y el grafo instancia como una representación particular
aplicación. Los participantes opinaron que la herramienta sería
de hechos acorde al esquema. A manera de evaluación, se pidió a
los participantes que encontraran en los grafos dos datos útil en su trabajo. Además, durante el desarrollo de la prueba se
particulares.3. Presentación de las operaciones del lenguaje. 4. observó que a los participantes les toma más tiempo (y
probablemente esfuerzo) interpretar el grafo instancia, esto
Actividad de identificación de operaciones aplicadas en cuatro
casos. confirma que es una buena alternativa usar el esquema para guiar
la formulación de las consultas.
El moderador de la prueba podía aceptar comentarios y preguntas
del usuario en cualquier momento y responder a ellas. 3.3 Pruebas de Evaluación
La etapa de evaluación incluye tres pruebas que se describen a
Se obtuvieron los siguientes resultados: continuación.
Los participantes identificaron y entendieron la información La primera prueba de evaluación se realizó con un experto en
representada en el esquema y la instancia y respondieron diseño de interfaces y un grupo de usuarios de diverso perfil. El
acertadamente a las preguntas de la parte 2. Dos de los experto es diseñador gráfico, especializado en estética, con
participantes tardaron en identificar la relación Encounter entre experiencia en publicidad, y desarrollo y diseño de estilos y de
los nodos Physician y Patient, por la notación que se usó en esos páginas web. En la prueba se presentaron dos diseños con
prototipos para ese tipo de nodo. Sus observaciones se tuvieron en diferencias en la organización de la pantalla y en los colores
cuenta para cambiar la representación de los nodos (Figura 38). usados en el grafo y para marcar las clases de interés, nodos con
Los participantes hicieron comentarios como “Me parece condición y los otros nodos y arcos. Con base en la opinión
recopiladas se eligió la organización de la pantalla, se cambiaron

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 414 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
algunos colores, se modificaron los iconos de los botones de los ejemplos datos por los médicos en la entrevista preliminar, los
operación y de control, y se cambiaron los nombres de los cuales se clasificaron según su nivel de complejidad. La
operadores. complejidad se definió en términos de las operaciones necesarias
para realizar las consultas sobre un modelo de grafos. Para las
En la segunda prueba cuatro participantes, dos profesionales y dos actividades se eligieron consultas de diverso nivel de complejidad.
estudiantes de carreras relacionadas con áreas de la salud, También se tuvo en cuenta que la formulación de la consulta en la
evaluaron el prototipo del lenguaje visual de consulta propuesto. interfaz gráfica de SPARQL se realiza dibujando el patrón de
El plan de esta pruebas constó de: (a) Apertura: El moderador consulta, nodo por nodo y arco por arco, por lo tanto se limitó la
explica el propósito de la prueba y las partes que incluye. El cantidad de nodos que se requieren en las consultas.
participante diligencia un cuestionario pre-test, que recoge Adicionalmente, se limitó el número de operaciones necesarias
información de su perfil de formación, y firma el formato de para formular la consulta en la interfaz gráfica (por ejemplo las
consentimiento para realizar la prueba. (b) Demostración de la
consultas no requieren union), para disminuir la cantidad de
herramienta, se hace con una base de datos sobre películas y series elementos de SPARQL que el usuario debía aprender.
de televisión. El moderador explica las opciones, botones y
operaciones de la herramienta, y desarrolla tres ejemplos de Las consultas que se propusieron en las cuatro actividades de la
formulación de consultas. (c) El usuario realiza cuatro actividades. prueba comparativa y la segunda prueba fueron:
Para este caso se toma una base de datos clínicos. En cada • Se requieren las enfermedades (incluyendo su descripción)
actividad se pide al participante que formule una consulta. La registradas en la historia familiar del paciente con identificación
complejidad de las consultas crece de una actividad a la siguiente. 723628; incluir la relación familiar si está disponible.
Las consultas se eligieron buscando poner a prueba las • Se requiere encontrar los diagnósticos (incluyendo su
operaciones de filtro, y los diálogos de clarificación de caminos y descripción) dados al paciente con identificación 723628 por
de condiciones de filtro del lenguaje de consulta propuesto. Los médicos con especialidad en “ENDOCRINOLOGY” y, si está
participantes cuentan con un tiempo máximo de 15minutos para disponible, las fechas de las consultas y las medicinas que le
formular la consulta de cada actividad. recetaron en ese momento (incluyendo descripción y dosis).
Si bien la tasa de respuestas correctas fue baja, estas pruebas • De los pacientes que han tenido un diagnóstico de
permitieron detectar varias oportunidades de mejoramiento de la “OSTEOMYELITIS” desde el año 2000 (desde 01-01-2000), se
interfaz. Primero, la versión que se evaluó no incluía el diálogo de requieren todos los datos de: identificación del paciente,
clarificación de caminos, la herramienta marcaba todos los diagnósticos recibidos, y procedimientos que se le han realizado
caminos posibles entre los nodos marcados, con el fin de hacerlos (incluir la descripción de los diagnósticos y procedimientos).
evidentes al usuario, y se esperaba que el usuario quitara la marca Note que estos datos incluyen los otros diagnósticos, además de
de aquellos que no eran necesarios. Se notó que los usuarios OSTEOMYELITIS, y todos los datos que se piden son
tienden a aceptar por defecto las sugerencias que la herramienta independientes de la fecha en que se registraron.
hace. Entonces, el patrón principal de consulta incluía caminos • Se requiere encontrar los pacientes que han tenido un estudio de
que no se requerían. Segundo los nombres de las operaciones de imágenes diagnósticas con anotación “GALLSTONES” y se les
filtro no ayudaban a que el usuario recordara la diferencia entre ha realizado el procedimiento “COLECTOMY”. De estos
los filtros. Por tanto, en algunos casos el usuario aplicaba una pacientes se requiere la fecha de nacimiento, género y raza, si
operación que no correspondía con lo solicitado en la consulta. están disponibles. Estos resultados pudieron realizarse en
Tercero, dado que la operación de filtro con datos adicionales no encuentros diferentes, pero el paciente cumple con ambas
cambia el esquema, el usuario dudaba sobre si se había o no condiciones.
aplicado una operación. Cuarto, se identificaron varias La primera es una consulta sencilla, que incluye una condición
validaciones que se podían agregar en cada operación, de manera (identificación del paciente) y datos opcionales (relación familiar).
que se prevengan posibles errores. Por ejemplo, si el usuario La segunda consulta incluye la conjunción de dos condiciones de
marca condiciones sobre algunos atributos y luego elige el filtro y datos opcionales. La tercera consulta también incluye la
operador Select a portion, las condiciones marcadas no tienen conjunción de dos condiciones de filtro y datos opcionales.
incidencia en la operación, por tanto es posible que el operador Adicionalmente, para formular la consulta en SPARQL se
esté confundiendo la operación que requiere aplicar. Además de requieren dos variables que representen los diagnósticos, y dos
las opiniones relacionadas estos aspectos, los participantes para la descripción de los diagnósticos. De esta manera se
concordaron en que la herramienta sería de gran utilidad en su incluyen otros diagnósticos adicionales a los de la condición de
campo profesional, que es fácil de usar, que se requiere una fase filtro. En GraphTQL esto se logra usando el operador Filter +
de entrenamiento corta, y que la representación de los datos en el additional data. La cuarta consulta incluye la conjunción de dos
grafo es fácil de entender. condiciones de filtro y datos opcionales. Para formular la consulta
en SPARQL el patrón de consulta debe incluir dos variables para
Finalmente, la tercera fue una prueba comparativa que incluyó los encuentros, uno que se conecte con cada condición. En
una interfaz gráfica para la formulación de consultas en SPARQL GraphTQL basta con responder a las preguntas del diálogo de
y GraphTQL. Se realizó con cuatro participantes, tres desambiguación. De manera que en el encuentro (nodo Encounter
profesionales de la salud y un estudiante de medicina. Dos en la Figura 38-(a)) se aplica una disyunción, y en el paciente una
participantes evaluaron GraphTQL y otros dos, la interfaz gráfica conjunción.
de consulta. La prueba siguió el mismo plan de la segunda prueba,
y adicionalmente se pidió a los usuarios que respondieran un
cuestionario SUS (System Usability Scale) [67].
Las cuatro actividades de prueba fueron las mismas para la
segunda y tercera pruebas. Estas consultas se eligieron a partir de

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 415 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
Tabla 2. Resultados de la prueba comparativa. representación gráfica de los elementos de un lenguaje de consulta
Errores con sintaxis en texto.
Consultas Tiempo por
por Puntaje Las herramientas de exploración y análisis, entre ellas Gephi [69],
Herramienta correctas consulta
consulta SUS GlyphLink [72], CGV [85], y Shamir y Stolpnik [84], ofrecen
(%) (avg.)
(avg.) diferentes mecanismos para explorar los datos. En cuanto a las
Interfaz 51,3 formas de seleccionar y filtrar la información, se encuentran, en
0,0% 4,5 10min 53sec
SPARQL general, operaciones relacionadas con la topología del grafo que
GraphTQL 62,5% 0,9 3min 56sec 72,5 permiten explorar, a partir de un nodo particular los nodos vecinos
(k-vecinos) y encontrar caminos entre nodos, y filtros que
permiten seleccionar nodos y arcos que tienen una propiedad
La Tabla 7 muestra los resultados obtenidos. Se nota que la
particular o son de cierto tipo, y en algunos casos permiten
formulación en SPARQL es difícil si no se tiene un entrenamiento
componer las expresiones de filtro por medio de operadores
prolongado. A pesar de ello, los participantes que evaluaron la
lógicos, sin embargo los filtros siguen siendo locales (aplican
interfaz gráfica para consultas en SPARQL se sintieron confiados
sobre cada nodo o arco).
al usar la herramienta.
Barzdins et al. [68], QUBLE [78], Sadanandan et al. [83] y
También se notan diferencias importantes en el tiempo requerido
Catarci et al. [71] proponen lenguajes visuales de consulta que
para la formulación de las consultas y en el número de errores. El
usan un grafo como representación visual para soportar la
número de errores se refiere al número de tipos de error diferente,
interacción con el usuario en la formulación de la consulta. Por su
por ejemplo si una consulta incluye tres datos opcionales y los tres
parte, Aasman y Cheetham [65], Groppe et al. [74], y
datos se incluyeron dentro del patrón de consulta como
SparqlFilterFlow [76] proponen interfaces gráficas para SPARQL.
obligatorios, se cuenta solamente un error. Todos los participantes
Estos sistemas se caracterizan por ofrecer mayor expresividad,
coincidieron en que una herramienta que les permita formular este
pues incluyen más operaciones (i.e. cláusulas optional y union) y
tipo de consultas sería muy útil en la atención a pacientes, la
los filtros se definen sobre resultados de patrones de consulta, con
administración y la investigación médica.
lo cual se puede seleccionar nodos conectados a otros nodos que
La prueba con la interfaz gráfica para SPARQL permitió observar cumplen cierta condición, además, incluyen operaciones lógicas
que la necesidad de explorar el grafo de datos para construir el (y/o) y en algunos casos filtros globales. Sin embargo,
patrón de consulta tiende a introducir errores porque se equivoca generalmente no hay manipulación directa sobre el diagrama del
la dirección de los arcos o se asume alguna representación de una grafo, los patrones se deben construir desde cero a partir de listas
relación de manera diferente a la que se establece en los datos. Se de objetos y propiedades. Por tanto, requieren que el usuario haga
observó que en general los participantes no tienen en cuenta el una exploración del grafo de datos para conocer cómo están
concepto de datos opcionales. Y que pueden confundir nodos que organizados los datos, lo cual puede resultar engorroso
representan una variable con nodos que representan un valor especialmente cuando hay datos incompletos. Adicionalmente, el
literal. Además, aunque los participantes identificaron en cuales usuario no tiene retroalimentación durante la formulación de la
consultas requerían definir más de una variable para representar consulta y debe entender el lenguaje pues las construcciones del
objetos del mismo tipo, no encontraron la forma de especificar los mismo se trasladan a una notación visual. Esto genera un mayor
patrones de manera que se diera respuesta a la consulta requerida. nivel de complejidad porque, por ejemplo, pequeñas diferencias
en la forma como se agrupan cláusulas como optional y union, y
De otra parte la prueba con GraphTQL permitió identificar que la ubicación de los filtros dentro de esos grupos puede generar
era necesario hacer más claridad en la pregunta de grandes variaciones en el resultado de la consulta.
desambiguación para el caso en que el usuario necesita elegir
diferentes operadores lógicos (conjunción/disyunción) en Finalmente, solamente los sistemas para consulta de ontologías
diferentes puntos de bifurcación del patrón exigido en la consulta. funcionan sobre una representación de nivel conceptual.

4. TRABAJOS RELACIONADOS En cuanto al enfoque de diseño centrado en el usuario, algunos

trabajos [1, 4, 5, 10] no mencionan el uso de una metodología
Batini et al. [70] define los Lenguajes Visuales de Consulta
para el desarrollo de un sistema con características de usabilidad,
(Visual Query Language) como sistemas de consulta de bases de
y tampoco reportan resultados de pruebas de usabilidad. En [8, 12,
datos que usan una representación visual para describir el dominio
14, 19, 20] reportan evaluaciones sumativas, realizadas al final del
de interés y para expresar las solicitudes sobre los datos. Las
desarrollo de la herramienta. En general estas evaluaciones miden
representaciones visuales son, usualmente, iconos, tablas,
el número de tareas realizadas correctamente, el tiempo requerido
diagramas, o una combinación de ellos.
para realizar las tareas, y la percepción de los participantes con
El desarrollo de sistemas visuales de consulta sobre modelos de respecto a algunas características de la usabilidad. Dos trabajos
grafos ha seguido dos vertientes principales: De una parte, se reportan un mayor involucramiento de los usuarios durante el
encuentran las herramientas de exploración y análisis de datos, ciclo de desarrollo del proyecto. Tominski et al. [85] reportan que
que se concentran principalmente en mecanismos de recibieron retroalimentación de los usuarios durante el desarrollo
agrupamiento (clustering) y visualización de grandes volúmenes del sistema. Además, que los comentarios sobre la interfaz y los
de datos, y el cálculo de propiedades del grafo (i.e. centralidad, mecanismos de interacción de los usuarios que empezaron a
índice de influencia). De otra parte están los sistemas que se utilizar la herramienta, influenciaron el desarrollo del sistema en
enfocan en el lenguaje de consulta, entre los cuales se incluyen varios aspectos, entre otros, el orden de las vistas, el uso de
aquellos que proponen un nuevo lenguaje y las interfaces gráficas parámetros de visualización, y la forma de realizar el
para lenguajes existentes. Estas últimas, proveen una desplazamiento y zoom. Sin embargo, los autores no reportan la
realización de pruebas de usabilidad. Por su parte, Catarci et al.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 416 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
[71] describen los experimentos de evaluación de usabilidad de un [5] M. Bastian, S. Heymann, and M. Jacomy. Gephi: An Open
sistema que permite el acceso a datos de fuentes heterogéneas, por Source Software for Exploring and Manipulating Networks.
medio de consultas expresadas sobre una ontología. Los autores In Intl. AAAI Conf. on Weblogs and Social Media, 2009.
reportan que el proyecto siguió la metodología de diseño centrada [6] C. Batini, T. Catarci, M. F. Costabile, and S. Levialdi. Visual
en el usuario, e involucró a los usuarios desde las primeras etapas. Query Systems: A Taxonomy. In Procs. of the IFIP TC2/WG
Durante el desarrollo del proyecto aplicaron evaluaciones 2.6 Second Working Conf. on Visual Database Systems II.
formativas y sumativas, entre ellas: una encuesta para obtener North-Holland Publishing Co., 1991.
información base para analizar el contexto de la aplicación; una
encuesta y una evaluación observacional sobre el primer y [7] T. Catarci, T. Di Mascio, P. Dongilli, E. Franconi, G.
segundo prototipos; y finalmente, una encuesta, una evaluación Santucci, and S. Tessaris. Usability evaluation in the
observacional y un experimento controlado sobre el prototipo SEWASIE project. In 11th Intl. Conf. on Human-Computer
final y el producto terminado. Interaction (HCII), 2005.
[8] M. Cayli, M. C. Cobanoglu, and S. Balcisoy. GlyphLink: An
5. CONCLUSIONES Y TRABAJO FUTURO interactive visualization approach for semantic graphs.
Este artículo presenta la aplicación del diseño centrado en el Journal of Visual Languages & Computing, 24(6), Dec.
usuario (UCD) en el desarrollo de un lenguaje visual de consulta 2013.
de nivel conceptual. Como parte de este proceso se realizaron un
conjunto de experimentos con los usuarios finales, profesionales [9] G. L. Glandon, D. H. Smaltz, and D. J. Slovensky. Austin
de la salud. Se presentaron los resultados obtenidos de la and Boxerman's Information Systems For Healthcare
aplicación de una entrevista preliminar, una prueba exploratoria, y Management. Health Administration Press/AUPHA, 7
pruebas de usabilidad y comparación con una interfaz gráfica para edition, 2008.
formular consultas en SPARQL. Estas últimas, se realizaron sobre [10] J. Groppe, S. Groppe, and A. Schleifer. Visual query system
un prototipo funcional del lenguaje visual de consulta, for analyzing social semantic web. In Proc. Of the 20th Intl.
desarrollado en Java, usando Jung13 para el despliegue de los Conf. Companion on World Wide Web, WWW '11. ACM,
grafos. 2011.
Se describieron las principales decisiones de diseño que surgieron [11] J. Gulliksen, B. Göransson, I. Boivie, J. Persson, S.
a partir del enfoque UCD y de los resultados de estas Blomkvist, and A. Cajander. Key Principles for User Centred
experiencias. La aplicación del UCD permitió conocer mejor las Systems Design. Intl. Journal Behaviour & Information
necesidades del usuario y tomar decisiones de diseño con criterios Technology, 22(6), 2003.
diferentes, más amplios, de los que se aplican en un proceso de [12] F. Haag, S. Lohmann, and T. Ertl. SparqlFilterFlow:
desarrollo tradicional. Estas decisiones reflejan la importancia que SPARQL Query Composition for Everyone. In V. Presutti,
tiene la interacción entre el usuario y la aplicación para el E. Blomqvist, R. Troncy, H. Sack, I. Papadakis, and A.
resultado que buscamos. Tordai, editors, The Semantic Web: ESWC 2014 Satellite
Actualmente se están realizando pruebas de rendimiento de la Events, volume 8798 of LNCS. Springer International
ejecución de las operaciones para, a futuro, proponer mecanismos Publishing, 2014.
de optimización de la ejecución de las consultas. Además, a [13] J. Hidders. Typing Graph-Manipulation Operations. In Proc.
futuro, se aplicará el DCU para elegir metáforas que se of the 9th Intl. Conf. on Database Theory, ICDT. Springer-
complementen, y ofrecer al usuario alternativas para la Verlag, 2002.
visualización de los datos que son resultado de las consultas y [14] H. H. Hung, S. S Bhowmick, B. Q. Truong, B. Choi, and S.
para incluir la posibilidad de aplicar funciones agregadas, ya sea Zhou. QUBLE: blending visual subgraph query formulation
sobre la representación de grafo o sobre otras representaciones with query processing on large networks. In SIGMOD '13.
con que se visualicen los resultados. ACM Press, 2013.
6. REFERENCES [15] T. Kamada and S. Kawai. An algorithm for drawing general
[1] J. Aasman and K. Cheetham. RDF Browser for Data undirected graphs. Information Processing Letters, 31(1):7-
Discovery and Visual Query Building. In Procs. of the 15, Apr. 1989.
Workshop on Visual Interfaces to the Social and Semantic [16] G. Klyne and J. Carroll. Resource Description Framework
Web, 2011. (RDF): Concepts and Abstract Syntax.
[2] R. Angles and C. Gutierrez. Survey of Graph Database http://dret.net/biblio/reference/rdfconcepts, 2004.
Models. ACM Computing Surveys, 40(1), Feb. 2008. [17] J. Nielsen. Usability Engineering. Morgan Kaufmann, 1993.
[3] A. Bangor, P. Kortum, and J. Miller. An Empirical [18] J. Rubin, D. Chisnell, and J. Spool. Handbook of Usability
Evaluation of the System Usability Scale. Intl. Journal of Testing: How to Plan, Design, and Conduct Effective Tests.
Human-Computer Interaction, 24(6), 2008. Wiley Publishing, Inc., 2008.
[4] G. Barzdins, E. Liepins, M. Veilande, and M. Zviedris. [19] A. A. Sadanandan, K. W. Onn, and D. Lukose. Ontology
Ontology Enabled Graphical Database Query Tool for End- Based Graphical Query Language Supporting Recursion. In
Users. In H.-M. Haav and A. Kalja, editors, Databases and Proc. of the 14th Intl. Conf. on Knowledge-based and
Information Systems V: Selected Papers from the Eighth Intl. Intelligent Inf. and Eng. Systems: Part I, volume 6276 of
Baltic Conf., DB&IS 2008. IOS Press, 2009. LNCS, 2010.
13
http://jung.sourceforge.net

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 417 |
 Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos
[20] A. Shamir and A. Stolpnik. Interactive visual queries for [21] C. Tominski, J. Abello, and H. Schumann. CGV An
multivariate graphs exploration. Computers & Graphics, interactive graph visualization system. Computers &
36(4), June 2012. Graphics, 33(6):660-678, Dec. 2009.

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 418 |
Lenguaje Visual de Consulta sobre Grafos de Datos: Un enfoque desde el Diseño Centrado en el Usuario
M. C. Pabón | C. A. Collazos

XVI Congreso Internacional de Interacción Persona-Ordenador | Usability and User Experience (II)
| 419 |
SOFTWARE, ARCHITECTURE AND INTERACTION
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías

An Agile Information-Architecture-Driven Approach for the

Development of User-Centered Interactive Software
Luis A. Rojas and José A. Macías
Escuela Politécnica Superior, Universidad Autónoma de Madrid,
Tomás y Valiente 11, 28049 Madrid, España
[email protected], [email protected]

ABSTRACT methodological proposals to develop it. There are currently

For the most part, Information Architecture processes include sets several methodologies for the IA development [2,3,4,5,6].
of activities and techniques to be carried out by the development However, these proposals are based on traditional development
team to create interactive applications effectively, involving process models, making it difficult to carry them out in changing
usability concerns at every development step. In fact, plenty of environments, where an agile and quick response is often
process models have already been proposed to bridge the gap required.
between User-Centered Development and Information On the other hand, agile methodologies emerge as a response to
Architecture, empowering the development team to build usable the need of adapting quickly to changing environments.
applications successfully. However, the combination of User- Nevertheless, this also reports difficulties when incorporating
Centered Development and Information Architecture paradigms User-Centered Design (UCD) in agile environments [7], which is
sometimes results in cumbersome process models containing lots an overriding factor when developing interactive software.
of phases and activities to be considered, which increases the Several studies have addressed the need of involving end-users
cycle time to have partial and validated software increments into the agile development, supplying specific recommendations
readily. As less effort has been devoted to speed up the usable but without providing a comprehensive or complete vision, which
Information Architecture development, the aim of this paper is to has been reported by [8] as an incentive to develop new
address such problem. To do so, we present Scrum-UIA, an agile methodologies for the integration of the UCD and the agile
and usable development process driven by the Information paradigm.
Architecture. This process is intended to develop web applications The aim of this paper is to address such drawbacks by providing
by splitting up responsibilities and tasks, and decreasing the time an agile methodology, called Scrum-UIA (Scrum driven by
to perform technical activities, in order to readily obtain usable Usable Information Architecture), for the user-centered
software increments. development of interactive applications, also involving IA as a
Categories and Subject Descriptors building block to guide and drive the development to make it
agilely adaptable to changing environments. Our proposal features
H.1.2 [Information Systems]: User/Machine Systems - Human the integration of UCD in Scrum to carry through an agile
factors; H.5.2 [Information Interfaces and Presentation]: User approach. Also, we provide a set of agile AI techniques to support
Interfaces – User-Centered Design and Prototyping development tasks. In addition, an end-user vision is incorporated
and considered throughout the whole process.
General Terms This paper is organized as follows: Section 2 presents a
Management, Design, Human Factors. comparative analysis of methodologies for IA and a discussion of
Keywords agile ones. Next, section 3 introduces the practices used in agile
and UCD integration, as well as specific recommendations to
Information Architecture, Agile Development, User-Centered
integrate UCD in the Scrum methodology. Then, section 4
Design, Usability
describes our proposed methodology for agile UCD-IA
1. INTRODUCTION development based on Scrum. Finally, section 5 presents
Today, even though we live immersed in the information society, conclusions and future work.
information in many organizations is still not properly managed, 2. ANALYSIS AND DISCUSSION ON
which affects negatively the costs of the organization in terms of
errors and inefficiencies, and also the client’s perception in terms RELATED METHODOLOGIES
of service quality [1]. Furthermore, environments move quickly In this section, a comparative analysis is performed to check the
and are constantly evolving in today's world, which is an current features of the different methodologies for the IA
additional factor that increases the complexity of information development (section 2.1). On the other hand, the feasibility of
management in organizations. integrating user-centered activities into agile methodologies is
also addressed (section 2.2).
The current scenario is at once an opportunity and an obligation
for both researchers and information professionals, allowing to
address the different challenges, opportunities and critical aspects
of the Information Architecture (IA). Martin et al. [1] reported
that one of the critical aspects of IA is the need for

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 420 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías

2.1 Comparative Analyses of Information process model (i.e., waterfall) that needs numerous controls and
policies/standards to be implemented.
Architecture Methodologies
Five IA methodologies [2,3,4,5,6] have been selected according to The results of the comparative analysis are presented in Table 1.
their popularity and broad usage. In order to have a common As we can see in Table 1, most of the proposals do not cover all of
framework to analyze and compare the main features of these the lifecycle phases of IA development, focusing primarily on the
methodologies for the IA development, the following criteria has analysis and design phases. It is important to highlight that
been used: User-Centered criterion, IA Elements Covered, Level proposals [6] and [5] are the only ones that present
of Description, Scope of the Proposal and Flexibility and recommendations of activities and techniques for the IA
Adaptability. These will be briefly described below. evaluation (Scope of the Proposal criterion: E). The proposal [4]
also indicates IA evaluation, but it is only described and shallowly
Table 1. Comparative analysis of IA methodologies. defined (-). It is worth noting that methodologies [5] and [6] are
the only ones that provide a user-centered approach to IA
Met User- IA Elements Level of Scope Flex. development. By contrast, in most of the proposals this aspect is
hod Cente Covered Descr. of the and carried out partially, as the end-users are only included in the
red Prop. Adpt. initial phases and not during the whole process.
[2] Partial -Navigation High A, D, No
ly -Organization I (-), Finally, methodologies [6] and [2] are the only ones that have an
-Labeling M (-) adequate level of proposal description (Level of Description
-Search and criterion: High) and cover all IA aspects (IA Elements Covered
recovery criterion) in the different lifecycle phases of IA development
-Information (Scope of the Proposal criterion). However, none of the analyzed
presentation methodologies present flexibility and adaptability characteristics
[3] Partial -Navigation Low A, D No to respond in an agile and flexible way to changing environments,
ly -Organization or they require considerable effort to be adapted (as commented
-Labeling before in the description of Flexibility and Adaptability criterion).
[4] Partial -Navigation Low A,D, No Such drawback can be addressed through the flexibility and
ly -Organization E (-) adaptability offered by agile methodologies, which allow to
-Labeling respond quickly to changing environments. However, agile
[5] Yes -Organization Average E No methodologies include specific issues that can prevent against
-Labeling usage in user-centered product design, which is an essential factor
[6] Yes -Navigation High A, D, E, No to consider for the IA-driven development.
-Organization I, M Therefore, it becomes necessary to analyze the feasibility of agile
-Labeling methodologies to integrate user-centered activities, in order to be
-Search and able to clearly discern which agile methodologies are more
recovery suitable for the development of user-centered interactive software.
-Information
presentation 2.2 Feasibility of Agile Methodologies to
The User-Centered criterion is used to identify whether the Integrate User-Centered Activities
proposals design their products focused on the needs of end-users. This section presents a study and analysis of several agile
This criterion can take Yes, Partially or No values to indicate that methodologies in order to know the extent to which these can
the proposals are fully, partially or not user-centered, respectively. integrate user-centered activities. This will enable to find out both
The IA Elements Covered criterion is used to identify the aspects the aspects to envision a hypothetical integration support and the
of the IA that proposals attempt to cover. According to Erlin et al. strengths that make it impossible.
[9] the aspects of the IA that are involved in the development of a
product correspond to: navigation, organization, labeling, search
and recovery, and information presentation. The Level of
Description criterion is used to identify the level of detail in
which methodologies for the IA development are described. This
criterion can take High, Average or Low values to indicate that
the proposals are described with sufficient, moderate or general
detail, respectively. The Scope of the Proposal criterion is used to
identify the lifecycle phases of IA development that proposals
attempt to cover. The considered phases are: analysis (A), design
(D), evaluation (E), implementation (I), and management (M). Figure 1. Overview of the feasibility of agile methodologies to
The phases that are incompletely covered (without providing or integrate user-centered activities.
prescribing the necessary information to carry them out) are This way, the seven most-popular agile methodologies were
pointed out with an (-). Finally, the Flexibility and Adaptability analyzed against a set of reference attributes based on the
criterion is used to identify whether methodologies are able to principles of ISO 9241-210 [10]. The agile methodologies
adapt to changing environments and respond quickly. In order to analyzed were Scrum, Extreme Programming (XP), Adaptive
assess this criterion, IA methodologies are analyzed to know Software Development (ASD), Feature-Driven Development
whether they require extensive planning or have a development (FDD), Agile Modeling (AM), Dynamic Systems Development
Method (DSDM) and Crystal Clear (CC).
XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction
| 421 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
Figure 1 depicts the result of the analysis in relation to the degree practices to actively involve end-users. For instance, DSDM
of feasibility of agile methodologies to integrate user-centered presents the user’s role descriptions and the use of prototypes.
activities. Axis X represents the ISO 9241-210 attributes used to Similarly, Scrum facilitates end-users to prioritize the list of
evaluate the agile methodologies, whereas axis Y indicates the requirements and provides a review process that facilitates end-
degree to which the agile methodologies comply with such user involvement. On the other hand, the DSDM methodology has
attributes (Yes, Partially or No). some weaknesses that prevent its application for the stated
As shown in the Figure 1, all methodologies comply with the purpose. To cite a few, DSDM methodology requires a specific
Iterative attribute, so that the project is planned and executed institutional framework for software development process, which
iteratively. is neither cheap nor easy to implement, and it also demands
significant change of consciousness in any organization. By
Regarding the Active Involvement of End-Users attribute, in 5 out contrast, Scrum methodology presents a framework that is easy to
of the 7 analyzed methodologies (Scrum, XP, FDD, AM and CC) implement, providing flexibility and adaptation to end-user and
end-users are partially involved in the process. The DSDM agile business requirements.
methodology is the only one that allows the active participation of
end-users by proposing roles that are directly assumed by them. On the other hand, AM and FDD methodologies provide less
Although 4 out of the 7 analyzed methodologies (Scrum, XP, AM facilities for the UCD integration. These methodologies have in
and CC) also describe roles related to end-users, those are more common their orientation towards optimization of coding
related to aspects concerning functional requirements of the techniques and modeling for systems development, as well as the
system. It is also important to highlight that Scrum includes a use of such methods for technical purposes.
review process (Sprint Review), which would facilitate the direct Therefore, the Scrum agile method has been selected as ideally
involvement of end-users. suited to integrate UCD in a development environment that
With respect to Multidisciplinary Knowledge attribute, 5 out of requires agility and end-user focusing.
the 7 analyzed methodologies (Scrum, XP, ASD, DSDM and CC)
consider work teams with different skills and knowledge.
3. INTEGRATING UCD AND AGILITY
This section describes and analyzes the different practices that are
However, the team building is primarily based on the search of
commonly used to integrate UCD in agile methodologies (section
efficiency in the development and maintenance of functionalities
3.1). Furthermore, specific recommendations that enable the
from the technical perspective of a software programmer. In the
integration of UCD in the Scrum methodology are also reviewed
case of Scrum methodology, it explicitly states that the work team
(section 3.2).
formation will be based on all competencies needed to accomplish
the work without depending on others being not part of the team. 3.1 Practices and Common Artifacts
In relation to Evaluations with End-Users attribute, 5 out of the 7 Da Silva et al. [11] identified, by means of a systematic review of
analyzed methodologies (Scrum, XP, ASD, DSDM and CC) bibliography, what needs, artifacts and common practices are used
accomplish assessments, which are partially focused on end-users. to support collaboration between designers and developers in the
In fact, these methodologies consider specific roles on behalf of integration of UCD and agility. Silva da Silva et al. analyzed 58
end-users, even considering roles as client or expert user with papers concluding that the most important practices and artifacts
knowledge about the system procedures or functional correspond to: little design up front (LDUF), low fidelity
requirements, which would allow partial involvement of end-users prototypes, users testing, user stories, inspection methods, one
in assessments. Nevertheless, agile assessments are mainly sprint ahead and big picture. Similarly, Jurca et al. [12]
oriented towards unit testing and system integration, as most of conducted a systematic mapping study to identify relevant
the agile approaches are principally focused on the client rather research and understand Agile-UCD combination. Some of the
than on the end-user. It is worth noting that Scrum defines a recommended practices and artifacts identified were the
specific event (Sprint Review) in order to verify each product following: concept maps, low fidelity prototypes, interviews,
increments with the participation of stakeholders. This feature scenarios and meetings with users. Recently, Brhel et al. [13] also
allows the participation of end-users in evaluations. conducted a systematic review of the literature on aspects of
As for the last evaluated End-Users-Driven Design attribute, in 5 integration of UCD in agile methodologies. The identified papers
out of the 7 analyzed methodologies (Scrum, XP, ASD, DSDM were analyzed using a coding system of four levels: process,
and CC) the design is in part based on understanding end-users, practices, people and technological dimensions. Thus, the most
tasks and environments. In 3 out of the 7 analyzed methodologies common practices identified were: prototypes, scenarios, usability
(Scrum, DSDM and CC) there exists the use of non-functional evaluation (expert), user objectives [14], usability testing (user)
prototypes and participation of various roles related (directly or and user stories, among others. Finally, Jia et al. [15] conducted a
partially) to end-users. This facilitates the inclusion of study with the aim of exploring how usability techniques were
environmental and end-user tasks aspects. However, the priority is integrated during the software development in Scrum projects. In
set on getting a usable version of the software. In this respect, it is this case, the most used usability techniques found in Scrum
important to stress that in Scrum the requirements are dynamically projects were: workshops, low fidelity prototypes, interviews and
managed by the Product Owner role, which has a profile oriented meetings with users.
to the end-user’s needs. Hence, this would lead the design based Summarizing all, Table 2 identifies correspondences between
on understanding end-users, tasks and environments. techniques, reported by [11,12,13], used to integrate UCD in agile
All in all, DSDM and Scrum are principally the two agile methodologies. In addition to this, the correspondence between
methodologies enabling a better adoption of user-centered the techniques mentioned above and the usability techniques used
activities. These methodologies have in common that both of them in the Scrum methodology [15] has been identified as well. These
involve end-users by considering specific roles and using correspondences allow to identify matches between usability

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 422 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
techniques used in the Scrum and techniques used to integrate end-user needs and context. Therefore, selected requirements
UCD in agile methodologies. should fit usability concerns and be prioritized accordingly. To
address these drawbacks, Budwig et al. [19] proposed the creation
As shown in Table 2, low fidelity prototyping, scenarios, heuristic
of a Product Backlog for issues related to UCD, which helps the
evaluation, usability testing, people, workshops and interviews are UCD team allocate corresponding resources to projects. Similarly,
the most frequent techniques found in the systematic review of the Singh [18] proposed to structurally maintain the Product Backlog
literature regarding UCD and agile integration. This consolidates but incorporating elements including greater awareness of
such techniques as mandatory when integrating aspects of usability, that is, a higher priority for the requirements that impact
usability in agile projects. Moreover, it is important to note that on usability, especially regarding potential acceptance criteria for
user stories and guidelines techniques are not mentioned in [15] requirements. Furthermore, Kuusinen [20] proposed that UCD
as common techniques in Scrum projects because these are mainly specialists were given more influential roles in regard to product
associated to XP methodology.
level decisions in order to improve managing the big picture and
Table 2. Summary, according to reviewed bibliography, understanding and fulfilling end-user needs.
describing correspondences between techniques used to Usability Evaluation Management: Lárusdóttir et al. [21]
integrate UCD in agile methodologies and usability techniques indicated that it is difficult to find a good moment for UCD
used in the Scrum methodology. evaluation in Scrum. On the one hand, a very early assessment in
Techniques Used to Integrate UCD in Usability the project is complicated, because the available features are still
Agile Techniques insignificant to have a UCD assessment. On the other hand, when
Used in characteristics are significant to be evaluated, it is difficult to
[11] [12] [13] make important changes because some parts of the product have
Scrum [15]
Low fidelity Low fidelity Prototyping Low fidelity already been delivered and there is not enough time to evaluate
prototyping prototyping prototyping before the next delivery. There have been different proposals
Scenarios Scenarios Scenarios Scenarios attempting to minimize this drawback. Among others, Felker et al.
Inspection Cognitive Usability Heuristic [17] proposed to schedule assessments before knowing what is
methods walkthrough evaluation evaluation going to be evaluated, analyze end-user feedback just after UCD
variants (expert) assessments, and carry out this at the end of Sprint. Lárusdóttir et
Users Usability Usability al. [22] suggested using informal ways to involve end-users in the
testing testing (user) evaluation evaluation and apply different methods to successfully perform
each of the user-centered assessments.
with users
Personas Personas Personas Completing the Contextual Inquiry Beforehand: The contextual
Workshops Focus groups Workshops inquiry is a method to inspect and understand end-users and their
Interviews Interviews Interviews workplace, tasks and preferences. Rannikko [23] recommended
Meetings Meetings the contextual inquiry to be completed before starting the software
with users with users development. Felker et al. [17] reported on a successful usage of
Contextual Field studies such guideline, so the authors noted that possessing the results of
inquiry a contextual inquiry was incredibly helpful and it allowed them to
User stories User stories focus on the design and implementation, helping establish initial
Guidelines Guidelines priorities.
Lastly, it can be stated that all these techniques have in common Close Collaboration: Lárusdóttir et al. [21] suggested that the
the ability to adapt to changing environments that require a rapid UCD specialists should work closely with developers in Scrum
response to the needs of end-users and business value. Teams. Moreover, Kuusinen [20] stated that it was necessary to
identify the right moment for the UCD specialists to work. These
3.2 Specific Recommendations to Integrate issues have been addressed in different ways, among these, it has
UCD in Scrum been suggested that the UCD should occur in parallel tracks to
In this section, the different practices and recommendations used implementation [24], set the UCD teams to work in one or two
to integrate UCD in the Scrum methodology have been collected Sprints ahead of the development teams [19] and design a Sprint
and analyzed. ahead of implementation [17,22].
Definition of "Done" for UCD-Related Tasks: Kniberg [16] Big Picture of the Project: The term big picture refers to a
highlighted the importance of the fact that Product Owner and holistic view of the whole project in Scrum. Lárusdóttir [21]
Development Team should agree on a clear definition of "Done". reported that the big picture of user experience is usually missing
This would facilitate obtaining a common understanding in Scrum projects. On the one hand, one of the reasons why this
regarding the scope and demands of the requirements presented happens is that programmers have the responsibility to deliver a
by the Product Owner, as well as the tasks that will be carried out small piece of software, but often they do not feel responsible for
by the Development Team. Felker et al. [17] proposed to use a the user experience or the entire system. On the other hand, it has
different definition of "Done" for UCD-related tasks. Authors been reported that the big picture of user experience is not present
suggested that this strategy would facilitate the monitoring of because the responsibility for particular activities of user
work in order to know the right moment to move on to the next experience in Scrum projects is often not clearly defined [21]. To
task. address these difficulties, Budwig et al. [19] proposed to quarterly
incorporate, throughout a common Sprint cycle, activities oriented
Product Backlog Management: Singh [18] noticed that one of the to update the big picture, in order to have a clear vision of the
key challenges to usability in the Scrum projects is the study of design to be carried out in the proyect and keep up the overall

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 423 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
coherence. Another proposal is to use overall quality goals to help Figure 2. Detail of Scrum-UIA.
deliver the overall design direction [8]. Finally, Lárusdóttir et al. The team is composed following the same traditional structure of
[21] indicated the need for strategic vision and user experience Scrum (a Product Owner, the Development Team, and a Scrum
objectives to be defined before starting the current project, that is, Master) but the Information Architect is incorporated as a primary
before the Sprint. However, the strategic vision should also be role in order to lead the contextual inquiry, support the Product
considered when defining what will take place in different Sprints Backlog management, promote the IA-driven incremental
[8]. Therefore, different authors recommend that a view of user development, ensure UCD and encourage end-user involvement.
experience must be considered before starting the implementation,
but also it needs to be applied during the iterations of the Scrum As shown in Figure 2, our proposal is based on establishing a
project. contextual inquiry as the starting point of the project with the aim
Assign Responsibility for End-User Concerns: The results of the of studying and analyzing the needs of end-users and prioritizing
work of Cajander et al. [8] showed that the responsibility for the the contents, in order for the gained knowledge to provide the
end-user’s perspective is not clear in Scrum projects, and end-user basis to set up a big picture of the project. Requirements included
perspective is often neither discussed nor described in the in the Product Backlog are improved by incorporating output from
projects. However, the end-user’s perspective is often present the contextual inquiry (end-user and content priorities). This is
through informal feedback used to understand the context of use carried out in coordination between the Product Owner and the
and report design. Cajander et al. [8] aimed to strengthen the Information Architect.
emphasis on the end-user’s perspective by clarifying and The Sprint planning is carried out after the requirements
explicitly communicating the responsibility of working through management through the Sprint Planning Meeting event. In the
usability. This includes both who will work with usability and Sprint planning the work to be done in each Sprint is defined.
who is responsible for the quality of the final product. However, Thus, the highest prioritized requirements of the Product Backlog
this proposal does not solve what could be done in the context of are selected for the Sprint Backlog. (This is explained in detail in
Scrum, where there are no formal responsibilities for any quality Section 4.1)
aspects, such as security, privacy and performance. Cajander et al.
[8] provided some examples of the organizational support needed: The Product Backlog’s items selected for this Sprint together with
sufficient mandate, support from the management, organizational the plan to carry them out made up the Sprint Backlog. The
competence as well as an adequate position in the team to be able Development Team splits requirements selected for the Sprint
to contribute to better usability. Backlog into specific development tasks, and those are developed
during the Sprint execution. Moreover, specific development tasks
Systematize the Process of End-User Inclusion: Cajander et al. can be associated to techniques for Agile IA development that are
[8] indicated that general agile processes do not support end-user auto-assigned by the Development Team.
participation. Rather, end-users are informally involved. Often
this is done on an ad hoc basis, and mostly based on personal Daily Scrum meetings are carried out during the Sprint execution
initiative and knowledge of the team members about the end- every day in order to inspect and adapt the daily work
user’s perspective rather than being systematically planned in the accomplished by the Development Team, and also to manage
Scrum process. Cajander et al. [8] suggested that it could be individual development tasks so that the visibility and consistency
useful to systematize the process through showing end-user with the big picture can be maintained. The Sprint Review
involvement and design feedback as general activities in the meeting takes place at the end of Sprint and is carried out
development process. involving both end-users and other secondary stakeholders, in
order to review the potential product Increment generated by the
4. PROPOSAL Development Team. Finally, the Scrum Team analyzes the
In order to address commented drawbacks taking also into working practices during the iteration and seeks improvement
consideration analyzed proposals to integrate the UCD in the opportunities through the Sprint Retrospective meeting. Overall,
Scrum methodology, an IA-driven approach is proposed with the our proposal is based on three essential components:
aim to integrate agility into the user-centered development process
of interactive software. The proposal, called Scrum-UIA (Scrum • Contextual inquiry-driven Product Backlog management.
driven by Usable Information Architecture), is based on the • Information architecture-driven Sprint planning.
Scrum methodology, and it includes roles, events, artifacts and • End-user-driven inspection and Continuous Improvement
associated rules [25], as well as a combination of practical and processes.
specific recommendations, as analyzed before in the literature, to These essential components are described below in detail.
integrate UCD in the Scrum process (see Figure 2).
4.1 Contextual Inquiry-Driven Product
Backlog Management
The objective of this component is twofold: first, to ensure that
the end-user’s perspective can be discussed, described and
considered (problem reported by [8]) throughout the Scrum
process. And the second is to provide a big picture of the project
to help obtain a global and inclusive vision of the product
regarding usability priorities, content, business value and end-
users.
In order to fulfill the objectives indicated above, we propose to
initiate the project with a contextual inquiry (see Figure 3), as

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 424 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
recommended in [23] and successfully implemented in [17], with models closer to the solution domain [28]. The results obtained
the aim of obtaining knowledge about the priorities of end-users are then used in the Sprint Planning Meeting.
and content. This simultaneously provides the basis to set up the
big picture of the project to support the Product Backlog 4.2 Information Architecture-Driven Sprint
management. Planning
Lárusdóttir et al. [21] reported that Scrum projects do not have a This component has three main objectives: the first is to ensure
big picture and, according to [26], this is perceived as a problem (during the Sprint Planning Meeting) that requirements
among UCD professionals. We propose to address this problem development is driven by IA priorities in an agile and user-
by specifying a low fidelity vision of the following artifacts: centered way. The second is to promote that the development is
content model, context model, site map and mockups. These performed incrementally through the different fidelity levels in
artifacts make it possible to report the design of contextual evolution with IA deliverables (blueprints, wireframes, content
navigation of a site, determine the critical content and visualize mapping, inventory and content models). And the third is to
relationships between pages and other content components. Thus, provide a common understanding through a clear definition of
these artifacts can be processed to obtain an overview of a site and “Done”, as recommended [17], regarding the scope and demands
facilitate discussion of the organization and content management, of the requirements set by the Product Owner (in collaboration
as well as access priorities desired by end-users [2,27]. with the Information Architect) and the tasks that are required to
be performed by the Development Team according to specific and
selected techniques.

Figure 3. Contextual inquiry-driven Product Backlog

Figure 4. Information Architecture-driven Sprint planning.
management.
As shown in Figure 4, the requirements of the Product Backlog
As a result, suggested artifacts can support the management of the
having the higher priority are selected to be included in the Sprint
project’s big picture by providing a clear view of the most critical
Backlog. Each of the selected requirements (which simultaneously
content elements of a site, and also facilitate discussion among the
contain elements regarding the priority of end-users, business
team members on the aspects of end-users and contents related to
value and content) is split into specific development tasks by the
business needs.
Development Team. Each requirement priority has an IA-driven
Therefore, it is proposed to develop the suggested artifacts before weighing provided by the Information Architect during the
starting the current project [21] during the contextual inquiry. requirements management, in order to specify IA priorities for the
Moreover, it is suggested to quarterly incorporate work to update incremental development.
the artifacts throughout a common Sprint cycle [19] and use
Table 3: Recommended techniques for agile IA development.
overall quality goals to support the global design management [8].
Furthermore, it is suggested to obtain and specify end-users Techniques IA Meth. A D E I M
acceptance criteria through interviews, Personas and meetings Affinity diagram [4,6] X
techniques that are usually used in Scrum projects [15]. Background investigation [2] X
Benchmarking [2,6] X X X
This way, we propose, similarly to Kuusinen’s suggestion [20], to Card Sorting [2,4,5,6] X X X X
provide the Information Architect with the facility for defining,
Consistency inspection [6] X
estimating and prioritizing the Product Backlog collaborating in
Consolidated evaluation [6] X X
coordination with the Product Owner. In this case, the Information
Diagramming [4] X
Architect provides the vision of IA and usability obtained from
Entity–relationship model [6] X
contextual inquiry, and the Product Owner provides business
requirements and prioritization. Feedback analysis [2,5,6] X X
Field studies* [6] X X
Finally, we propose, in a similar manner to [18], to maintain the Focus group discussion [2,6] X X
same structure of Product Backlog but incorporating elements to Goodness rating [5] X
include greater consideration on content priority, end-users Heuristic evaluation* [2,5,6] X X
acceptance criteria , business value priorities, as well as Interface design patterns [6] X X X
identification and association of requirements with the big Interviews* [2,3,6] X X
picture’s elements. In particular, the Information Architect Low fidelity prototyping* [6] X X
identifies, among Product Backlog items, high-level conceptual Meetings* [2,6] X X
representations of content that evolve towards the solution domain Mock-up prototype [6] X X
and facilitate the IA-driven incremental development – i.e., Participatory design [2,6] X X
content models created in the problem domain evolving towards Personas* [2,4,6] X X

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 425 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
Predictability and efficiency [5]
X
evaluation
Questionnaires* [4,5] X X
Scenarios* [2,6] X X
Speeded sentence verification [5] X
Sponsor-driven structure [5]
X
evaluation
Storyboards [6] X
Structure evaluation [6] X
Usability evaluation* [6] X
Survey [5] X X
Workshops* [2,6] X X Figure 5. End-user-driven inspection and continuous
A set of agile techniques for IA development (see Table 3) is improvement processes.
provided to support the development of specific tasks during the During the Sprint execution a “Done”, useable, easy-to-use and
Sprint execution. The agile techniques for IA development are potentially releasable product Increment is created. As shown in
auto-assigned by the Development Team, as well as the tasks to Figure 5, the Sprint execution is supported by a set of activities,
carry out for each of the specific development tasks. techniques and products for IA development in an agile and user-
centered way, as commented before.
We have obtained the agile techniques for IA development from a
second analysis of the five IA development methodologies Every day, the Daily Scrum takes place during the Sprint
[2,3,4,5,6]. The second analysis was performed in order to obtain execution. The Daily Scrum is done by inspecting the work since
a set of activities, techniques and products for IA development the last Daily Scrum and forecasting the work that needs to be
considering an agile and user-centered approach. In addition the done before the next one. Moreover, we encourage the
coincidences between the resulting techniques and the usability Information Architect participation in the Daily Scrum to review
techniques used in Scrum, reported by [15], are identified. individual tasks in order to preserve the big picture and identify,
In Table 3, the aforementioned set of agile techniques for IA from early stages, the fulfillment of the end-user’s acceptance
criteria.
development is presented, as well as the recommended phases
where they can be applied: analysis (A), design (D), evaluation A Sprint Review is carried out at the end of the Sprint execution
(E), implementation (I) and management (M). Furthermore, the in order to inspect the Increment and sort out the Product Backlog
techniques presented by [15] are pointed out with an asterisk (*) if needed. It is proposed to strengthen end-users involvement
in order to demonstrate the high correspondence with the Agile IA through an end-user-driven Sprint Review that, according to [8],
techniques. Also, each associated IA methodology is provided could be also beneficial. In this way, the end-user’s direct
denoting its corresponding bibliographical reference. involvement is materialized through formally assigning the
prerequisite that end-users must interact with the potential product
Finally, the Development Team performs the Sprint execution
Increment during Sprint Review. That is to say, the Development
according to the stated planning and the techniques to ensure
Team demonstrates the work that is “Done” through the end-
usability during the development, having in mind the big picture
user’s direct involvement interacting with the Increment.
of the project and a common understanding of the requirements.
Moreover, it is suggested to reflect on the results of these
4.3 End-User-Driven Inspection and ceremonies just after the evaluations [17] and apply different
methods to successfully perform each of the user-centered
Continuous Improvement Processes assessments [22].
The last issue of our proposal is aimed at enhancing traditional
inspection and continuous improvement processes in Scrum Finally, the Sprint Retrospective is held with the aim of reviewing
(Daily Scrum, Sprint Review and Sprint Retrospective) as a way aspects concerning the practices and Agile IA techniques used by
of encouraging UCD and end-users involvement. On the one the Scrum Team during Sprint execution, and also creating a plan
hand, individual tasks are inspected to keep up with the big for improvements to be enacted during the next Sprint. The
picture and track compliance of end-users acceptance criteria from Information Architect uses this event as an opportunity to review
the early stages during the Sprint (Daily Scrum). On the other the performance of the practices and techniques used in the
hand, the potential product Increment is evaluated through the development of the IA, in order to identify improvement
straight involvement of end-users and other secondary opportunities regarding responsiveness to the demands of agility
stakeholders (Sprint Review). And last but not least, all team and management of issues related to end-users. The idea is to
members reflect on the completed Sprint to find out what improve the process, incorporating ideas for next Sprints.
improvements could be made in the next one – i.e., process
improvement (Sprint Retrospective).
5. CONCLUSIONS
This paper presents an IA-driven approach for the agile
development of usable software, which considers the dynamic
demands of the environments where organizations operate, as well
as the needs of end-users. Concretely our approach, named
Scrum-UIA, aims at integrating agility and UCD by contributing
the following issues:
• The end-user’s perspective is discussed, described and
considered during the Scrum process through a Contextual
Inquiry-Driven Product Backlog Management.
XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction
| 426 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
• Development of requirements is driven by IA priorities in an (Doctoral dissertation, University of Regensburg, Germany),
agile and user-centered way through an Information 2005.
Architecture-Driven Sprint Planning. Furthermore, a set of agile [7] Salah, D., Paige, R. F., and Cairns, P. A systematic literature
techniques for IA development is provided to support the review for agile development processes and user centred
development of specific tasks during the Sprint execution. design integration. In Proceedings of the 18th International
• Compliance with the end-user’s acceptance criteria is checked Conference on Evaluation and Assessment in Software
from the early stages of the Sprint, and the potential product Engineering (May 2014), p. 5.
Increments are evaluated through an End-User-Driven
Inspection and Continuous Improvement Processes. [8] Cajander, Å., Larusdottir, M., and Gulliksen, J. Existing but
not explicit-The user perspective in scrum projects in
In addition, this paper presents different contributions involving practice. In Human-Computer Interaction–INTERACT 2013
analysis and discussion on the following issues: (2013), 762-779.
• A comparative analysis checking the current features of the [9] Erlin, E., Yunus, Y., and Abdul Rahman, A. The evolution of
methodologies for the IA development shows that none of them information architecture. Institute of Electrical and
present flexibility and adaptability characteristics to respond in Electronics Engineers (2008).
an agile and flexible way to changing environments, and require
considerable effort to be adapted. [10] DIS, I. 9241-210: 2010. Ergonomics of human system
interaction-Part 210: Human-centred design for interactive
• A analysis to know the feasibility of integrating user-centered
systems. International Organization for Standardization
activities into agile methodologies, showing that the Scrum
(ISO), Switzerland. (2009).
methodology presents a framework that is easy to implement,
providing flexibility and adaptation to end-user and the business [11] Da Silva, T. S., Martin, A., Maurer, F., and Silveira, M. S.
requirements. User-Centered Design and Agile Methods: A Systematic
• A analysis of the practices that are commonly used to integrate Review. In AGILE (2011), 77-86.
UCD in agile methodologies, showing that low fidelity [12] Jurca, G., Hellmann, T. D., and Maurer, F. Integrating Agile
prototyping, scenarios, heuristic evaluation, usability testing, and User-Centered Design: A Systematic Mapping and
people, workshops and interviews are the most frequent Review of Evaluation and Validation Studies of Agile-UX.
techniques used in the UCD and agile integration. In Agile Conference (AGILE) (July 2014), 24-32.
As future work, we expect to build an easy-to-use CASE tool, [13] Brhel, M., Meth, H., Maedche, A., and Werder, K. Exploring
which can be used to implement Scrum-UIA, in order to Principles of User-Centered Agile Software Development: A
automatically provide scheduling, recommendation, activities and Literature Review. Information and Software Technology
techniques to carry out an agile UCD-IA development, also (2015).
dealing with end-user demands, priorities, usability, business
value and content during the agile development process. In [14] Losada, B., Urretavizcaya, M., and Fernández-Castro, I. A
addition, we expect to apply the proposal to a real case and thus guide to agile development of interactive software with a
obtain improvement clues. “User Objectives”-driven methodology. Science of Computer
Programming, 78, 11 (2013), 2268–2281.
6. ACKNOWLEDGMENTS [15] Jia, Y., Larusdottir, M. K., and Cajander, Å. The usage of
This work has been supported by the funding projects «Go-Lite» usability techniques in Scrum projects. In Human-Centered
(TIN2011-24139) and «Flexor» (TIN2014-52129-R) granted by Software Engineering (2012), 331-341.
the Spanish Government, and «eMadrid» (S2013/ICE-2715)
[16] Kniberg, H. Scrum and XP from the Trenches. Lulu (2007).
granted by the Madrid Research Council.
[17] Felker, C., Slamova, R., and Davis, J. Integrating UX with
7. REFERENCES scrum in an undergraduate software development project. In
[1] Martin, A., Dmitriev, D., and Akeroyd, J. a resurgence of Proceedings of the 43rd ACM technical symposium on
interest in information architecture. International journal of Computer Science Education (2012), 301-306.
information management, 30, 1 (2010), 6-12. [18] Singh, M. U-SCRUM: An agile methodology for promoting
[2] Morville, Peter and Rosenfeld, Louis. Information usability. In Agile, 2008. AGILE'08 (August 2008), 555-560.
architecture for the world wide web. O’Reilly, 2006. [19] Budwig, M., Jeong, S., and Kelkar, K. When user experience
[3] Sharlin, Micole, Tu, Evelyn, and Bartus, Thomas. Guide to met agile: a case study. In CHI'09 Extended Abstracts on
Creating Website Information Architecture and Content. Human Factors in Computing Systems (April 2009), 3075-
Princeton University (2009). 3084.
[4] Lamar, L. Introduction to a user interface design/information [20] Kuusinen, K. Improving UX Work in Scrum Development:
architecture process for Web sites. In Professional A Three-Year Follow-Up Study in a Company. In Human-
Communication Conference (2001), 185-197. Centered Software Engineering (2014), 259-266.
[5] Toub, Steve. Evaluating information architecture: a practical [21] Lárusdóttir, M. K., Cajander, Å., and Gulliksen, J. The big
guide to assessing web site organization. Argus Center for picture of UX is missing in Scrum projects. In Proc. I-
Information Architecture (2000). UxSED (2012).
[6] Reichenauer, A. LUCIA: Development of a comprehensive [22] Lárusdóttir, M., Cajander, Å., and Gulliksen, J. Informal
information architecture process model for websites. feedback rather than performance measurements–user-

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 427 |
 An Agile Information-Architecture-Driven Approach for the Development of User-Centered Interactive Software
L. A. Rojas | J. A. Macías
centred evaluation in Scrum projects. Behaviour & [26] da Silva, T. S. A framework for integrating interaction
Information Technology, 33, 11 (2014), 1118-1135. design and agile methods (Doctoral dissertation, Pontifical
[23] Rannikko, P. User-centered design in agile software Catholic University of Rio Grande do Sul). 2012.
development (2011). [27] Brown, D. M. Communicating design: developing web site
[24] Sy, D. Adapting usability investigations for agile user- documentation for design and planning. New Riders, 2010.
centered design. Journal of usability Studies, 3 (Feb. 2007), [28] Rojas, L. A. and Macías, J. A. Bridging the gap between
112-132. information architecture analysis and software engineering in
[25] Schwaber, K. and Sutherland, J. The Scrum guide. Scrum interactive web application development. Science of
Alliance (2011). Computer Programming, 78, 11 (2013), 2282-2291.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 428 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero

Hacia la Caracterización de la Calidad de Interacción

Cristina Roda Víctor López-Jaquero Francisco Montero
Grupo de Investigación LoUISE Grupo de Investigación LoUISE Grupo de Investigación LoUISE
Dpto. de Sistemas Informáticos Dpto. de Sistemas Informáticos Dpto. de Sistemas Informáticos
Universidad of Castilla-La Mancha Universidad of Castilla-La Mancha Universidad of Castilla-La Mancha
02071, Albacete, España 02071, Albacete, España 02071, Albacete, España
[email protected] [email protected] [email protected]

ABSTRACT que permita desarrollar este tipo de aplicaciones sensibles al

La Calidad de Interacción es un factor muy a tener en cuenta en contexto: Adaptación.
cualquier tipo de aplicación y, por supuesto, el diseño de una La adaptación al contexto de uso de interacción que nos rodea se
adaptación a nivel de interfaz de usuario de una aplicación ha convertido en una rutina diaria cuando interactuamos con
concreta, también debería considerar aspectos relativos a la nuestros dispositivos, por ejemplo cuando rotamos el dispositivo,
calidad. Descuidar la evaluación de la calidad puede producir la interfaz de usuario se reestructura de nuevo, o cuando un
fácilmente aplicaciones que no cumplan con las expectativas de determinado widget nos informa de la previsión meteorológica del
usuarios y desarrolladores. En este sentido, trabajos previos han lugar en el que nos encontremos, gracias a la geolocalización.
guiado el camino a seguir hacia la consideración de la calidad en
un marco de adaptación de interfaces de usuario. Concretamente, En la literatura clásica sobre adaptación, se describen dos tipos
se ha ilustrado cómo se debe contemplar la Calidad de Adaptación principales de aplicaciones [1]:
en cada etapa del framework ISATINE, analizando las múltiples
• Aplicación Adaptable, cuando el usuario es el actor encargado
facetas de este concepto, y proponiendo su descomposición en
de realizar la adaptación de la interfaz de usuario.
cuatro tipos de calidad de interacción: Calidad Esperada de la
• Aplicación Adaptativa, cuando la propia aplicación es la que
adaptación, Calidad Deseada de la adaptación, Calidad
realiza la adaptación automáticamente.
Alcanzada de la adaptación, y Calidad Percibida de la adaptación.
Además, se han descrito qué tipos de calidad de adaptación están De esta forma, existen diversos marcos de adaptación que guían al
involucrados en cada etapa de ISATINE. Pero, ¿cuál es la idea diseñador en el proceso de crear experiencias de adaptación
que tienen, tanto usuarios como desarrolladores, sobre calidad de agradables. El proceso de adaptación más ampliamente aceptado
interacción? ¿Utilizan los mismos términos para referirse a ella? es el propuesto por Dieterich en 1993 [4], aunque, como veremos
Estas son las preguntas que guían este trabajo, el cual presenta un más adelante, este marco presenta algunas deficiencias. Para
estudio sobre la caracterización de la Calidad de Interacción, solucionar estas limitaciones, se propone el framework ISATINE
concretamente sobre la Calidad Esperada y la Deseada, con el [17]. Estos y otros marcos de adaptación se describen con más
objetivo de entender mejor las preferencias de calidad de los detalle en la sección 2.
usuarios que permita a los desarrolladores diseñar productos
software mucho más atractivos. Obviamente, estos marcos de adaptación y, en general, el proceso
de adaptación, deben proporcionar cierto nivel de calidad de
Categorías interacción, entendida como [10] la capacidad de un producto
H.5.2 [Interfaces de Información y Presentación]: Interfaces de software de satisfacer necesidades expresadas o implícitas cuando
Usuario – evaluación/metodología, diseño de pantalla, diseño se utiliza bajo condiciones específicas. Esta característica de
centrado en el usuario. calidad garantiza una adaptación apropiada y precisa para una
situación dada. Del mismo modo que se desarrolla una interfaz de
Términos Generales usuario (y una aplicación software) para lograr cierto grado de
Diseño, Factores Humanos, Teoría. calidad de interacción, también debe ocurrir lo mismo con las
capacidades de adaptación de la aplicación. Este último tipo de
Keywords calidad está relacionado con la Calidad de Adaptación (Quality of
Interfaz de Usuario, Calidad de Interacción, Proceso de Adaptation, QoA) y se refiere a la medida (o incremento) en que
Adaptación, Calidad de Adaptación, ISATINE, Estudio. un conjunto de adaptaciones produce una interfaz de usuario para
alcanzar objetivos específicos con usabilidad en un contexto de
1. INTRODUCCIÓN uso concreto [17].
En los últimos años, hemos sido testigos del creciente uso de Sin embargo, como ya se ha visto en trabajos previos [22],
aplicaciones que facilitan las tareas habituales al usuario, como actualmente la calidad no se considera de forma apropiada en los
comprar billetes de avión o entradas para un espectáculo, o la procesos de adaptación y, por tanto, no siempre se proporciona
variedad de personas que utilizan una misma aplicación. Estos son una buena experiencia de adaptación al usuario. De esta forma, se
sólo algunos ejemplos que ilustran la necesidad de crear ha propuesto integrar la QoA en el marco ISATINE con el
aplicaciones software adecuadas para cualquier tipo de usuario, objetivo de tener en cuenta la calidad de interacción a través de
cualquier tipo de plataforma, y capaces de trabajar en cualquier todas las etapas del marco de adaptación. Esta consideración de la
situación. Desafortunadamente, es imposible diseñar y prever calidad ayudará en la mejora de la experiencia de adaptación en
cada una de las situaciones que puedan surgir durante la cada paso del proceso de adaptación, y no sólo después de la
interacción, por lo que se hace necesario introducir un mecanismo ejecución de la adaptación, como ocurre habitualmente.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 429 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
En este sentido, este artículo presenta un resumen del trabajo propuesto por Oppermann en 1994 [19] quien, al igual que Lorenz
previo realizado en aras de considerar la integración completa de (2000), dividió el proceso de adaptación en tres fases: aferencial,
la QoA en ISATINE, para lo cual se describen los distintos tipos inferencial y eferencial, es decir, la recogida de la información de
de calidad de interacción y su relación con cada etapa de contexto (principalmente interacción del usuario), el
ISATINE. Este resumen previo permitirá contextualizar mejor el procesamiento e inferencia de dicha información, y la
estudio presentado posteriormente. implementación y presentación de la adaptación al usuario final,
La justificación de por qué es necesario un estudio para respectivamente. Además, este autor destaca la importancia de
caracterizar la Calidad de Interacción es bien sencilla: entender definir el momento preciso para comenzar y evaluar la adaptación.
mejor las preferencias de calidad de los usuarios, es decir, qué Sin embargo, como se mencionó anteriormente, en el contexto de
factores o atributos son los más importantes para ellos a la hora de adaptación, el proceso de adaptación más ampliamente aceptado
determinar si un producto software tiene suficiente calidad o no. es el que propuso Dieterich en 1993 [4]. Concretamente, los
Así, disponer de este conocimiento permitirá a los desarrolladores cuatro pasos que conforman este proceso son:
diseñar adecuadamente dichos productos software de acuerdo a
las preferencias y necesidades de los usuarios. 1. Iniciativa. Una de las entidades involucradas en la
interacción (usualmente el usuario o el sistema) sugiere su
Este trabajo se organiza de la siguiente forma: la sección 2 intención de realizar una adaptación. Por ejemplo, cuando
presenta distintos procesos de adaptación propuestos a lo largo de estamos utilizando Gmail, éste es capaz de detectar si la
los años y la necesidad de un nuevo proceso de adaptación como conexión a Internet es lenta y sugerir una vista mucho más
ISATINE para abordar las deficiencias encontradas en dichos básica.
procesos; la sección 3 presenta los conceptos de calidad y Calidad 2. Propuesta. Si la aplicación necesita una adaptación, se deben
de Interacción, sus tipos y las relaciones entre ellos, así como el proponer diversas adaptaciones que se puedan aplicar con
concepto de Calidad de Adaptación y su inclusión en ISATINE. éxito en el contexto de uso actual. En el ejemplo anterior,
La sección 4 presenta un estudio sobre la primera brecha surgida Gmail sugiere cambiar desde su interfaz habitual a un modo
al caracterizar la calidad de interacción y una discusión al respecto HTML básico, mitigando la carga y disminuyendo el tiempo
(sección 5). Finalmente, la sección 6 presenta las conclusiones y de respuesta. Esta propuesta se muestra al usuario por medio
trabajo futuro. de un mensaje de texto que también es un hiperenlace:
“Cargar vista básica en HTML (para conexiones lentas)”.
2. CONTEXTUALIZANDO EL ESTUDIO 3. Decisión. Como puede ser que existan distintas propuestas de
Como primer paso para contextualizar este trabajo, a continuación
adaptación derivadas de la etapa anterior, se debe seleccionar
se describen distintos procesos encaminados a guiar un proceso de
aquella que mejor se ajuste a la necesidad de adaptación
adaptación.
detectada. En el ejemplo, el usuario es responsable de
2.1 Procesos de Adaptación cambiar o no el modo de Gmail, haciendo clic en el enlace
Una adaptación en sí no tiene por qué ser adecuada y puede, por correspondiente.
tanto, causar el rechazo del sistema adaptativo por parte del 4. Ejecución. El último paso consiste en la ejecución de la
usuario. Por ejemplo, el brillo de la pantalla de un teléfono móvil adaptación previamente elegida. En el ejemplo, si el usuario
se intensifica automáticamente debido a las condiciones de selecciona el modo HTML básico, Gmail proporcionará una
iluminación actuales, lo que conlleva un incremento del consumo vista mucho más simple, eliminando algunas de las
de batería. Quizá esta adaptación de IU puede ser rechazada por el características del modo estándar, pero soportando
usuario si su prioridad es ahorrar energía porque el dispositivo funcionalidades básicas como lectura o escritura de correos.
tiene poca carga de batería.
2.2 Evaluación de la Adaptación por Etapas
Por consiguiente, se hace necesario un proceso que guíe la
Además de estos trabajos que muestran la importancia de utilizar
adaptación para que ésta sea apropiada, entendiendo como
un proceso estructurado que guíe la adaptación de una IU
proceso [8] una secuencia de tareas que, cuando se ejecuten
concreta, es interesante también destacar otros trabajos que
correctamente, producirán el resultado deseado. Más
contribuyen de algún modo a la idea de evaluar un proceso de
formalmente, el estándar ISO 9000 [11] define un proceso como
adaptación con el fin de saber más acerca de su calidad. Por
un conjunto de actividades interrelacionadas o interactivas que
ejemplo, Paramythis et al. [20] están de acuerdo con nosotros en
transforman entradas en salidas. Algunos de los beneficios de
cuanto a realizar una evaluación distinta para cada aspecto que
utilizar un enfoque basado en procesos son [11] la habilidad de
conforma un proceso de adaptación. Proponen un marco para
centrarse en la efectividad y la eficiencia del proceso; resultados
sistemas adaptativos que guía una evaluación por capas de cada
mejorados, consistentes y previsibles; o fomentar la participación
fase del proceso de adaptación, así como de todo el proceso en su
activa de los individuos y dejar claras sus responsabilidades
conjunto, y presentan algunos criterios de evaluación (usabilidad,
dentro del proceso.
validez, privacidad, etc.) y métodos (minería de datos, grupos
A lo largo de los años, se han propuesto diferentes marcos o foco, test de usuario, etc.) para evaluar cada etapa por separado.
procesos de adaptación que persiguen el objetivo de guiar a los En este sentido, las principales capas de adaptación a ser
diseñadores en la definición de las capacidades de adaptación. Por evaluadas serían: Recogida de datos de entrada, Interpretación de
ejemplo, Cockton en 1987 [2] propuso un proceso de adaptación los datos recogidos, Modelado del estado actual del mundo,
de dos etapas. La primera de ellas es el diagnóstico, en la que el Decidir sobre la adaptación, y finalmente Aplicar la adaptación.
sistema imagina las necesidades y habilidades del usuario El trabajo presentado por Paramythis et al. es similar a la idea que
basándose en información registrada. Seguidamente, en la proponemos en [22], aunque existen ciertas diferencias. Nuestro
segunda etapa, se aplica el tratamiento encontrado para remediar punto de vista es mucho más formal ya que está centrado en la
la situación detectada. Otro proceso de adaptación fue el calidad, dando una definición explícita de Calidad de Adaptación

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 430 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
y haciendo que este concepto sea el principal foco cuando se la necesidad de cubrir el resto de etapas no tenidas en cuenta. Esta
evalúa un proceso de adaptación. Por otra parte, estos autores no idea nos permite expandir la taxonomía de Dieterich, ya que es la
cubren todo el proceso de adaptación como hacemos nosotros con más extendida, dando lugar al marco de adaptación ISATINE [17]
ISATINE, ya que su framework se centra únicamente en la que tiene en cuenta todas las Etapas de Acción definidas en el
ejecución de la adaptación, como también ocurre en la propuesta modelo de Norman.
de Dieterich. Además, nuestra propuesta se basa en un proceso de Por otro lado, el proceso de Dieterich muestra otras deficiencias,
adaptación previamente estudiado y consolidado como es considerando la más relevante para nosotros que la QoA no está
ISATINE, el cual se describe con más detalle en la siguiente contemplada en absoluto, es decir, su proceso de adaptación no
sección. permite evaluar la adaptación para saber si ha sido apropiada o no,
Otro trabajo relevante es el presentado por Gena y Weibelzahl [6] ya que sólo se centra en la ejecución de la misma. De esta forma,
que propone un enfoque de ingeniería de usabilidad para diseñar y el trabajo presentado en [22] mejora el marco de ISATINE con la
evaluar sistemas basados en web adaptativa. En este sentido, estos integración de la QoA, solucionando estos fallos detectados en el
autores proponen varias técnicas de evaluación de acuerdo a la proceso de Dieterich. Nótese que ISATINE considera tres tipos de
etapa del ciclo de vida en la que se encuentre, es decir, fase de actores involucrados en el proceso de adaptación: el Usuario (U),
requisitos, fase de evaluación preliminar, o fase de evaluación el Sistema (S) y un Tercero (T), que puede ser, por ejemplo, un
final. Además, para cada técnica de evaluación, se especifica la agente externo encargado de actualizar el sistema o un sistema
dimensión importancia para la web adaptativa con el objetivo de externo que interactúa con el nuestro.
ayudar a los investigadores a elegir la mejor opción para una A continuación, se describen las siete etapas de ISATINE, como
situación dada, indicando de qué manera dicha técnica puede resultado de la especialización de las siete etapas de acción del
beneficiar a los sistemas basados en web adaptativa. Estos autores modelo de Norman:
proponen diferentes métodos para dar soporte a la evaluación de
un sistema adaptativo, pero centrándose sólo en la evaluación de 1. Objetivos de la adaptación de la interfaz de usuario. Cualquier
sistemas basados en web. Nuestro enfoque, por el contrario, es entidad (U, S, o T) puede ser la responsable de definir y
mucho más genérico y flexible y puede ser aplicado a cualquier mantener actualizados una serie de objetivos que garanticen la
tipo de sistema. Asimismo, Gena y Weibelzahl proponen métodos adaptación de la interfaz de usuario. Aunque la adaptación
para evaluar la adaptación en cada etapa del ciclo de vida, frente a debe ser siempre en beneficio del usuario, puede ser realizada
nuestra idea de evaluar cada etapa dentro de un proceso de con respecto a cualquier aspecto del contexto de uso, es decir,
adaptación como ISATINE. Como se mencionó anteriormente, la con respecto al propio usuario, la plataforma o el entorno físico
siguiente sección presenta una visión general del marco de en que el usuario está llevando a cabo sus tareas. Los objetivos
adaptación ISATINE. pueden ser auto-expresados, expresados por la máquina, local
o remotamente, dependiendo de su localización: en la mente
2.3 ISATINE del usuario (U), en el sistema local (S), o en un sistema remoto
Como se ha comentado con anterioridad, se propuso un nuevo
(T). Continuando con el ejemplo de Gmail, esta aplicación es
framework de adaptación, llamado ISATINE [17], que especializa
responsable de mantener un cierto nivel de funcionalidad
el modelo mental de Norman de interacción de usuario [18]. Este
dentro de su plataforma (S), como algunas características
modelo mental descompone cualquier interacción de usuario en
básicas de lectura y composición de correos a pesar de, por
siete Etapas de Acción:
ejemplo, problemas de conexión como conexiones a Internet
1. Formación del Objetivo, el usuario da forma a un objetivo en lentas.
su cabeza. 2. Iniciativa para la adaptación. Esta etapa se refina a su vez en
2. Formación de la Intención, para alcanzar el objetivo, el usuario formulación de la petición de adaptación, detección de la
forma alguna intención. necesidad de adaptación, y notificación de petición de
3. Especificación de una Acción, la intención se transforma en adaptación, dependiendo de su localización, respectivamente
una serie de acciones. U, S o T. Por ejemplo, T puede ser el responsable de iniciar
4. Ejecución de una Acción, una acción en un momento una adaptación cuando una actualización de la IU está
determinado es seleccionada y ejecutada. disponible o cuando se produce un cambio en el contexto que
5. Percepción del Estado del Mundo, después de que la acción no puede ser detectado por el propio sistema, como un evento
haya sido ejecutada, se perciben los resultados producidos por externo. En el ejemplo, la especificación de esta etapa es muy
la misma. similar a la etapa Iniciativa del proceso de adaptación de
6. Interpretación del Estado del Mundo, los resultados percibidos Dieterich, es decir, cuando utilizamos Gmail, éste puede
generan una interpretación en la mente del usuario sobre cómo detectar si la conexión a Internet es lenta y sugerir una vista
ha cambiado el Mundo. básica para su IU.
7. Evaluación de la Salida, dependiendo de esta interpretación, el 3. eSpecificación de la adaptación. Esta etapa puede ser a su vez
usuario evalúa si la acción que ejecutó coincide con su objetivo descompuesta en especificación por demostración, por
inicial. computación, o por definición, dependiendo de la entidad que
produce la adaptación, respectivamente U, S o T. Cuando un
En este punto, si intentamos hacer corresponder las cuatro etapas usuario desea adaptar la interfaz, debe ser capaz de poder
propuestas por Dieterich con el modelo de Norman, se puede ver especificar las acciones necesarias para hacer dicha adaptación,
que la Iniciativa corresponde con la Intención, la Propuesta y la como por ejemplo programando por demostración o definiendo
Decisión son dos etapas involucradas en la Especificación de una las operaciones necesarias. Cuando es el sistema el responsable
Acción, y las etapas de Ejecución de ambas propuestas coinciden. de esta etapa, debe ser capaz de calcular una o varias
Por tanto, es fácil ver que el proceso de adaptación de Dieterich propuestas de adaptación dependiendo de la información del
sólo cubre algunas de las etapas del modelo de Norman, creando contexto disponible. Cuando es un tercero el que especifica la

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 431 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
adaptación, puede enviar al sistema directamente una 3. UN VISTAZO A LA CALIDAD
definición de las operaciones que deben ser aplicadas. En el Una vez se ha presentado el marco de adaptación ISATINE, es
ejemplo actual, Gmail (S) es responsable de especificar la momento para definir y describir más a fondo y formalmente el
adaptación, ya que es él el que detecta la conexión lenta, concepto de calidad (sección 3.1), concretamente en el contexto
sugiriendo por tanto un cambio en la vista de la aplicación, de la interacción persona-ordenador, así como sus tipos (sección
desde su interfaz estándar a un modo HTML básico. 3.2), y más específicamente en el ámbito de la adaptación de una
4. Aplicación de la adaptación. Esta etapa indica qué entidad interfaz de usuario (sección 3.3). Esto permitirá entender mejor lo
aplicará la adaptación especificada en la etapa anterior. Como que significa dotar de calidad de interacción a un proceso de
la adaptación se aplica siempre sobre la interfaz de usuario, adaptación como ISATINE (sección 3.4), es decir, entender lo que
dicha interfaz debe facilitar siempre un mecanismo que la es realmente la QoA.
soporte. Si el U desea aplicar la adaptación, por ejemplo, a
través de opciones o personalización, debe ser también posible 3.1 Calidad y Calidad de Interacción
a través de algún mecanismo de la interfaz de usuario. En el El término calidad es bastante ambiguo en su definición, debido
ejemplo, el sistema es responsable de cambiar el modo de principalmente a su uso habitual en múltiples contextos y, por
Gmail, después de que el usuario decida hacerlo en la etapa tanto, posee diferentes interpretaciones y definiciones, como las
anterior, por tanto aquí, S es la entidad que aplica la propuestas en [16] y [3]. Éste último define la calidad como “el
adaptación. importantísimo catalizador que marca la diferencia entre el éxito y
5. Transición a la adaptación. En esta etapa se describe qué el fracaso”. Una definición similar fue propuesta en [21].
entidad garantizará una transición suave desde la interfaz de De manera más formal, la IEEE en su estándar 610.12 [9] define
usuario original a la adaptada. Por ejemplo, si S es el la calidad como “el grado en que un sistema, componente o
responsable de esta etapa, puede proporcionar algunas técnicas proceso satisface los requisitos especificados y las necesidades o
de visualización que mostrarán los pasos ejecutados durante la expectativas del cliente o usuario”. En la familia de estándares
transición, por ejemplo, a través de animación, morphing o ISO 9000 [10], también podemos encontrar otra definición de
visualización progresiva [23]. En el ejemplo, Gmail es calidad: “capacidad de un producto software para satisfacer
responsable de asegurar una transición suave entre la interfaz necesidades explícitas o implícitas cuando se utiliza bajo unas
estándar y la básica, una vez que se ha hecho clic en el enlace condiciones determinadas”.
para cambiar la vista.
6. INterpretación de la adaptación. Esta etapa especifica qué Sin embargo, la evaluación, el diseño y, especialmente, los
entidad producirá información significativa suficiente para procesos de adaptación requieren una definición de lo que
facilitar la compresión de la adaptación al resto de entidades. determina si un producto es bueno o malo desde la perspectiva del
Normalmente, cuando S realiza una adaptación sin usuario final y, por tanto, se requiere una definición de Calidad de
proporcionar una explicación, U no comprende necesariamente Interacción. Así, el término Usabilidad constituye una definición
por qué se ha realizado la adaptación. Recíprocamente, cuando para calidad de interacción ampliamente aceptada hoy día. El
U realiza una adaptación, debe indicar al sistema cómo estándar ISO 9241-11 [12] define la usabilidad como “el grado en
interpretarla. Siguiendo el ejemplo, una vez que la adaptación que un producto puede ser utilizado por usuarios concretos para
se ha aplicado y el usuario está visualizando la vista básica en alcanzar objetivos concretos con efectividad, eficiencia y
HTML de Gmail, se muestra un mensaje en la parte de arriba satisfacción, en un contexto de uso específico”. Más
de la pantalla que dice “En estos momentos estás utilizando la recientemente, el estándar ISO/IEC 25010:2011 [13] descompone
vista Gmail en modo HTML básico”. El usuario puede también la calidad en calidad del producto software y calidad de
pinchar sobre un enlace para descubrir el porqué del cambio. interacción. Ambos modelos de calidad pueden utilizarse para dar
De esta forma, el usuario está al tanto de por qué ha cambiado soporte a la especificación y evaluación del software desde
la IU. diferentes perspectivas [14][13][15], asociadas a la adquisición,
7. Evaluación de la adaptación. Esta etapa especifica qué entidad los requisitos, el desarrollo, el uso, la evaluación, el soporte, el
tiene la responsabilidad de evaluar la calidad de la adaptación mantenimiento, la garantía de calidad y la auditoría de software.
realizada, de forma que sea posible comprobar si se cumplen Así, en este último estándar internacional, la Calidad de
los objetivos inicialmente especificados. Por ejemplo, si S Interacción se define como “la calidad en uso” de un sistema, es
mantiene una planificación interna de los objetivos, deber ser decir, el impacto que dicho producto software tiene sobre los
capaz de actualizar su planificación de acuerdo a las usuarios.
adaptaciones aplicadas hasta el momento. Si los objetivos están Otro término reciente y alternativo al de usabilidad y que también
en la mente del usuario, pueden ser también evaluados en es muy utilizado por diseñadores y desarrolladores de interfaces
función de lo acontecido en las etapas anteriores. En este caso, de usuario es el de Experiencia de Usuario (User eXperience,
la explicación de la adaptación realizada contribuye además a UX). Este concepto está ligado a las percepciones y respuestas del
la actualización de los objetivos. También sería posible la usuario cuando utiliza un producto, sistema o servicio (ISO/DIS
colaboración entre S y U para este propósito. En el ejemplo, 9241-210:2008).
Gmail es responsable de evaluar la adaptación de su modo
visual, dando la opción al usuario de volver atrás a la vista 3.2 Tipos de Calidad de Interacción
estándar (el usuario rechaza la adaptación) o dejando la vista Como se ha visto hasta el momento, la calidad y, concretamente,
básica en HTML como vista por defecto (el usuario aprueba la la calidad de interacción dependen de cada usuario y sus
adaptación). objetivos. Por ejemplo (adaptado del ISO/IEC 9126-1:2001):
• Un usuario puede identificar la calidad con la idoneidad de un
producto software, utilizando criterios asociados a la
usabilidad y la calidad en uso. En este contexto, los usuarios

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 432 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
con discapacidad pueden asociar la calidad con la accesibilidad calidad de interacción del producto después de su entrega, es
de su software. decir, la percepción de la calidad que tiene el
• Un adquiridor reconoce la calidad de un producto software por usuario/adquiridor final. La usabilidad, calidad en uso,
medio de medidas externas de funcionalidad, fiabilidad, funcionalidad o experiencia de usuario son conceptos
usabilidad y eficiencia, o de calidad en uso. asociados a este tipo de calidad.
• Un mantenedor generalmente establece una correspondencia Finalmente, se puede concluir que los tipos de calidad anteriores
entre calidad y mantenibilidad. están relacionados entre sí. Concretamente, la calidad Esperada y
• Una persona responsable de la implementación del software la Deseada son complementarias. Así, un producto software será
en diferentes entornos y plataformas, asocia la calidad con la de calidad cuando la calidad Esperada y la Deseada estén
portabilidad. altamente solapadas, es decir, su percepción sea similar. De esta
• Un desarrollador puede identificar la calidad de un producto forma, este elevado nivel de calidad de interacción será percibido
software por medio de medidas internas de cualquier por los usuarios cuando la calidad Deseada y la Alcanzada
característica de la calidad. (competencia de los desarrolladores y diseñadores) se centren en
• Un evaluador puede asociar la calidad de un producto software la consecución de las necesidades del usuario. En este sentido,
con la productividad, la satisfacción y la efectividad de dicho resulta de interés realizar un primer estudio (sección 4) que
producto. muestre si existen diferencias significativas entre la calidad
Además de estos seis puntos de vista de la calidad de interacción, Esperada y la Deseada en cuanto a los términos empleados para su
podemos preguntarnos cómo podemos describirla. El trabajo caracterización, ya que sospechamos que puede existir un brecha
propuesto en [5], sugiere describir la calidad de interacción desde conceptual muy clara entre la percepción de usuarios con y sin
cuatro perspectivas diferentes: experiencia en calidad de interacción.
• Calidad Esperada, la calidad de interacción que el cliente o Los seis puntos de vista de la calidad de interacción pueden
usuario necesita, es decir, los elementos o requisitos necesarios reagruparse a su vez en dos puntos de vista más genéricos:
en la especificación del sistema. Usuario – Interacción, y Desarrollador – Adaptación. El primero
• Calidad Deseada, el grado de calidad que el experto en calidad de ellos incluye a los puntos de vista del usuario y el adquiridor,
de interacción quiere conseguir para la versión final del mientras que el segundo incluye a los puntos de vista del
sistema. Este tipo de calidad se deriva de la Calidad Esperada, mantenedor, persona responsable de la implementación,
ya que está relacionada también con la especificación. desarrollador y evaluador. La Tabla 1 muestra las relaciones
entre estos dos puntos de vista más genéricos y los distintos tipos
• Calidad Alcanzada, la calidad obtenida a partir de una
de calidad de interacción. En este sentido, la calidad Percibida y la
implementación dada del sistema. Idealmente, debe satisfacer
Esperada dependen directamente de las preferencias de los
la Calidad Deseada.
usuarios, sus habilidades y su conocimiento (punto de vista de
• Calidad Percibida, la percepción sobre los resultados que el
Usuario – Interacción), mientras que la calidad Alcanzada y la
cliente o usuario tiene, una vez que el sistema ha sido Deseada están relacionadas con los distintos entornos, plataformas
entregado.
y contextos en los que se utiliza la aplicación (punto de vista de
En este punto, es interesante establecer relaciones entre los Desarrollador – Adaptación).
diferentes puntos de vista de la calidad de interacción, y sus cuatro Tabla 1. Puntos de vista de la calidad de interacción y tipos
tipos. De esta forma, sabremos qué puntos de vista se asocian con
cada tipo de calidad de interacción: Calidad
Esperada Deseada Alcanzada Percibida
• La Calidad Esperada puede ser tratada desde el punto de vista Vistas

del usuario, ya que se refiere a los requisitos del sistema, es Usuario

√ √
decir, satisfacer las necesidades del usuario. Este tipo de Interacción
calidad de interacción también está relacionado con la vista del Desarrollador
adquiridor dado que también es adaptable a las necesidades del √ √
Adaptación
cliente.
• La Calidad Deseada puede ser tratada desde los siguientes Todas estas relaciones identificadas entre los distintos actores
puntos de vista: mantenedor, persona responsable de la involucrados en un proceso de interacción y las múltiples facetas
implementación, desarrollador y evaluador, ya que de la calidad de interacción constituyen un pilar fundamental para
representan a expertos en calidad que quieren alcanzar una comprender adecuadamente lo que significa interacción y QoA,
calidad de interacción final con respecto a la mantenibilidad, la como se verá en las siguientes secciones.
portabilidad, medidas internas de calidad, y la efectividad del
sistema, respectivamente. 3.3 Calidad de Adaptación
• La Calidad Alcanzada, como la calidad de interacción deseada, A lo largo de esta sección, se describe cómo se puede considerar a
también puede ser tratada desde los puntos de vista del la QoA en las diferentes facetas de la adaptación. Concretamente,
mantenedor, persona responsable de la implementación, las facetas cubiertas son aquellas incluidas en el marco de
desarrollador y evaluador, ya que representan a expertos en adaptación ISATINE.
calidad que finalmente logran cierta calidad de interacción de En trabajos previos [17], hemos definido la QoA como el grado
acuerdo a la mantenibilidad, la portabilidad, medidas internas con el que un conjunto de adaptaciones produce interfaces de
de calidad, y la efectividad del sistema, respectivamente. usuario que permiten al usuario lograr objetivos concretos con
• La Calidad Percibida, como la calidad de interacción usabilidad (ISO 9241-11 [12]; ISO/IEC 25010 [13]) en contextos
esperada, también puede ser tratada desde los puntos de vista de uso específicos. Dentro del contexto de uso consideramos las
del usuario y el adquiridor, dado que está centrada en la características del usuario, de la plataforma (hardware y software),

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 433 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
del entorno físico donde tiene lugar la interacción, y de la tarea del usuario, dado que puede realizarse, por ejemplo, por motivos
que el usuario está realizando. de marketing.
En este sentido, Gjørven, Eliassen y Aagedal [7] proponen que la 2. Iniciativa de la adaptación (Iniciativa). Esta fase también está
QoA se use para valorar cuán diferentes son las salidas de una relacionada con la Calidad de Adaptación Esperada dado que el
adaptación real de las salidas de un servicio de adaptación ideal. actor encargado de iniciar la adaptación (usuario, sistema o
Así, un servicio se define como el trabajo realizado, especificado tercero) expresa su voluntad de comenzar el proceso de
por un subconjunto de eventos de salida, y aquello que provocó la adaptación, debido a que el estado de la interfaz no le permite
petición del trabajo, especificado por un conjunto de entradas. De alcanzar sus objetivos en el contexto de uso actual.
esta forma, un servicio de adaptación se define como aquel cuyo 3. eSpecificación de la adaptación (Especific.). En esta fase se
evento de entrada es un disparador de la adaptación y cuyos describen las adaptaciones que pueden llevarse a cabo para lograr
eventos de salida son un conjunto de servicios que potencialmente la QoA Esperada, por lo que esta etapa está relacionada con la
han sido modificados o producidos durante la adaptación. Calidad de Adaptación Deseada.
Por otro lado, también es posible definir la Calidad de Adaptación 4. Aplicación de la adaptación (Aplicación). La aplicación de las
retomando el concepto de calidad de interacción definido adaptaciones especificadas en la fase previa también debe
anteriormente. En este caso, la QoA se define como el cambio en realizarse de acuerdo a la Calidad de Adaptación Deseada.
el nivel de calidad de interacción, es decir, la QoA es el
incremento Δ(calidad de interacción), el cual se calcula restando 5. Transición de la adaptación (Transición). Es obvio que el
el antiguo valor de calidad de interacción, f(x), a su nuevo valor, cambio de la interfaz de usuario original a la adaptada debe
f(x + Δx), donde x hace referencia a la calidad de interacción: realizarse considerando criterios de calidad alcanzada. En esta
fase, el actor encargado de la adaptación debe poder dar cuenta de
Δ(calidad de interacción) = f(x + Δx) - f(x) la calidad lograda con la adaptación y facilitar que se retomen las
actividades que venían realizándose antes de la adaptación. El tipo
Esta expresión matemática sirve como herramienta para
de calidad que se considera en esta fase es la Calidad de
calcular la QoA, obteniendo un valor concreto y Adaptación Alcanzada.
cuantificable, lo que permite que, aplicada a distintos
productos software, éstos puedan ser más fácilmente 6. INterpretación de la adaptación (Interpret.). Siguiendo la
transición de la adaptación, surge la necesidad de ser capaz de
evaluables y comparables en términos de calidad de
justificar el nivel de calidad logrado, fruto de la realización de la
interacción. En la siguiente sección, se verá cómo aplicar adaptación. Por tanto, esta fase está asociada con la Calidad de
este concepto de QoA al marco de adaptación ISATINE. Adaptación Alcanzada, dado que representa lo que los actores
involucrados en la adaptación deberían interpretar o explicar
3.4 Calidad de Adaptación en ISATINE acerca de lo ocurrido.
En la sección 3.2, se han presentado cuatro tipos diferentes de
calidad de interacción: Esperada, Deseada, Alcanzada y Percibida. 7. Evaluación de la adaptación (Evaluac.). En esta fase, se
Concretamente, en el contexto de adaptación, las distintas comprueba si los objetivos de la adaptación inicialmente
perspectivas de la QoA pueden definirse de la siguiente forma: especificados se han cubierto o no. Esto puede ser interpretado
como la Calidad de Adaptación Percibida por el usuario final.
• Calidad de Adaptación Esperada, los objetivos y requisitos de
la adaptación que el usuario espera que proporcione una La Tabla 2 resume todas estas relaciones, es decir, qué tipos de
aplicación. QoA se ven involucrados en cada etapa de ISATINE, así como los
• Calidad de Adaptación Deseada, la especificación de la puntos de vista de calidad relacionados. De esta manera, conocer
adaptación y las políticas de selección utilizadas para elegir qué tipo de adaptación debemos soportar en cada etapa del
entre las adaptaciones posibles, y así lograr cierto grado de proceso de adaptación resulta crucial para ser capaces de ofrecer
calidad derivado de la QoA Esperada. QoA en cada etapa de una forma adecuada. Así, el diseñador será
capaz posteriormente de proporcionar los medios apropiados para
• Calidad de Adaptación Alcanzada, la calidad realmente
evaluar realmente la QoA de una forma cuantitativa en cada etapa,
obtenida para una implementación dada de una adaptación y su
según el correspondiente tipo de QoA, aplicando distintas técnicas
interpretación justificada.
y métricas. La propuesta de esta serie de técnicas y métricas está
• Calidad de Adaptación Percibida, la calidad percibida por el
siendo actualmente objeto de estudio.
usuario o el sistema, una vez que se ha hecho uso de la IU
adaptada. Tabla 2. Puntos de vista, tipos de QoA y etapas de ISATINE
Una vez definidas las distintas perspectivas de la QoA, es QoA
interesante relacionar estos cuatro tipos con las distintas fases del Esperada Deseada Alcanzada Percibida
Vistas
marco de trabajo ISATINE. El objetivo de esto es clarificar qué 1. Objetivos
Usuario
tipo de QoA debe considerarse en cada momento. 2. Iniciativa
7. Evaluac.
Interacción
1. Objetivos de la adaptación (Objetivos). Esta fase está Desarrollador 3. Especific. 5. Transición
relacionada con la Calidad de Adaptación Esperada ya que 4. Aplicación 6. Interpret.
Adaptación
representa lo que se espera de la adaptación por parte de un
usuario, un sistema o un tercero. Por ejemplo, en una página web 4. CARACTERIZANDO LA CALIDAD DE
que soporte actividades de comercio electrónico es necesario que INTERACCIÓN: UN ESTUDIO
las ofertas sigan estando presentes, aunque se adapte la interfaz. El objetivo de este estudio es determinar si existen diferencias
Cabe destacar que la adaptación no siempre se aplica en beneficio
significativas entre la calidad Esperada y Deseada en términos de
caracterización y denominación. Nuestra esperanza es que los

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 434 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
resultados derivados de este análisis ayuden a los desarrolladores oscilaban entre 22 y 52 años (Media = 33.71, Desviación Estándar
a entender mejor las preferencias de calidad de los usuarios con el = 9.06).
único propósito de diseñar productos software que resulten De manera similar al estudio anterior sobre calidad Esperada, este
atractivos. Este estudio empírico identifica una brecha preliminar estudio estaba enfocado en términos de caracterización y
entre lo que consideran los usuarios y los desarrolladores sobre preferencias de la calidad de interacción, pero con la
calidad de interacción. A continuación, se analizan dos particularidad de que en este caso los participantes poseían un
dimensiones de la calidad de interacción en ISATINE, conocimiento específico sobre interacción. Concretamente, los
concretamente la calidad Esperada y la Deseada. sujetos estaban familiarizados con los distintos estándares
Nótese que no se han incluido por el momento las otras dos internacionales de calidad, y la mayoría de ellos habían
dimensiones de la calidad de interacción (Alcanzada y Percibida) desarrollado productos software con diversos requisitos, tanto
ya que forman parte de la evaluación clásica de usabilidad. funcionales como no funcionales.
4.1 Calidad Esperada De nuevo, se formuló una única pregunta a los participantes: Por
Un total de 31 personas sin experiencia específica en interacción favor, ¿Podrías escribir, al menos, cinco
completaron este estudio sobre la calidad Esperada. Los sujetos palabras (positivas o negativas) relacionadas
eran alumnos de nuevo ingreso cursando el primer curso del grado con tu idea de calidad de interacción? Como se
en Informática. Aproximadamente el 84% de los encuestados eran puede observar, esta cuestión es similar a la planteada en el
hombres (n=26) mientras que el resto, un 16%, correspondía a estudio anterior. De esta forma, es posible realizar fácilmente una
mujeres (n=5). Las edades de dichos sujetos oscilaban entre 19 y comparativa sobre un mismo aspecto, enfocada desde dos puntos
25 años (Media = 19.87, Desviación Estándar = 1.41). de vista distintos. En este caso, ambos estudios se centran en
conocer la idea que tienen, tanto usuarios expertos como
Los resultados fueron analizados en términos de caracterización y inexpertos en interacción, sobre la calidad de interacción, es decir,
preferencias de la calidad de interacción. Se formuló una la calidad Esperada frente a la calidad Deseada.
única pregunta a los participantes. Esta
cuestión era: Por favor, ¿Podrías escribir, al
menos, cinco palabras (positivas o negativas)
relacionadas con tu idea de calidad de
interacción?
Las respuestas fueron analizadas calculando el
porcentaje de palabras repetidas. Estos
porcentajes se han representado utilizando un
tag cloud o nube de etiquetas (véase Figura 1).
Estas nubes de etiquetas son representaciones Figura 2. Caracterización de la calidad de interacción por
visuales que indican la frecuencia de uso de sujetos con experiencia en interacción
una determinada palabra dentro de un texto
concreto, incluyendo sitios web, artículos, Los resultados de este estudio se analizaron y presentaron de la
bases de datos, o discursos. Dentro de la nube, misma forma que en el estudio anterior. La Figura 2 muestra la
la frecuencia de cada palabra se representa representación en forma de nube de etiquetas sobre dichos
incrementando el tamaño de la fuente y la resultados analizados. Como se verá en el siguiente apartado, hay
saturación del color asociado a dicha palabra. pocas palabras que aparezcan como respuesta en ambos estudios,
como intuitivo, eficiente y seguro. A priori, esto nos hace
sospechar que la idea de calidad de interacción de un determinado
producto software puede venir determinada por el tipo de usuario
que utiliza dicho producto, es decir, dependerá de la experiencia
concreta que tenga el usuario a la hora de interactuar con
aplicaciones software.
5. DISCUSIÓN
Analizando los resultados de ambos estudios, se han observado
múltiples diferencias con respecto a la caracterización de la
calidad de interacción. Estas diferencias están directamente
relacionadas con el conocimiento adquirido sobre interacción
persona-ordenador, e informática en general, de los participantes.
Figura 1. Caracterización de la calidad de interacción por Como era de esperar en el caso de expertos en interacción,
sujetos sin experiencia en interacción
muchos de los términos encontrados tras analizar sus respuestas
4.2 Calidad Deseada corresponden a aquellos empleados en los estándares de calidad,
De forma paralela, se llevó a cabo otro estudio sobre la calidad como por ejemplo, usable, eficiente, efectivo, fácil de aprender, o
Deseada. Un total de 24 personas con experiencia específica en satisfacción. Por el contrario, los sujetos menos expertos en
interacción persona-ordenador e informática completaron este interacción utilizaban una gran variedad de términos para referirse
estudio. Estos sujetos eran expertos en interacción de diferentes a la calidad de interacción y no estandarizados, haciendo especial
universidades españolas. Aproximadamente el 67% de los énfasis en fácil de usar, rápido, confiable, portable y eficiente.
encuestados eran hombres (n=16) mientras que el resto, un 33%,
correspondía a mujeres (n=8). Las edades de estos sujetos
XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction
| 435 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
De forma muy preliminar, estos estudios han permitido identificar entre los distintos puntos de vista de la calidad, los tipos de QoA y
empíricamente la brecha que existe entre la percepción que tienen las diferentes fases de ISATINE.
individuos con y sin conocimientos sobre calidad de interacción. Finalmente, este trabajo se ha centrado en uno de los puntos
Por ejemplo, en interesante destacar cómo la palabra portable no débiles de ISATINE, concretamente en la brecha encontrada en la
ha sido uno de los términos utilizados por los expertos, pero sí por definición de calidad Esperada y Deseada. Esta deficiencia se ha
los usuarios sin experiencia en interacción. Quizá una de las identificado, estudiado y analizado por medio de un estudio en
razones para explicar esto es que hoy en día su uso está forma de encuesta, donde los participantes eran sujetos con y sin
ampliamente extendido, a pesar de ser un término bastante experiencia previa en interacción. Este estudio identifica los
técnico. distintos términos que ambos tipos de sujetos han dado para
Por tanto, con respecto a las distintas propuestas sobre marcos de describir la calidad de interacción. Después de llevar a cabo la
adaptación de interfaces de usuario (véase sección 2), se hace encuesta, pudimos comprobar que nuestra hipótesis inicial era
necesaria la implementación de mecanismos que permitan cerrar correcta, ya que existe realmente una diferencia en cuanto a la
esta brecha entre la calidad Esperada y la Deseada. Al minimizar idea que ambos grupos de individuos tienen respecto a la calidad
esta separación entre ambos tipos de calidad, se conseguirá que, de interacción, es decir, en cuanto a su manera de expresar
tanto usuarios como desarrolladores, entiendan la calidad de verbalmente este concepto. Si consideramos que en ISATINE,
interacción del mismo modo. Esto permitirá crear adaptaciones dependiendo de la etapa, se puede utilizar un tipo de QoA distinto,
con menor probabilidad de ser rechazadas finalmente por el unas veces más orientado al usuario, y otras al diseñador o
usuario, dado que ambas partes, es decir, usuario y desarrollador, desarrollador, debemos tener en cuenta esta brecha conceptual en
estarán de acuerdo en cuanto a los objetivos de la adaptación y la cuanto al entendimiento de lo que es la calidad. Considerando
necesidad de llevarla a cabo (calidad Esperada), lo que se traduce esto, se podrá proporcionar información significativa sobre la
finalmente en una especificación y aplicación de la adaptación QoA en cada etapa, dando un completo y buen soporte a la QoA.
adecuadas (calidad Deseada). Este trabajo sienta las bases de un trabajo futuro centrado en cómo
Como es obvio, esta brecha también se encuentra en la calidad considerar la QoA a través de un proceso de adaptación,
Esperada y la calidad Deseada identificadas para el proceso de detallando qué técnicas y métricas encajan mejor dentro de cada
adaptación ISATINE (véase sección 3.4). Por lo tanto, los etapa. Al lograr esto, entenderemos mejor qué es la calidad de
resultados derivados de este estudio son extrapolables a ISATINE. adaptación y cómo puede ser utilizada realmente en un proceso
La mayoría de los resultados de este estudio son consistentes con completo de adaptación, lo que nos permitirá proporcionar un
hallazgos previos encontrados en la literatura, relativos a usuarios modelo completo de calidad de adaptación que guiará a cualquier
y desarrolladores. Sin embargo, este estudio proporciona diseñador durante el proceso de diseño.
información más detallada sobre la caracterización de la calidad Por otro lado, se propone examinar los diferentes términos
de interacción (criterios). utilizados, tanto por usuarios expertos como inexpertos en
interacción, para determinar si se refieren a conceptos similares, y
6. CONCLUSIONES Y TRABAJO FUTURO así unificar y establecer los términos más utilizados para referirse
A lo largo de este trabajo, se ha visto la importancia de disponer a la calidad de interacción.
de un proceso que guíe la adaptación con el principal propósito de
crear adaptaciones adecuadas que no impliquen el rechazo de la AGRADECIMIENTOS
IU. Además, este proceso de adaptación debe ir más allá de la Este trabajo ha sido financiado por el Ministerio Español de
simple ejecución de la adaptación e incluir también aspectos Economía y Competitividad, y por los fondos FEDER de la Unión
relativos a la evaluación de la misma. En este sentido, ISATINE Europea bajo la subvención para el proyecto insPIre (TIN2012-
es un marco de adaptación que proporciona estas características, 34003). También ha sido financiado por el Ministerio Español de
cubriendo desde el inicio del proceso en que se crea la adaptación, Educación, Cultura y Deporte, gracias a la beca FPU
hasta la evaluación de lo que el usuario obtiene de la aplicación de (FPU12/04962).
la adaptación.
También se ha visto cuán importante es considerar la calidad de
REFERENCIAS
[1] Benyon, D. and Murray, D. 1993. Developing adaptive
interacción dentro de este proceso de adaptación, de la misma
systems to fit individual aptitudes. Proc. of the 1st int. conf.
manera en que se considera la usabilidad en el diseño de IUs. Para
on Intelligent user interfaces (Orlando, USA), 115–121.
alcanzar esto, es necesario que el proceso de adaptación tenga en
cuenta la QoA en cada una de sus etapas. De esta forma, se ha [2] Cockton, G. 1987. Some Critical Remarks on Abstractions
profundizado en el concepto de calidad de interacción, for Adaptable Dialogue Managers. Proceedings of Third
descomponiéndolo en cuatro tipos de calidad, dependiendo del Conference of the British Computer Society Human-
punto de vista desde el cual se describa. Computer Interaction Specialist Group (1987), 325–343.
A partir de esta noción de calidad de interacción, se ha definido y [3] Crosby, P.B. 1979. Quality is free: The art of making quality
refinado un nuevo concepto llamado Calidad de Adaptación certain. McGraw Hill Custom Publishing.
(QoA) con el objetivo de entender mejor cómo este término aplica [4] Dieterich, H. et al. 1993. State of the Art in Adaptive User
a las distintas etapas de un proceso de adaptación, concretamente Interfaces. Adaptive User Interfaces: Principles and Practice.
a ISATINE. Así, dependiendo de la etapa de ISATINE en la que Elsevier B.V. 13–48.
nos encontremos, podemos hablar de un tipo u otro de QoA, es
decir, Calidad de Adaptación Esperada, Calidad de Adaptación [5] García Frey, A. 2013. Quality of Human-Computer
Deseada, Calidad de Adaptación Alcanzada, o Calidad de Interaction : Self-Explanatory User Interfaces by Model-
Adaptación Percibida. Finalmente, se han establecido relaciones Driven Engineering (Ph. D thesis). Université de Grenoble.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 436 |
 Hacia la Caracterización de la Calidad de Interacción
C. Roda | V. López-Jaquero | F. Montero
[6] Gena, C. and Weibelzahl, S. 2007. Usability Engineering for [16] Juran, J.M. and Godfrey, A.B. 1999. Juran’s quality
the Adaptive Web. The Adaptive Web. Springer. 720–762. handbook. McGraw Hill.
[7] Gjørven, E. et al. 2006. Quality of Adaptation. Proceedings [17] López-jaquero, V. et al. 2008. Quality of Adaptation : User
of the International Conference on Autonomic and Cognitive Models in Adaptation Quality Assessment.
Autonomous Systems (ICAS ’06) (Washington, USA, 2006). Computer-Aided Design of User Interfaces VI (London,
[8] Humphrey, W.S. 1988. Characterizing the software process: 2008), 265–275.
a maturity framework. IEEE Softw. 5, 2 (Mar. 1988), 73–79. [18] Norman, D.A. 1986. Cognitive Engineering. User Centered
[9] IEEE Computer Society 1991. IEEE Std. 610 - A System Design. Lawrence Erlbaum Associates. 31–61.
Compilation of IEEE Standard Computer Glossaries. [19] Oppermann, R. (Institute for A.I.T. (Fit)) 1994. Adaptively
[10] ISO 2000. ISO 9000 - Quality management systems - supported Adaptability. International Journal of Human-
Fundamentals and vocabulary. Computer Studies. 40, 3 (1994), 455–472.

[11] ISO 2008. ISO 9000 Introduction and Support Package: [20] Paramythis, A. et al. 2010. Layered evaluation of interactive
Guidance on the Concept and Use of the Process Approach adaptive systems: Framework and formative methods. User
for management systems. Modelling and User-Adapted Interaction. 20, 5, 383–453.

[12] ISO/IEC 1998. ISO 9241-11: Ergonomic requirements for [21] Pressman, R.S. 1992. Software engineering: a practitioner’s
office work with visual display terminals (VDTs) - Part 11: approach. McGraw-Hill.
Guidance on usability. [22] Roda, C. et al. 2014. Avanzando Hacia la Consideración de
[13] ISO/IEC 2008. ISO/IEC 25010, Software engineering - la Calidad de Adaptación en ISATINE. XV International
Software product quality - Requirements and Evaluation Conference on Human Computer Interaction (Puerto de la
(SQuaRE) Quality model. Cruz, Tenerife, España, 2014), 312–315.

[14] ISO/IEC 2001. ISO/IEC 9126-1: Software engineering - [23] Schlienger, C. et al. 2006. Les transitions visuelles
Product quality - Part 1: Quality model. différenciées. Proc. of the 18th int. conf. on Association
Francophone d’Interaction Homme-Machine - IHM '06, 59–
[15] ISO/IEC 2007. ISO/IEC 9126-4: Software Engineering - 66.
Software product quality - Part 4: Quality in use metrics.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 437 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón

Extending MOOC ecosystems using web services and

software architectures
Juan Cruz-Benito Oriol Borrás-Gené Francisco J. García-Peñalvo
GRIAL Research Group GATE, Technical University of Madrid GRIAL Research Group, Department
Department of Computer and Calle Ramiro de Maeztu. 7 of Computer and Automatics
Automatics, University of Salamanca 1st floor (Rectorado A) Research Institute for Educational
IUCE, Paseo de Canalejas 169 28040 Madrid, Spain Sciences (IUCE), University of
37008, Salamanca, Spain [email protected] Salamanca
[email protected] Paseo de Canalejas 169
37008, Salamanca, Spain
[email protected]

Ángel Fidalgo Blanco Roberto Therón

Technical University of Madrid Department of Computer and Automatics
Calle Alenza nº4, office 718 Faculty of Sciences, University of Salamanca
(M Building) 7th floor Plaza de los Caídos s/n
28003, Madrid, Spain 37008, Salamanca, Spain
[email protected] [email protected]

ABSTRACT be updated and adapted to new systems, platforms and paradigms

This paper present a research project that tries to extend the in eLearning. In these new types of learning platforms and
MOOC ecosystems by integrating external tools like social paradigms can be highlighted the MOOCs, because they expand
networks. This integration is developed by using a software the traditional limits in students’ interaction with teachers,
architecture that mediate between the different systems and contents and online learning platforms. Furthermore, MOOCs
platforms establishing communication workflows and analyzing leverage other platforms (even those that are not purely intended
the information retrieved. This kind of system is applied in a real to be applied in education) like the social networks and other
case, and it allows teachers and managers of the MOOC platform online tools, applying by this way multi-platform and multi-
to get enhanced information and insights about users interaction context approaches that can improve and upgrade the learning
with contents and MOOC tools, as well as some metrics experience [20].
impossible to retrieve or calculate manually in this kind of It is because of this use of multiple tools and multiple context that
eLearning platforms with high amounts of users. is necessary to design and implement new ways of interaction
analysis and platforms that allow to perform it. These new ways
Categories and Subject Descriptors and platforms will manage the acquisition of knowledge regarding
H.3.5 [Online Information Systems]: Web-based systems. H.5 the learning and interaction with platforms, establishing
[Information Interfaces and Presentation]: General K.3.1 convergence of knowledge between different learning vectors and
[Computers Uses in Education]: Collaborative learning. context, to finally allow teachers and managers to learn, explore
and implement possible improvements that help in the learning
General Terms process, the design of content and the motivation of students.
Measurement, Human Factors.
The goal of this paper is to explain a modular software
Keywords architecture implemented to allow teachers and managers of a
iMOOC, MOOC, Moodle, Web Services, Software Architectures, MOOC retrieve knowledge about how users enrolled in a MOOC
HCI, eLearning. course utilize some tools external to the MOOC platform, getting
by this way insights about what did users on these external tools,
1. INTRODUCTION what kind of interaction they perform inside them, and thus,
The analysis of interaction among users and systems provide great discover possible improvements and solutions for eLearning
insights about how users use, understand and take advantage of processes to be applied later in the MOOC platform and its
tools and platforms they utilize to perform any kind of task. courses.
The fact of analyze the interaction and try to extract valuable To explain these contents, the paper is divided into the following
knowledge from it, have real application in many areas of sections: section 2 (Aims and goals) presents the main aims and
knowledge and business, as in digital marketing, in education goals of the research presented. Section 3 (iMooc Platform)
(Learning Analytics), etc. Although in some fields as education, describes real MOOC platform where is being tested the software
this type of behavior analysis, and interaction analysis is architecture proposed and the integration with other online tools.
increasingly common, the approaches and tools developed should Section 4 (Software Architecture Proposal) explains the software

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 438 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
architecture designed to tackle the integration with other tools and participants. At this point, the resources generated from both
gain knowledge about the MOOC users and their interaction with strategies need organized. Finally, the “cooperative layer” shows
contents inside or outside the MOOC platform. Section 5 the outcomes and the content generated with the cooperation of
(Services and Crawlers) explains how is being developed the instructors and participants of the course [18]. For more
software architecture integration with other tools and platforms connectivist orientation course based on the interaction between
(eLearning platform and social networks). Section 6 (Results) participants will require the use of social tools, a good selection of
shows partial results of the application of this software these is a good way to engage students and promote their
architecture retrieving data from the integrated environments. participation in the course [1] using these tools we can create
Finally, the Section 7 (Conclusions) presents several conclusions virtual learning communities (VLC), it is defined as communities
about the research work and potential work for the future of people who share common values and interests, and that are
connect via different communication tools that such networks
2. Aims and goals offer, whether synchronous or asynchronous [5]. The
The MOOC courses are characterized by a high drop out rates incorporation of VLC may provide greater interaction between
primarily due to the heterogeneity of participants is why further participants, support and guidance to people with difficulties and
information on their activity on the platform will assist meet the may increase collaborative processes between participants.
shortcomings of course adapted to the student needs.
Over the life of people not only learn in schools or Universities,
It aims to create a model to obtain the maximum information from so do in other situations in life, this idea is called lifelong learning
the MOOC platform combining it with the results obtained in the MOOC one of the developments associated with this item. Based
external elements that support the learning of the course, such as on the concept of lifelong learning are identified three types of
social networks, to give a feedback to the educational process, learning associated with this concept [16]. Their differences [15]
analyze performing users to adapt the content to their interaction, are based on the following items: where does the learning take
study the behavior to provide the extra functionality MOOC place, in the case of online learning, the platform becomes as the
integrating user feedback analysis. institutional place for learning; if the learning is structured or
unstructured; whether the learning is guided or not by faculty; and
Thus the analysis of interaction in media is emphasized to obtain a
if there is any certification [2].
deeper comprehension of users or the use of content analysis
possible interest. Detecting those topics that interest students Taking these differences the Formal learning is that occurs in an
while those resources contributed by students for future editions organized and structured environment and leads to validation and
are reused. certification, it is the most institutionalized [2]. The following
types of learning take place in less organized spaces. Then, with
It is important to locate the types of learning according to the
opposite sense to the formal is informal learning, which is
proposed activities. It is in the non-formal and informal learning
voluntary and unstructured. It is based on the intrinsic motivation
where these foster interactions between users
[11] of the student. In this type of learning, the student chooses
Because of the massiveness of these many interactions resources the way to acquire the knowledge, the learning is given
and resources making it necessary to use tools to filter and collect everywhere and applies to any situations in common life [15] and
all the information generated to recover this. This paper is based is not evaluated. Finally there is the non-formal learning, which is
on the use of hashtags associated with publications to quickly structured and guided by the faculty, but is generally voluntary
retrieve the generated content. Hashtag is a character string with a and is usually not evaluated. MOOC elements can be associated
topic associated preceded by a # sign and is used to mark tweets with the three types of learning, finding the more formal part in
[14], part of the text of these. Although their use is widespread in the course platform (xMOOC) and the informal part and non-
twitter have extended to other social networks such as Google+. A formal community-associated learning (cMOOC).
study of hashtags is posed for the types of learning identified in
the elements and course activities. 3.2 Description and main features of the
platform
3. iMOOC Platform In 2013, because of the agreement between the Technical
3.1 The iMOOC approach University of Madrid, the University of Zaragoza and the
MOOC courses offer new opportunities for learning, features like University of Salamanca, the platform iMOOC or intelligent
massiveness of participants, peer-to-peer interactions, free-of- MOOC [32] emerges. Based on Moodle 2.6.5 platform [27] was
charge, openness or scalability [28]. There are two main types of chosen for its versatility. The main MOOC platforms such as edX
courses MOOC, the xMOOC with a behavioral approach (occurs [13], Coursera [6] or MiriadaX [24] focused on Spanish speaking
in traditional online courses) and cMOOC with a connectivism require a single path for the course. Given the heterogeneity of the
and networking based approach [17]. Taking advantage of participants this can be in many cases the abandonment of them to
features of both types of MOOC can apply the cooperative model take the part that interests them, one of the main distinguishing
of Fidalgo et. al. [18] which can be defined by three layers: The features of the iMOOC platform in front those with a more
first one is the “technological layer” linked to the platform where traditional approach is adaptivity of courses for students. This
will find the course content and the social network that will adaptivity is based on three variables: Depending on the user
support the learning community. The second layer named profile, according to an itinerary chosen by the student or for
"training strategy" refers to the instructional design of the course. student progress within the course and the knowledge that he is
This layer is divided, according to the two types of MOOC gained. To achieve this functionality is necessary to use Moodle
exposed, into a “behavioral strategy” focused on the acquisition of features such as conditionals and groups, as well as external
basic common knowledge and into a “connectivist strategy” plugins to create groupings. On the other hand iMOOC provides
dedicated to the generation of educational resources by

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 439 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
an appropriate environment to an Informal and cooperative • Voluntary exercises along the videos. In both cases this
learning. activity takes place in the learning community.
Will make use of the tools offered by the platform such as • Discussion proposed and discussed in the community.
profiles, forums, workshops and secondly external social
networks for informal. • Two videoconferences using Google Hangouts,
streamed live, where some course participants could
3.3 MOOC course “Social Networking and make a brief presentation to other peers about an
Learning” educational project related to his social networks
To take advantages of iMOOC platform in early 2015 a course Table 1. Relationship of course activities on Google +
based on a test offered by the faculty version along two editions in
the Iberoamerican MOOCs platform Miriada X [4] was launched. Activity Category Hashtag
The course of one-month duration began January 12 and ended on Search examples of Activities #RSEEjemplosRRS
February 8, leaving an extra week to allow students to finish it. A
social networks and exercises S
Total of 793 students were enrolled for the course, more than 400
started it, and 183 students finally accomplish the goal. Bad practices in the use Activities #RSEMalasPractica
The VLC was created using the tool communities of Google +, of social networks and exercises s
space where students can interact with each other and with content Measuring influence in Activities #RSEMiKlout
in the network sharing it. Here they can also discuss, submit social networks using and exercises
questions and publish the result of voluntary exercises throughout
Klout
the course. To do this has been created 9 default categories
(presentations, announcements, discussions, questions, resources, Using Twitter in Activities #UsosTwitterEnseñ
Activities and exercises, application examples, contests and more) education and exercises anza
with which to classify each publication, besides the possibility to
use hashtags. Discussion about the Discussion
possibility of replacing a s
This course is based on the cooperative model [18] at which has
been added a fourth layer called "Gamification layer" that learning management
interacts with the other three layers, which aims to improve system (LMS) through a
motivation [3]. Depending on types of learning can be divided the social network
course study into three parts, the first section concerns the
Discussion on digital Discussion
iMOOC platform itself, which takes place the more formal or
formal part of the course and the other two sections relate to the identity s
community of external learning the course where interactions Hangout #RSEHangout
among participants are established and where they are generated
and share new resources. These activities can also be performed on Twitter although the
official platform is Google +. Course evaluation is based on four
The objective of this course is to provide participants, mostly questionnaires one per module that participants must overcome to
teachers of basic digital skills in social networks to implement in obtain a certificate of participation offered for free.
their classrooms with students creating virtual learning
communities. Throughout 4 modules consist of lessons, it is So this part can be associated to a non-formal learning, since such
theoretically analyzed in depth the social web and learning courses by not offering an official certifications, for example to
communities are studied thoroughly Facebook and Twitter allow for recognition of credits, can not be considered as formal
networks and finally reviewed with less detail other valid learning.
networks for educational use with and support tools to manage About informal learning this occurs in the community of Google +
social networks. The modules are divided into small lessons that and is associated with the interactions among participants not
address a specific topic and the content is based on a short video addressed by the teaching staff, similar to learning which could
as well as additional information that complements the video. result in informal settings outside the classroom such as between
As adaptive part participants can choose between five educational colleagues in an office. Three types of interactions:
itineraries: Full course for teachers (additional lessons are given • Creating new hashtags and subsequent interaction
the keys to implement networks studied), complete course for
non-teachers (only the different networks are studied), Twitter • Discussion proposed by participants
(only one module on the network), Facebook (only one module
this network) and one special itinerary. The special itinerary was • Resources contributed
addressed to students who had participated in any of the two The last two actions are included in the categories created for this
previous editions, with an extra module focused on learning publication ("debates" and "resources").
communities.
4. Architectural proposal
Following the previous experience of the authors in similar cases,
The course consists of a series of directed activities and proposals where they apply software architectures to extend the
by the faculty to enhance knowledge of the lessons and interaction functionality of eLearning ecosystems [7-10; 21], authors decided
between course participants: to use the core of a software architecture they built in 2014.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 440 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
Several layers compose this core, one to retrieve data from each interaction with the information retrieved depends strictly on the
external platform or tool, other that wipes and stores the resources Moodle provides.
information retrieved, another to push analyzed information to
other platforms, and others that enable searches and interaction 5. Services and crawlers
between information and users. In order to implement the information workflows shown in the
Figure 1 between the software architecture and the different
The core of the architecture is a system that acts as a mediator systems and social networks, researchers need to establish the
between the different social networks and learning platforms that proper communication channels for the information. These
will be interconnected (Figure 1). This mediator system communication channels are based, in this case, in services and
communicates with each external tool through using web services crawlers:
(REST APIs commonly) and crawlers; retrieving data and
information from them and analyzing the information in order to • The services are facilities provided by third-party
convert the raw data in valuable information for teachers and software to facilitate the communication and
managers of the iMOOC platform (based on Moodle). interconnection with other systems, applications or
clients. In this case, researchers have used services for
In order to implement this software architecture and its layers, has communicating with Moodle and Twitter.
been used for development several technological components and
technologies that are listed below: • The crawlers are software applications that find
automatically information in third-party systems when
• Django Framework [12]: This web framework is used to they do not provide services for pull and push
build the software layers and to coordinate the information between systems. In this case we are
information workflows between the components and working on crawlers for getting information from
systems of the architecture. Google+ Communities (Google+ does not provide API
• MongoDB [25]: This NoSQL database is used to store or other services to get and post activities and other
the data without the traditional restrictions of the SQL information within the communities).
databases, and allows to adapt the database storage
schemas to each kind of content retrieved from external In the subsections below are explained how have been used these
tools and platforms [7]. services and crawlers, and how they are implemented within the
• REST APIs: these web services are used to serve as software architecture.
communication channels between components and
systems involved in the MOOC ecosystem. In case of 5.1 Moodle
those tools and systems that do not provide this kind of Moodle provides several API services and API architectures;
facilities, will be used crawlers to retrieve information allowing users and third party applications and systems interact
(this will be explained in the following section). with courses, administration settings, users and configuration
information. The API used in this work is based on
Representational State Transfer (REST) architecture [19], and it
allows several actions in both directions of communication (GET
and POST actions, as well as DELETEs, etc.); the full
documentation and functionalities of this API can be found at
[26].
For example, these API endpoints and functionality allow
managers and teachers of the (i)MOOC course to make automatic
checks about the tasks completed by users, automatized (and
simply) assessment about their participation in the MOOC, etc. In
a regular course on Moodle, this usage of the API is not a key
aspect, most of these checks and assessment is performed
manually by the teachers, but in a MOOC course with more than
700 hundreds (in this case, several thousands in bigger MOOC
courses) turns out this resources as a key factor to evaluate the
users’ interaction with the MOOC and for assessing their learning
within the course.
Bellow is presented an example code that allow teachers to
Figure 1. Software architecture proposal retrieve the full list of users enrolled in the course; this result list
The main idea behind the framework, is that teachers and of users enrolled, for example in the case of the iMOOC course
managers could use the website provided by the architecture to was used to check what users filled their profile with the links to
interact with the information and data retrieved from the external their personal social networks profiles, which was proposed as an
tools, so all the assessment and evaluation of the users learning in activity of the Twitter in education lesson. As previously
the MOOC could be centralized in the architecture. The other explained, authors implemented the software architecture using
possible approach, is that the architecture push the analyzed Python language, so the code is formatted in the phytonic way and
information again to the Moodle platform, allowing teachers and includes the main software library used, Requests [30] that allows
managers to allow them assess and evaluate the learning without to implement easily the API consumption.
leave the MOOC platform, in that case, the data visualization and import requests, json

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 441 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
AllMyPlus (http://www.allmyplus.com/) that allows them to
parameters = {'wsfunction': core_enrol_get_enrolled_users', 'courseid':'id',
'moodlewsrestformat':'json', 'wstoken':'xxxxxx'}
retrieve information of the learners community related to MOOC.
url = "http://gridlab.upm.es/imooc/" This is not the best solution, because it convert the ideal automatic
process indeed in a manual process, so the authors are trying to
response = requests.get(url, params=parameters) develop a crawler that enables them to retrieve information
if response.status_code == 200: directly from Google+ website or AllMyPlus website.
results = response.json()
for result in results: 6. RESULTS
print result By using the software architecture and the other tools
else:
print "Error code %s" % response.status_code (AllMyPlus), was possible to retrieve information about users’
posts on social networks, information about their profiles on the
5.2 Twitter iMOOC platform, etc. As example below of the utility of this kind
Regarding the Twitter data retrieval implementation, the authors of software architecture supporting and expanding MOOC
have implemented collector that gets tweets on live based on their functionalities, are shown several metrics retrieved from the
hashtags. This implementation is possible thanks to the Twitter complete learning ecosystem:
REST APIs [31] and Tweepy library for Python [29]. Using both
facilities (specially the Twitter Streaming API) authors built a • During the course were recorded in the Google+
software that is able to retrieve in real time tweets tagged [23] community 302 publications belonging to 140 users, 57
with the any of hastaghs proposed in the MOOC course and of whom have used a hashtag. Table 2 shows the times
storing the tweets in the software architecture database (enabling that have been used hashtags and how many of them are
by this way MOOC user matching, etc). As example of how is depending on the type of learning.
done this data retrieval, below is attached a simplified version of • During the course were retrieved more than 200
the code: interactions of MOOC users with the content and
hashtags in the social network Twitter.
from __future__ import absolute_import, print_function
from tweepy.streaming import StreamListener
• Also, to evaluate the completion of the MOOC
from tweepy import OAuthHandler activities, for example as mentioned in previous section,
from tweepy import Stream related to the number of users that fill their MOOC
consumer_key="xxxx"
profiles with the links to their social networks profiles.
consumer_secret="xxxx" In this case, the number of users that accomplish this
access_token="xxxxx" activity was 275, a 33’86% of the total users.
access_token_secret="xxxxx" • Hashtags usage and temporal evolution (Figure 2) of
class StdOutListener(StreamListener): proposed hashtags by the teachers (non-formal) and
def on_data(self, data): those used by students initiative (informal).
try:
print(data) Table 2. Distribution of posts and contents in Google+
return True community
except:
pass Tot Differe # Users
def on_timeout(self): al # nt # misspelled using #
sys.stderr.write("Timeout, sleeping for 60 seconds...\n") Non- 128 8 8 37
time.sleep(60) formal
return
Informa 144 82 - 43
if __name__ == '__main__': l
l = StdOutListener() Total 272 90 - 23 (both
auth = OAuthHandler(consumer_key, consumer_secret) types)
auth.set_access_token(access_token, access_token_secret)
stream = Stream(auth, l)
stream.filter(track=['#RSEEjemplosRRSS',#UsosTwitterEnseñanza", Table 3. Interactions with MOOC contents and proposed
"#RSEMiKlout "]) hashtags in Twitter
About Twitter integration in the system, should be highlighted
that the MOOC managers and teachers must get permission of the Total interactions Twitter
users about storage their tweets, or simply anonymize the personal
data present in each tweet (name and username, location, etc.), Publications 108
because the social network specify in their policy rules this Replies 17
restriction.
Retweets 42
5.3 Google+
About Google+, the situation is totally different. This social Favorites 45
network provides APIs and methods to retrieve information about
users, posts, comments, etc. [22], but it does not allow to retrieve
information from the users communities within the social
network. This disables the possibility of use the same way to get
information about conversation and interactions in the
communities, regarding this, teachers and managers from the
MOOC course were searching other tools that let them to retrieve
the desired information; for example, they use currently the tool

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 442 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
[2] Bjornavold, J. and European Centre for the Development of
Vocational Training, 2008. Validation of non-formal and
informal learning in Europe: a snapshot 2007 Office for
Official Publications of the European Communities,
Luxembourg.
[3] Borrás Gené, O., Martínez Núñez, M., and Fidalgo Blanco,
Á., 2014. Gamification in MOOC: challenges, opportunities
and proposals for advancing MOOC model. In Proceedings
of the Second International Conference on Technological
Ecosystems for Enhancing Multiculturality, F.J. García-
Peñalvo Ed. ACM, New York, USA, 215-220. DOI=
http://dx.doi.org/10.1145/2669711.2669902.
[4] Borrás-Gené, O., 2013, Aplicación de las redes sociales a la
Figure 2. Evolution of informal and non-formal hashtags enseñanza: Comunidades virtuales. . Available
related to MOOC contents usage https://www.miriadax.net/web/aplicacion-rrss-ensenanza.
Retrieved 2/04/2013
7. CONCLUSIONS [5] Cabero, J., 2006. Las TIC y las inteligencias múltiples.
This paper explains a software architecture designed and Infobit. Revista para la difusión y uso educativo de las TIC
developed to extend the MOOC ecosystems functionalities and 13, 75.
utilities by integrating external tools like social networks. This
integration is built by using a software architecture that mediate [6] Coursera, 2015, Coursera MOOCs Homepage. Available
between the different systems and platforms establishing http://coursera.org/. Retrieved 3/05/2015
communication workflows and analyzing the information [7] Cruz-Benito, J., García-Peñalvo, F.J., and Therón, R., 2014.
retrieved. This kind of system is applied in a real case, and it Defining generic data collectors for Learning Analytics:
allows teachers and managers of the MOOC platform to get Facing up the heterogeneous data from heterogeneous
enhanced information and insights about users interaction with environments. In Proceedings of the International
contents and MOOC tools, as well as some metrics impossible to Conference on Advanced Learning Technologies (ICALT)
retrieve or calculate manually in this kind of eLearning platforms 2014 (Athens, Greece2014), IEEE. DOI=
with high amounts of users. http://dx.doi.org/10.1109/ICALT.2014.108.
In order to demonstrate the utility of this kind of software [8] Cruz-Benito, J., Garcia-Peñalvo, F.J., Theron, R., Maderuelo,
architectures, showing also the possibilities and new metrics that C., Perez-Blanco, J.S., Zazo, H., and Martin-Suarez, A.,
could be gathered using it, authors show some data gathered from 2014. Using software architectures to retrieve interaction
the iMOOC platform and social networks, showing in these data information in eLearning environments. In Computers in
how the application of this software architecture can help to Education (SIIE), 2014 International Symposium on, 117-
measure elements difficult to estimate because the vast amount of 120. DOI= http://dx.doi.org/10.1109/SIIE.2014.7017715.
users enrolled in MOOC courses or because they implicate the
[9] Cruz-Benito, J., Therón, R., García Peñalvo, F.J., Maderuelo,
usage of external tools like the social networks that teachers can
C., Pérez-Blanco, J.S., Zazo, H., and Martin-Suarez, A.,
not track without this kind of tools and systems.
2014. Monitoring and feedback of Learning Processes in
8. ACKNOWLEDGMENTS Virtual Worlds through analytics architectures: A real case.
In Sistemas y Tecnologías de Información. Actas de la 9ª
Authors would like to thank to the GRIAL and LITI research
Conferencia Ibérica de Sistemas y Tecnologías de
groups by their support in this research, also they would like to
Información, Á. Rocha, D. Fonseca, E. Redondo, L.P. Reis
thank to the Technical University of Madrid by its financial
and M.P. Cota Eds. AISTI (Asociación Ibérica de Sistemas y
support throughout the “Diseño y desarrollo de MOOC
Tecnologías de Información), Barcelona, España, June, 18-
universitarios” (Ref. PT1415-05000) Project and to the Vice
21, 2014, 1126-1131.
Chancellor for Academic Policy at the University of Salamanca
for financing the project under the Teaching Innovation program [10] Cruz-Benito, J., Therón, R., García-Peñalvo, F.J., and Pizarro
ID2014/0281 that allowed the authors to develop the software Lucas, E., 2015. Discovering usage behaviors and
architecture described in this research work. engagement in an Educational Virtual World. Computers in
Human Behavior 47, 18-25. DOI=
9. REFERENCES http://dx.doi.org/10.1016/j.chb.2014.11.028.
[1] Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C., [11] Csikszentmihalyi, M. and Hermason, K., 1995. Intrinsic
Parada G, H., Muñoz-Organero, M., and Rodríguez-De-Las- Motivation in Museums-What Makes Visitors Want to
Heras, A., 2013. Analysing the Impact of Built-In and Learn. Museum news 74, 3, 34-37& 59-62.
External Social Tools in a MOOC on Educational
Technologies. In Scaling up Learning for Sustained Impact, [12] Django Software Foundation, 2015, Django Web
D. Hernández-Leo, T. Ley, R. Klamma and A. Harrer Eds. Framework. Available https://www.djangoproject.com/.
Springer Berlin Heidelberg, 5-18. DOI= Retrieved 15/03/2015
http://dx.doi.org/http://dx.doi.org/10.1007/978-3-642-40814- [13] Edx, 2015, Edx MOOCs Homepage. Available
4_2. https://www.edx.org/. Retrieved 3/05/2015

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 443 |
 Extending MOOC ecosystems using web services and software architectures
J. Cruz-Benito | O. Borrás-Gené | F. J. García-Peñalvo | A. Fidalgo Blanco | R. Therón
[14] Efron, M., 2010. Hashtag retrieval in a microblogging Training Processes in Health Area Using Virtual Worlds.
environment. In Proceedings of the 33rd international ACM The Scientific World Journal 2014.
SIGIR conference on Research and development in [22] Google, 2015, Google+ Platform APIs. Available
information retrieval ACM, 787-788. https://developers.google.com/+/api/. Retrieved 1/03/2015
[15] Eshach, H., 2007. Bridging in-school and out-of-school [23] Huang, J., Thornton, K.M., and Efthimiadis, E.N., 2010.
learning: Formal, non-formal, and informal education. Conversational tagging in twitter. In Proceedings of the 21st
Journal of science education and technology 16, 2, 171-190. ACM conference on Hypertext and hypermedia ACM, 173-
[16] Faure, E., 1972. Learning to be: The world of education 178.
today and tomorrow. Unesco. [24] Miriadax, 2015, MiriadaX MOOCs Homepage. Available
[17] Fidalgo Blanco, Á., García-Peñalvo, F.J., and Sein-Echaluce http://miriadax.net/. Retrieved 3/05/2015
Lacleta, M.L., 2013. A methodology proposal for developing [25] Mongodb, 2015, Website. Available
adaptive cMOOC. In Proceedings of the First International https://www.mongodb.org/. Retrieved
Conference on Technological Ecosystem for Enhancing
Multiculturality, F.J. García-Peñalvo Ed. ACM, New York, [26] Moodle, 2014, Web service API functions. Available
USA, 553-558. https://docs.moodle.org/dev/Web_service_API_functions.
Retrieved 1/03/2015
[18] Fidalgo Blanco, Á., Sein-Echaluce Lacleta, M.L., and
García-Peñalvo, F.J., 2013. MOOC cooperativo. Una [27] Moodle, 2015, Website. Available http://moodle.org.
integración entre cMOOC y xMOOC. Cooperative MOOC. Retrieved 3/04/2015
An integration between cMOOC and xMOOC. In II [28] Núñez, M.M., Gené, O.B., and Blanco, Á.F., 2014. Social
Congreso Internacional sobre Aprendizaje, Innovación y community in MOOCs: practical implications and outcomes.
Competitividad, CINAIC 2013, Á. Fidalgo Blanco and M.L. In Proceedings of the Second International Conference on
Sein-Echaluce Lacleta Eds. Fundación General de la Technological Ecosystems for Enhancing Multiculturality
Universidad Politécnica de Madrid, Madrid, 481-486. ACM, 147-154.
[19] Fielding, R.T., 2000. Architectural Styles and the Design of [29] Project, T., 2015, Tweepy on Github. Available
Network-based Software Architectures. In Irvine University https://github.com/tweepy. Retrieved 1/03/2015
of California.
[30] Requests, 2015, Requests: HTTP for Humans. Available
[20] García-Peñalvo, F.J., Cruz-Benito, J., Borrás-Gené, O., and http://docs.python-requests.org/. Retrieved 1/02/2015
Fidalgo Blanco, Á., 2015. Evolution of the Conversation and
[31] Twitter Inc, 2015, REST APIs. Available
Knowledge Acquisition in Social Networks related to a
https://dev.twitter.com/rest/public. Retrieved 1/03/2015
MOOC Course. In Proceedings of the HCI International (Los
Angeles, CA, USA, In press 2015), Springer. [32] Universidad Politécnica De Madrid, Universidad De
Zaragoza, and Universidad De Salamanca, 2015, iMOOC.
[21] García-Peñalvo, F.J., Cruz-Benito, J., Maderuelo, C., Pérez-
Available http://gridlab.upm.es/imooc/. Retrieved 1/03/2015.
Blanco, J.S., and Martín-Suárez, A., 2014. Usalpharma: A
Cloud-Based Architecture to Support Quality Assurance

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 444 |
 Arquitectura para la interacción en un videojuego para el entrenamiento de la voz de personas con discapacidad intelectual
M. Corrales | D. Escudero | V. Flores | C. González | Y. Gutiérrez

Arquitectura para la interacción en un videojuego para el

entrenamiento de la voz de personas con discapacidad
intelectual.
Mario Corrales, David Escudero, Valle Flores, Yurena Gutiérrez
Cesar González Universitat Autònoma de Barcelona
Universidad de Valladolid Campus de la UAB
Campus Miguel Delibes s/n 08193 Cerdanyola, Barcelona
47011-Valladolid [email protected]
[email protected],
[email protected],[email protected]
[email protected]
ABSTRACT discapacidad intelectual [5,6] desarrollando una interfaz de
En este artículo se presenta una arquitectura software que resuelve usuario acorde con dichas problemáticas.
aspectos relativos a la interacción persona computador en un Debido a que esta herramienta está enfocada principalmente al
videojuego para la mejora de la comunicación oral de personas entrenamiento de aspectos lingüísticos, se hace necesaria la
con discapacidad intelectual. Se describen una serie de aspectos interacción multimodal entre el usuario y el videojuego, utilizando
importantes a tener en cuenta en este tipo de aplicaciones como entradas y salidas tanto sonoras como visuales. La interacción
son la falta de memoria de los usuarios, la necesidad de uso multimodal es controvertida porque su uso resulta poco práctico
concurrente de la aplicación por parte de alumno y de los en la mayoría de aplicaciones [9]. Se presenta aquí un caso en el
profesores, la necesidad de guardar registros de la interacción y que la interacción vocal es necesaria y donde contribuye además a
generación de los correspondientes informes de uso, la adaptación reforzar la eficacia de la interacción convencional.
al usuario y la entrada/salida multimodal. La solución es un
Otro aspecto clave a tener en cuenta es el género del videojuego a
videojuego basado en la metáfora de historia gráfica, cuya fase de
utilizar. Nos hemos decantado por las aventuras gráficas, ya que
pruebas de usabilidad con usuarios reales se avanza en este este tipo de videojuegos permiten simular un elevado número de
trabajo.
situaciones que creen un entorno significativo idóneo para
Categories and Subject Descriptors contextualizar los actos de habla o las diferentes producciones
lingüísticas. Pero hay que tener en cuenta que algunas personas
H.5.2 [Information Interfaces and Presentation (e.g., HCI)]:
con discapacidad intelectual, o diversidad funcional cognitiva
User Interfaces; K.8.0 [Personal Computing]: Games; J.4 [Social
entre otras dificultades, presentan importantes limitaciones con
and Behavioral Sciences]: Psychology
sus procesos de memoria [3], por lo que hay que evitar que el
General Terms usuario tenga que recordar gran cantidad de información, algo
Human Factors, Design común en este género de videojuegos. En nuestro enfoque,
empleamos recursos de diseño de interfaz que vengan a suplir las
Keywords limitaciones de memoria de los estudiantes.
Keywords are your own designated keywords.

1. INTRODUCCIÓN
Las personas con discapacidad intelectual presentan una serie de
problemas relacionados con la utilización del lenguaje [4].
Existen herramientas con las que los logopedas y maestros en
audición trabajan con dichas personas, aunque éstas se reducen a
meras actividades individuales en las que se trabajan aspectos
concretos del lenguaje [1]. Sin embargo, el uso de las TIC en este
contexto está muy poco desarrollado, principalmente por las
dificultades que entraña la interacción entre personas con
discapacidad intelectual con sistemas informáticos. Por otro lado,
en los últimos años, el uso de videojuegos orientados a la
educación está sufriendo un incremento muy importante, ya que
estas herramientas permiten integrar mecánicas educativas en
entornos lúdicos que ayudan a que los alumnos desarrollen dichas
dinámicas con una motivación mayor a la que se produce cuando
se realizan en contextos estrictamente educativos [8]. El presente
trabajo propone un videojuego donde se integran estos aspectos,
poniendo especial interés en los problemas que se han detectado
en el uso de herramientas informáticas por parte de personas con

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 445 |
 Arquitectura para la interacción en un videojuego para el entrenamiento de la voz de personas con discapacidad intelectual
M. Corrales | D. Escudero | V. Flores | C. González | Y. Gutiérrez
Trainer pueda hacer que Player repita o suspenda determinados
ejercicios en función del aprovechamiento del mismo.
El rol de apoyo del profesor es fundamental para explotar al
máximo la potencialidad educativa del videojuego. El profesor
además de apoyar y guiar al usuario en el proceso, determinará los
niveles de dificultad que el usuario puede ir superando, animará a
éste a continuar cuando encuentre dificultades en una tarea o
actividad, complementará las ayudas proporcionadas en el
videojuego para la realización de las actividades y para su avance
en el mismo, ayudándole a resolver tanto las dificultades del
usuario tanto en el manejo del mismo, como sus dificultades de
comprensión de la narrativa o de las actividades.
2.1 Interacción multimodal
A la hora de diseñar el videojuego se han seguido los
planteamientos del aprendizaje multimedia, es decir, el
aprendizaje utilizando imágenes y sonido [7]. De esta forma, se
trata de conseguir que los usuarios no dependan tanto del canal
verbal para la recibir la información relevante, sino que se pueda
complementar dicha información con imágenes, ya que esta
modalidad de información presenta menos problemas de
procesamiento para estas personas [3]. De hecho, se ha
demostrado que el uso de imágenes para apoyar y complementar
la información verbal es una estrategia educativa más eficaz que
el uso exclusivo de información verbal [2].
Se trata de una aplicación con un interfaz multimodal, tanto desde
el punto de vista de la entrada como el de la salida. La entrada se
realiza utilizando voz y también el ratón o el touch si se emplea la
versión para móviles y tabletas. En los ejercicios de entrenamiento
de la voz, el jugador deberá utilizar la voz de forma obligatoria.
Figura 3: Flujo de entrada/salida de la aplicación
La salida del juego emplea los modos visual y sonoro. El
Nuestra aproximación pretende que la herramienta sirva de enlace videojuego incluye gráficos 2D que describe las escenas donde se
entre el educador o profesional y el usuario. El educador tendrá desarrolla la acción, los recursos que describen el estado, los
capacidad para manejar la aplicación de forma coordinada con avatares que representan al jugador y al asistente virtual. El canal
usuario, no sólo configurando las actividades de acuerdo al perfil sonoro de salida se dedica a la voz en off que narra la aventura, a
requerido, sino realizando acciones con el interfaz en paralelo al los personajes que intervienen en la escena y a los apoyos del
trabajo del alumno. La herramienta dejará registro de la actividad asistente virtual. En todo caso, se utiliza voz grabada para incluir
del usuario, y realizará informes que permitan al profesor conocer un componente de emotividad que la voz sintética aún no alcanza.
el aprovechamiento de su uso.
Todos estos condicionantes determinan la definición de la interfaz 2.2 Lógica y control
y la arquitectura de la herramienta. En las siguientes secciones, La arquitectura del juego se muestra en la Figura 2, la cual se
analizamos la relación entre videojuegos y educación y detallamos estructura en torno a tres componentes separadas. El módulo de
la arquitectura del videojuego. Después, explicamos algunas lógica es el responsable de acceder a los módulos
consideraciones de interfaz y desarrollo. Finalizamos realizando correspondientes para presentar las escenas, actividades y ayudas
una discusión sobre el trabajo futuro que tiene que ver que corresponden en cada momento. También se encarga de
principalmente con la estrategia de pruebas del sistema. reconocer el nivel del usuario y adaptar las dinámicas a dicho
nivel. El trainer podrá también acceder a este módulo para
2. ARQUITECTURA DE LA APLICACIÓN corregir las predicciones del mismo.
La Figura 1 muestra un esquema funcional del flujo entrada/salida El módulo de interfaz gestiona la presentación de las distintas
del sistema. Dos usuarios interactúan con el programa: el usuario actividades del juego. Hay actividades de entrenamiento de la voz
player y el usuario trainer. Player es típicamente un niño o un y otras exclusivamente gráficas. Se incluyen actividades gráficas
joven con necesidades específicas en el ámbito del lenguaje, para hacer más entretenido el juego y para diagnosticar, en tiempo
concretamente en lo que hace referencia a la percepción y real, el nivel de comprensión del jugador. También es responsable
producción prosódica. Trainer es típicamente un asistente de dar apoyo y realimentación al jugador durante el desarrollo de
(profesor, logopeda, familiar…) que apoya al player durante la la partida. En particular es responsable del avatar y del mapa que
sesión de trabajo. El juego puede utilizarse sin la presencia del indica al jugador los pasos que ha cubierto en su aventura.
Trainer. Cuando Trainer y Player participan juntos en la dinámica
del juego, el primero intervendría para asistir al Player en el
correcto uso de la voz, pero también para configurar la
herramienta para adaptar el juego a las necesidades del Player.
Por último, las dinámicas incluyen opciones especiales para que el

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 446 |
 Arquitectura para la interacción en un videojuego para el entrenamiento de la voz de personas con discapacidad intelectual
M. Corrales | D. Escudero | V. Flores | C. González | Y. Gutiérrez
alejados de las tareas más cotidianas. Por ello, según avanza el
juego, los escenarios realistas se irán transformando en escenarios
con toques fantásticos.

Figura 5: Escenario autobús

Para potenciar la sensación de inmersión dentro del juego, se le
ofrece al jugador la oportunidad de seleccionar un avatar al
comienzo del juego. Dicho avatar representa la imagen del
jugador en el juego, lo que le permite identificarse con el
Figura 4: Arquitectura de la aplicación personaje de la historia. Además, se ha incluido un asistente que
guía al jugador en el transcurso del juego. Dicho asistente guía al
jugador para que sepa a qué escenario debe ir o qué objeto debe
La interacción del player y del trainer queda grabada. El histórico conseguir. También recordará el objetivo actual si el jugador se
de la interacción incluye las muestras de voz. Este banco de datos queda atascado en algún punto del juego. Esto último es
es monitorizado por un módulo de datos que puede ofrecer importante ya que se pretende que el jugador complete el máximo
información filtrada al trainer sobre los problemas detectados en de actividades posibles, por lo que fomentar la permanencia en el
el habla del player y sobre si se detectan mejorías en su habla a lo juego se vuelve un factor muy importante.
largo del uso del sistema. También permite guardar la partida al
player para continuarla posteriormente. Otro aspecto importante que se ha tenido en cuenta es la inclusión
de ayudas visuales sobre los escenarios del juego, para ayudar a
La arquitectura propuesta pretende desacoplar en la medida de lo los jugadores a identificar el siguiente paso a realizar y evitar que
posible los distintos elementos que componen el videojuego, a dichos jugadores no consigan avanzar en la historia.
saber: el motor del juego (game engine), el guión de la historia
(storyline), las actividades (activity), los escenarios (background), 3.2 Tipo de actividades
el gestor de datos (storage manager) y la gestión de la interfaz. De Como hemos comentado anteriormente, el objetivo principal del
esta manera se simplifica el desarrollo del videojuego y la videojuego es que el usuario trabaje sus competencias
creación de las distintas narrativas por las que discurre el juego. comunicativas. Para ello, se han definido 3 tipos de actividades
El desarrollo se ha realizado usando Adobe Flash Professional enfocadas a mejorar algún aspecto relevante en dichas
CC. Actualmente el videojuego se puede ejecutar en cualquier competencias.
dispositivo en el que se pueda instalar Adobe Air: PC, Mac,
• Actividades de comprensión: Dichas actividades están
tableta Android, etc.
enfocadas tanto a la comprensión léxico-semántica
3. INTERFAZ Y ACTIVIDADES como a la mejora de la percepción prosódica en
contextos concretos, como realizar una pregunta o pedir
3.1 Narrativa e inmersión en el juego algo educadamente. Para ello se introduce al usuario en
Debido a que la principal función del videojuego es ayudar a las diferentes diálogos con personajes del juego, en los que
personas con discapacidad intelectual a mejorar el uso del tendrá que elegir entre diferentes opciones con las que
lenguaje, los escenarios, personajes y objetos con los que el continuar el diálogo.
jugador interactuará son representaciones del mundo real, para
• Actividades de producción: Estas actividades están
que dichos jugadores puedan identificar situaciones cotidianas y
dirigidas a la producción oral por lo que se les insta a
sean capaces de utilizar lo aprendido durante el juego en
ejercitar el habla teniendo en cuenta aspectos
situaciones reales. Así el jugador pasará durante la aventura por
prosódicos, como la entonación, la expresión de
escenarios cotidianos como una biblioteca, un autobús o una
emociones o el énfasis silábico. Esto se consigue
tienda de material escolar. Sin embargo, la historia narrada
también realizando diálogos con personajes del
también incluye aspectos imaginarios, ya que ayudan a que el
videojuego, en los que el usuario tendrá que repetir
jugador se sienta retado a alcanzar determinados objetivos
diferentes frases en adecuación al contexto del diálogo.
• Actividades visuales: Estas actividades tienen 2
objetivos. Por un lado, están las actividades dirigidas a
XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction
| 447 |
 Arquitectura para la interacción en un videojuego para el entrenamiento de la voz de personas con discapacidad intelectual
M. Corrales | D. Escudero | V. Flores | C. González | Y. Gutiérrez
mejorar aspectos muy concretos de la prosodia, todo usuario, una segunda persona puede interactuar con el sistema
acompañado de una respuesta visual a la entrada sonora para garantizar el aprovechamiento del uso de las dinámicas por
del usuario. Por otro lado, se introduce actividades parte del jugador.
diseñadas para añadir diversidad al aspecto jugable, que Un test de usabilidad completará las pruebas heurísticas que se
ayudan a reducir la sensación de monotonía al jugar. han realizado hasta ahora. El objetivo es evaluar los aspectos de
Además potencian otras capacidades cognitivas como la comprensión de la historia, el seguimiento de las directrices para
percepción espacial, la identificación de formas o la ir superando las distintas actividades, la preferencia de
relación entre causa y consecuencia, entre otras. determinadas ilustraciones, la permanencia del jugador con el uso
Cada actividad ofrece en su resolución un feedback positivo o del videojuego, etc. También los profesionales de los centros
negativo, según los resultados obtenidos por los usuarios en cada (logopedas, educadores, profesores, etc.) complementarán
actividad. Esto es importante ya que un videojuego debe proponer formularios de evaluación. Durante la prueba, se anotan todos
retos jugables y los jugadores deben ser conscientes de los aquellos aspectos relativos a la actitud del jugador en diferentes
resultados de su interacción. Sin embargo, debido a las momentos, problemas técnicos con el uso de la herramienta, nivel
dificultades que presentan los usuarios objetivos, es importante no de comprensión de las actividades y otros. Por último, se les
causarles frustración para que no abandonen el uso del pregunta por la experiencia con el videojuego con el objetivo de
videojuego. Para ello, los errores se tratan de manera positiva. Por saber si les ha gustado la historia, las imágenes y las actividades,
un lado, siempre se les permite avanzar independientemente de los si lo han encontrado fácil y si se imaginan para qué puede
resultados, variando únicamente el feedbak mostrado. Por otro servirles, entre otras cosas.
lado, a pesar de darles feedback negativo cuando se equivocan
para que sean conscientes del error, este feedbak se complementa 5. AGRADECIMIENTOS
con un refuerzo positivo que les alienta a seguir jugando y no Este trabajo ha sido financiado por el Proyecto Juguem a
desmoralizarse, haciéndoles ver que cometer un error es algo comunicar millor! La millora de la competència prosòdica com a
totalmente normal. via d'integració educativa i d'inclusió social de l'alumnat amb
necessitats educatives especials derivades de la discapacitat.
3.3 Adaptación al usuario PZ611683-2013ACUP00202. Financiado por ReserCaixa,
Debido a la variedad de usuarios que se integran dentro de la Fundación la Caixa. Agraceder la colaboración en el proyecto a
discapacidad intelectual y sus respectivas capacidades cognitivas, Valentín Cardeñoso Payo, a Lourdes Aguilar, a Ferran Adell y a
es necesario que el juego se pueda adaptar a las necesidades de los Patricia Sinobas.
usuarios. Dentro del grupo de usuarios objetivo, se ha
diferenciado los perfiles de lector y no lector, ya que la dificultad 6. REFERENCIAS
para la lectura es un aspecto relevante en dichos usuarios. Por lo [1] Proyecto comunica.
tanto, las actividades se han complementado con ayudas visuales http://dihana.cps.unizar.es/alborada/cientec.html. Accesed:
y sonoras adicionales. 2015-03-27.
La clasificación de los usuarios en un perfil u otro debe ser [2] Buckley, S. and Sacks, B. An overview of the development
determinada actualmente por el personal de apoyo que acompaña of teenagers with down syndrome (11-16 years). Down
a los usuarios al jugar, aunque se pretende que este proceso sea Syndrome Issues and Information, 2002.
automático en un futuro gracias a las actividades de diagnosis [3] Chapman, R. and Hesketh, L. Language, cognition, and
iniciales que se establecerán como parte de la creación del perfil short-term memory in individuals with down syndrome.
por parte del usuario, es decir, junto a la personalización del Down Syndrome Research an Practice, 7(1):1-7, 2001.
avatar.
[4] Chapman, R.S. Language development in children and
Otro aspecto de la adaptabilidad del sistema es la capacidad de adolescents with down syndrome. Mental Retardation and
ajustarse al nivel del usuario. Se han definido 3 niveles de Developmental Disabilities Research Reviews, 3(4):307-312,
dificultad para cada actividad definida anteriormente. Estos 1997.
niveles de dificultad no afectan a la mecánica general de las
actividades, si no al grado de dificultad de las frases en los [5] González Sánchez, J., Cabrera, M. and Gutiérrez, F. Diseño
diálogos, el número de opciones propuestas por el videojuego en de videojuegos aplicados a la educación especial. In VIII
las actividades de comprensión, la ayuda o ausencia de ayuda en Congreso Internacional de Interacción Persona-Ordenador
las actividades visuales, etc. (INTERACCIÓN-2007), pages 35-45, 2007.
[6] Lanyi, C.S. and Brown, D.J. Design of serious games for
4. CONCLUSIONES Y TRABAJO FUTURO students with intellectual disability. IHCI, 10:44-54, 2010.
En este artículo se presentan los condicionantes de una interfaz de
[7] Mayer, R.E. Multimedia learning. Psychology of Learning
usuario derivadas de una funcionalidad relacionada con el
and Motivation, 41:85-139, 2002.
entrenamiento de la comunicación oral, y de las particularidades
del tipo de usuario. El canal visual y el canal sonoro se [8] McFarlane, A., Sparrowhawk, A. and Heald, Y. Report on
complementan para reforzar los mensajes y para proponer the educational use of games. TEEM (Teachers evaluating
ejercicios de entrenamiento de la voz. Los refuerzos que se educational multimedia), Cambridge, 2002.
incluyen en la interfaz facilitan el seguimiento de la historia [9] Oviatt, S. Ten myths of multimodal interaction.
gráfica por parte del usuario. Concurrente con la interacción del Communications of the ACM, 42(11):74-81, 1999.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 448 |
 Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y Accesibilidad
G. Moreno | V. Castillo | K. Williams | N. Menéndez

Caracterización de las Empresas Desarrolladoras de

Software en Panamá en Materia de Usabilidad y
Accesibilidad
Ing. Giankaris Moreno, Ing. Vanessa Castillo, Lic. Kaiser Williams, Ing. Nyder Menéndez
Universidad Tecnológica de Panamá
Panamá, Panamá
(507) 560 – 3602
{giankaris.moreno; vanesa.castillo; kaiser.williams; nyder.menéndez}@utp.ac.pa

RESUMEN Tomando en consideración las definiciones presentadas

En este artículo se muestra un primer acercamiento a las empresas previamente acerca de usabilidad y accesibilidad, se procedió a
desarrolladoras de software en Panamá en materia de usabilidad y realizar una investigación que nos permita obtener y establecer las
accesibilidad, el principal objetivo era tener una idea de cual es el primeras evidencias sobre la situación actual de las empresas
estado actual de la disciplina conocida como Interacción Humano desarrolladoras de software en la República de Panamá, en cuanto
Máquina (HCI – “Human Computer Interaction”) en Panamá. a la utilización de usabilidad y accesibilidad se refiere.
Para ello se llevaron a cabo entrevistas y encuestas a una muestra
de empresas dedicadas exclusivamente a desarrollar software en
2. METODOLOGÍA
Panamá y otras empresas que poseen internamente un Con el fin de conocer si las empresas panameñas desarrolladoras
departamento de desarrollo de software para la misma empresa. de software aplican en sus modelos de desarrollo características
propias de HCI (específicamente usabilidad y accesibilidad), se
Palabras Claves procedió a realizar una investigación sobre el proceso de
Accesibilidad; Desarrollo de Software Panamá; Usabilidad desarrollo de dichas empresas.
Para realizar la investigación mencionada anteriormente, se utilizó
1. INTRODUCCIÓN una metodología que constaba con una investigación a priori
Con el pasar del tiempo el desarrollo de software ha realizado sobre las empresas desarrolladoras de software en la República de
importantes cambios en las metodologías empleadas en el Panamá, posteriormente se le aplicó una encuesta a
desarrollo de procesos y aplicaciones. En sus inicios se hablaba desarrolladores de software pertenecientes a estas empresas y
del ciclo de vida del proceso de desarrollo y existían diversos luego se procedió a realizar el análisis de los resultados obtenidos.
modelos entre los que podemos mencionar el Modelo en Cascada, A continuación se describirán con mayor detalle cada uno de los
Modelo Espiral, Desarrollo Iterativo e Incremental entre otros. pasos mencionados con anterioridad:
Actualmente el modelo que está acaparando la atención de los
desarrolladores es el Modelo de Desarrollo Ágil (Metodologías 2.1 Investigación sobre las empresas de
Ágiles), el cual está más orientado a las personas que a los desarrollo de software existentes
procesos.[8] Para hacer la caracterización de las empresas desarrolladoras de
Al orientar el desarrollo en las personas más que en los procesos, software en Panamá fue necesario realizar una búsqueda de las
se busca centrar tanto el desarrollo como el diseño, en el usuario; empresas dedicadas en el país a desarrollar software, fue por ello
con lo cual se presta especial atención a la usabilidad de la que se utilizó el directorio de empresas de la Cámara Panameña
aplicación que se desea desarrollar. de Tecnologías de Información, Innovación y
La usabilidad según la norma ISO 9241-11 se define como: “el Telecomunicaciones (CAPATEC) del año 2013.
grado en que un producto puede ser usado por determinados CAPATEC es la entidad del país que involucra a empresas
usuarios para conseguir objetivos específicos con efectividad, que generan conocimiento e innovación, utilizando el valor
eficiencia y satisfacción en un contexto de uso específico” [9, 11]. agregado como base de su negocio, empresas de desarrollo de
Los factores principales que deben considerarse al hablar de aplicaciones informáticas, empresas de servicios que utilizan las
usabilidad son la facilidad de aprendizaje, la efectividad de uso y tecnologías de información y telecomunicaciones como
la satisfacción con las que las personas son capaces de realizar sus herramienta fundamental para su modelo de negocio, entre las que
tareas [2] gracias al uso del producto con el que están trabajando, se incluye a las prestatarias de servicios de telecomunicaciones,
factores que descansan en las bases del Diseño Centrado en el llamadas telefónicas, servicios de transmisiones de datos y
Usuario (DCU).[4] similares; así como también las empresas dedicadas a la
Por otra parte la accesibilidad se puede definir como: “el grado en producción de componentes (productores de hardware), usuarios
que un objeto, sitio, servicio, elemento puede ser utilizado por una de software, empresas de servicios sustitutos o complementarios
persona independientemente de sus capacidades físicas o (outsourcing, aplicaciones a la medida, integración de sistemas),
técnicas” [1, 5, 6]. empresas incubadoras de tecnología (parques tecnológicos,
Centros de estudios) [3]. El directorio de empresas de CAPATEC

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 449 |
 Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y Accesibilidad
G. Moreno | V. Castillo | K. Williams | N. Menéndez
identifica dos categorías de empresas relacionadas con el De ser sí la respuesta, ¿Qué herramientas utiliza para la
desarrollo de software: aquellas en las que su actividad principal evaluación de la accesibilidad?
es el desarrollo de software y las que tienen como principal
actividad brindar otros servicios pero poseen un departamento de 2.3 Análisis de Resultados
desarrollo de software para la propia empresa. En total se realizó Luego de recabar la información obtenida con la encuesta
la investigación a las trece empresas que aparecían registradas realizada, se procedió a catalogar los resultados obtenidos para las
como empresas dedicadas al desarrollo de software (diez características evaluadas.
empresas) y aquellas que poseen un departamento interno de
desarrollo de software (tres empresas). 3. RESULTADOS OBTENIDOS
Una vez aplicada la encuesta se procedió a realizar el análisis de
2.2 Encuesta las respuestas obtenidas para cada uno de los aspectos evaluados.
Para recolectar la información sobre las empresas se confeccionó A continuación se muestra la evaluación de las entrevistas
una encuesta cuyo propósito fue obtener información de aspectos realizadas.
vinculados al desarrollo de software, los mismos están basados en
el Modelo de Procesos de la Ingeniería de la Usabilidad y de la 3.1 Análisis de los Requerimientos
Accesibilidad (MPIu+a)[10]. Dichos aspectos son: análisis de Son pocas las empresas que realizan un estudio a profundidad de
requerimientos, diseño, implementación, lanzamiento, la población a la cual va dirigida el sistema. Sólo un 25% de las
prototipado, usabilidad y accesibilidad. A continuación empresas entrevistadas realiza estudios de la población a la que va
presentamos las preguntas formuladas para evaluar cada aspecto: dirigido el desarrollo de la aplicación. Algunos expresaron que la
falta de tiempo impide este estudio, ya que las empresas piden sus
2.2.1 Análisis de los Requerimientos sistemas con carácter de urgencia y estos estudios conllevan
Antes de comenzar a desarrollar el sistema requerido ¿Se lleva a mucho tiempo.
cabo un estudio de la población a la cual va dirigida? Por otra parte un 75% de las empresas realizan entrevistas previas
¿Se realizan entrevistas previas a todas las personas involucradas a las personas involucradas en el sistema, más no se les aplicaba a
en el uso del sistema? todas las personas involucradas sino a una parte ya que debido al
tema de tiempo de entrega del producto no era posible.
¿Se clasifican (en roles y/o perfiles) los usuarios del sistema?
En el aspecto de clasificación de los usuarios del sistema, un 87%
2.2.2 Diseño de las empresas indicaron que sí clasifican los usuarios, ya que
¿En base a que fuente ustedes determinan las tareas a realizar? eso les facilita el proceso de desarrollo desde el inicio. El otro
13% de las empresas indicaron que no realizan la clasificación de
Al momento de diseñar el sistema ¿qué estilo de metodología
los usuarios ya que solo consideraban el tema de los accesos al
utilizan?
sistema y no a un nivel más profundo de roles.
2.2.3 Implementación
¿Antes de iniciar la codificación se ha definido y realizado un
3.2 Diseño
La Figura 1 muestra las Fuentes que emplean las empresas para
análisis de los requisitos de forma exhaustiva?
determinar las tareas; en su mayoría indicaron que aplican
2.2.4 Lanzamiento entrevistas por ser la más común, sencilla y menos costosa, y le
¿Se practica la retroalimentación de parte del usuario para obtener permite a los diseñadores obtener una mayor cantidad de
observaciones de fallos, defectos o virtudes? información. Otro aspecto importante que resaltar es que el 47%
de las empresas manifestó que utilizan la metodología de diseño
De ser sí la respuesta, ¿Esta retroalimentación es utilizada para: apoyada en estándares generales, el 29% emplea la metodologías
implementar mejoras, planificar el mantenimiento, definir fechas basadas en estándares aceptables mientras que un 24% indicó que
para futuras revisiones del producto, otros? emplean otras metodologías en la que mencionaron RAD (Rapid
Application Development), la cual apoya el diseño de sus
2.2.5 Prototipado sistemas, algunas empresas por aspectos relacionados a sus
¿Incluyen ustedes la creación de algún prototipo dentro de las políticas internas no mencionaron cual herramienta emplean.
fases de desarrollo del sistema?
De ser sí la respuesta, ¿En qué etapa(s) se crea(n)?
¿Qué estilo de prototipado se utiliza?
2.2.6 Usabilidad
¿Aplican ustedes alguna evaluación de la usabilidad del producto
en desarrollo?
De ser sí la respuesta, ¿Qué técnicas utiliza para la evaluación de
la usabilidad?
2.2.7 Accesibilidad
¿Aplican ustedes algún tipo de evaluación de accesibilidad al
producto en desarrollo?

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 450 |
 Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y Accesibilidad
G. Moreno | V. Castillo | K. Williams | N. Menéndez
Figura 1 Fuente que determina las tareas a realizar

3.3 Implementación
El 38% de las empresas no define ni realiza un análisis de
requerimientos de forma exhaustiva antes de iniciar la
codificación. Explicaron que sí lo realizan pero no de manera
exhaustiva ya que requiere de mucho tiempo. Solo es presentado
los requerimientos primarios y en el desarrollo van definiendo
otros requerimientos. Estos es así para minimizar tiempo,
alegaban las empresas.

3.4 Lanzamiento
El 100% de las empresas entrevistadas manifestó realizar durante
el ciclo de vida del sistema algún tipo de retroalimentación con el
usuario. El 65% de estas empresas indicaron que las
retroalimentaciones ayuda a proveer mejoras en las futuras
versiones del sistema, mientras que a otras les permite planificar
Figura 2 Evaluación de la usabilidad
mantenimientos.

3.5 Prototipado 4. CONCLUSIÓN

Un 87% de las empresas emplean cierto tipo de prototipo en algún En Panamá, actualmente, es muy poco lo que se conoce e
punto del desarrollo del sistema. Comentaban que esto les implementa en el desarrollo de software en materia de usabilidad
permitía percibir si las ideas de ellos concordaban con la de los y accesibilidad. Una de las principales razones por la cual las
usuarios. EL 94% de las empresas concordaron que al inicio y empresas desarrolladoras de software en Panamá no aplican
durante el proceso es el mejor momento para crear un prototipo, pruebas de usabilidad y accesibilidad a sus productos de software
ya que de esta manera el cliente va observando cómo va a quedar es el alto costo en tiempo y recursos que representa aplicar dichas
el sistema antes que sea terminado; el 6% restante que mencionó pruebas durante y posterior al desarrollo de los mismos
que al final crean el prototipo no están claramente familiarizados
con las metodologías utilizadas en HCI, las cuales indican 5. REFERENCES
claramente que el prototipado se realiza antes y durante la etapa [1] C. A. Vanegas and S. P. Nuñez, "Accesibilidad en el diseño
de desarrollo del sistema. de páginas web," Vínculos, vol. 9, pp. 177-190, 2013.
El 58% de las empresas indicaron que el estilo de prototipo que [2] C. M. Barnum, Usability testing essentials: ready, set... test!:
más se utilizaba (aunque algunas empresas usan varios Elsevier, 2010.
simultáneamente dependiendo del sistema a diseñar) era el [3] CAPATEC. (2014). Cámara Panameña de Tecnologías de
prototipado de software, ya que este permitía que los integrantes Información, Innovación y Telecomunicaciones. Available:
se fueran familiarizando con la herramienta a utilizar. Un 21% http://www.capatec.org.pa/acerca-de-capatec/
indicó que empleaban prototipos de papel mientras que el otro
[4] D. A. Norman, The design of everyday things: Basic books,
21% emplea los bocetos.
2002.
3.6 Usabilidad [5] D. Conforto, "Accessibility: Discussing Human-Computer
En la Figura 2 podemos observar que el 75% de las empresas no Interaction on the Web," in Computers and Education, ed:
aplica ningún tipo de evaluación de usabilidad a sus sistemas Springer, 2003, pp. 127-137.
creados, algunas expresaron no creer necesario realizarlas y otros [6] J. Abascal, "Interacción persona-computador y
que el costo que esto acarrea para las empresas que utilizan el discapacidad," Revista Minusval, número especial dedicado a
sistema no lo desean costear. El otro 25% indicaron que sí la” Discapacidad y las Nuevas Tecnologías”, Ministerio de
aplicaban evaluación de usabilidad en sus productos de software, Trabajo y Asuntos Sociales, pp. 18-21, 2002.
emplean combinaciones de técnicas para dar mejor soporte a los
resultados, las más utilizadas son el recorrido de usabilidad, [7] R. A. Española. (2014). Real Academia Española (23 ed.).
observación de campo y entrevistas. Available: http://lema.rae.es/drae/?val=caracterizar
[8] R. A. G. Bustos, "Métodos de desarrollo de software: El
3.7 Accesibilidad desafío pendiente de la estandarización," Theoria, vol. 12,
Muy pocas empresas toman en cuenta que tan accesibles son sus pp. 23-42, 2003.
sistemas. En esto se percibió que es por falta de expertos que
sirvan de apoyo a los programadores. El 19% de las empresas [9] S. ISO, "9241-11. 1998," Ergonomic Requirements for
manifestaron que sí evalúan la accesibilidad de sus sistemas e Office Work with Visual Display Terminals (VDTs)–Part II
indicaron que solo han utilizado las herramientas W3C CSS Guidance on Usability.
Validator y W3C HTML Validator Service. Ellos conocen de [10] T. Granollers, J. Lorés, and F. Perdrix, "Modelo de proceso
otras herramientas pero esa ha sido la que ha resultado más de la Ingeniería de la Usabilidad. Integración de la ingeniería
cómoda para ellos trabajar. del Software y de la Usabilidad," in Proceedings of the
Workshop de investigación sobre nuevos paradigmas de
interacción en entornos colaborativos aplicados a la gestión y
difusión del Patrimonio cultural, COLINE, 2002, pp. 11-12.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 451 |
 Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y Accesibilidad
G. Moreno | V. Castillo | K. Williams | N. Menéndez
[11] X. F. Grau, "Principios Básicos de Usabilidad para Ingenieros Software," in JISBD, 2000, pp. 39-46.

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 452 |
Caracterización de las Empresas Desarrolladoras de Software en Panamá en Materia de Usabilidad y Accesibilidad
G. Moreno | V. Castillo | K. Williams | N. Menéndez

XVI Congreso Internacional de Interacción Persona-Ordenador | Software, Architecture and Interaction

| 453 |
ACCESSIBILITY AND SEMANTIC WEB
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros

The Effects of Cross-modal Collaboration on the Stages of

Information Seeking
Dena Al-Thani, Tony Stockman, Anastasios Tombros
School of Electronic Engineering and Computer Science
Queen Mary University of London
Mile End road, London, E1 4NA, UK
{d.al-thani, tony.stockman, a.tombros }@ qmul.ac.uk

ABSTRACT We term the process under investigation cross-modal,

Previous studies of users with visual impairments access to the collaborative Information Seeking (CCIS). The purpose of this
web have focused on human-web interaction. This study explores study is to better understand the CCIS process and its effects on
the under investigated area of cross-modal collaborative stages of the individual IS process presented by Marchionini and
information seeking (CCIS), that is, the challenges and White [14].
opportunities that exist in supporting visually impaired (VI) users To date, very few studies [24][26] have examined the process of
to take an effective part in collaborative web search tasks with CIS or attempt to draw a framework and models that describe its
sighted peers. We conducted an observational study to investigate processes, therefor in our paper we take the Marchionini and
the process with fourteen pairs of VI and sighted users in co- White [14] model of individual information seeking and map its
located and distributed settings. The study examined the effects processes to the individual and collaborative IS activities
of cross-modal collaborative interaction on the stages of the performed. The way their model introduces the IS activity as a
individual Information Seeking (IS) process. The findings showed process that includes discreet stages will help to inform our
that the different stages of the process were performed understanding of how users employing different modalities go
individually most of the time; however it was observed that some about performing each stage of the IS process both individually
collaboration took place in the results exploration and and collaboratively. The project is motivated by the observation
management stages. The accessibility challenges faced by VI that many activities in both educational and work settings involve
users affected their individual and collaborative interaction and teamwork, and that internet searching often forms an important
also enforced certain points of collaboration. The paper concludes component of such activities. Specifically, we wish to understand
with some recommendations towards improving the accessibility what barriers may exist to visually impaired searchers taking part
of cross-modal collaborative search. in CCIS, and what approaches are currently employed by CCIS
participants to overcome or work around these barriers.
1.1 Categories and Subject Descriptors
The paper starts with a brief overview of related work on the
H.5.3 [Group and Organization Interfaces]: Collaborative
accessibility of the single user IS process and a summary of the
computing; K.4.2 [Social Issues]: Assistive technologies for
current research on CIS. We then present our motivation and
persons with disabilities
research questions, before describing the details of the study and
1.2 General Terms results obtained. The effects of cross-modality on the awareness
Human Factors, Experimentation, Design and division of labour in CIS involving VI and sighted
collaborators have been reported elsewhere [1], where the
1.3 Keywords patterns of behavior to achieve awareness and strategies to divide
Collaborative information seeking; cross-modal interaction, labor are described. This paper focuses on examining the effects
information seeking process; accessibility; web search of cross-modality on the structure of the IS process and
identifying the stages in which collaboration occurred and the
1. INTRODUCTION reasons for it taking place. This paper concludes by discussing
In the context of Information Seeking (IS), observational studies the implications of the results to inform the design of a tool that
reveal that group members often collaborate when searching for supports CCIS activities.
information, even if they were not explicitly asked to do so
[15][12] The activity that involves a group of people searching in 2. Related Background
a common information seeking task is called Collaborative 2.1. Accessible Information Seeking
Information Seeking (CIS). Over the past few years, research in Despite the fact that issues surrounding web accessibility have
this area has been gaining much interest. This attention on multi- attracted increased attention in research and in the commercial
user IS has always assumed all group members are using visual world [3][7], the area of accessible IS is rarely examined. The
displays. This focus on the visual modality limits the relevance of sequential nature of screen-readers imposes many challenges on
previous research to users employing other interaction modes for visually impaired (VI) web users. These challenges range from
accessing and managing retrieved results. This paper presents the the lack of context to overload of short-term memory. Studies
results of an exploratory study conducted to investigate the effect have highlighted these challenges and proposed a set of
of the presence of two different modalities on the process of CIS.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 454 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
guidelines to be considered when designing an accessible search behaviours in information seeking, there were also important
engine [2][5]. While these studies focused on the usability differences. The differences are related to the contextual aspects
aspects of the problem, a recent study by Sahib et al. [22] associated with social factors. The results of the studies were
examined the challenges that this problem imposes on the similar and both concluded that Kuhlthau’s process did not
different stages of the IS process and hence on the behavior of the completely address the social dimension of CIS.
VI information seeker. Golovchinsky et al. [6] proposed a taxonomy of CIS collaboration
In the comparative analysis of Sahib et al. [22] an observational in which they introduced four dimensions of collaboration. The
user study was conducted with 15 VI and 15 sighted participants. focus of this taxonomy is on technical models of collaboration
The participants were asked to perform a complex task which rather than social models, they proposed four different
required a high level of cognitive effort (i.e., detailed planning of dimensions of collaboration, which are intent, depth of
a vacation). The results of the comparative study identified major mediation, concurrency and location. (1) Intent: Explicit vs.
differences between the IS behavior of VI and sighted Implicit: When implicit collaboration is supported, the search
participants. These differences were particularly apparent in the engine uses data from previous anonymous users with similar
query formulation and results exploration stages. One of the main information needs or similar behavior to offer recommendations
barriers highlighted was the inaccessibility to screen-reader users to users. Recommender systems thus support implicit
of query-level support features provided by search engines at the collaboration. In contrast, in systems that support explicit
query formulation phase. Also, In the search exploration stage the collaboration, users explicitly work together in the query
number of results viewed by VI participants (mean of 4.27 (SD= formation and results exploration stages. Microsoft’s
2.15) web search results viewed) was considerably lower than the SearchTogether [17] system is an example of an application that
number of results viewed by sighted participants (mean of 13.40 supports explicit collaboration. (2) Depth of Mediation: This
(SD= 7.39)). These findings led to the development and dimension refers to the level in which the mediation of
evaluation of a search interface [23] that aim to tackle the issues information seeking is applied in a system. Pickens et al [20]
identified in their study with special attention to the results introduced Cerchiamo, in which CIS is mediated at the
exploration and managements stages of the single user IS algorithmic level. The Cerchiamo collaborative search engine
process. divides the labour between two collaborators. One collaborator is
the “preceptor” who investigates new fields of information, while
2.2. Collaborative Information Seeking the other collaborator is the “surveyor,” who looks at and
Collaborative information seeking is defined as the activity explores each new field in detail. (3) Concurrency: Synchronous
performed by a group of people with a shared information need or vs. Asynchronous: Concurrency, which does not actually relate to
‘goal’ [15]. A survey by Morris [15] is referred to as one of the time, means that the system should allow the actions of a user to
earliest studies that encouraged increased attention in this area. be conveyed in some way to other team members. In other words,
Her survey, which she revisited lately [16] did not attempt to systems should support awareness between collaborative users
examine users’ behavior, yet it provided a wealth of data within a group engaged in different information seeking
regarding the prevalence of collaborative web search and the activities. (4) Location: Co-located vs. Distributed: Distributed
tasks motivation and tools involved. collaboration may require additional communication channels
such as instant messaging, offline messaging services and voice
Despite the extensive research in this field in the past few years, chat.
there is no consensus over a single model or framework that
describes the CIS process. Though there have been a number of
research attempts to develop models either to describe the CIS 3. Research Questions and Motivation
environment [24] or to classify the systems supporting it [6]. The increased interest in CIS reflects the fact that it is a more
Shah [24] proposed a layered model of information seeking. The frequent activity in our daily lives. However, there has not
model contains four layers which are information, tools, users previously been any attempt to consider the way CIS activities
and results. The information layer refers to the different resources may be different when collaborators use different interface
and formats of information contained in the entire search space. modalities, which is the focus of the current study. The questions
The tools basically refer to the search engines and the we wished to examine are as follows:
functionality they provide. The user layer includes the users, their Q1: What stages of the information seeking process are done
profiles and any mechanisms available for personalization. The collaboratively and how?
final layer is the results, ultimately the product of the search Studies have revealed that though IS stages are typically done
process, including all relevant information, users’ comments and individually. Nevertheless collaborators may choose to work
metadata. together at many points in the process [26][27]. These studies
Studies [8][26] examined the applicability of Kuhthau’s [11] also stressed that collaborative activities are generally ill
process of individual information seeking in the context of a structured [13]. Hence, very few efforts have been made to define
group. Hyldegard [8] observed a group of 10 students over 14 a model of CIS activity [8][26]. In addressing this question, we
weeks when performing information seeking activities and Shah aim to explore how often collaboration occurs at each stage, how
and Gonzalez-Ibanez [26] conducted a laboratory study involving much collaboration takes place and what techniques are used to
42 pairs performing a general exploratory search task. Both support it? Furthermore, we wish to examine the effects of cross
concluded that though there are evident similarities in the general modality on group performance and techniques employed to
stages of the process between individual and collaborative address issues arising from the use of different interaction modes.
For example, previous research on VI users IS behaviour has

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 455 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
highlighted that most challenges are encountered during the session was to organize a business trip to the United States (US)
results exploration phase [22]. These difficulties arise because while the task in the distributed session was to organize a holiday
examining large search result sets using a screen reader can be a trip to Australia. They were given dates of engagements in
lengthy process, due to the sequential nature of speech and other different cities and times when leisure or work activities needed
limitations relating to the navigation of complex information with to be identified. Participants were asked to organize the travel,
a speech-based screen reader [19][28]. accommodation and activities in these cities. In advance of each
Q2: What are the strategies and techniques employed to manage study we made sure that participants had not visited the cities
search results? before. The complexity of the two tasks was counterbalanced to
This question also explores the management of search results in make them approximately equal in their level of difficulty. They
the presence of a common goal between group members who use were balanced for subtasks and amount of information retrieved.
different access modalities. It seeks to identify approaches and
techniques used to organize, exchange and manage search results. 4.3. Sessions
Both the co-located and distributed sessions took place at the VI
4. Observational Study participants’ workplace as the intention was to observe the
We observed 14 pairs of users, each pair comprising one sighted participants in real world settings. For the same reason they were
and one VI partner, performing two CCIS tasks. For one of the asked to use their own PCs and the web browser and search
tasks the partners were co-located, while in the other they were engines they normally used. In the distributed sessions,
located separately. Task order and location were balanced to participants were seated in remote locations and told that they
counter learning effects. could use one or more of the following methods to communicate:
email, instant messaging, shared documents, or any tool they
found suitable. While in the co-located setting, participants were
4.1. Participants seated in the same room and asked to communicate verbally,
We recruited 28 participants, 14 sighted and 14 VI, via mailing though they were free to use additional methods if desired.
lists; table 1 contains their demographic data and the
technologies they used. Three VI users employed headphones for During the first session, participants were briefed about the
speech output, while the other five used speakers. All the VI purpose of the study and asked to fill a pre-study questionnaire
users used the speech-only version of the Jaws 1 screen-reader. which collected their demographic data, information about the
Two pairs were colleagues for more than two years. None of the technologies they use for this type of task and their level of
other pairs had worked together on a regular basis. experience with web searching. In each session, they were
provided with a brief document giving information about the trip
Table 1. Demographic and technology information about they were required to organise, including dates when they needed
participants to be in different places and details of the types of activities they
Visually Impaired Sighted Participants were required to book.
Participants Following that, participants were asked to perform the tasks and
Age 2(21-29), 4 (30-39), 2(18-20), 3(21-29), 3 about 35 minutes into their work the principle researcher asked
3(40-49), 5 (50-59) (40-49), 5 (30- them to stop. We intentionally did not inform them in advance
39),1(50-59) about the amount of time they have to perform the task as we
were not interested in examining the influence of time pressure
Gender 8 Male,6 Female 8 Male, 6 Female in this study. We concluded each session with a brief semi-
Browser 12 IE, 8 Safari, 5 6 IE, 4 Firefox structured interview to discuss the participants’ experience of the
Used Firefox 3 Safari, 1 Chrome task.
(Multiple All sessions were videotaped, having obtained the approval of the
Answers) participants. During the tasks, the screens of both participants
were captured using screen recording software. The VI
Frequency 3 Daily , 2 Once a 2 Weekly, 3 once a
participants’ screens were captured using a video camera, as we
of CIS week, 5 once a month month , 6 Once in the
noticed in a pilot of the experiment that screen recording
Activity 1 Once in the past six past six months, 3
software sometimes reduced the responsiveness of screen-
months, 3 Never Never
readers. Additionally the principle researcher made notes of
observations during the sessions.
4.2. Tasks
Previous CIS research has identified that simple information 4.4. Data Collection
look-ups and fact finding tasks do not benefit from CIS activity, The main source of data was the video recordings of the
while multi-facetted and exploratory search tasks are likely to be interactions between partners and their interactions with the
more appropriate for use in CIS investigations [13][17]. search engines and the post-study interviews. After transcribing
Therefore, participants were asked to collect relevant information the videos, we used the Open and Axial coding phases of
for two exploratory tasks that were designed to be realistic work grounded theory [4]. Open coding is the process of generating
and leisure tasks respectively. The task used in the co-located initial concepts from the data while axial coding is when the data
is put together to establish connections between the different
1
http://www.freedomscientific.com/ concepts and categories. The selective coding process includes

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 456 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
the formalisation of the data into theoretical frameworks. the observed sessions, a sum of 17 instances were recorded where
However, for this study, we stopped our data analysis after open the VI participants asked for assistance from their sighted
and axial coding as we wanted only to explore the behaviour of partners in the co-located setting. , 13 of these cases were
the collaborating searchers, as opposed to developing a new accessibility issues while four of them were related to navigating
theory. large result sets.
The coding scheme captured indicators of each IS process stage. In all distributed sessions, a common pattern for the execution of
In relation to the interactions between partners, the coding the stages was observed. After the initial division of labour, both
scheme captured instances of collaborative IS activities and the participants performed the Information seeking tasks individually
reasons for these taking place. Semi-structured interviews were and shared the results via email or instant messages. Unlike the
conducted individually with each participant to complement the process in the co-located session, in the distributed sessions there
data collected during the study. On the quantitative data, we was no evidence that participants discussed division of labour
carried out statistical testing at p<0.05 with a two-tailed unpaired later in the process. It was observed. However, if one participant
t-test. completed all the tasks assigned to him/her, they would decide to
complete their partner’s outstanding tasks. Additionally, there
5. ANALYSIS were virtually no interactions between partners relating to
making sense of retrieved results. There were only three requests
5.1. Stages of the Collaborative Process for assistance recorded and all were access related.
In general, the process started with a stage in which the pair
divided the tasks to be performed. At this stage, usually one of 5.2. Stages of the Information Seeking
the participants took the lead and assigned tasks to themselves
and to their partner. During this process, the other partner might Process
either agree on the task s/he is being given or suggest another For the most part, the separate stages of the information seeking
task. For instance, visually impaired experienced web users process were done individually. Nevertheless in the co-located
sometimes anticipated that certain tasks were likely to require a sessions, a number of instances were recorded in which query
longer time for them to complete, therefore they sometimes formulation, results exploration, query re-formulation and the
suggested they performed other tasks. Seven VI participants in search result management stages were accomplished
the co-located setting and 10 VI participants in the distributed collaboratively.
setting preferred searching for a tourism site to booking a hotel
room, because the latter task involved filling an online form. In 5.2.1. Query formulation
the interviews, VI participants explained that from previous When a participant is assigned a particular task, he/she
experience of filling web forms they knew that this process can immediately opened a search engine and entered a query
sometimes be lengthy or not feasible due to the presence of keyword. Usually the initial query would be broad and once a
inaccessible form elements. relevant result set is found, the participant might choose to
narrow down the search to a more specific query with more
In the co-located sessions, an iterative process was observed. This
keywords to obtain the information they need. However, this was
process mainly involved three stages. In the first stage the pair
not the case with VI users, as the average length of the initial
spent from 2 to 5 minutes looking into and discussing the task.
query entered by sighted participants was 2.65 (SD: 0.84) in the
The discussion at this stage mainly related to an initial division
co-located setting and 2.93 (SD: 0.54) in the distributed setting,
of labour. At this stage the task was divided into smaller sub-
while the average length of the initial query entered by the VI
tasks. However in the majority of cases, partners only decided on
participants was 3.37(SD: 0.96) in the co-located setting and 3.31
who would do each of the first sub-tasks. In stage 2, after each
(SD: 0.95) in the distributed setting. In both settings, the average
partner had been assigned a sub-task, each participant started to
length of queries by sighted participants is shorter than that of VI
perform the information seeking process individually. Once a
users. The result were statistically significant in the co-located
piece of information was found (e.g. once a sub-task was
setting at ( t (26))=2.1058, p=0.045) and not statistically
completed), the participants usually paused and notified their
significant in the distributed setting at ( t(26)= 1.2818, p=
partner about the completion of this sub-task by discussing the
0.2112). This result agrees with a previous comparative study
outcome and search results found (Stage 3). The discussion in
[22] of the search behavior of VI and sighted users. In interviews
stage 3 always revolved around three main aspects: division of
conducted as part of [22], VI users confirmed that they often try
labour, making sense of the results and reviewing the remaining
to express their complete information need in a long, precise
sub-tasks. Stages 2 and 3 were then repeated until the task is
query, in an attempt to reduce the number of results they need to
completed. However, in some cases a participant interrupted
browse to reach the desired result.
his/her partner during a task. Two main reasons were noticed for
such behaviour. One reason was that the participants would need Returning to the present study, a number of instances of
to browse search results together either to collaboratively make collaboration were observed at this stage; participants sometimes
sense of the retrieved information or in some cases VI suggested query keywords for his/her partner. In all co-located
participants would face difficulties in viewing large volumes of experiments, the average of 0.36 (SD: 0.66) instances of
search results, due to the limitations of speech-based screen- suggesting query terms have been recorded, while only one case
readers, and so asked their sighted partners for assistance. The was recorded in the distributed setting. In situations where the
other reason was that some websites were inaccessible and it was participant was unable to find results that satisfied the
impossible for the VI partner to complete the task individually. In information need, his/her partner usually suggested another query

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 457 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
keyword. This suggestion was either based on prior knowledge, browser window whereas VI users tended to open multiple
or based on the context of the task. For instance in the windows to keep track of retrieved information. In the co-located
conversation extract below, one participant was finding a hotel in setting, the most used note taking tool was Microsoft word. 10 VI
Los Angeles (L.A). This participant suggested the query keyword participants and 7 sighted participants have used it. The next
for her partner, who was looking for a restaurant to dine in L.A. most used tool by sighted participants was using pen and paper;
She suggested that the restaurant had to be near the hotel, as four sighted users requested a pen and paper to note down
shown in the excerpt below: retrieved information. Additionally two VI participants and one
SP: “I will look for a place to dine in L.A.” sighted user used the Microsoft note pad application. One
participant established skype communication and added the
VP: “You can Google restaurants near Beverly Hills”
retrieved information into the Skype session. In most of the
5.2.2. Search Result Exploration conducted sessions, both participants would store the retrieved
The number of search results explored by sighted users (average information in one of the previously mentioned forms. However,
7.14, SD: 3.37) is statistically significantly higher than that for VI in three sessions only one participant noted down the retrieved
participants (average 3.92, SD: 2.12) with ( t(26)= 2.798, information and the other participants entirely depended on their
p=0.009) in the co-located setting. In the distributed setting, partners. In two sessions it was the sighted users who kept track
although the difference was smaller, it was still statistically of the retrieved information and stored it, while in one session it
significant ( t(26)= 2.32, p= 0.03). It was recorded that the was the VI user who organized and stored the retrieved
average number of search results explored by sighted users information in a word file.
(average 6.79, SD: 2.38) and by VI users (4.71, SD: 2.64). In the distributed setting, the most used note taking tool was
Collaboratively exploring a set of search results was commonly Microsoft word with nine VI participants and five sighted
observed in the co-located setting only. In all sessions, an average participants. Three sighted participants requested pen and paper
of 3.75 (SD: 1.25) instances of exploring results collaboratively to take notes, and two VI participants used the Microsoft notepad
were recorded. The average number of search results viewed application. In the distributed setting, three VI participants and
collaboratively is 0.5(SD: 1.38). All such collaboratively obtained four sighted participants preferred storing their notes and
results were triggered by the VI partner needing to explore more retrieved information using the communication tool, which was
results faster. An example of comments taken from two different either email or chat messaging. In these situations, one team
studies in which the VI partner asked the sighted partner to assist member usually kept track of the information shared in the
when exploring the search results is shown below communication tool by storing them in a word processing
From Study #3 co-located session: VP: “Could you application. Four VI participants and three sighted participants
just glance at these results yourself?!” kept track of the retrieved in information received from their
From Study #2 co-located sessions: VP: “It is listing a partners and stored them in a Microsoft word file.
number of places, Can you see L.A. there?” It was observed that the information noted down or exchanged by
participants was of five types. These were: a website link, a
5.2.3. Query Re-formulation website link with details, details about the sub-task, simply
This stage occurs when the user is not satisfied with the initial keywords that refer to the information or copying a part of the
retrieved set of results and chooses to submit a new query. The webpage. Figure 1 shows the percentage of each identified
new query might be a term from prior knowledge or from category in the co-located and the distributed settings.
information that was just found. Most of the time this stage was
done individually with an average of 1.07 (SD: 1.14) in the co-
located setting and an average of 0.7 (SD: 1.24) in the distributed
setting by VI participants. Re-formulation of queries by sighted
participants had an average of 2.93 (SD: 2.47) in the co-located
setting and an average of 1.29 (SD: 1.98) in the distributed
setting. It was observed that this stage was performed
collaboratively in only 3 instances in the co-located setting. In
these instances the partner interrupts and suggests a query term
when one partner is not satisfied with the set of results. An
excerpt of a conversation that captures query re-formulation
accomplished collaboratively is shown below:
From Study #4 SP:” I think, perhaps Virgin Atlantic
doesn’t have direct flights to Las Vegas”.
VP: “Yes, this is what I was thinking about”.
SP: “Let us try another keyword; perhaps you can
Google direct flights to Las Vegas”.

5.2.4. Managing Search Results

Since the task was conducted in one session, users did not
employ favorites or bookmarks to keep track of required
information. Sighted users tended to open multiple tabs within a

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 458 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
As shown in Figure 1, the majority of information kept by both
sighted and VI users in both settings were either website links
with details (52% in the co-located setting and 59% in the
distributed setting) or details only (25% in the co-located setting
and 16% in the distributed setting). Moreover, the amount of
information kept by VI users is nearly half the amount of
information kept by sighted users. In fact in the distributed
setting, sighted users exchange rate of information to VI users
was 2:1. In the post-study interviews, seven VI participants
highlighted the difficulties of having to switch between three
different applications: the web browser, the email client and the
note taking tool during the process.
In the co-located setting the retrieved information was noted
down but was not exchanged between the participants by any
Co-located Distributed
means. The participants were merely verbally notifying their
VI Sighted VI Participant Sighted
participant participant Participant partners about their progress or asking about their partner’s
progress as a means of updating their awareness information.
Entering query 2.38 2.08 2.51 1.37
term [1:12] [1:59] [2:01] [0:58] Whilst in the distributed setting, partners exchanged information
by email or instant messaging as well as using note taking tool.
Exploring 3:58 2:11 3:17 2:07
search results [2:39] [1:49] [1:52] [ 1:17]
1.4 5.3 Time intervals
Browsing 14:29 14:19 10:44 11:49 We observe the time spent by participants on each stage. This
websites [8:48] [8:47] [6:47] [6:08]
includes time spent entering a query, times spent viewing search
Managing 2:50 5:57 1:59 2:50 results pages, time spent browsing websites, and time spent
information [2:25] [3:06] [1:52] [3:52]
managing information. In addition, time spent dealing with an
Chat 0:00 0:00 06:56 8:36
error (whether it’s a connection error, interface error or
[3:25] [4:37]
accessibility issue) and time spent switching from one application
Encountering 0:23 0:00 0:20 0:01
error [0:43] [0:28] [ 0:02]
to another. Table 2 shows the average time interval spent in each
stage in both settings. The figures shows that the most apparent
Switching 1:45 0:35 1:21 0:47
applications [0:19] [0:31] [0:34] [0:27]
differences between the two groups of users were in the results
exploration stage, retrieved information management stage,
communication stage and switching from one application to
another. In the results exploration stage in both settings VI users
spent on average a longer time then their sighted partners.
Though the differences were not statistically significant with t-
test results ( t(13) = 2.05, p= 0.06) in the distributed setting and (
t(13) = 1.95, p=0.7) in the co-located setting. According to our
observations the main reason that made the VI participants spend
on average more time in this stage is the serial nature of speech
that would make the process of going through search results
longer.
Table 2. The average time interval spent (in seconds) in each
stage in both settings by both groups of users in the study
(Mean [standard deviation])

In managing retrieved information, sighted participants spent

longer time in both settings. However, the differences are not
statistically significant using at ( t(13)=1.95, p= 0.72) in the co-
located setting at ( t(13) =2.05, p= 0.06) in the distributed
setting. Additionally, it was observed that VI users spent
considerably more time switching from one application to
another. The applications were internet explorer, the email client
or instant chat application in the distributed setting and word
processing document or note pad. In the post-study interviews
eight VI participants have highlighted the difficulties of having to
Figure 1. The total percentage occurrence of each type of switch between the three different applications.
information kept or exchanged in both co-located and
The average time consumed browsing web search results by both
distributed sessions
groups was significantly higher in the co-located setting with t-

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 459 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
test results (t(26)= 2.27, p= 0.03). As in the distributed setting experiments, only one team member was taking notes, while in
participants spent a considerable about of time using the email seven of the distributed studies only one team member was
client or instant chat messaging service to provide their partners taking notes.
with updates about their progress. The observations showed that the amount of information kept and
noted down by sighted participants was higher than the amount of
6. Findings and Discussion retrieved information kept and exchange by VI participants. This
is likely to be the result of two factors. The first being that
6.1. CIS Process sighted users viewed more results than VI users and hence they
Clearly identifying the stages of the CIS process was not among kept and exchanged more retrieved information. The second
the formulated research questions; however having an insight of factor is related to the cognitive overhead and time delays that VI
the stages would be of benefit and can help in identifying the users encounter when switching between the web browser and an
phases which are influenced by the presence of two different external application used to take notes. This itself is likely to
modalities. According to the literature, the process of CIS is not increase the cognitive load on VI users and hence slow down the
well-defined and can largely differ according to the task process. The effect of this factor was more apparent in the
performed. London [13] introduced a general model of distributed setting where VI users were required to switch
collaborative activity. The model comprises three main phases: between three applications: the email client or instant chat
(1) the problem setting phase in which collaborators spend time application, the web browser and note taking application. The
understanding the problem and identifying resources required for amount of information kept and exchanged by sighted users was
solving it. (2) A direction setting phase which involves more than double the information kept and exchanged by VI
organizing group activities and agreeing on actions, and (3) the participants reported in the analysis section.
implementation phase in which collaborators complete the task
assigned to them. He emphasize that this stage can differ
according to application area and group size.
7. Implications and future work
The results and findings of our study clearly indicate that there
We observed a similar structure in the current study. The pairs are a number of ways that the CCIS process could be made more
started by discussing and making sense of the given task. They accessible and that the tools used currently do not address the
then assigned different subtasks to each other and started process adequately. The motivation to improve this situation is
conducting the information seeking task individually. As seen in strengthened by the frequency of team working both in education
the analysis section, it was observed that in cases when the and employment [15], of which web searching often forms an
partners were colleagues, the VI partner would delegate the task important part. This section discusses the implications of the
that might contain an inaccessible interaction to the sighted current study for the design of technical solutions to support CIS
partner. This action contributed to enhancing the performance of in a cross-modal context, some of which will form future work in
the pairs and their efficiency in completing the task. this project.
There was clear evidence of similarities between the stages of the
individual IS process and stages of collaborative IS as reported by 7.1. Providing an Overview of Search
Hyldegard [8] and Shah and Gonzalez-Ibanez [26]. Even though Results
participants often performed the IS task individually before The search results exploration was a stage that was done
sharing the results with their partners, as shown in the analysis collaboratively in many instances. The reason behind this sort of
section, some stages were performed collaboratively for various collaboration is that the VI participants needed help from their
reasons. The frequency of collaboration largely differed; it sighted partners to navigate through a large volume of search
occurred mostly in the search results exploration stage in the co- results. This also was highlighted by studies that compared VI
located sessions and in the results management stages in both individual information seeking behaviour with sighted
settings. information seeking behavior [9][22]. These studies have stressed
that this stage is the most challenging and time consuming stage
6.2. Result Exploration for VI users. In fact, this stage is assessed to be two times longer
Collaboration was triggered when the VI participant would ask when performed by VI users than sighted users [9]. The
the sighted participant to assist in going through a large volume sequential nature of speech rendered by screen-readers over loads
of search results. Examination of large sets of search results by the user’s short-term memory and provides no means of
Speech-based screen-readers can be time consuming and imposes representing spatial layout and contextual information of web
a number of challenges such as short term memory overload and pages [19]. Developing a mechanism that provides VI group
a lack of contextual information about the data presented members with an overview of search results and the ability to
[19][22][28]. Additionally, current screen-readers provide almost zoom in on particular information of interest could help in
no mechanism for overviewing a set of search results. increasing the VI group members’ independence in CCIS
activities.
6.3. Results Management
The results management stage was also done collaboratively. The 7.2. Improving the Management and
motivation behind users’ collaboration in this stage was that they Sharing of Search Results
were encouraged to collaboratively work together and produce It was observed in the study that managing search results was one
one outcome at the end of the task. In three of the co-located of the main obstacles faced by VI users. This was more apparent

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 460 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
in the distributed setting where the user is required to switch screen-reader voices and/or spatially distributing the auditory
between three different applications. Moreover, observations in representations of queries and comments made by different group
both settings highlighted differences in individual approaches members.
employed by sighted and VI users when managing search results.
A recent publication by Sahib et al. [23] described an integrated 8. Conclusions
tool that allows VI users to keep track of their search progress, This paper describes an exploratory study that examines the
and manage their search results. An evaluation of the tool with VI effect of cross-modal collaboration on the stages of information
participants resulted in high satisfaction rates as they found it seeking in co-located and distributed settings. The findings show
easier to handle search results within the tool as it removed the that there is a clear influence of the different modalities and
overhead of switching between a number of applications. settings on the different stages of information seeking. The most
Extending this approach to a collaborative context, including apparent collaborative issues occurred in the results exploration
tracking the search progress of all collaborating searchers and the and management stages. Some of these problems have an
management of results at the individual and group levels, appears underlying accessibility issue caused by to the limitations in the
to be worth investigating, to see if similar or additional way information is presented and navigated using speech-based
advantages accrue. However, a utility that allows collaborators screen readers. This paper concludes by proposing some design
to recall the websites visited and query keywords entered by their implications that can be considered when designing a tool that
partners is clearly not sufficient, as our findings showed the supports CCIS. The results of the study highlight that
majority of information exchanged regarding search results considerably more work needs to be done to improve the
included website links and details of the information retrieved. accessibility of both individual and cross-modal information
Therefore, a tool to support CCIS needs to provide better seeking.
integration of the whole process as well as supporting the sharing
of websites and details of results found. An additional complexity 2. References
in providing an integrated tool in the context of CIS, as opposed [1] Al-Thani, D., Stockman, T., Tombros, A., (2013) Cross-
to individual IS, is that this tool will be shared between users Modal Collaborative Information Seeking (CCIS): An
employing different interaction modes, and must be accessible Exploratory Study. In Proceeding of the 27th International
through these different modes. British Computer Society Human Computer Interaction
Conference. London, UK .
A tool like SearchTeam 2, which is a commercially available
website for collaborative search, provides the collaborators with a [2] Andronico, P., Buzzi, M., Castillo, C., & Leporini, B. (2006).
common place to share details of websites, links and comments. Improving search engine interfaces for blind users: a case
This is similar to a feature that Diigo [10], a collaborative study. Universal Access in the Information Society.
bookmarking website, provides that allows users to share [3] Caldwell, B. (2008). Web content accessibility guidelines
bookmarks. (WCAG) 2.0. W3C.
[4] Corbin, J. M., & Strauss, A. C. (2008). Basics of Qualitative
7.3. Improving Awareness of the Search Research: Techniques and Procedures for Developing
Query Terms and Search Results Grounded Theory (Third ed.). Sage Publications, Inc.
Allowing collaborators to know the query terms entered and
[5] Craven, J., Brophy, P., & Britain, G. (2003). Non-visual
results viewed by their partners would certainly inform them
Access to the Digital Library: The Use of the Digital Library
about their partner’s progress toward the completion of a task.
Interfaces by Blind and Visually Impaired People.
Additionally, having a view of partners search results can allow
Manchester Metropolitan University, Centre for Research in
sighted users to collaborate with their VI partners while going
Library and Information Management.
through large amounts of search results. A tool like Coagmento
[24] provides a mechanism to allow users to be aware of their [6] Golovchinsky, G., Pickens, J., and Back, M. A taxonomy of
team members’ activities. This is done by dedicated views that collaboration in online information seeking. In JCDL
are updated with query terms used and web pages viewed by Workshop on Collaborative Information Retrieval, 2008.
collaborators. Co-sense is a Microsoft developed CIS system that [7] Harper, S., & Yesilada, Y. (2008). Web accessibility: a
is aimed to support sensemaking in collaborative web search foundation for research. Springer.
tasks. It provides different views of the collaborators’ activities in
terms of query keywords; documents retrieved and shared [8] Hyldegard ,J. (2009). Beyond the search process–Exploring
comments [20]. In the CCIS context, features and views provided group members’ information behavior in context. Information
by interfaces such as Coagmento and Co-Sense, could be tailored Processing & Management, 45(1), 142–158.
to be screen-reader friendly, by supporting rapid navigation to [9] Ivory, M. Y., Yu, S., & Gronemyer, K. (2004). Search result
different views using keystrokes and providing keyword exploration: a preliminary study of blind and sighted users’
summaries of activities. The WeSearch system [17] allows decision making and performance. In Proceeding CHI’04
sharing queries and comments within the group. The queries and extended abstracts on Human factors in computing systems.
comments are colour coded by collaborators. This could be
implemented within the context of CCIS by using different [10] Kelly, R. M., & Payne, S. J., 2014. Collaborative web search
in context: A study of tool use in everyday tasks. In
Proceedings of ACM conference on Computer-Supported
2
http://searchteam.com/ Cooperative Work.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 461 |
 The Effects of Cross-modal Collaboration on the Stages of Information Seeking
D. Al-Thani | T. Stockman | A. Tombros
[11] Kuhlthau, C. C. (1991). Inside the search process: development in information retrieval, SIGIR ’08 (pp. 315–
Information seeking from the user’s perspective. Journal of 322). New York, NY, USA.
the American Society for Information Science. [21] Paul, S. A., & Morris, M. R. (2009). CoSense: enhancing
[12] Large, A., Beheshti, J., & Rahman, T. (2002). Gender sensemaking for collaborative web search. In Proceeding of
differences in collaborative Web searching behavior: an the 27th international conference on Human factors in
elementary school study. Information Processing & computing systems, CHI ’09. New York, NY, USA.
Management, 38(3), 427–443. [22] Sahib, N. G., Tombros, A., & Stockman, T. (2012) A
[13] London, S. (1995). Collaboration and community. comparative analysis of the information seeking behavior of
Richmond, VA, Pew Partnership for Civic Change, visually impaired and sighted searchers. Journal of the
University of Richmond. American Society for Information Science and Technology.
[14] Marchionini, G., & White, R. (2008). Find what you need, [23] Sahib, N., Tombros, A., Stockman, T. (2013) Evaluating a
understand what you find. International Journal of Human search interface for visually impaired users. Submission to
Computer Interaction, 23(3), 205–237. the Journal of the American Society for Information Science
[15] Morris, M. R. (2008). A survey of collaborative web search and Technology.
practices. Proceedings of the twenty-sixth annual SIGCHI [24] Shah, C. (2009). Toward Collaborative Information Seeking
conference on Human factors in computing systems, In (CIS). In Proceeding of 1st collaborative information
Proceeding CHI ’08 (pp. 1657–1660). New York, NY, retrieval workshop, JCDL.
USA. [25] Shah, C. (2010). Coagmento-a collaborative information
[16] Morris, M. R. (2013). Collaborative search revisited. seeking, synthesis and sense-making framework. Integrated
Proceedings of the 2013 conference on Computer supported demo at CSCW, 6–11.
cooperative work (pp. 1181–1192). New York, NY, USA. [26] Shah, C., & González-Ibáñez, R. (2010). Exploring
[17] Morris, M. R., & Horvitz, E. (2007). SearchTogether: an information seeking processes in collaborative search tasks.
interface for collaborative web search. In Proceeding of the In Proceeding of the American Society for Information
20th annual ACM symposium on User interface software Science and Technology, 47(1), 1–7.
and technology (pp. 3–12). [27] Shah, C., & Marchionini, G. (2010). Awareness in
[18] Morris, M. R., Lombardo, J., & Wigdor, D. (2010). collaborative information seeking. Journal of the American
WeSearch: supporting collaborative search and sensemaking Society for Information Science and Technology, 61(10),
on a tabletop display. In Proceeding of the 2010 ACM 1970–1986.
conference on Computer supported cooperative work. [28] Stockman, T., & Metatla, O. (2008). The influence of
[19] Murphy, E., Kuber, R., Strain, P., McAllister, G., & Yu, W. screen-readers on web cognition. In Proceeding of
(2007). Developing sounds for a multimodal interface: Accessible design in the digital world conference (ADDW
conveying spatial information to visually impaired web 2008), York, UK.
users. In Proceeding of the International Conference on
Auditory Display (pp. 26–29).
[20] Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., &
Back, M. (2008). Algorithmic mediation for collaborative
exploratory search. In Proceeding of the 31st annual
international ACM SIGIR conference on Research and

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 462 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo

Modelado de perfiles de usuario accesibles para servicios

académicos basados en MOOCs
Francisco Iniesto Covadonga Rodrigo
Departamento de Lenguajes y Departamento de Lenguajes y
Sistemas Informáticos Sistemas Informáticos
E.T.S. de Ingeniería Informática E.T.S. de Ingeniería Informática
Universidad Nacional de Educación a Universidad Nacional de Educación a
Distancia (UNED) Distancia (UNED)
[email protected] [email protected]

RESUMEN millones de personas en todo el mundo puedan seguir diferentes

Los MOOCs son un ejemplo de la gran evolución que están iniciativas educativas [1], [2]. Pero también es cierto que el
viviendo en la actualidad los entornos flexibles de aprendizaje, diseño pedagógico y visual desarrollado en los MOOCs junto a
entornos que facilitan que los estudiantes aprendan desde aspectos claves de factores relacionados con la arquitectura de la
cualquier lugar, a su propio ritmo y con un acceso abierto a los información, usabilidad e interacción entre participantes puede
contenidos. Este escenario está revolucionando el campo de estar teniendo un impacto negativo en la participación de los
eLearning y provoca la aparición de nuevos tipos de estudiantes que se demuestra en bajos porcentajes de finalización
aplicaciones/servicios académicos relacionadas con este sistema en este tipo de cursos [3].
de aprendizaje. Este trabajo presenta la estrategia seguida en la En este sentido este nuevo tipo de aprendizaje en abierto y en
definición de un perfil de aplicación basada en una selección línea llamado MOOC parece disponer de características
concreta de ciertos estándares de metadatos con el objetivo de especiales que los hacen adecuados para ayudar a la renovación
concretar un perfil de usuario que almacene información acerca de la educación inclusiva y llegar a todos los ciudadanos: bajo
de sus preferencias de accesibilidad y que aporte un valor coste, flexibilidad temporal y de ritmo de aprendizaje,
añadido en el diseño de nuevos servicios académicos útiles a completamente en línea,…Por tanto aparecen como un potencial
personas con diversidad funcional y basados en el uso de los instrumento que ayude a alcanzar la inclusión social, la cual sólo
MOOCs. se puede conseguir mediante la incorporación de estrategias que
incluyan a los grupos vulnerables en la alfabetización digital
Categorías y Descriptores completa, entre ellos, las personas con diversidad funcional.
H.1.2 [Information Systems]: User/Machine Systems - human Es cierto que las TIC ofrecen numerosas posibilidades a personas
factors, human information processing. H.5.2 [Information con discapacidad visual, auditiva y de movilidad reducida con el
Interfaces and Presentation]: User Interfaces - standardization, fin de mejorar su bienestar, promover su formación y por lo tanto
prototyping, user-centered design. K.3.1 [Computers and su potencial para entrar en el mercado laboral [4], [5]. Por lo
Education]: Computer Uses in Education - Collaborative tanto, una estrategia para hacer los cursos MOOCs más
learning, Distance learning. K 4.2 [Computers and Society accesibles es la incorporación de los estándares de metadatos
Issues]: Social Uses - assistive technologies for persons with relacionados con la accesibilidad, tanto desde el punto de vista
disabilities, handicapped persons/special needs de definición del usuario y sus preferencias, como desde la
vertiente de los propios recursos educativos y las plataformas que
Términos Generales los contienen.
Design, Human Factors, Standardization.
El análisis sobre diferentes estándares de metadatos que
contemplan la accesibilidad presentado en el siguiente artículo
Palabras clave ofrece ideas para adecuar el modelado de perfiles de usuario en
Accesibilidad, MOOC, estándares, metadatos, perfilado de
el diseño de nuevos servicios académicos basados en MOOCs de
usuario.
forma que satisfaga sus preferencias de la mejor manera posible
y automatizada, siendo el marco general de este trabajo el diseño
3. INTRODUCCIÓN de un sistema recomendador de cursos MOOC ajustados a las
Los cursos, en línea, masivos y en abierto (MOOCs - Massive necesidades formativas del usuario y de las adaptaciones
Online Open Courses) han conseguido popularizar los relacionadas con sus necesidades especiales. En primer lugar se
fundamentos de la educación abierta (Open Education) presenta el estudio concreto de los aspectos de accesibilidad en
ofreciendo una ventana gratuita de formación a un público MOOCs, a continuación se muestran los diferentes estándares
globalizado. El impulso está llegando a las instituciones de sobre los que se ha trabajado y el sub-grupo que se ha
educación superior, muchas de las cuales están migrando sus seleccionado, mostrando finalmente las principales conclusiones
plataformas educativas cerradas a nuevos entornos de del estudio.
aprendizaje en abierto demostrando que la evolución de la
educación en abierto en Internet es un hecho permitiendo que

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 463 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo

4. CUESTIONES DE ACCESIBILIDAD EN 5. ESTÁNDARES DE ACCESIBILIDAD:

MOOCS PERFIL DE APRENDIZAJE Y RECURSOS
Las plataformas MOOC son básicamente aplicaciones web que: EDUCATIVOS
proporcionan herramientas específicas para programar el
currículo académico, ofrecen una gran variedad de material
multimedia, permiten la comunicación síncrona y asíncrona entre
instructores y estudiantes, además enlazan diversos modos de
evaluación y retroalimentación con el estudiante. En cuanto a los
elementos de la interfaz visual para iniciar o cerrar la sesión,
navegar en los cursos y contenidos, los entornos MOOC disponen
de múltiples capas a través del las cuales los usuarios con
discapacidad deben poder interactuar. También es importante
tener en cuenta que los materiales de aprendizaje en línea se
utilizan a menudo con una tecnología específica la cual puede
hacerlos menos disponibles para las personas que tiene
capacidades de acceso limitado o que están utilizando equipos no
estandarizados. Estas barreras pueden incluir elementos de la
interfaz, de la plataforma en la que los materiales están
integrados y la manera en que los usuarios interactúan con estos
objetos [6].
Un entorno de eLearning efectivo debe tener en cuenta cada una
de las habilidades de los alumnos, junto con los objetivos de
aprendizaje, dónde se lleva a cabo el aprendizaje y qué
dispositivos específicos utiliza el alumno. En este contexto es
estratégico describir las preferencias y necesidades del alumno
por medio de un perfil, puesto que la forma en que este perfil
interactúa con la plataforma eLearning y los objetos que contiene
afectará a la experiencia de aprendizaje de los usuarios con
diferentes capacidades funcionales, como por ejemplo se realiza Figura 1. Estándares de metadatos para mejorar la
en el proyecto de accesibilidad en educación superior EU4ALL accesibilidad.
[7]. Con el uso de los estándares los usuarios del sistema pueden Varias estrategias se pueden aplicar para mejorar el nivel de
especificar qué tipo de adaptación o recurso alternativo prefieren accesibilidad en las plataformas MOOC y servicios en su
o necesitan, por ejemplo, el usuario puede preferir información conjunto, algunas de ellas están relacionadas con la accesibilidad
textual sobre los recursos visuales o recursos auditivos, sobre el al añadir repositorios de materiales educativos a través del uso
texto o las imágenes, etc. de esquemas de metadatos específicos y la definición del perfil
de usuario y preferencias [8].
Ante todo hay que distinguir entre dos tipos de normas y
estándares: aquellas que se utilizan desde el punto de vista del
usuario para definir su perfil, el cual contiene información acerca
de sus preferencias de visualización y aquellos estándares
dedicados a proveer meta-información a los recursos educativos
para que se ajusten a dichas preferencias del usuario, como se
muestra en la Figura 1.

5.1 Estándares de perfil de aprendizaje.

Algunos estándares se han definido para determinar las
preferencias del estudiante y las necesidades que ayudarán al
usuario a personalizar los dispositivos y servicios para
estudiantes con diversidad funcional.
El Consorcio IMS (Global Learning Consortium) ha desarrollado
una especificación para abordar el desarrollo del perfil del
estudiante (IMS LIP) [9], dedicado a describir las características
generales del usuario mediante la definición de un conjunto de
paquetes que se pueden utilizar para importar y extraer datos
desde un servidor de información de estudiantes compatible con
IMS. El paquete de información accesible del aprendizaje IMS
para el LIP (ACCLIP) [10] es el subconjunto de IMS LIP que

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 464 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo
permite a los estudiantes especificar sus preferencias de 5.2 Estándares de recursos educativos.
accesibilidad y necesidades en términos gráficos, fonéticos o de En el momento de utilizar Internet como medio de comunicación
dispositivo. Este perfil proporciona un medio para describir cómo para publicar contenido multimedia en formato audiovisual es
los alumnos interactúan con un entorno de eLearning centrándose necesario tener en cuenta diferentes aspectos:
en los requisitos de accesibilidad. Por lo tanto un conjunto de
preferencias de los usuarios puede ser utilizado de acuerdo con • Tecnológico: los agentes de usuario que deben hacer posible el acceso
los diferentes contextos de uso del entorno de eLearning, a la información, la tecnología para desarrollar y editar los recursos, las
herramientas de autor para facilitar la producción de materiales
pudiendo personalizar la visualización de los contenidos de accesibles o la adaptación de los ya producidos.
aprendizaje, seleccionar el dispositivo de entrada o de salida • Dispositivos de ayuda: cuando un usuario accede a un recurso
preferido, etc. disponible en Internet se puede acceder directamente o mediante un
dispositivo específico: lector de pantalla, ratón especializado, teclado
Las preferencias del usuario en términos de accesibilidad en los virtual, lupa, etc.
estándares IMS se pueden agrupar de la siguiente manera: • Metodologías inclusivas existentes y estándares educativos: en este
• Información en pantalla: este conjunto describe las preferencias de sentido, los lenguajes de marcado XML tienen que ser mencionados
los usuarios para tener información de lo que se muestra o se presenta. junto con el uso de metadatos que proporciona la capacidad de
Por ejemplo, es posible definir preferencias relacionadas con los textos adaptación de los contenidos de acuerdo al perfil del usuario.
(fuentes y colores), video (resolución), ratón (puntero, movimiento), Con el fin de mejorar la accesibilidad de los contenidos
etc. eLearning la especificación de metadatos Access-For-All
• La información de control: este conjunto define las preferencias del (ACCMD) [13] fue desarrollada por IMS en 2004. En ella se
usuario para controlar el dispositivo: teclado (virtual), preferencias de describe el contenido de aprendizaje mediante la identificación
zoom y el reconocimiento de voz. que se dispone de los tipos de recursos en un objeto de
• Información del contenido: este conjunto define las preferencias del aprendizaje, que puede ser utilizado para presentar el mismo
usuario para visualizar los contenidos de aprendizaje.
contenido a un estudiante determinado o a través de diferentes
• Privacidad y protección de datos: cada elemento ACCLIP tiene
medios. Los metadatos se pueden utilizar para describir los tipos
metadatos relacionados con esta información. La privacidad y la
integridad de los datos se considera muy importante ya que la y las relaciones entre un recurso original y sus formatos
información intercambiada puede estar estrechamente relacionada con adaptados disponibles. Mediante la interpretación de los perfiles
la discapacidad del usuario. de usuario que permite elegir el contenido apropiado y su
Mientras que el estándar IMS se centra en la definición de las combinación con los metadatos de ACCMD se pueden describir
características del contenido, la norma ISO relacionada se centra las alternativas textuales que están disponibles para las
en la diversidad funcional de los usuarios, es decir, especifica los correspondientes imágenes, descripciones de audio para videos,
sentidos y la forma a través de los cuales se accede a los transcripciones o subtítulos para las pistas de audio, alternativas
contenidos. En este sentido la segunda parte de la norma de visuales para texto y una variedad de otros formatos alternativos
accesibilidad ISO/IEC24751:2008 (Tecnología de la información que coincidan con las preferencias del usuario. En base a
- adaptabilidad individualizada y accesibilidad en eLearning, ACCMD estos recursos alternativos adecuados pueden ser
educación y formación-Parte 2: "Acceso para todos" necesidades recuperados y presentados al usuario, un alumno con
personales y preferencias para la entrega digital) [11] está discapacidad visual, por ejemplo, que vea un vídeo que había
dedicada a la descripción de las necesidades personales y sido utilizado en el perfil ACCLIP previamente, recibirá
preferencias de los alumnos. automáticamente ese video con descripciones de audio, mientras
que un alumno con discapacidad auditiva recibirá el mismo video
De acuerdo con los estándares, los alumnos pueden declarar
pero con subtítulos incluidos en la presentación.
explícitamente sólo un modo de acceso alternativo para cada
recurso de aprendizaje y no permite cambios: por ejemplo, un Por otra parte, la tercera parte de la norma de accesibilidad
usuario ciego puede preferir audio-descripción pero si esas ISO/IEC 24751:2008 (Información tecnológica – adaptabilidad
alternativas no están presentes en su perfil no puede elegir una individualizada y accesibilidad en eLearning, educación y
descripción de texto en su lugar. Por ello se ha desarrollado formación-Parte 3: "Acceso para todos" descripción de recursos
recientemente un nuevo estándar de IMS de acceso para todos digitales) [14] se dedica a describir los recursos que constituyen
(AfA) necesidades personales y preferencias (PNP) 3.0 [12], con un contenido eLearning con un enfoque similar a IMS ACCMD,
el objetivo de resolver este tipo de problemas y dejar que el ya que son normas que tienen el mismo objetivo: proporcionar
estudiante pueda especificar múltiples solicitudes de adaptación información sobre alternativas a los recursos originales.
para cada modo de acceso existente. Sin embargo, IMS PNP Cualquier recurso presentado en un contenido de aprendizaje
tiene algunas restricciones al elegir el tamaño o la calidad de los electrónico puede ser identificado con una forma original y una o
recursos de vídeo y audio. Por ejemplo, no es posible solicitar más formas adaptadas, dependiendo de su tipo de soporte
una versión anterior de un videoclip o archivo de audio para tecnológico.
adaptarse al dispositivo del usuario. Por lo tanto sería deseable
Una limitación de estas normas surge cuando los autores de
un perfil de calidad específico para los recursos de aprendizaje,
contenido eLearning quieren ofrecer alternativas tanto a todo el
así como normas de clarificación para describir mejor la lista de
contenido original como a cada parte que compone el recurso:
opciones alternativas.
imágenes incluidas en un documento, los textos formateados, etc.
De acuerdo con estas normas no es posible disminuir esa
granularidad y declarar esas partes de texto formateado como
recursos originales si no están en archivos separados, ni tampoco

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 465 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo
un subconjunto de recursos adaptados puede ser declarado como • AccessModeRequired, modo de acceso que el usuario busca, ya sea
una alternativa a un solo recurso. Por ejemplo, una secuencia de una adaptación o un recurso original como reemplazo para un modo de
acceso diferente, se permite el uso del atributo "existingAccessMode"
archivos de audio no se puede identificar como un único recurso
que define el acceso existente, por ejemplo "visual" y el atributo
auditivo, un video con lengua de signos no se puede definir como "adaptationRequest " que indica la preferencia en el modo de acceso de
una alternativa a la misma y una secuencia de imágenes no puede los usuarios, por ejemplo" textual ".
ser declarada como una alternativa a un video. • AdaptationTypeRequired, la naturaleza o género de la adaptación
requerida para un modo de acceso específico, tiene los mismos
El estándar de IMS acceso para todos (AfA) de descripción de atributos para permitir el acceso siendo igualmente "visual" el ejemplo
recursos digitales (DRD) 3.0 [15] propone objetivos para resolver de un modo de acceso y un tipo de adaptación que el usuario prefiere
estos problemas cambiando radicalmente el punto de vista: ahora podía ser por ejemplo "audio-descripción".
es posible declarar uno o más modos de acceso para cada recurso, • AtInteroperable, el recurso es compatible con las ayudas técnicas, en
definir adaptaciones accesibles existentes y determinar si cumplimiento con las WCAG 2.0. [19]
proviene del recurso original específico. • EducationalComplexityOfAdaptation, identifica si el recurso está
simplificado o enriquecido con relación a otro recurso que presenta el
El proyecto de accesibilidad de metadatos (AMP) [16] es un mismo contenido intelectual.
subconjunto de metadatos que surgió con la idea de definir un • HazardAvoidance, recursos que tienen dicha característica por las
conjunto de metadatos para permitir la búsqueda y cuales no deben ser entregados a un usuario, por ejemplo "luces
descubrimiento de recursos web que se adapten a las necesidades intermitentes".
y preferencias de los usuarios. Se trata de encontrar una solución • InputRequirements, sistema de entrada única suficiente para
controlar un recurso, por ejemplo, si queremos recursos que son
a la falta de propiedades para identificar el carácter accesible de
plenamente utilizables con un teclado.
los recursos que son útiles proponiendo definir la semántica que
• LanguageOfAdaptation, la preferencia por el lenguaje de la
los describa de manera que se facilite su descubrimiento por adaptación de los recursos educativos.
conveniencia. • LanguageOfInterface, la preferencia por el idioma de la interfaz de
La iniciativa de metadatos de recursos para el aprendizaje usuario.
(LRMI) [17] es una iniciativa desarrollada para facilitar la • AdaptationDetailRequired, detalle de alguno de los tipos de requisito
de adaptación, también contiene "existingAccessMode", podría ser
publicación, la búsqueda, el descubrimiento y la oferta de "auditiva", y permite el uso de "ADAP-tationRequest" para expresar la
recursos educativos en la web, después de haber desarrollado un alternativa deseada, si existen, por ejemplo "palabra por palabra".
marco común de metadatos para describir los recursos de • AdaptationTypeRequired, la naturaleza o género de la adaptación
aprendizaje. Está enfocada a beneficiar el trabajo automático de requerida como un re-colocación para un modo de acceso específica,
los motores de búsqueda de forma que éstos puedan proporcionar que tiene el valor "existingAccessMode" como un ejemplo "visual" y el
resultados más acertados, permitiendo a educadores y estudiantes "adaptationRequest" podría tener el valor "PDF".
descubrir los recursos que se ajustan mejor a su particular
situación de aprendizaje.
6.2 MOOCs y metadatos de recursos
6. ESTÁNDARES DE ACCESIBILIDAD Y educativos.
MOOCS Se han escogido trece de un total de diecinueve elementos
Para definir y modelar el perfil de usuario se han escogido los relacionados con los aspectos educativos de los recursos:
estándares más recientes y más completos de IMS como es • AccessMode, modo de acceso a través del cual se describe el
acceso para todos (AfA) y sus aspectos de la PNP y DRD, como contenido intelectual de un recurso o adaptación, por ejemplo,
se ha hecho por ejemplo en el proyecto METALL3, ya que "visualmente".
permiten definir colecciones en lugar de un único valor para cada • AccessModeAdapted, el modo de acceso del contenido intelectual del
caso (multiplicidad) [18] como se muestra en la Figura 2. A recurso que está siendo adaptado, por ejemplo "visual".
continuación se detallan los metadatos seleccionados para • AccessModeOrnamental, descripción del contenido ornamental del
recurso o adaptación.
modelar el perfil de aprendizaje y los recursos educativos, los
• AdaptationDetail, el detalle de uno o más valores del tipo de
criterios seleccionados han sido aquellos que tratan el modo de
adaptación, por ejemplo si el objeto está grabado con voz humana en
acceso requerido por el usuario, el tipo de adaptación que lugar de voz sintetizada.
necesita, aquellos que conllevan información sobre el • AdaptationMediaType, identifica el tipo de medio del recurso
enriquecimiento del recurso educativo y finalmente los descrito.
relacionados con el idioma. La selección tiene en cuenta el • AdaptationType, la naturaleza o género de la adaptación, por ejemplo
enfoque de uso en el diseño del sistema recomendador. "texto alternativo"
• ApiInteroperable, indica que el recurso es compatible con la API de
6.1 MOOCS y metadatos para perfilar el accesibilidad que se hace referencia, por ejemplo "ARIAv1".
• AtInteroperable, el recurso es compatible con las ayudas técnicas, en
aprendizaje. cumplimiento con las WCAG 2.0. [19]
Del conjunto total de dieciséis elementos se han escogido diez • ControlFlexibility, identifica un único método de entrada que es
elementos que tienen que ver con los aspectos educativos del suficiente para controlar el recurso descrito, podría ser como ejemplo
perfil: plenamente utilizable con el teclado.
• DisplayTransformability, identifica una característica de
visualización del recurso descrito que puede ser modificado mediante
3 programación, por ejemplo si se permite ajustar el tamaño de fuente a
https://access.ecs.soton.ac.uk/projects/metall petición del usuario.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 466 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo
• EducationalComplexityOfAdaptation, identifica si el recurso está http://www.eua.be/Libraries/Publication/MOOCs_Update_J
simplificado o enriquecido con relación a otro recurso que presenta el anuary_2014.sflb.ashx
mismo contenido intelectual.
• Hazard, una característica del recurso descrito que no debe ser [3] Tyler-Smith, K. 2006. Early attrition among first time
entregado a algunos usuarios, por ejemplo, "luces intermitentes". eLearners: A review of factors that contribute to drop-out,
• LanguageOfAdaptation, idioma del contenido intelectual del recurso. withdrawal and non-completion rates of adult learners
undertaking eLearning programmes. Journal of Online
learning and Teaching, 2(2), 73-85.
[4] i Díaz, M. P. and Bonjoch, M. R. 2007. ¿Y después del
trabajo, qué?: más allá de la integración laboral de las
personas con discapacidad. Revista de Educación, (342),
329-348.
[5] Vila, M., Pallisera, M. and Fullan, J. 2007. Work
integration of people with disabilities in the regular labour
market: What can we do to improve these processes.
Journal of Intellectual & Developmental Disability, Volume
32, Issue 1:10-18.
[6] Iglesias, A., Moreno, L., Martínez, P., y Calvo, R. 2011.
Evaluating the accessibility of three open-source learning
content management systems: A comparative study.
Computer Applications in Engineering Education, 22 (2),
320-328.
Figura 2. Perfil de usuario y preferencias. [7] Boticario, J.G., Rodriguez-Ascaso, A., Santos, O.C.,
Raffenn,e E., Montandon, L, Roldán, D, Buendía, F. 2012.
7. CONCLUSIONES Accessible Lifelong Learning at Higher Education:
Las plataformas MOOC deben cumplir con las normas de Outcomes and Lessons Learned at two Different Pilot Sites
accesibilidad como el resto de las plataformas de eLearning, no in the EU4ALL Project. Journal of Universal Computer
sólo con respecto a la interfaz web sino también tener en cuenta
Science, vol. 18, no. 1 (2012), 62-85.
las capacidades funcionales de los estudiantes y sus objetivos de
aprendizaje. Para lograr la mejor adecuación del entorno al [8] Sanchez-Gordon, S., Luján-Mora, S. 2015. Adaptive
usuario es necesario describir sus preferencias y necesidades por Content Presentation Extension for Open edX. Enhancing
medio de un perfil que atienda a su diversidad funcional y que MOOCs Accessibility for Users with Disabilities. ACHI
modele también la experiencia del aprendizaje de usuarios con 2015: The Eighth International Conference on Advances in
capacidades diferentes. El “acceso para todos” (AfA) en sus Computer-Human Interactions. IARIA, 2015. ISBN: 978-1-
normas PNP y DRD ofrece la posibilidad a los alumnos para que 61208-382-7
puedan especificar qué tipo de adaptación y/o recurso alternativo [9] IMS Learner Information Profile (IMS LIP), 2001.
prefieren o necesitan. http://www.imsglobal.org/profiles/lipinfo01.html
En el diseño del sistema recomendador los siguientes pasos [10] IMS Learner Information Package Accessibility for LIP
consisten en refinar el modelado del perfil de usuario y su (ACCLIP), 2003.
enriquecimiento para finalmente poder ser cruzado con un vector http://www.imsglobal.org/accessibility/acclipv1p0/imsacclip
de características de accesibilidad de los diferentes cursos _infov1p0.html
MOOC.
[11] ISO/IEC 24751:2008 accessibility standards (Information
technology–Individualized adaptability and accessibility in
8. AGRADECIMIENTOS eLearning, education and training–Part 2: “Access for all”
Se agradece el soporte de la Cátedra de Tecnología y personal needs and preferences for digital delivery), 2008.
Accesibilidad UNED - Fundación VODAFONE y a la red Global http://www.iso.org/iso/catalogue_detail?csnumber=43603
OER Graduate Network (GO-GN). [12] IMS Access for All (AfA) Personal Needs and Preferences
(PNP) 3.0, 2012.
9. REFERENCIAS http://www.imsglobal.org/accessibility/afav3p0pd/AfA3p0_
[1] Haggard, S. 2013. Massive open online courses and online PNPinfoModel_v1p0pd.html
distance learning: review. GOV.UK Research and analysis. [13] IMS Access-For-All. Meta-Data (ACCMD), 2004.
www.gov.uk/government/publications/massive-open-online- http://www.imsglobal.org/accessibility/accmdv1p0/imsaccm
courses-and-online-distance-learning-review d_infov1p0.html
[2] Gaebler, M. 2014. MOOCs Massive open online courses. [14] ISO/IEC 24751:2008 accessibility standards (Information
EUA occasional papers. EUA European University technology–Individualized adaptability and accessibility in
Asociation. eLearning, Education and training–Part 3: “Access-for-all”
digital resource description), 2008.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 467 |
 Modelado de perfiles de usuario accesibles para servicios académicos basados en MOOCs
F. Iniesto | C. Rodrigo
http://www.iso.org/iso/home/news_index/news_archive/new [17] Learning Resource Metadata Initiative (LRMI), 2014.
s.htm?refid=Ref1217 http://www.lrmi.net/
[15] IMS Access for All (AfA) Digital Resource Description [18] Centelles Velilla M., Vázquez Guzmán C., Ribera M. and
(DRD) 3.0, 2012. Pérez Pineda I. 2014. Asignación de metadatos de
http://www.imsglobal.org/accessibility/afav3p0pd/AfA3p0_ accesibilidad a vídeos docentes, INNOVADOC (Documents
DRDinfoModel_v1p0pd.html d'Innovació Docent).
[16] IMS Accessibility Metadata project (AMP), 2013 [19] World Wide Web Consortium W3C, Web Content
http://a11ymetadata.imsglobal.org/ Accessibility Guidelines (WCAG) 2.0, 2008.
http://www.w3.org/TR/WCAG20/

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 468 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera

Creación de documentos EPUB accesibles por usuarios

no técnicos: un largo camino por recorrer
Jordi Roig Mireia Ribera
Estudiante de la Universitat de Barcelona Universitat de Barcelona
c/Melcior de Palau, 140 c/Melcior de Palau, 140
08010 - Barcelona 08010 - Barcelona
+34 934034017 +34 934034017
[email protected] [email protected]

ABSTRACT herramientas de ofimática y conversiones posteriores [17][15].

Este artículo presenta una evaluación cuantitativa de los Grandes proyectos de accesibilidad [12] han potenciado la
resultados de conversión de documentos ofimáticos a EPUB. creación de herramientas de conversión como DAISY Pipeline
Como conclusión se observa que los resultados de la conversión [5], RoboBraille [24] u otras; en Internet abundan las guías
no cumplen los estándares y no preservan la información de para crear documentos accesibles a partir de MS Word o MS
accesibilidad y que las herramientas llevan retraso en la PowerPoint. Las directrices ATAG 2.0 (Authoring Tools
adopción del nuevo formato EPUB3. Accessibility Guidelines) [21] del Consorcio World Wide Web,
sobre herramientas de autor, tratan las conversiones en dos
únicos criterios de éxito. El criterio B.1.2.1, titulado
Categories and Subject Descriptors “Restructuring and Recoding Transformations (WCAG)”, pide
I.7 Document and text processing, I.7.2 Document preparation, preservar la información de accesibilidad, o en caso contrario a)
I.7.4 Electronic publishing avisar a los autores, b) activar un chequeo automático de la
accesibilidad, o c) sugerir a los autores realizar este chequeo.
General Terms El criterio B.1.2.4, titulado “Text alternatives for non-text
Documentation content are preserved”, sólo se ocupa de las alternativas
textuales, e implica que si el contenido no-textual se conserva
en la transformación también debe conservarse su alternativa.
Keywords El primer criterio de éxito supondría que en la conversión a
Transformación de documentos, EPUB3, Calibre, Sigil,
EPUB se conserve la accesibilidad o bien se informe a los
accesibilidad digital, ATAG 2.0
autores; el segundo criterio supondría que en la transformación
de imágenes se conserva su texto alternativo si lo hay.
10. INTRODUCCIÓN El artículo se divide en una sección de objetivos (sección 2),
EPUB es un formato para codificar documentos electrónicos, no metodología (sección 3), resultados (sección 4) con una
sólo libros sino también revistas, artículos, informes… [3] su valoración de las conversiones y finalmente en una sección de
versión 3, EPUB3 [13], se ha diseñado teniendo en cuenta que reflexión y conclusiones finales (sección 5).
las maneras de acceder a la información son diversas y pueden
incluir leer, escuchar, visualizar el texto con letra ampliada, 11. OBJETIVOS
visualizar los vídeos con subtítulos… con una visión inclusiva, En este trabajo hemos querido revisar si es posible generar un
pensada para todos los públicos [9] [25], a partir de la libro EPUB según los estándares y buenas prácticas de
experiencia del formato DAISY [16]. La adopción por parte de accesibilidad a partir de la edición desde una herramienta
EPUB3 de los lenguajes HTML5, CSS3, junto a las ofimática con una conversión posterior o con herramientas
posibilidades que ofrece JavaScript, permiten a los usuarios WYSWYG sin conocimientos de código.
acceder a auténticas publicaciones electrónicas multimedia e
interactivas. Por lo que respecta a la accesibilidad EPUB3
incorpora la nueva estructura semántica de HTML5 (section,
article, aside, nav, etc.) con el añadido de la portabilidad y
mecanismos más ricos de navegación, permite la incorporación
de audio sincronizado con el texto y las especificaciones
recomendadas para los lectores facilitan la personalización de
la experiencia lectora por parte de los usuarios con algún tipo
de discapacidad.
Para crear un documento EPUB 3 en una editorial se suele
partir de un contenido estructurado en XML y transformaciones
XSL que permiten además crear otras versiones del mismo
contenido como PDF, Mobi, etc. Pero para los autores, y en
general para el gran público, su creación suele basarse en

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 469 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
Editor Conversor Versión Plataforma
EPUB testeada

Sigil 0.8.1 No requiere 2 OS X

Windows
Linux
Pages 5.5.2 No requiere 2 OS X

De los editores de texto mencionados, únicamente Pages

dispone nativamente de un conversor al formato EPUB. Los
demás editores, MS Word y LibreOffice Writer, para publicar
un documento como EPUB deben usar un complemento o usar
una herramienta externa. En el caso de LibreOffice Writer se ha
seleccionado el complemento Writer2epub (W2E)
[http://extensions.openoffice.org/en/project/writer2epub]. En el
caso de MS Word se ha seleccionado una herramienta externa,
Figura 1. El proceso de autoría de un EPUB a partir de el programa Calibre [http://calibre-ebook.com/], compatible con
herramientas ofimáticas o WYSWYG Windows y OS X y muy popular [11].
Finalmente para este editor se ha evaluado también la
12. METODOLOGÍA conversión en dos pasos: con el complemento Save as DAISY
Para evaluar la validez de la creación de EPUBs por autores no
[http://www.daisy.org/project/save-as-daisy-ms-word-add-in] de
técnicos en primer lugar hemos hecho una selección de
MS Word de Windows y con el programa de edición y autoría
herramientas de autor con una alta adopción, tras ello hemos
de EPUBs Tobi [http://www.daisy.org/project/tobi] también de
elegido los elementos a testear y hemos creado un documento
Windows, ambos creados desde la comunidad de accesibilidad
para cada uno de ellos con cada una de las herramientas
[6]. Estas herramientas no se han seleccionado por su adopción
seleccionadas.
sino por su especial atención a la codificación semántica. El
12.1 Selección de herramientas de autor complemento Save as Daisy convierte el documento de texto al
El criterio principal para la elección de las herramientas ha formato Digital Talking Books [18] y el programa Tobi
sido su disponibilidad y alta adopción. Así en primera instancia convierte éste último al formato EPUB. El resultado de todas
se han seleccionado el editor de texto MS Word para la las combinaciones citadas, con excepción de Save as DAISY y
plataforma Windows; el editor de texto Pages para la Tobi, es un documento en formato EPUB2.
plataforma OS X; y el editor de texto LibreOffice Writer
disponible tanto en la plataforma Windows, como en OS X
12.2 Juego de pruebas
Para testear la conversión se han establecido unos elementos de
como en Linux, y muy extendido en esta última plataforma.
contenido del documento básicos y se han creado unos juegos
Tras revisar diversos fórums sobre el tema y con el deseo de
de muestra. En esta investigación no se han analizado
dar una visión global finalmente también se ha añadido el
elementos estructurales como prefacio, prólogo u otros, ni
editor EPUB Sigil, multiplataforma, por ser uno de los pocos
metadatos. Tampoco se han tenido en cuenta aquellos
editores EPUB dirigido a una audiencia no profesional que
elementos más propios de documentos técnicos como texto a
cuenta con una interfaz de texto. Sigil tiene ahora mismo una
dos columnas, fórmulas matemáticas o gráficos.
alta adopción, aunque seguramente se discontinuará a favor del
editor Calibre [23] y cuenta además con la ventaja de ser un
programa gratuito traducido a muchos idiomas, hecho que lo
12.2.1 Elementos analizados
hace atractivo a una gran masa de la población. Para evaluar la conversión de documentos, se han analizado
diversas publicaciones (artículos, documentos docentes…) [20]
Tabla 1. Herramientas utilizadas así como guías de creación de documentos ofimáticos
Editor Conversor Versión Plataforma accesibles [19] y se han listado los elementos más comunes:
testeada encabezados, imágenes, listas y tablas, considerados como
EPUB
elementos básicos; notas, como elemento importante en los
MS Word Daisy 2.5 + Tobi 3 Windows libros electrónicos pero secundario; y estilos de texto y enlaces
15.0.4 2.4 considerados como elementos complementarios.
MS Word Calibre 2.20 2 OS X
14.4.7 12.2.1.1 Elementos básicos
Writer 4.3.5 W2E 1.1 2 OSX Encabezados. Son elementos primordiales para la navegación
del documento por parte de los usuarios con discapacidades
Windows visuales y motrices, y para obtener una visión global del
Linux documento en el caso de los usuarios con discapacidad
cognitiva.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 470 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
Imágenes. La accesibilidad de las imágenes es fundamental Listas. Se analizan listas ordenadas, no ordenadas y mixtas -
para que los usuarios con discapacidades visuales puedan combinación de ambas-. Para las viñetas y la numeración
acceder a su contenido. únicamente se consideran los símbolos clásicos y nombres
Listas. Son un elemento estructural del documento, y al igual arábigos respectivamente.
que en los encabezados, facilitan la navegación para los A la correcta conversión de cada tipo de lista le corresponde un
usuarios con discapacidades visuales y motrices. Por otra parte, 6,7% del 20% asignado a las listas.
para los usuarios con discapacidad cognitiva son un recurso Tablas. Solamente se consideran tablas simples con fila de
clave para presentar el contenido de una forma clara. encabezado, título y resumen de la tabla.
Tablas. A pesar de ser un elemento que no está bien resuelto A la correcta conversión de cada elemento (estructura de tabla
desde el punto de vista de la accesibilidad, sobre todo por lo bien formada, encabezado, título, y resumen) le corresponde
que respecta a las tablas complejas, es considerado como un 5% del 20% asignado a las tablas.
elemento básico ya que es un elemento muy común en los
documentos. 12.2.2.2 Elementos secundarios
Notas. Se incluyen notas a pie página y notas al final del
12.2.1.2 Elementos secundarios documento. Hay que tener en cuenta que en el formato EPUB,
Notas. Aunque las notas son en sí un contenido secundario, no existe el pie de página como tal, pues la “página” sólo existe
éstas pueden ser muy relevantes para la comprensión del texto en los EPUB de maquetación fija.
y para dar explicaciones complementarias en el caso de las A la correcta conversión de cada tipo de nota le corresponde un
personas con discapacidades cognitivas. Las notas deben estar 5% del 10% asignado a las notas.
construidas correctamente a nivel de código para que los 12.2.2.3 Elementos complementarios
usuarios con discapacidades visuales que utilizan lectores de
pantalla puedan gestionarlas adecuadamente. Estilos de texto. Se toman en consideración los estilos
tipográficos más habituales: la negrita, la cursiva y el
subrayado. A la correcta conversión de cada estilo le
12.2.1.3 Elementos complementarios
corresponde un 1,6% del 5% asignado a los estilos de texto.
Estilos de texto. Resaltar las palabras o enunciados Enlaces. Se incluyen tanto los enlaces que redirigen a
principales facilita la comprensión del texto a los usuarios con direcciones web como enlaces a correos electrónicos. Hay que
discapacidades cognitivas o con dificultades de lectura. tener en cuenta que en el entorno de los EPUBs los enlaces no
Enlaces. Una correcta formación de los enlaces e incorporación son muy comunes puesto que representan enlaces externos y a
de atributos aportan información sobre el contenido del enlace menudo no son gestionados correctamente por los lectores de
y su comportamiento a los usuarios con discapacidades visuales libros electrónicos.
que utilizan lectores de pantalla. En oposición al entorno web A la correcta conversión de cada tipo de enlaces le corresponde
los enlaces en libros electrónicos son poco usuales pues no se un 2,5% del 5% asignado a los enlaces.
pueden resolver en el mismo programa de lectura.

12.2.2 Ponderación 12.2.3 Documentos de test

Como texto de referencia para hacer el análisis de los
Inspirados por la evaluación de EPUBTest,
[http://epubtest.org/] hemos asignado un valor a cada uno de los elementos se ha escogido el cuento infantil de Los tres cerditos,
elementos en función de la relevancia de la correcta conversión en su versión inglesa, The three little pigs.
a la accesibilidad usable del resultado. Es decir, a los
elementos considerados principales se les asigna un 20% de la
calificación global, a los secundarios un 10%, mientras que a
los complementarios sólo afectan a un 5% de la nota global
cada uno de ellos.
12.2.2.1 Elementos básicos
Encabezados. Se consideran los seis niveles de encabezados
del lenguaje HTML, de h1 a h6.
A una correcta conversión de cada nivel le corresponde un
3,3% del 20% asignado a los encabezados.
Imágenes. Se exploran, según las pautas de accesibilidad,
cuatro posibles escenarios:
• Imágenes decorativas sin necesidad de descripción
• Imágenes que requieren texto alternativo
• Imágenes con epígrafe que requieren texto alternativo
• Imágenes con que requieren una descripción más
detallada
A la correcta conversión de cada tipo de imagen le corresponde
un 5% del 20% asignado a las imágenes.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 471 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
Figura 2. Muestra de un documento de test de Pages, especificaciones EPUB2 y EPUB3 definidas por el
con una imagen International Digital Publishing Forum.
A partir de este texto se han creado documentos independientes Los documentos resultantes de la conversión y los documentos
conteniendo un único elemento de los escogidos para la EPUB teóricos se han comparado tanto visualmente como a
evaluación (ver Tabla 2 y Figura 2). En consecuencia, para nivel de código para identificar qué atributos se mantenían a
cada elemento existen cuatro documentos ofimáticos – la través de la conversión. La evaluación, dado el bajo número de
versión MS Word básica, la versión MS Word editada con la documentos, y la falta de investigaciones previas que
extensión Save as DAISY, la versión de Pages y la de permitieran automatizar la tarea, se ha realizado manualmente
LibreOffice Writer -y un documento EPUB creado directamente por los autores.
con Sigil-.
Tabla 2. Documentos de test
Nombre del archivo Contenido
13. RESULTADOS
A continuación se detallan los resultados de las comparaciones
encabezados_editor_conversor.ex 6 niveles de encabezados
entre los documentos convertidos reales y los documentos
t
convertidos ideales elemento por elemento, para hacer
enlaces_editor_conversor.ext un fragmento del cuento finalmente una valoración de las distintas herramientas
con una palabra en utilizadas.
negrita, otra subrayada y
otra en cursiva 13.1 Elementos
Encabezados.
estilos_texto_editor_conversor.ex un fragmento del cuento
t con un enlace a un sitio Todas las herramientas convierten correctamente los
web y otro enlace a un encabezados a excepción de Pages que sólo convierte los
mail encabezados hasta el tercer nivel. Este error es debido a una
limitación en el propio editor de texto, que no permite la
imagenes_editor_conversor.ext tres imágenes relacionadas
creación de encabezados superiores al nivel 3.
con el cuento; una que
hace la función de En la evaluación final cada nivel de encabezado bien
decorativa, otra con texto convertido se ha ponderado cómo un 1/6 de la conversión de
alternativo y otra con texto encabezados. Así todas las herramientas obtienen un 100% de
alternativo y epígrafe ponderación, excepto Pages que obtiene un 50%.
listas_editor_conversor.ext una lista ordenada, no Imágenes
ordenada y mixta Según los cuatro escenarios descritos anteriormente (ver
marca_agua_editor_conversor.ext marca de agua sección 3.2.1.1):
(sólo en MS Word y LibreOffice) • Imágenes decorativas:
notas_final_editor_conversor.ext un fragmento del cuento Las imágenes decorativas no deben aparecer en las
con tres notas al final del presentaciones no visuales pues no aportan contenido
documento relevante ni realizan ninguna función de interacción, y
notas_pie_editor_conversor.ext un fragmento del cuento por ello no requieren de texto alternativo. Únicamente
con tres notas a pie de MS Word + Save as DAISY + Tobi puede transformar
página imágenes con el atributo “alt” vacío, las demás
tablas_editor_conversor.ext una tabla que sigue las herramientas por defecto añaden el nombre del fichero
pautas de accesibilidad como texto alternativo si éste no aparece en el documento
origen. Como alternativa, los editores de texto MS Word
Cada uno de estos documentos se ha creado siguiendo buenas
prácticas de creación de documentos accesibles [19], y se ha y LibreOffice Writer permiten añadir imágenes como
convertido a EPUB excepto en el caso de Sigil, que ya crea un marcas de agua pero estas desaparecen incluso de la
EPUB directamente. presentación visual tras la transformación.
Tanto los documentos de test como el resultado de las • Imágenes con texto alternativo
conversiones respectivas se ofrecen en abierto en el web del Todos los editores permiten añadir texto alternativo a sus
grupo de trabajo Adaptabit imágenes menos Sigil -que automáticamente pone el
[http://bd.ub.edu/grups/adaptabit/Interaccion2015/documents.zip
nombre del fichero como texto alternativo-. Ahora bien,
] para su posible reutilización en investigaciones posteriores.
después de la transformación sólo lo preservan Pages y
12.3 Evaluación MS Word + Save as DAISY + Tobi. El conversor W2E no
Para cada contenido de los documentos de muestra, se han incorpora el texto alternativo que se especificó en el
creado unos documentos EPUB teóricos según las editor de texto sino que lo cambia automáticamente por
el nombre del fichero. Por su parte, Calibre renombra
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 472 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
todas la imágenes como “Imagen”, “Imagen1”, etc., por ordenadas y mixtas-; Calibre transforma correctamente las
orden de aparición de las imágenes en el documento. listas ordenadas y las no ordenadas. En cambio, Pages no crea
listas verdaderas, sino que las simula mediante estilos CSS.
• Imágenes con texto alternativo y epígrafe Finalmente, LibreOffice Writer + W2E tienen un problema
En EPUB3 se puede indicar el epígrafe con la etiqueta grave de formación de código de listas, pues, generalmente
figcaption (ver Figura 3), y la mayoría de editores de añaden un ítem vacío a la lista y genera un código HTML mal
texto ofrece funciones para incluir un epígrafe en las formado con dos etiquetas sin cerrar. Véase como muestra el
imágenes. Pero en las conversiones Pages no convierte el código de la Figura 3: en el último ítem de lista se puede
epígrafe como parte de la imagen sino como texto observar que falta la etiqueta de cierre, así como también falta
la etiqueta de cierre de la lista.
independiente, y únicamente MS Word + Save as DAISY
+ Tobi resuelve esta situación con una buena conversión, En la evaluación final cada tipo de lista bien convertida se ha
pues en la edición con la extensión Save as DAISY se ponderado con 1/3 de la nota final de listas. Así Sigil y Save as
ofrece una función adicional para indicar los epígrafes de DAISY + Tobi obtiene el 100% de puntuación y Calibre
obtienen un 66%.
las figuras.
Tablas
Sigil no ofrece la opción de crear tablas desde la interfaz de
texto, las tablas se pueden crear únicamente mediante la
interfaz de código. A pesar de que las demás herramientas en el
documento en origen permiten indicar encabezado, resumen y
epígrafe para las tablas, en la conversión ninguna combinación
de herramientas preserva el resumen de la tabla. La
combinación MS Word+ Save as DAISY +Tobi preserva la
estructura, el título y los encabezados en las tablas, pero añade
un código superfluo en las celdas y muestra las tablas sin
bordes (véanse las etiquetas de párrafo en el código de la
Figura 4). Pages, LibreOffice Writer + W2E y MS Word +
Calibre no distinguen entre celdas de encabezado y celdas de
datos, y las transforman todas como celdas de datos.

Figura 3 Conversión Save as DAISY+Tobi de una

imagen con texto alternativo y epígrafe
• Imagen con descripción detallada
Este escenario no está resuelto en los editores origen pero
sí en EPUB3, gracias al atributo aria-describedby.
En la evaluación final cada uno de los escenarios planteados se
ha puntuado con 1/4 de la conversión de imágenes. Así, Save as
DAISY + Tobi obtiene un 75% de ponderación, Pages un 25%
y las otras herramientas obtienen un 0%.
Listas

Figura 4. Conversión de tablas con DAISY

Por lo que respecta al epígrafe, éste desaparece después de la
transformación en Pages y en LibreOffice Writer + W2E,
mientras que MSWord + Calibre lo presentan como un párrafo
independiente de la tabla.
En la ponderación final cada elemento de la tabla bien
convertido se ha ponderado con 1/4 de la calificación total de
Figura 3. Conversión errónea de listas no ordenadas con las tablas. Sigil se valora pues con un 0%, Pages, W2E y
W2E. Calibre con un 25% y sólo Save as DAISY + Tobi aprueban con
un 75%.
Sigil y MS Word + Save as DAISY + Tobi transforman
correctamente los tres tipos de listas analizadas -ordenadas, no Estilos de texto.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 473 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
Las combinaciones LibreOffice + W2E y MS Word + Save as ofimáticas permiten indicar un campo “Tema” que las
DAISY + Tobi transforman la negrita y la cursiva en etiquetas herramientas de conversión añaden en el enlace URL del
semánticas (strong y em) mientras que Calibre y Sigil optan por correo electrónico. En el editor WYSWYG de Sigil no aparece
etiquetas de presentación (b e i). esta posibilidad, y si se quiere incluir el tema hay que añadirlo
Por lo que respecta al estilo subrayado, todas las herramientas manualmente.
lo transforman haciendo uso de los estilos CSS, a excepción de Cada tipo de enlace se ha ponderado con un 50% de la nota
Sigil que emplea la etiqueta <u> -actualmente obsoleta-. final, y todas las herramientas obtienen el 100% de la
Finalmente, Pages codifica estos estilos mediante hojas CSS y calificación.
no mediante código semántico HTML tal y como muestra la
figura 5.

Figura 5. Conversión de estilos de texto con Pages.

Para cada uno de los estilos analizados en la puntuación final
se ha asignado 1/6 de la nota total de estilos si en la conversión
se utilizan etiquetas de presentación y 1/3 si se utilizan
etiquetas semánticas.
Así, W2E y Save as DAISY + Tobi obtienen la máxima
calificación, Pages queda con un triste 16%, Sigil le sigue con
una puntuación de 33% y Calibre con un aprobado justo del
50%.
Notas.
Aunque los editores de texto permiten insertar notas a pie de
página, en el formato EPUB esta opción no existe. Para dar una
solución a las notas originales a pie de página Pages y W2E
generan en el fichero EPUB una página HTML nueva en la que
ubican todas las notas que en los editores de texto eran notas a
pie de página, en cambio, Save as DAISY + Tobi las convierte
como notas al final del documento y Calibre crea una página
HTML nueva para cada una de las notas. Por lo que respecta a
las notas al final del documento, Pages y W2E las transforman
del mismo modo que las del pie de página, es decir,
ubicándolas en una página aparte, mientras que Save as DAISY
+ Tobi da error en la conversión y únicamente Calibre las
transforman correctamente al final del cuerpo del texto. El
editor Sigil no proporciona un modo directo para crear notas
desde la interfaz de texto.
La conversión de cada tipo de notas se ha ponderado en la
evaluación final con 1/2 de la calificación global de las notas.
Así Calibre obtienen una calificación de 100%, mientras que
Pages, W2E y Save as DAISY + Tobi se quedan en el 50%, y
Sigil con un 0%.
Enlaces.
Todos los correctores transforman los enlaces correctamente,
tanto los que redirigen a un sitio web como los que permiten
enviar un correo electrónico. En éstos últimos, las herramientas

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 474 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
En la tabla 3 se pueden consultar los valores agregados.
Tabla 3. Evaluación numérica de la conversión
Encabezados Imágenes Listas Tablas Notas Estilos de texto Enlaces Total
20 % 20 % 20 % 20 % 10 % 5% 5% 100 %
Pages 10 % 5% 0% 5% 5% 0,7 % 5% 30,7 %
W2E 20 % 0% 0% 5% 5% 5 % 5% 40 %
Sigil 20 % 0% 20 % 0% 0% 1,5 % 5% 46,5 %
Calibre 20 % 0% 13,3 % 5% 10 % 2,5 % 5% 55,8 %
Save as
DAISY+ 20 % 15 % 20 % 15 % 5% 5 % 5% 85 %
Tobi

Este editor obtiene una puntación global de 46.5 sobre 100.

13.2 Valoración de las herramientas
Ante este resultado cabe recordar que Sigil es propiamente un
Pages
editor EPUB y por lo tanto, con ciertos conocimientos de
Aunque es el único editor ofimático analizado que integra la código –básicamente HTML y CSS- puede conseguirse la
posibilidad de exportar nativamente al formato EPUB es el que máxima puntación. No obstante, en este estudio se han
peor puntuación obtiene de todos los conversores. En los analizado sólo las funcionalidades del editor WYSWYG de
elementos considerados básicos, esto es, encabezados, Sigil, según el uso que podría darle un usuario sin
imágenes, listas y tablas, está muy lejos de realizar conocimientos de código.
conversiones aceptables. Por lo que respecta a los elementos
considerados secundarios, es decir, notas, estilos de texto y
enlaces, solamente en el últimos obtiene la máxima puntación.
Dicho esto, en una evaluación únicamente con criterios
visuales, es decir, sin tener en cuenta la corrección del código,
la mejor conversión sería sin duda la del editor Pages, pues sus
conversiones son muy fieles al documento ofimático. Las
imágenes, tablas, listas, y otros elementos aparecen con mucha
fidelidad tras la conversión. Parece que en el conversor de
Pages los esfuerzos técnicos de transformación se han dirigido
al lenguaje CSS, dejando en un segundo plano la corrección del
código HTML.

Figura 7. Comentario del lector iBooks. Este tipo de errores

pone de manifiesto la necesidad de informar a los usuarios
sobre la calidad de las transformaciones.
En su editor de texto, Sigil no ofrece la posibilidad de crear
tablas ni notas y en las imágenes ofrece pocas posibilidades.
Figura 6. Conversión de listas en Pages Los demás elementos, a excepción de los estilos de texto que
no los marca semánticamente, los realiza correctamente.
Writer2epub
Si Sigil mejorara el gestor de imágenes e incorporara las tablas
Este conversor obtiene una puntuación global de 40 sobre 100. y las notas a sus funcionalidades WYSWYG sería sin duda la
De los elementos básicos sólo en los encabezados realiza una herramienta recomendada para la creación de documentos
buena conversión, mientras que en los demás elementos – EPUB accesibles por parte del usuario no técnico.
imágenes, listas y tablas- las conversiones son absolutamente Desgraciadamente el desarrollo de Sigil en los últimos tiempos
deficientes hasta tal punto que, en el caso de las listas, los ha tenido altos y bajos [10] y su continuación está en
lectores EPUB deben corregir el código (ver Figura 7). entredicho.
Únicamente las conversiones de los elementos secundarios - Calibre
notas, estilos de texto y enlaces- son correctas. Cabe esperar
que en futuras versiones del conversor [2] estos errores se Este conversor consigue una puntación global de 55.8 sobre
solucionen. 100. Las conversiones de imágenes y tablas -5 de 40 puntos
posibles- penalizan mucho el resultado global. Una mejora en
Sigil estos elementos beneficiaría muchísimas conversiones pues

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 475 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
Calibre no es solo una de las herramientas más usadas para las propuestas en [16] o [3] para modelizar las conversiones
crear EPUBs, sino que es la base de otras herramientas como entre diferentes formatos de documentos digitales.
RoboBraille [http://www.robobraille.org/], el Convertidor
[http://www.elconvertidor.com/ebook-converter.php] Online 15. AGRADECIMIENTOS
Converter [http://ebook.online-convert.com/es/convertir-a-epub] Éste trabajo ha sido financiado por el proyecto Mejora de la
u otros. Sorprende que Calibre todavía no haya actualizado las Experiencia de Usuario para Infraestructuras de Datos
conversiones a EPUB3 pues, al contrario que Sigil, Calibre Abiertos Gubernamentales Sostenibles (InDAGuS-UX)
cuenta con una comunidad de desarrolladores muy activa. (TIN2012-37826-C02-02) del Ministerio de Economía y
Competitividad.
Save as DAISY + Tobi
Como se ha dicho en el apartado 3.1 la elección del 16. REFERENCIAS
complemento Save as DAISY y la aplicación Tobi no se debe a [1] Ballesteros, E., Ribera, M., Pascual, A., & Granollers, T.
su alta adopción por parte de los usuarios no técnicos sino por (2014). Reflections and proposals to improve the
su especial atención a la codificación semántica. No es efficiency of accessibility efforts. Universal Access in the
casualidad entonces que obtengan la puntuación más alta de Information Society, (01). Doi:10.1007/s10209-014-0356-
todos los conversores analizados, 85 sobre 100. No obstante, 1
debe mencionarse que la instalación de estas dos herramientas, [2] Calciani, L. (2015). “Download di W2E ripristinato”.
sobretodo en el caso de Tobi, no es tan sencilla como puede ser http://writer2epub.it/blog/
la de las otras herramientas analizadas, pues requiere disponer
de una máquina virtual Java, instalar Net framework, etc. En el [3] CEN, 2008. Document Processing for Accessibility
caso del complemento Save as DAISY para crear un documento Workshop. CEN/ISSS CWA 15778:2008. CWA 15554.
altamente accesible deben seguirse unas recomendaciones ftp://ftp.cenorm.be/PUBLIC/CWAs/DPA/accessible-
adicionales para los pies de figura, notas, etc. que aumentan CWA15778-2008-Feb.pdf
levemente la dificultad de creación del documento original. [4] Daily, L., 2011. “Digital Monograph Technical Landscape:
Ambas herramientas tienen planificadas actualizaciones [7] y Exemplars and Recommendations”.
cuentan con una organización sólida tras ellos y financiación. http://jiscpub.blogs.edina.ac.uk/final-report
[5] DAISY Consortium “DAISY Pipeline 2 | DAISY
Consortium”.
14. CONCLUSIONES http://www.daisy.org/pipeline2
El formato EPUB, desde su adopción por IDPF como estándar
oficial en septiembre de 2007, ha tenido una gran difusión [6] DAISY Consortium, 2015. “Tools & Services | Daisy
como formato de publicaciones digitales [25], pero esta Consortium”.
adopción no ha llegado aún al gran público autor de http://www.daisy.org/tools
documentos, pues entre las herramientas ofimáticas más [7] Deltour, R., 2014. “Work Plan 2015”.
adoptadas, solamente Pages puede exportar nativamente a https://docs.google.com/document/d/1wVVbSVHV6Fmxn
EPUB sus documentos, mientras que tanto Pages como los RiwwkknWJ4YH1rJbhPri8JxU0oe6XI/edit
demás editores sí pueden exportar a PDF. [8] DIAGRAM Center, 2015.” Image Guidelines for EPUB3”.
Por otro lado, son escasos los complementos o herramientas http://diagramcenter.org/59-image-guidelines-for-epub-
externas gratuitas que conviertan documentos ofimáticos al 3.html
formato EPUB3. La mayoría todavía lo hace al formato EPUB2
[9] Garrish, M. 2012. Accessible EPUB 3: best practices for
[14] a pesar que las especificaciones de EPUB3 fueron
creating universally usable content. O’Reilly.
aprobadas en octubre de 2011 e incluyen cambios notables
http://shop.oreilly.com/product/0636920025283.do
respecto su versión anterior. Ello puede ser debido a la
dificultad de los pequeños desarrolladores para encontrar los [10] Goyal, K., 2009. “Calibre’s History”
recursos necesarios para mantenerse al día de las nuevas http://calibre-ebook.com/about#history
versiones o estándares [22] [1]. [11] Goyal, K., 2015. “Calibre. Usage statistics”
Finalmente cabe resaltar, como se muestra en los resultados, http://calibre-ebook.com/dynamic/calibre-usage
que las conversiones de estas herramientas no son aceptables - [12] GPII “Document Transformation Infrastructure -
especialmente en lo que concierne a los criterios de wiki.gpii”.
accesibilidad- y hacen inviable la creación de documentos http://wiki.gpii.net/w/Document_Transformation_Infrastru
EPUB de calidad por el usuario no técnico. El autor de un cture
documento no conoce de antemano ni recibe ningún tipo de
retroacción sobre la calidad de la transformación y las [13] Gylling, M., et al., 2014. “EPUB 3 Overview:
características de accesibilidad introducidas en el documento de Recommended specificacion”
origen por norma general se pierden por el camino. http://www.idpf.org/epub/301/spec/epub-overview.html

Por otra parte las directrices ATAG 2.0 se quedan muy cortas [14] IDPF, 2015. “EPUB Solutions”.
en sus exigencias y descripción de los problemas de http://epubzone.org/solutions
conversión, y se requerirían descripciones más completas, como [15] Keegan, L; Stevns, T, 2012. SCRIBE. A model for
implementing Robobraille in a higher education
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 476 |
 Creación de documentos EPUB accesibles por usuarios no técnicos: un largo camino por recorrer
J. Roig | M. Ribera
institution. In Lecture Notes in Computer Science, vol. [21] Richards, Jan; Spellman, Jeanne; Trevinarus, J. 2013.
7382, pp. 77–83 Authoring Tool Accessibility Guidelines (ATAG) 2.0:
http://www.springerlink.com/index/10.1007/978-3-642- Candidate recommendation 7 november 2013.
31522-0 http://www.w3.org/TR/ATAG20/
[16] Ko S.-K., Kang, M.-S., Sohn, W.-S., Lim, S.-B., & Choy, [22] Schember, J. 2014a. Sigil Status Update.
Y.-C., 2002. Conversion of eBook Documents Based on http://sigildev.blogspot.com.es/2014/09/sigil-status-
Mapping Relations, In ECDL '02 Proceedings of the 6th update.html
European Conference on Research and Advanced [23] Schember, J. 2014b. Sigil’s Spiritual Successor.
Technology for Digital Libraries, pp. 32–46. http://sigildev.blogspot.com.es/2014/02/sigils-spiritual-
http://dl.acm.org/citation.cfm?id=646635.700070 successor.html
[17] Meester, B. De, Nies, T. De, Ghaem, H., & Vander, M. [24] Synscenter Refsnaes and Sensus Aps. Convert a File |
2014. A digital-first authoring environment for enriched e- RoboBraille.
books using EPUB 3. Information Services & Use, 34, http://www.robobraille.org/
259–268. DOI:10.3233/ISU-140748
[25] Zichuhr, K., 2014. E-reading rises as device ownership
[18] NISO Committee AQ on Digital Talking, & Book, 2005,. jumps. Pew Internet.
Specifications for the Digital Talking Book. National http://www.pewinternet.org/files/2014/01/PIP_E-
Information Standards Organization, Bethesda, Maryland reading_011614.pdf
NISO Press, Bethesda, Maryland, U.S.A.
http://www.daisy.org/z3986/2005/Z3986-2005.html
[19] Ribera, M., Pascual Almenara, A., Salse, M., Masip
Ardévol, L., Granollers I Saltiveri, T., López González,
J.M., Comas I Aleix, A.C. (2010). Guia de contingut
digital accessible. Universitat de Lleida.
http://diposit.ub.edu/dspace/handle/2445/29018
[20] Ribera, M., Salse, M., Alcalà Ponce de León, M.,
Splendiani, B., Centelles Velilla, M., Llerena, I., …
Térmens i Graells, M. (2012, November 22). Plantillas
para la creación de material docente accesible con
herramientas ofimáticas y LaTex. CIUD (Congreso
International Universidad y Discapacidad) Fundación
ONCE. http://diposit.ub.edu/dspace/handle/2445/33169

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 477 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert

Exploring language technologies to provide support to

WCAG 2.0 and E2R guidelines
Lourdes Moreno Paloma Martínez Isabel Segura- Ricardo Revert
Labda Group, Labda Group, Bedmar Labda Group,
Computer science Computer science Labda Group, Computer science
department department Computer science department
Universidad Carlos III Universidad Carlos III department Universidad Carlos III
de Madrid de Madrid Universidad Carlos III de Madrid
[email protected] [email protected] de Madrid [email protected]
[email protected]

ABSTRACT WCAG 2.0 accessibility guidelines [47] created by W3C are

Part of citizenship faces accessibility barriers when they read and concerned as a standard for the industry and many important
understand texts containing long sentences, unusual words, principles from the standpoint of an E2R are highlighted in these
complex linguistic structures, etc. Readability and understanding guidelines. However, the capability of the WCAG 2.0 to estimate
should be considered when texts are created. In order to make accessibility of a website for E2R is not complete in all situations
online texts more accessible, there are initiatives as Easy-to-Read [1].
(E2R) and Web Content Accessibility Guidelines (WCAG) 2.0, WCAG 2.0 contain some accessibility guidelines regarding the
however they do not cover every need that arises in this regard. text related to the readability and understanding. Some of these
In addition to accessibility guidelines and E2R rules, technology guidelines have the highest conformance level (Level of
supporting the authorship of texts is required because the Conformance AAA). WCAG 2.0 is standard ISO among others
transformation of a text to an easier text to read and and is the work of reference in the most of the regulatory
understandable is impossible. As a solution to the need to have frameworks and laws in the countries. These laws require a
(semi)automatic support to comply with accessibility and E2R Level AA, this can cause these requirements (of Level AAA) are
guidelines, the application of Natural Language Processing not raised. If these guidelines are not fulfilled, there are user
(NLP) resources and methods is proposed. In this paper, E2R groups affected with accessibility barriers. These user groups are
guidelines are introduced and a subset of WCAG guidelines people with any kind of cognitive disability. One of them is older
regarding readability and understanding has been obtained. In people group [31]. Figures indicate that in the coming decades
addition, a review on NLP technology concerning accessibility is will be a large number of older citizens and at the same time,
given. To illustrate this proposal for improving accessibility active citizens in the use of ICT. Consequently, the age related a
using PLN approaches, a use case for simplifying drug package disability is a very important factor to keep in mind. If
leaflets in Spanish is introduced. accessibility barriers regarding the text are not stopped, the
digital divide will increase in the near future.
Categories and Subject Descriptors WCAG 2.0 documentation provides useful techniques in
K.4.2 [Computers and Society]: Social Issues -handicapped conjunction with to satisfying these guidelines regarding the text;
persons/special needs, I.2.7 [Natural Language Processing]: however, these guidelines are complicated requirements to carry
Text Analysis out in web applications and web sites, especially in large-scale
web sites. Despite the existence of a complexity to comply with
Keywords these accessibility requirements related to the accessible text,
Web accessibility, Easy-to-Read, disability, Natural Language there are techniques and technologies from different disciplines
Processing, Text Simplification, Lexical Simplification that can be used to support these requirements of the
accessibility standards. Just as there are works of the discipline
of Software Engineering and Web Engineering to integrate
1. INTRODUCTION accessibility in the design process [27] [32], there are important
The problem of readability is under investigation since 1920s,
works on the scope of the Natural Language Processing (NLP)
when educators discovered a way to use vocabulary difficulty and
that may offer support to comply with the guidelines regarding to
sentence length to predict the difficulty level of a text [14].
the accessible text [39] [5] as proposed in this work. The paper
Reading and understanding texts containing long sentences,
aim is to introduce opportunities to integrate these NLP
unusual words, and complex linguistic structures among others
approaches into authoring tools, as well as possible services to be
can be very hard for persons with cognitive or learning
able to offer to improve accessibility in text content as the
disabilities [41].
readability and understanding.
Reading difficulties and the impact of different linguistic
In this paper, E2R guidelines are introduced in the section 2. In
deficiencies especially in learning are known [15]. In order to
order to help professionals who have an interest to fulfil these
make online texts more accessible and readable to these groups
guidelines, section 3 an overview of techniques, methods and
of readers, E2R guidelines [17] were produced. Moreover, the

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 478 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert
tools of NLP focused on approaches simplification of text is From these guidelines and initiatives arise specific guidelines,
introduced. In section 4, a set of WCAG 2.0 accessibility which can help address the increased demand for E2R content on
requirements related to this issue is presented. In order to clarify the Web. In general terms these guidelines are: use the simplest
this support of NLP methods to improve the accessibility, a proof and most common words should, that long words should be
of concept in the drug package leaflets is presented in the section avoided, should be avoided use of abbreviations and that the
5. Finally, conclusions and future work are presented. same term should be consistently used to refer to the same
concept. Besides, use short sentences, avoid complex sentences
2. EASY-TO-READ (E2R) Guidelines with dependent clauses, use active language and avoid passive
Readers with cognitive disabilities among others often voice.
experience difficulties understanding complex words and Table 1: Main different aspect between E2R guidelines
deriving their meaning from the context [6]. In order to make
online texts more accessible to these groups of readers, their IFLA guidelines Inclusion Europe 2009
content should follow E2R guidelines [17] and thus be uncovered standard
of complex words or phrases that could be replaced with more It mainly concentrates on It mainly concentrates on
commonly used words [13]. These adaptations are carried out People with very different People with intellectual
with the use of text simplification techniques which will be linguistic needs as the target disability.
presented in this paper. groups.
In the past decades, in many countries guidelines on E2R have Not planned to develop detailed To provide precise and
developed [21] [20]. Usage examples of these guidelines are, for guidelines for E2R. normative rules for producing
example, in Spanish (Discapnet) 4 or in English with (Journal of E2R.
Inclusion Europe) 5 and news with simple summaries with access
'easy reading' (Simple English Wikipedia) 6. Highlight the System
Simplex project; it is aimed at producing a text simplification These guidelines help to produce texts E2R, but they are aimed
system for Spanish 7. only to text content. According to user studies, the most
important principles in addition to E2R guidelines itself are
2.1 E2R target groups related to page structure and the amount of information on a web
The importance of understandable content and accessible page [1]. For this reason, accessibility requirements of WCAG
information for people with learning difficulties has received 2.0 must be taken into account such as will indicate in section 4.
increased attention in last decade. People with intellectual and
learning disabilities are the main E2R target group. Additionally, 3. NLP APPROACHES FOR TEXT
pre-lingually deaf persons, deafblind persons, persons with SIMPLIFICATION
dementia belong to the group [34]. Individual cognitive abilities Natural language processing (NLP) is a discipline devoted to
such as attention span and memory also play a role as can be the develop technology to understand natural language in a similar
case of older people. Good practices in E2R benefit non- way human being does it. It has different applications such as
alphabetized people and immigrants with a different native machine translation, information retrieval, information extraction
language [28]. Furthermore, research with application de E2R from unstructured data, summarization, question answering and
based technology with good results for people with aphasia, many others. Tokenization, part-of-speech tagging, syntactic
dyslexia, and autism are found [38] [12]. parsing, named entity recognition, relation extraction among
If authors produced texts that are simple to read and understand, others processes are integrated in different architectures to
the result is not only more accessible to people with cognitive analyse texts.
disabilities, but also for all users [41]. In recent years research has been done in the area of text
simplification where NLP processes are applied with the
2.2 E2R goals objective of transforming a text in an equivalent one but more
There is still no overall acceptance on the common linguistic accessible to people with any kind of cognitive disability. This
principles of E2R. They vary from detailed and strict standards of section describes related works carried out in the area of text
Inclusion Europe 2009 to holistic and loose guidelines of IFLA simplification. For the purpose of this work three NLP processes
2010 [25]: that could be applied to text simplification tasks are described:
language detection, abbreviations detection and topic detection
• to the definition of the people needing E2R
• to the definition of text genres that are supposed to be Language detection consists on identifying the language of a text
simplified. and is useful if language-dependent NLP processes are used. It is
Also, E2R goals of both guidelines differ in some fundamental also helpful for example, when screen readers are used. A usual
aspects (see Table 1). method is to find out it is to check if language-specific
characters, e.g. Dutch if string “ik” appears, German is “ich”
or “β” is used, Polish if “czy” or “ń”, “Ł”, “ź” are included in
4 words. These conditions can be easily implemented as rules in a
www.noticiasfacil.es
5
http://www.e-include.info/
6
http://simple.wikipedia.org/
7
http://www.simplext.es/
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 479 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert
text tokenizer . Other approaches, [7], use n-grams 9 frequency
8
taking into account the position of the sentences (important
distributions. All languages have words that occur more sentences appear in specific places, for instance in introduction
frequently than others (Zipf´s Law 10), therefore if two texts of a and conclusions sections of a paper) or giving more score to
same language are compared then they should have similar n- words of headings. Also cue words depending of the type of
grams frequency distributions. This method has limited success it summary and patterns could be considered..
is applied to language detection of short texts.
The algorithm has four steps: (1) score all the words of the
Concerning abbreviations, this is an active area in processing document according to the TF*IF measure, (2) score all the
biomedical texts due to the fact that many acronyms and sentences in the text by adding the scores of the words from
abbreviations in scientific literature, clinical reports, etc. these sentences, (3) take out the sentences with top N scores and
Abbreviation detection and expansion is also a useful technique (4) display the extracted sentences in the original order. If the
that contributes to enhance text understanding in the accessibility objective is to entitle a paragraph the algorithm should extract in
field. Main approaches to recognized abbreviations and step 3 only the top scored sentence. This sentence is the best
corresponding expansions use pattern-matching methods based candidate as title of the paragraph.
on rules and heuristics to detect upper alphanumeric strings with
The approaches reviewed in this section are useful resources to
the objective of identify Long form (short form) or Short form
comply with some WCAG 2.0 guidelines concerning the
(long form) utterances. For instance, in the utterance acquired
readability and understandability, some of them and their
immune deficiency syndrome (AIDS), AIDS is the short form and
correspondence with WCAG 2.0 success criteria are shown in
acquired immune deficiency syndrome is the long for (or
Section 4.
definition). [35] describes a method that combines linguistic
information (POS tags, noun phrases and stop words) with
statistical information (frequency of terms, length of terms,
3.1 Text Simplification
First works in text simplification started 20 years ago [8]. It is
frequency of nested terms) to extract terms that appear frequently
based on transforming a text in an equivalent text that is easier to
next to an abbreviation in a collection of texts. The hypothesis is
read and probably easier to understand by a target audience. In
if a sequence of words co-occurs frequently with an abbreviation
automatic text simplification there are different levels of
and the sequence does not occur with other near words then it is
difficulty in texts to be considered.
assumed that there is a “abbreviation-definition” relationship.
There is a need of content adaptation to some groups of people
Third NLP technique is automatic text summarization or topic
with disabilities due to information is not equally accessible to
detection. This technique is of great interest due to the fact that
everyone. It is highly unlikely that professionals elaborate
editors of web contents could require tools to create titles of
adapted texts and NLP techniques could help to simplifying texts
paragraphs and sections that faithfully represent the content they
by automatizing some tasks. In this way, it is possible to help
head.
content editors to generate adapted contents. On the other hand,
The goal of automatic text summarization is to obtain a set of text simplification is essential in several types of texts: News,
sentences that reflects its content, A summary could be only a Government and administrative information, laws and rights, etc.
sentence, such as the case of a headline for news or it could be
There are three subtasks of text simplification, Saggion et al.
several sentences, as in abstracts of scientific publications. A
(2011) [40]: (1) syntactic simplification that divides complex
good definition of a summary is [44]: A reductive transformation
sentences in simplest sentences, (2) lexical simplification whose
of source text through content reduction by selection and/or
objective is to replace complex vocabulary by common
generalisation on what is important in the so usually, people
vocabulary and (3) clarification that provides definitions and
make summaries by exploring the document, making judgments
explanations. These tasks are not completely automatic, they
of relevance and finally, generating the abstract. This is a
have to be manually reviewed in some cases.
difficult task to evaluate because there is no a clear idea about
what is a good summary. As it has been previously mentioned, there are authors that
distinguish between readability and understandability, these
Automatic text extraction is the most used approach to obtain
concepts capture different aspects of the complexity of the text.
summaries. One common approach is to use a surface-base
Readability gives an evaluation about the structure of sentences
methods that considers that relevant sentences of a text have a
(it concerns syntax and consequently requires syntactic
big amount of important words. The importance of a word is
simplification approaches). On the other side, understandability
calculated with a TF*IDF measure that relies on how frequent is
captures the lexical aspects [3] and lexical simplification
a word in a document and in how many documents from a
approaches are required.
collection the word appears. This measure can be enhanced
Concerning syntactic simplification it consists on transforming
complex and long sentences into simplest and independent
8
NLP process devoted to identify sentences and tokens of a text sentences eliminating coordination (of clauses, verbs, etc.),
(words, punctuation marks, etc.) dropping subordination utterances (relative clauses, gerundive
9
A n-gram is a sequence of n characters that are part of a string and participle utterances), resolving anaphora and transforming
(including blanks). For instance, bi-grams of the word “book” passive into active voice. First a parser is used to obtain a
are _b, bo, oo, ok, k_; tri-grams are _bo, boo, ook, ok_, etc. dependency tree that represents the syntactic structure of the
10
Given a corpora of utterances, the frequency of any word is sentence (noun, prepositional and verbal phrases and how they
inversely proportional to its rank in the frequency table
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 480 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert
are related to). Then, rule-based approaches are used in syntactic Finally, there are specific works to simplify numerical
simplification. Rules can be automatically learned from expressions. Bautista and Saggion (2014) [4] propose a rule-
annotated corpora of text (syntactic trees of sentences where based lexical component that simplifies numerical expressions in
original sentences are related to their simplified sentences), [49] Spanish texts. This work makes news articles more accessible to
or handcrafted rules [8] [43]. The rules include split, drop, certain readers by rewriting difficult numerical expressions in a
copying and reordering operations over syntactic trees. simpler way.
Related to lexical simplification, this task consists on replacing
words (taking into account the context) and complex utterances 3.2 Support Technology
by easier words or phrases. A heuristic used is that complex The use of tools to assist authors and support the linguistic
words have a low frequency. Moreover, lexical resources as analysis could be explored further [28]. Implementing E2R asks
Wordnet, [16], are used to extract synonyms as candidates to for research and discussion of automated tools helping with
replace a complex or difficult word. Combining a lexical analyses grammar/semantic as well as navigation/interaction in
resource and a probabilistic model is an approach that has been order to support users and authors of web content [33].
tried [11]. Probabilistic models are obtained from lexical Different NLP tools which have been integrated in existing text
simplifications have previously done applying E2R guidelines, as editor are found in some systems; these tools have the aim of
in the Simple Wikipedia. McCarthy and Navigly (2007) [30] providing support for text simplification experts who prepare
introduces work to propose candidates to replace a word using easy-to-read texts in English [19]. Other related tool is
contexts. In Semeval 2012, English Lexical Simplification SPARTA2 [41], a Word plug-in for text simplification in Italian
challenge 11 with ten participant systems, the evaluation results language.
showed that proposals based on frequency give good results
comparing to other sophisticated systems. 4. WCAG 2.0: READABILITY AND
But before simplifying we have to know le level of readability of UNDERSTANDABILITY
a text by using complexity measures. There are simple measures In this section, Web Content Accessibility Guidelines (WCAG)
based on frequency of words in texts as well as length of phrases, 2.0 standard is introduced, and a subset of accessibility
FOX index [18], Flesch-Kinaid [24] measures are used in requirements relating text content of WCAG 2.0 is presented.
English. In Spanish texts, [2] describes several indexes to These accessibility requirements can be supported by NLP
measure the structural complexity of a text: the number or verbal methods to achieve compliance.
predicates in subordinate clauses, and the index of sentence
recursion (a measure that counts the number of nested clauses in 4.1 Layers of Guidance
the text). To measure the lexical complexity two indexes are WCAG 2.0 defines how to make Web content more accessible to
proposed: an index of low frequency words (the number of people with disabilities. The professionals that adopt WCAG
content words 12 with low frequency13/ Total of lexical words) vary widely and include web designers and developers, policy
and an index of lexical density (number of distinct content words makers, purchasing agents, teachers, and students. In order to
/total of discourse segments 14. Finally, other indexes such as the meet the different needs of this audience, several layers of
average length of sentences and average length of words guidance are provided including overall principles, general
(syllables) although they are criticized. guidelines and testable success criteria with techniques are
provided.
These indexes have to be validated by final users. Knowing the
There are four principles to be considered accessing and using
readability level of a document, users have the opportunity to
Web content. Anyone who wants to use the Web must have
choose the most suitable text, from a collection of documents
content following these principles: Perceivable (Information and
delivering the same information [41].
user interface components must be presented to users in such a
This paper describes in section 5 a prototype for lexical way they can perceive them), Operable (User interface
simplification of drug package leaflets. In particular, each drug components and navigation must be operable), Understandable
effect is replaced by a simpler synonym. The MedDRA resource (Information and user interface must be understandable), Robust
is used to obtain the set of synonyms for each effect. The ( Content must be robust enough that it can be interpreted
prototype proposes two possible simpler synonyms: the preferred reliably by a wide variety of user agents, including assistive
term proposed by MedDRA and the MedDRA synonym with the technologies).
highest frequency in a collection of documents gathered from the Under each of the principles there are guidelines and success
MedLinePlus website. criteria that help to address these principles for people with
disabilities. The guidelines are not testable, but provide the
framework and overall objectives to help authors understand the
11
http://www.cs.york.ac.uk/semeval-2012/task1/ success criteria and better implement the techniques. For each
12 guideline, testable success criteria are provided to allow WCAG
A content word is a word with meaning (nouns, verbs, 2.0 to be used where requirements and conformance testing are
adjectives and adverbs) necessary. In order to meet the needs of different groups and
13
There are several resources/corpora to query the frequency of a different situations, as it has been previously mentioned there are
word. In Spanish, for instance, there is the CREA corpus, three levels of conformance: A (lowest), AA, and AAA (highest).
http://corpus.rae.es/creanet.html
14
A discourse segment is a sentence or a phrase.
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 481 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert

4.2 WCAG success criteria concerning text (Level A). equivalent purpose

content 2.4.2 Page Titled Web pages have titles that describe topic or
(Level A). purpose.
After analyzing the WCAG 2.0, we propose a subset of success
criteria for the textual content (see Table 2). As indicated above, 2.4.4 Link Purpose (In The purpose of each link can be determined from
in E2R guidelines two fundamental concepts are always present: Context): the link text alone or from the link text together
understandable and readability. Some authors treat these with its programmatically determined link context
(text type)
concepts in a combined way, but others make a distinction
between readability and understandability. These concepts 2.4.6 Headings and Headings and labels describe topic or purpose.
Labels (Level AA).
capture different aspects of the complexity of the text: “a text
could be highly readable, since the syntax is extremely simple, 2.4.9 Link Purpose A mechanism is available to allow the purpose of
but extremely hard to understand because of the lexicon used” (Link Only) (Level each link to be identified from link text alone,
[41]. In section 3 have been described different NLP strategies to AAA). except where the purpose of the link would be
ambiguous to users in general.
simplify texts depending on whether you want to analyze (text type)
understandable or readability. One of the guidelines of WCAG
2.4.10 Section Section headings are used to organize the content.
2.0 is 3.1 (Readable: Make text content readable and Headings (Level
understandable); this guideline comprises various success AAA).
criteria. Readability concept has a direct correspondence with the
3.1.1 Language of The default human language of each Web page
success criteria 3.1.5 (Reading Level), however, for
Page (Level A). can be programmatically determined.
understandable, several success criteria should be considered as
3.1.3 (Unusual Words) and (3.1.4 Abbreviations) among others. 3.1.2 Language of The human language of each passage or phrase in
Parts (Level AA). the content can be programmatically determined.
As proposed in this work, NLP methods can be used in order to
3.1.3 Unusual Words A mechanism is available for identifying specific
evaluate or detect that a WCAG 2.0 success criteria is not
(Level AAA). definitions of words or phrases used in an
satisfied, besides providing a reparation proposal for the success unusual.
criteria failure. Works in the accessibility field propose the use
NLP techniques in order to help users to better understand web 3.1.4 Abbreviations A mechanism for identifying the expanded form
(Level AAA). or meaning of abbreviations is available.
contents as well as the operation of the user interface [29] are
found. 3.1.5 Reading Level When text requires reading ability more advanced
(Level AAA). than the lower secondary education level after
For example in the case of following success criteria 2.4.2 (Page removal of proper names and titles, supplemental
Titled), 2.4.6 (Headings and Labels) and 2.4.10 (Section content, or a version that does not require reading
Headings), in the process that follows a person when interpreting ability more advanced than the lower secondary
education level, is available.
a text it is important the existence of well determined titles,
because these titles are activated during the processing of a text (*) Success criteria concerning to captioning (1.2.2, 1.2.4) were not
as help understanding the information through schemes that also considered because they follow specific standards on how to create
eliminate the ambiguity. For web documents that lack these captioning for the deaf.
important elements, NLP approaches presented in section 3 as
text simplification through the topic detection task can be used in 4.3 Additional accessibility requirements
order to repair these documents generating titles and headings. One important issue when E2R content is published in web page
is the fact that E2R audience is a very heterogeneous with
Similarly could give support to success criteria 3.1.3 (Unusual different needs. For some user groups an easier content is enough
Words), through the combined use of PLN approach and but some user groups need a user interface easier to use [1]. So,
dictionaries with definition, an extension in the users' browsers in addition to the linguistic features of E2R, more accessibility
using a glossary to enhance the original web content with these features should be explored [28]. In addition to language itself,
explanations could to be provided.. Another case is the use of the most important factors influencing accessibility of web pages
approach text simplification through the abbreviations detection for E2R users are page structure, navigation structures, the
task in order to detect abbreviations/acronyms and obtains their amount of information on a webpage and the way elements in a
corresponding expansions to be repairing it. The accessibility web page are highlighted.
requirement indicates that the expansion of an abbreviation must
be provided to the reader. So, this approach would help comply WCAG 2.0 document does not specify guidelines to these
with the success criterion 3.1.4 (Abbreviations). matters in a same precision level as it does to matters concerning
visual or auditory accessibility, because of that it is more
Finally, for success criteria 3.1.5 (Reading Level), as indicated challenging to use WCAG 2.0 with E2R user groups [1].
the syntactic simplification approaches are required. Although there WAI documentation in this regard [48], the
Table 2. WCAG 2.0 success criteria concerning text content accuracy of E2R criteria for digital information should be
(*) reviewed, especially regarding the coverage of E2R in WCAG
[28].
Code (Level Description
Conformance) Through an analysis carried out, a set of additional WCAG 2.0
1.1.1 Non-text Content Every non-text content that is presented to the success criteria has been obtained. This proposal contains
user has a alternative text that serves the important accessibility requirements regarding the presentation,

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 482 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert
navigation, structure, cognitive aspects in user task, etc. that and compliance with WCAG 2.0 success criteria of the guideline
must be taken into account in addition to those concerning the 3.1 is supported.
textual contents (see Table 2) and E2R guidelines. These
Lexical simplification (which basically consists in replacing
additional success criteria are: 1.4.8 (Visual Presentation), 2.4.5
complex concepts with simpler synonyms) has been applied to
(Multiple Ways), 3.2.3 (Consistent Navigation), 3.2.4
simplify texts from different domains (such as crisis management
(Consistent Identification), 2.2.3 (No Timing), 3.3.1 (Error
[45], health information [22], aphasic readers [6] [12], language
Identification), 3.3.2 (Labels or Instructions) and 3.3.5 (Help).
learners [37], but they have not used to simplify drug package
4.4 Discussion and conclusions leaflets.
After the analysis and study conducted, the following main
results are obtained: 5.1 System architecture
The architecture of the system is a pipeline in GATE 15, in which
A confusion of how to use the WCAG 2.0 has been detected. It is we can recognize two different modules (see Figure 1): a first
due to the lack of correspondence between the two leading module to recognize drug effects and a second one to simply
concepts in E2R guidelines (readability and understandability) these terms.
and success criteria of WCAG 2.0. This situation causes the
professional closely to the field of the accessibility conformity
WCAG does not know how to accomplish requirements E2R.
Aside from WCAG 2.0 success criteria regarding the text, further
accessibility features should be considered. These are navigation
structures, the amount of information on a web page and visual
presentation among others.
Moreover, in WCAG 2.0 support is not provided beyond the list
of related resources provided for each WCAG 2.0 success
criteria. As the solution, the PLN approaches with a use of E2R
resources provide the support needed.

5. PROOF OF CONCEPT: LEXICAL

SIMPLIFICATION OF DRUG PACKAGE
LEAFLETS
NLP approaches as simplifying the texts have been reviewed. In
this section, a proof of concept which shows the adequacy of an
NLP approach to make texts more accessible is presented.
The principal text source of information for patients about their
medications is package leaflet. This document provides
informative details about a medicine, including its appearance,
actions, side effects and drug interactions, contraindications,
special warnings, etc. Several studies [36][10] have shown that
there is an urgent need to improve the quality of drug package Figure 1 - Architecture of the system
leaflets because they are usually too difficult to understand by
patients, especially those with low literacy skills as well as 5.1.1 First module: NER
elderly people, and could result in misuse of medications. The first module performs the Named Entity Recognition (NER)
Posology and administration, contraindications and adverse drug task. Specifically, this module detects the mentions of drug
reactions seem to be the sections most difficult to understand effects in the package leaflets. To do this, we used
[26]. The vocabulary used is too specific, too technical. Also, MeaningCloud 16, a multilingual text analysis engine to extract
readers usually have trouble understanding long paragraphs, information from any type of texts. We have previously
especially those containing lists of side effects. Furthermore, developed several pipelines basesd on this technology to NER
drug package leaflets are usually written using a too small font (drugs, diseases, adverse effects among others) [42]. This tool
size (9 points), which does not seem to encourage patients to follows a dictionary-based approach to identify named entities. In
read these documents. Patient misunderstanding could be a particular, we used MedDRA17, a medical multilingual
potential source of drug related problems, such as medication terminology dictionary about events associated with drugs, as
errors and adverse drug reactions. dictionary to recognize drug effects. We used a plugin, which
integrates MeaningCloud into GATE.
We created a system with the objective of simplifying the lexical
difficulties that patients may encounter when facing with
package leaflets. The main goal of our system is to present the
information in an easy and clear way to read. Medical terms (in
15
particular, drug effects) are translated into lay terms, which http://gate.ac.uk/
16
patients can understand. In this way, accessibility is improved https://www.meaningcloud.com/
17
http://www.meddra.org/
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 483 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert

5.1.2 Second module: Lexical Simplifier (hemorrhage) (93) and gases (gases) (82) for flatulencia
Once the drug effects have been detected in the package leaflets, (flatulence) (21).
the second and main module identifies the effects whose names Table 3- Output example
are considered complex with the objective of replacing them by a
simpler synonym. Thus, the LexicalSimplifier module is aimed Muy frecuentes: diarrea e indigestión.

original
to check whether a given effect is a complex term or not. If it is a Frecuentes: náuseas, vómitos, dolor abdominal.
complex term, then it is replaced by an easier synonym, Poco frecuentes: hemorragia.
Raros: perforación gástrica, flatulencia, estreñimiento
following two different strategies: preferred term substitution and
most frequent term substitution.
Muy frecuentes: diarrea e dispepsia.
Preferred Term Substitution Frecuentes: náuseas, vómitos, dolor abdominal.

PT
Poco frecuentes: hemorragia.
The hierarchical structure of MedDRA allows to defining sets of Raros: perforación gástrica, flatulencia, estreñimiento
synonyms and providing a preferred term for each set. Therefore,
this first strategy consists in substituting an effect for its Muy frecuentes: diarrea e pirosis.
preferred term as described by MedDRA (if it is not itself). For Frecuentes: náuseas, vómitos, dolor abdominal.

freq
example, cefalalgia (cephalalgia) would be substituted for Poco frecuentes: sangrado.
cefalea (headache). Raros: perforación gástrica, gases, estreñimiento

Most Frequent Term Substitution

For this strategy, we take into account the frequency of At first, we decided to use Wikipedia as corpus to build the
appearance of all the different effects in a corpus of MedlinePlus index and to calculate the frequencies of effects, nevertheless, it
documents. A total of 1,536 documents were gathered from the was observed that many of effects were not found in Wikipedia
MedLinePlus website 18 using the Java HTML parser jsoup, 939 or their frequencies were not statistically significant. For this
belonging to drug package leaflets and 597 to general health reason, we finally decided to use a health-related site like
related articles about diseases, effects and diagnoses. We used MedLinePlus, instead of an open-domain resource like
Elasticsearch 19 to index the MedLinePlus documents. Wikipedia.
Elasticsearch is a flexible, powerful, open source, distributed and
real-time search and analytics engine, which runs on top of 6. CONCLUSIONS
Apache Lucene, offering complex search capabilities, high- For some people, it is difficult to infer the meaning of a word or
performance and being reliable. phrase from context when the word is unusual; for these users
In order to calculate the frequency of each effect in the corpus, the ability to read and understand may depend on the availability
we lean on the Elasticsearch index, obtaining the number of hits of specific definitions or the expanded forms of abbreviations.
for a particular effect, and then comparing it with the one of its Also, long sentences and complex linguistic structures can cause
synonyms. In this case, the complexity of the terms will be barriers in access to text content as indicated in WCAG and E2R
determined by the number of times they appear in the articles. guidelines. However, these guidelines despite providing
Our hypothesis is that complex terms should be less frequent resources and techniques, do not provide precise methods and
than simpler terms in the corpus. Therefore, an effect will be support (semi) automatic with which to address these
substituted for its synonym with highest frequency (if it is not accessibility issues concerning to text readable and
itself) in the corpus. For instance, catarro (nasopharyngitis), understandable. In this sense, the application of NLP resources
resfriado (cold), resfriado común (common cold) and síntomas and methods are proposed, which provide support for compliance
de resfriado (cold symptoms) can be considered synonyms with WCAG guidelines regarding readability and understanding.
according to our dictionary built from MedDRA. They appear 12, A review on this NLP technology has been presented.
48, 7 and 6 times respectively in the MedLinePlus corpus, With the aim of bringing the paper reader to the text
meaning that if any of them appeared in the text, it would be simplification approach, a prototype to simplify drug package
replaced by resfriado (cold), with 48 appearances. leaflet that implements a component for lexical simplification
Table 3 shows a passage of the adverse effects section of an has been shown.
Ibuprofen package leaflet.
In this system are being included other approaches to providing
In the first row we can see the original text from the package
support for more accessibility issues concerning abbreviations,
leaflet, with the detected effects highlighted in blue. The second
summaries, definitions of unusual words, etc. In addition,
row shows the output of the first strategy, Preferred Term
evaluations by experts and users are planned. The combination of
Substitution. We can see dispepsia (dyspepsia) replaces
using the approaches of text simplification with how to display
indigestión (indigestion) in the first line. The third row shows
the content simplified taking into account other important issues
the output of the second strategy, Most Frequent Term
as the presentation elements, page structure, navigation
Substitution. In this case, pirosis (pyrosis) (43 hits) stands for
structures etc. on the Web interface is a great challenge for us.
indigestión (32), sangrado (bleeding) (347) for hemorragia

7. ACKNOWLEDGMENTS
This work was supported by eGovernAbility-Access project
18
http://www.nlm.nih.gov/medlineplus/spanish/ (TIN2014-52665-C2-2-R)
19
https://www.elastic.co/)
XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web
| 484 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert

8. REFERENCES [17] Freyhoff, G., Hess, G., Kerr, L., Menzel, E., Tronbacke, B.,
[1] Älli S. Some Challenges for developing an Easy-to-Read Van Der Veken, K. (1998). Make it Simple, European
Website. Easy-to-Read on the Web. Symposium W3C. WAI. Guidelines for the Production of Easy-to-Read Information
2012 for People with Learning Disability; for authors, editors,
information providers, translators and other interested
[2] Anula, A. 2007. Tipos de textos, complejidad lingüística y persons.
facilitación lectora. In Actas del Sexto Congreso de
Hispanistas de Asia, Man-Ki, Jy-Eun, y Macías (eds.), [18] Gunning. R. The Technique of Clear Writing. McGraw-Hill,
pages 45–61, Seúl, República de Corea. New York, 1968.

[3] Barbieri T., Bianchi A., Sbattella L. (2005) MultiAbile: a [19] Hervás R., Bautista S., Rodríguez M., de Salas T., Vargas
multimodal learning environment for the inclusion of A., Gervás P.. Integration of Lexical and Syntactic
impaired e-Learners using tactile feedbacks, voice, Simplification Capabilities in a Text Editor. DSAI 2013
gesturing and text simplification. AAATE 2005, Lille, [20] Inclusion europe. (2009) Information for all. European
France. standards for making information easy to read and
[4] Bautista S. and Saggion H.. Can Numerical Expressions Be understand.
Simpler? Implementation and Demonstration of a Numerical [21] IFLA. (2010) Guidelines for easy-to-read materials.
Simplification System for Spanish. International Conference International Federation of Library Association and
on Language Resources and Evaluation (LREC-2014), 956- Institutions IFLA Professional Reports 120.
962
[22] Kandula, S., Curtis, D., & Zeng-Treitler, Q. (2010). A
[5] Bott S., Saggion H.:. Automatic Simplification of Spanish semantic and syntactic text simplification tool for health
Text for e-Accessibility. ICCHP (1) 2012: 527-534. 2013 content. AMIA Annual Symposium Proceedings (Vol. 2010,
[6] Carroll, G. Minnen, Y. Canning, S. Devlin, and J. Tait. p. 366). American Medical Informatics Association.
1998. Practical Simplification of English Newspaper Text to [23] Kerstin Matausch Annika Nietzio. Easy-to-Read and Plain
Assist Aphasic Readers. AAAI-98 Workshop on Integrating Language: Defining Criteria and Refining Rules. Easy-to-
Artificial Intelligence and Assistive Technology, pages 7–10 Read on the Web. Symposium W3C. WAI. 2012
[7] Cavnar William B. and Trenkle John M. 1994. N- [24] Kincaid J., Fishburn R. Rogers Jr., R., and Chissom B.
grambased text categorization. SDAIR’94, pages 161–175. Derivation of New Readability Formulas for Enlisted Navy
[8] Chandrasekhar, R. et al., (1996). Motivations and methods Personnel. Research Branch Report 8-75, Naval Air Station
for text simplifi-cation. COLING ’96, Conference on Memphis, illington,1975
ComputationalLinguistics- Volume 2. Stroudsburg, PA: [25] Leskela L. Guidelines or standards for Easy-to-read?. Easy-
Association of Computational Linguistics to-Read on the Web. Symposium W3C. WAI. 2012
[9] Chandrasekar R. and Srinivas B. Automatic Induction of [26] March, C. J., Prieto, R. M., Ruiz, A. A., Simón, L. P.,
Rules for Text Simplification. Knowledge-Based Systems, Barrio, C. I., & Danet, A. (2010). Quality improvement of
Elsevier, 10 (3):183 – 190, 1997. Knowledge-Based health information included in drug information leaflets.
Systems, Elsevier, 10 (3):183 – 190, 1997. Patient and health professional expectations]. Atención
[10] Davis, T. C., Wolf, M. S., Bass, P. F., Middlebrooks, M., primaria/Sociedad Española de Medicina de Familia y
Kennen, E., Baker, D. W., & Parker, R. M. (2006). Low Comunitaria, 42(1), 22.
literacy impairs comprehension of prescription drug warning [27] Martín A., Rossi G., Cechich A., Gordillo S.E:: Engineering
labels. Journal of General Internal Medicine, 21(8), 847-851 Accessible Web Applications. An Aspect-Oriented
[11] De Belder, Jan; Deschacht, Koen; Moens, Marie-Francine. Approach. World Wide Web 13(4): 419-440 (2010)
Lexical simplification, Itec2010, International Conference [28] Matausch K., Nietzio A.. Easy-to-Read and Plain Language:
on Interdisciplinary Research on Technology, Education and Defining Criteria and Refining Rules. Easy-to-Read on the
Communication Web. Symposium W3C. WAI. 2012
[12] Devlin, S., Tait J. The use of a psycholinguistic database in [29] Mbipom G. and Harper S.. 2009. The transition from web
the simplification of text for aphasic readers. Linguistic content accessibility guidelines 1.0 to 2.0: what this means
databases, 161-173. 1998 for evaluation and repair. SIGDOC '09. ACM, New York,
[13] Drndarevic B., Štajner S. and Saggion H. Reporting Simply: NY, USA, 37-44.
A Lexical Simplification Strategy for Enhancing Text [30] McCarthy D. and Navigli R.. 2007. Semeval- 2007 task 10:
Accessibility. Easy-to-Read on the Web. Symposium W3C. English lexical substitution task. International Workshop on
WAI. 2012 Semantic Evaluations (SemEval-2007)
[14] DuBay, W.H. 2004. The principles of readability. [31] Moreno L. and Martínez P. 2012. A review of accessibility
[15] EU HLG-report. (2012) Act now! Final report of EU High requirements in elderly users' interactions with web
Level Group of Experts on Literacy. applications. 13th International Conference on Interacción
Persona-Ordenador (INTERACCION '12).
[16] Fellbaum, Ch. editor. 1998. WordNet: An Electronic Lexical
Database. The MIT Press

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 485 |
 Exploring language technologies to provide support to WCAG 2.0 and E2R guidelines
L. Moreno | P. Martínez | I. Segura-Bedmar | R. Revert
[32] Moreno L., Valverde F., Martínez P., Pastor O., (2013). [40] Saggion, H., E. Gómez-Martínez, A. Anula,L. Bourg, and E.
Supporting accessibility in Web engineering methods: a Etayo. 2011. Text simplificationin simplext: Making texts
methodological approach, January, 2013, Journal of Web more accessible. Procesamiento del Lenguaje Natural, 46.
Engineering, RINTON PRESS, Vol.12 , No.3&4, [41] Sbattella, L., Tedesco R. Calculating text complexity during
[33] Miesenberger K., Matausch K., Petz A.. Including Easy to the authoring phase. Easy-to-Read on the Web. Symposium
Read, Legibility and Readability into Web Engineering. W3C. WAI. 2012
Easy-to-Read on the Web. Symposium W3C. WAI. 2012 [42] Segura-Bedmar, I, Revert , R and Martínez, P. (2014).
[34] Nomura, M., Nielsen, G.S., Tronbacke, B. (2010). Detecting drugs and adverse events from Spanish social
Guidelines for easy-to-read materials, IFLA Professional media streams, International Workshop on Health Text
Reports, No. 120, International Federation of Library Mining and Information Analysis (Louhi), 106-115
Association and Institutions [43] Siddharthan A.. 2002. Resolving attachment and clause
[35] Okazaki N. and Ananiadou S. 2006. A term recognition boundary amgiguities for simplifying relative clause
approach to acronym recognition. COLING/ACL. constructs. Student Research Workshop, 40th Meeting of
Association for Computational Linguistics, Stroudsburg, PA, the Association for Computacional Linguistics
USA, 643-650. [44] Sparck J. Automatic summarising: The state of the art.
[36] Parker, C. T. (2007). Literacy and misunderstanding Information Processing & Management. Volume 43, Issue 6,
prescription drug labels. Annals of internal medicine, November 2007, Pages 1449–1481.
147(4), 280. [45] Temnikova, I. (2012). Text complexity and text
[37] Petersen, S. E., & Ostendorf, M. (2007). Text simplification simplification in the crisis management domain (Doctoral
for language learners: a corpus analysis. In Proc. of dissertation, University of Wolverhampton).
Workshop on Speech and Language Technology for [46] Vari-Cartier, P.: (1981): Development and Validation of a
Education. New Instrument to Asses the Readability of Spanish Prose,
[38] Rello L, Baeza-Yates R., Dempere-Marco L., Saggion H.. Modern Language Journal, 65, 141-148.
Frequent Words Improve Readability and Short Words [47] W3C, WAI, Web Content Accessibility Guidelines (WCAG)
Improve Understandability for People with Dyslexia. 2.0. 2008. http://www.w3.org/TR/WCAG20/
INTERACT (4) 2013: 203-219
[48] W3C, WAI, Research Report on Easy to Read on the Web,
[39] Rello L. and Baeza-Yates R., Lexical Quality as a Measure Draft 2013 http://www.w3.org/WAI/RD/2012/easy-to-
for Textual Web Accessibility, Website Accessibility read/note/ED-E2R-20130417
Metrics, Symposium 5 December 2011,
http://www.w3.org/WAI/RD/2011/metrics/ [49] Zhu Z., Bernhard D., and Gurevych I. 2010. A monolingual
tree-based translation model for sentence simplification.
International Conference on Computational Linguistics,
pages 1353–1361.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 486 |
 Building a unified repository of interaction patterns
A. Palacios | R. García | T. Granollers | M. Oliva

Building a unified repository of interaction patterns

Alfons Palacios1,2 Roberto García2 Marta Oliva2

1 2 2
Escola Superior Politècnica Universitat de Lleida, Universitat de Lleida,
TecnoCampus Mataró-Maresme Jaume II 69, E-25001 Lleida, Spain Jaume II 69, E-25001 Lleida, Spain
(Universitat Pompeu Fabra) Fax +34 973 702 702 Fax +34 973 702 702
Av. Ernest Lluch 32 (Porta Laietana) [email protected] [email protected]
E-08302 Mataró, Spain
FAX +34 169 65 05 Toni Granollers2
[email protected] 2
Universitat de Lleida,
Jaume II 69, E-25001 Lleida, Spain
Fax +34 973 702 702
[email protected]
ABSTRACT and propose software components to build GUI that are
In recent years there have been many proposals of lists of understandable and usable (Tidwell, 2011) [ 2].
interaction patterns applicable to the design and development of We have studied and built a unified repository of interaction
web interfaces. Many of the patterns proposed in different lists patterns from proposals from (Tidwell, 2011),(van Welie, 2007)
are the same, perhaps with different names and different [ 3],(Toxboe, 2011)[ 4],(Yahoo, 2012),(Crumlish et al., 2009)[ 5] or
examples. Our purpose is to join these lists in a unified (Quince from Infragistics, 2012)[ 6]. We have also considered the
repository. work of (van Duyne et al., 2006) [ 7]) and (Scott and Neil, 2009)
[ 8].
This requires formalizing the definition of interaction patterns to
make them comparable, facilitate their classification or To classify the patterns we have used our proposal of Semantic
recommend them automatically based on a characterization of Web End-User Tasks in (Palacios et al., 2014) [ 9]. In Section 2, a
user needs, interaction context, etc. brief description of these tasks is done. Section 3 explains how
we have formalized and built the unified repository of interaction
In parallel, Web applications using Semantic Web technologies patterns and, finally, Section 4 presents the conclusions and the
are starting to proliferate. This implies in many cases a paradigm future work.
shift for user interaction that requires questioning which are the
more appropriate end user tasks, starting from “traditional” Web
interaction tasks but taking into account that they might change
2. A PROPOSAL OF SEMANTIC WEB
or raise new tasks. A good characterization of these tasks is then END-USER TASKS
the starting point to explore what patterns can be used to Several definitions of sets of end-user tasks for Web and
implement them. Semantic Web are presented in the work of (Heath et al, 2005)
[ 10], (Kellar et al, 2006) [ 11], (Battle, 2006) [ 12], (Mäkelä et al,
To facilitate this process, we have built a unified repository of 2007) [ 13], (Di Maio, 2008) [ 14] and (Sabou et al, 2007) [15].
interaction patterns classified by Semantic Web end-user tasks. From the analysis of this existing literature we have synthesized
We have built this repository using Semantic Web technologies. a set of generic end-user tasks that can assist Semantic Web
developers:
Categories and Subject Descriptors
H.5.2 [Information Interfaces and Presentation (HCI)]: User
2.1 Search
Interfaces – evaluation/methodology, graphical user interfaces This kind of tasks corresponds to those when a user poses a
(GUI), interaction style, standardization, theory and methods, query and obtains a set of results that might be rendered in
user-centered design, user interface management systems. different ways. We include here when the search might be
delayed or repeated in the future, like in monitoring scenarios.
General Terms 2.2 Browse
Design, experimentation, human factors, standardization. This task is performed when the user moves through the
information currently displayed. In the context of Web
Keywords information systems this is usually done by following the links
Human computer interaction (HCI), Semantic web, Interaction that connect the information to related information pieces.
patterns, Semantic web end-user tasks, Semantic web usability.
2.3 Annotate
In this task the user describes a resource by providing properties
1. INTRODUCTION and values that model its characteristics, its relations to other
An Interaction Pattern is the description of a solution to common resources, etc. This task includes providing a completely new
problems of usability and accessibility in using Graphical User
Interfaces (GUI) (Yahoo, 2012) [ 1]. Interaction Patterns model

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 487 |
 Building a unified repository of interaction patterns
A. Palacios | R. García | T. Granollers | M. Oliva
description but also complementing an existing one, modifying it
or deleting some or all of the attributes currently available.
2.4 Mashup
This task is about the user gathering different pieces of
information and combining them in order to get something more
than the simple aggregation of those pieces. Specific examples of
this task range from simple mashups such as combining a set of
resources that are geographically situated in order to, for
instance, which are the hotels near a venue, or resources with
temporal dimension that are arranged in a calendar or timeline in
order to facilitate scheduling.
2.5 Map
This task takes place when the user defines mappings among
terms from different vocabularies. It is not constrained to a
particular set of resources like in the case of the Mashup task,
and it does not operate at the level of particular resource
descriptions. On the contrary, in this task, the user is working at
the level of the vocabularies. These vocabularies might be used
in descriptions for many resources, some of which the user might
not be aware of it at the moment.
3. BUILDING A UNIFIED REPOSITORY
2.6 Share OF INTERACTION PATTERNS
This task considers uploading, publishing, updating and deleting The process of building the repository has been done in the
pieces of content with the intention of making them available to following steps: classification, formalization and implementation.
other users, who can access the content from a place and at a
time individually chosen by them. This last statement allows to 3.1 Classification
clearly distinguishing this tasks from the Communicate task, All patterns are classified taking into account the proposed
which is presented next. semantic web end-user tasks discussed in the previous section.
Many of them might be mapped directly from the Web domain to
2.7 Communicate the Semantic Web, like most of Welie’s patterns for Search and
This task is about sharing information directly with particular Browse for example. In any case, new features in Semantic Web
users, without the intention of making it available to other users. provide new opportunities in some patterns. Our analysis shows
The process is in this case driven by the user participating in this that the user tasks where the contribution of Semantic Web
task as the emitter. technologies might be more important are in Annotate, Mashup
and Map user tasks. For these end-user tasks –the newest and
2.8 Transact
closest to the Semantic Web- is difficult to find interaction
This task is associated with user actions that provoke a change in
patterns; new ones for Semantic Web have to study and define.
the state of a real-world entity or of a resource in a system
outside the scope of the system the user is interacting with. Furthermore, sometimes different sources explain the pattern
giving different names and different shades; we have been
In a previous study (Palacios et al, 2014) we presented examples
grouped identical or similar patterns in a single proposal.
of using these tasks in the particular case of the semantic web, as
they could be enriched with semantic content and we compared Therefore, the work of classifying includes:
this set of semantic web end user tasks with existing literature: a) Identify patterns proposals from different sources that
Table 2. Semantic web end user tasks and previous works are inherently the same, despite having different names
or descriptions.
b) Assign each interaction pattern to an end user task.
Sometimes an interaction pattern can be assigned a
more than one task. We have chosen one as primary
and leave the rest as secondary.
c) Choose a name and a definition for each set of identical
or similar patterns. A definition that can be formally
described in the future to enable the implementation of
recommendation systems or the automatic generation of
GUI.

3.2 Formalization
In order to implement a unified repository is important to define
a conceptual model for the inventory of interaction patterns.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 488 |
 Building a unified repository of interaction patterns
A. Palacios | R. García | T. Granollers | M. Oliva
The goal is to eventually create one or more ontologies that
represent the interaction patterns.
A first proposal of ontology is explained in Figure 1.

Figure 2 Unified repository in Rhizomer

The Rhizomer platform (García et al., 2008) [16] (W3C, 2013)
[17] is a Semantic Content Management System (SemCMS)
based on a Resource Oriented Approach (RESTful) [18] and
Figure 1 Ontology for an inventory of interaction patterns Semantic Web technologies. The Rhizomer project constitutes a
technological framework that can be used to build up semantic
The goal is to define a framework to merge the sets of interaction
web portals. Rhizomer manages RDF metadata in a user-friendly
patterns by different authors in a single inventory that allows
way. It facilitates not just the common Semantic Web to end-user
adding new patterns defined in the future.
interaction provided by semantic web browsers, it also provides
Centralization in a single inventory of all sources of interaction the reverse interaction path: end-users can create, edit and
patterns makes it easier for the research community on issues of remove semantic metadata. The whole interaction is performed
HCI and the Semantic Web, and, in general, the entire developer through a “classical” HTML interface in a usable way, which
community. minimises user efforts and maximises the benefits they obtain
In the model we relate the concepts of interaction pattern from their Semantic Web experiences. The overall intention is to
introduced in section 1 and semantic web end user task minimise the gap among computers and human beings in the
commented in section 2, defining subclasses for pattern-type context of Semantic Web. The semantic metadata management
consolidated proposal. Specific: part provided by Rhizomer is combined with an easy to use and
simple content management system based on a Wiki engine. The
• Task represents semantic web end user tasks. wiki provides the means to create and maintain information
• InteractionPattern are the proposed interaction objects that are described with semantic metadata.
patterns that have been studied (from one or several To work with Rhizomer we have to translate the conceptual
proposals by one or more authors). The patterns are model to RDF. Once translated the conceptual model to a RDF
classified by defining subclasses for equivalent patterns model, a repository for the inventory of interaction patterns have
of various proposals. Subclasses are instantiated with stayed at Rhizomer.
patterns from different sources. The repository where is located the inventory of interaction
• SetOfPatterns is each of the sets of interaction patterns using Rhizomer, can be found in :
patterns proposed by different authors to the developer http://indagus.udl.cat/iPatternsSemWeb/ .
community of GUI.
• Author contains information of all researchers or 3.4 Results
companies that have made proposals in the field of The core of this unified repository of interaction patterns is built
interaction patterns. from proposals of (Tidwell, 2011)(94 interaction patterns),(van
Welie, 2007)(131 patterns), (Toxboe, 2011)(54 patterns),(Yahoo,
3.3 Implementation 2012)(59 patterns),(Crumlish et al., 2009)(124 patterns).
Once we have formalized the unified interaction patterns Our analysis against related work specific for Web information
inventory, the next step is to implement it in a repository. systems shows that the user tasks where the contribution of
We implement this repository with semantic web technologies. Semantic Web technologies might be more important, because
We have used Rhizomer for implementing this repository. they a less consider or not considered at all, are Annotate,
Mashup and Map.
This is reflected in the number of patterns that we have
associated to each semantic web end-user task; while tasks as
Search (20 patterns from different sets), Browse (134 patterns),
Share (93 pattens), Communicate (12 patterns) or Transact (130
patterns) are the result of years of works to find solutions to the
interaction with the web, for end-user tasks newer and closer to
the semantic web as Annotate, Mashup or Map, we have only
found a few interaction patterns in these sets.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 489 |
 Building a unified repository of interaction patterns
A. Palacios | R. García | T. Granollers | M. Oliva
A future work is to define new interaction patterns that are e) Build a model of interaction patterns that automates the
specific for the Semantic Web and can be applied in new end- process of study, to centralize the proposals of different
user tasks and adapt existing ones to new features of the end-user sets of patterns in a single repository and provide the
tasks. basis for proposals for self-generation of user interfaces
for the semantic web. The model should allow us to
4. CONCLUSIONS AND FUTURE WORK validate the classification of patterns and decisions on
We have proposed a set of semantic web end user tasks is equivalences between patterns.
defined (Palacios et al., 2014); the set of tasks includes Search, f) Model the interaction needs of the semantic web end
Browse, Annotate, Mashup, Map, Share, Communicate and users to link needs with interaction patterns.
Transact and they have been described, and they have been used
to classify interaction patterns. g) Develop a recommender to help the process of design
and implementation of interfaces that work with the
Once the user tasks have been identified, it is really useful to
Semantic Web
have an inventory of interaction patterns that give support to
these user tasks as a guideline. There are many lists of
interaction patterns, though most of them focus on Web systems 5. ACKNOWLEDGEMENTS
or other interactive systems without adapting their proposal to Partially supported by the research project InDAGuS,
Semantic Web user tasks. Infrastructures for Sustainable Open Government Data with
Geospatial Features (Spanish Government TIN2012-37826-C02)
Our aim has been to build an inventory of Semantic Web
interaction patterns starting from existing inventories, e.g.
Tidwell’s, van Welie’s, Toxboe, the Yahoo! Design Pattern 6. REFERENCES
Library, or Crumlish & Malone's. This objective has been built
on top of these pattern libraries and classified them taking into
account the proposed Semantic Web end-user tasks.
[1] The Yahoo! Design Pattern Library.
Different sets of interaction patterns have different proposals of http://developer.yahoo.com/ypatterns, 2007-2012.
patterns that are similar or identical although they have different
names and definitions. A work has been done for detecting and [2] J. Tidwell. ”Designing Interfaces, second edition” O’Reilly,
grouping identical or similar patterns in unified proposals with a 2011. http://designinginterfaces.com.
single name and a single description. [3] M. van Welie. ”Welie.com :Patterns in Interaction Design”
To build a unified repository a conceptual model has been http://www.welie.com, 2007.
defined. In this conceptual model interaction patterns have been
[4] A. Toxboe. ”UI-Patterns :User Interface Desing Pattern
related to end user tasks and sets of patterns by different authors.
Library” http://ui-patterns.com 2007-2011..
Subclasses have been proposed to represent each of the unified
interaction patterns. Subclasses have been instantiated by [5] C. Crumlish and E. Malone. ”Designing Social Interfaces”
specific patterns of different sets. O’Reilly, 2009. http://designingsocialinterfaces.com 2007-
The conceptual model has been translated to RDF to build a 2009.
unified repository using the Rhizomer Platform, a Semantic [6] Quince from Infragistics, http://quince.infragistics.com
Content Management System based on Semantic Web 2012.
technologies.
[7] D.K. van Duyne, J.A. Landay and J.I. Hong. ”The Design of
Our future work is aimed at achieving the following challenges: Sites,second edition” Prentice Hall, 2007
a) Find a way to formalize the definition of each http://designofsites.com.
interaction pattern with ontologies that allow an
[8] B. Scott and T. Neil. ”Designing Web Interfaces” O’Reilly,
analysis in-depth.
2009.
b) Check the patterns that have been grouped using these
[9] A. Palacios, R. García, M. Oliva and T. Granollers.
ontologies.
”Semantic Web End-User Tasks” in Interaccion 2014 XV
c) Enrich patterns and end user tasks with new features of International Conference on Human Computer Interaction.
the Semantic Web, new opportunities should be International Conference Proceedings Series. ACM Digital
detected. Library, 2014..
d) Study and define new interaction patterns that are [10] T. Heath, M. Dzbor, E. Motta: "Supporting User Tasks and
specific for the new semantic end user tasks, there is Context: Challenges for Semantic Web Research", in: Proc.
room for novel interaction patterns that the Semantic ESWC2005 Workshop on End-User Aspects of the Semantic
Web might make possible. Our analysis shows that Web (UserSWeb), 2005.
user tasks where the contribution of Semantic Web
[11] M. Kellar, C. Watters, M. Shepherd. A Goal-based
technologies might be more important are Annotate,
Classification of Web Information Tasks. In 69th Annual
Mashup and Map.
Meeting of the American Society for Information Science
and Technology, Austin, US, 3-8 November 2006..

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 490 |
 Building a unified repository of interaction patterns
A. Palacios | R. García | T. Granollers | M. Oliva

[12] L. Battle, Preliminary Analysis of Users and Tasks for the

Semantic Web, Semantic Web User Interaction Workshop,
International Semantic Web Conference, Athens, GA, 2006.
[13] E. Mäkelä, K. Viljanen, O. Alm, J. Tuominen, O.
Valkeapää, T. Kauppinen, J. Kurki, R. Sinkkilä, T. Känsälä,
R. Lindroos, O. Suominen, T. Ruotsalo and E. Hyvönen.
Enabling the Semantic Web with Ready-to-Use Web
Widgets. FIRST - First Industrial Results of Semantic
Technologies Workshop, at the ISWC+ASWC Conference,
Busan, Korea, November 11-15, 2007.
[14] P. Di Maio, Toward Global User Models for Semantic
Technologies: Emergent Perspectives, in Proceedings of the
Third Asian Semantic Web Conference, ASWC’08
Bangkok, Thailand, 2008, pp. 141-152.
[15] M. Sabou, J. Garcia, S. Angeletou, M. D'Aquin and E.
Motta. Evaluating the Semantic Web: A Task-based
Approach in The 6th International Semantic Web
Conference and the 2nd Asian Semantic Web Conference,
2007.
[16] R. García, J.M. Gimeno, F. Pedrix, R. Gil, M. Oliva. The
Rhizomer Semantic Content Management System,
WSKS’08 1st World Summit on the Knowledge Society.
Lecture Notes in Artificial Intelligence, Vol. 5288, pp. 385-
394. Springer, 2008.
[17] World Wide Web Consortium (W3C). Software in
www.w3c.org : http://www.w3.org/2001/sw/wiki/Rhizomer
last update, 2013.
[18] Richardson, L., Ruby, S.: Restful Web Services. O'Reilly
2007.

XVI Congreso Internacional de Interacción Persona-Ordenador | Accessibility and Semantic Web

| 491 |
1º CONCURSO 2015
AIPO DE TFG/TFM
 Desarrollo de una herramienta para el diseño y ejecución de actividades enfocadas a ancianos con el tabletop NIKVision
C. Bonillo Fernández

Desarrollo de una herramienta para el diseño y ejecución

de actividades enfocadas a ancianos con el tabletop
NIKVision
Clara Bonillo Fernández
GIGA Affective Lab
Dep. Informática e Ingeniería de Sistemas Universidad de Zaragoza, España
[email protected]
ABSTRACT realizó una propuesta de actividades para cada una de las áreas
elegidas, las cuales son explicadas con más detalle en la
El principal objetivo de este trabajo ha sido desarrollar una sección 5.
herramienta que permita diseñar y ejecutar actividades para
personas mayores que aprovechen las ventajas que ofrecen los 3. DESCRIPCIÓN DE LA
tabletops, utilizando para ello el tabletop tangible NIKVision,
perteneciente al grupo GIGA Affective Lab de la Universidad
HERRAMIENTA
Tomando como base la propuesta de actividades previamente
de Zaragoza
mencionada, se realizó el análisis de la herramienta extrayendo
Lo que se ha intentado conseguir es que cualquier persona que los requisitos que esta habría de cumplir.
necesite crear una actividad para tabletop pueda hacerlo de una Gracias a dicho análisis de requisitos, se decidió modelar las
manera sencilla con la herramienta realizada, sin necesidad de actividades para tabletop como una secuencia de tareas (ver fig.
que esa persona necesite tener conocimientos de programación. 1). Cada tarea es un objetivo que el jugador ha de alcanzar para
Como objetivo secundario, se han desarrollado también un poder avanzar en el juego. Cada tarea está compuesta por:
conjunto de actividades como demostración de la validez de la • Un fondo: el tablero.
herramienta junto con una evaluación de la misma.
• Varias áreas: zonas del tablero o de una pieza de
juego en las que posicionar unas determinadas piezas de juego
1. ESTADO DEL ARTE: TABLETOPS Y tiene un significado.
TERCERA EDAD
• Varias piezas de juego: objetos utilizados en el juego.
Se ha realizado un estudio sobre ejemplos de actividades
Se colocan en las áreas; fuera de ellas la pieza no tiene ningún
usadas por personas mayores, con especial énfasis en aquellas
significado para el juego.
que hacen uso de tabletops, concluyéndose que aunque las
actividades estudiadas permiten a las personas mayores trabajar • Retroalimentación: elementos gráficos o sonoros que
sus capacidades cognitivas, no aprovechan todo lo que podrían muestran las consecuencias de las acciones del jugador.
las ventajas de tener un dispositivo tabletop, puesto que casi La estructura jerárquica de los diferentes elementos hizo que la
todas las actividades se realizan usando la mano o algún forma más adecuada de modelar las actividades fuera mediante
dispositivo que podría ser fácilmente sustituido por la mano, y un lenguaje de marcas como el XML. La herramienta
en los casos en los que se utilizan objetos, éstos son pocos y desarrollada es la encargada de cargar y ejecutar los ficheros
demasiado concretos. XML que contienen las actividades en el dispositivo tabletop.
Por esa razón, la principal ventaja de NIKVision reside en la
variedad de objetos que permite utilizar, de modo que
prácticamente cualquier juguete puede usarse sin necesidad de
crear nuevos objetos.

2. ÁREAS COGNITIVAS Y
ACTIVIDADES
El envejecimiento trae consigo diversos problemas. Algunas de
las áreas cognitivas que se ven más afectadas cuando una
persona envejece son: la atención, la memoria, la capacidad de
cálculo, el lenguaje, la orientación espacio-temporal y el
razonamiento. Por lo tanto, nuestro objetivo ha sido buscar
actividades que ayuden a potenciar dichas áreas para ayudar a
prevenir, en la medida de lo posible, su deterioro.
Este trabajo se ha centrado en las áreas cognitivas de la Figura 1: Jerarquía de los elementos
atención, la memoria y el razonamiento, puesto que se vio que
eran las que más podían aprovechar el uso de objetos sobre el La herramienta ofrece dos opciones para la ejecución de las
tabletop. Una vez seleccionadas las áreas en las que trabajar, se actividades: la primera opción es ejecutar una actividad

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 493 |
 Desarrollo de una herramienta para el diseño y ejecución de actividades enfocadas a ancianos con el tabletop NIKVision
C. Bonillo Fernández
concreta indicando la actividad a ejecutar en un fichero de 4.2 Actividades de memoria
arranque. La segunda opción es mostrar en el propio tabletop Como actividades de memoria se desarrollaron las actividades
un menú en el cuál se muestran todas las actividades “Lista de la compra” y “Viajes”.
desarrolladas, de modo que el usuario puede seleccionar la
actividad que desee ejecutar en ese momento de forma táctil En “Lista de la compra” se muestra en pantalla una imagen con
sobre el tabletop. Una vez dentro de la actividad, el usuario una lista de la compra con alimentos durante cinco segundos.
tiene las opciones de volver al menú, de pausar la actividad y Pasado ese tiempo, se quita la imagen de la lista poniéndose
de poder modificar en tiempo de ejecución las actividades. una imagen con una bolsa de la compra y el usuario ha de
seleccionar los alimentos correctos y situarlos en la bolsa. Los
La herramienta también guarda información sobre las objetos que se utilizan en esta actividad son diversos juguetes
actividades realizadas en un fichero de logs en el que se guarda de comidas y bebidas (ver fig 3. izquierda).
la fecha y hora de realización de la actividad, el tiempo
empleado en su resolución, las acciones correctas e incorrectas En la actividad “Viajes”, al usuario se le muestra un mapa de
realizadas y si la actividad se ha completado hasta el final o no. Europa y se reproduce una grabación con un itinerario a seguir
con distintos medios de transporte. El usuario ha de recordar el
4. ACTIVIDADES DESARROLLADAS itinerario y situar los transportes correspondientes en los países
A continuación se van a comentar las actividades que se correctos. Los objetos que se utilizan en esta actividad son tres
desarrollaron como ejemplo para cada una de las áreas juguetes de medios de transporte: tren, avión y barco (ver fig 3.
cognitivas. derecha).
4.1 Actividades de atención
Como actividades de atención, se desarrollaron las actividades
“¿Cuántos hay?” (atención dividida), “Marca los símbolos”
(atención selectiva), y “Tangram” (atención sostenida).
En “¿Cuántos hay?” al usuario se le muestra una imagen con
números entre el 0 y el 9. Nada más empezar la actividad,
suena una grabación que le dice al usuario qué número ha de
buscar. El usuario tendrá que ir situando fichas sobre todos los
números que encuentre de ese tipo, mientras que a la vez se Figura 3: Actividades de memoria. Izquierda: Lista de la
estará reproduciendo una grabación con una secuencia de compra. Derecha: Viajes.
golpes, de modo que el usuario, a la vez que marca los
5.2 Actividades de razonamiento
números, tendrá que decir cuántos golpes van sonando. Los
objetos que se utilizan en esta actividad son fichas que el Como actividades de memoria, se desarrollaron las actividades
usuario tendrá que colocar en los números correspondientes “Analogías” y “Completa la secuencia”.
(ver fig. 2 izquierda). En “Analogías” se dispone de un conjunto de fichas con dos
En “Marca los símbolos” se presenta un mapa con símbolos de imágenes cada una. A cada imagen le corresponde una pareja,
gasolineras, restaurantes, hoteles, farmacias...y se pide al que estará en otra ficha diferente. El objetivo de la actividad es
usuario que marque todos los que sean de un tipo concreto. Los que todas las fichas formen una cadena de modo que todos los
objetos que se utilizan en esta actividad son fichas que el extremos de las fichas estén con su correspondiente pareja. Los
usuario tendrá que colocar en los símbolos correspondientes objetos que se utilizan en esta actividad son las diferentes
(ver fig. 2 centro). fichas (ver fig 4. izquierda).
La actividad “Tangram” tiene dos modos: el fácil, en el que se En “Completa la secuencia” se usan fichas de dominó para
ve en qué posición está cada una las piezas que conforman la mostrar una secuencia en la que faltan algunas piezas. El
figura, y el difícil, en el que solo se muestra el contorno de la usuario ha de seleccionar la pieza que falta para completar la
figura. En la modalidad difícil también se trabaja el secuencia. Los objetos que se utilizan en esta actividad son las
razonamiento, al tener que deducir dónde va cada pieza. Los fichas de dominó (ver fig 4. derecha).
objetos con los que trabajamos en esta actividad son las siete
piezas del Tangram (ver fig. 2 derecha).

Figura 2: Actividades de atención. Izquierda: ¿Cuántos Figura 4: Actividades de razonamiento. Izquierda:

hay? Analogías.
Centro: Marca los símbolos. Derecha: Tangram Derecha: Completa la secuencia

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 494 |
 Desarrollo de una herramienta para el diseño y ejecución de actividades enfocadas a ancianos con el tabletop NIKVision
C. Bonillo Fernández

5. EVALUACIÓN
Una vez que se tuvo un prototipo implementado de la
herramienta, alumnos de la asignatura Diseño Centrado en el
6. CONCLUSIONES Y TRABAJO
Usuario de Grado de Ingeniería Informática de la Escuela de FUTURO
Ingeniería y Arquitectura de Zaragoza hicieron, como práctica La principal línea de trabajo futuro sería implantar la
opcional de la asignatura, el diseño de una actividad para el herramienta en un ámbito sanitario para que pueda ser probada
tabletop NIKVision. El objetivo principal que se buscaba era por personas mayores y así ver si su uso les permite mejorar
ver si la herramienta permitía realizar el diseño de actividades sus capacidades cognitivas y evitar su deterioro, así como
de manera sencilla para aquellas personas que no estuvieran comprobar la utilidad de los ficheros de logs generados durante
familiarizadas ni con el tabletop NIKVision ni con la la realización de las actividades.
herramienta mediante una evaluación de su usabilidad, y se
obtuvieron buenos resultados que indicaron que el manejo de la
herramienta era bastante sencillo.

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 495 |
 Análisis de requerimientos y prototipado de una aplicación accesible para personas ciegas basada en la API de Google Maps
R. Alcaraz Martínez

Análisis de requerimientos y prototipado de una aplicación

accesible para personas ciegas basada en la API de Google Maps
Rubén Alcaraz Martínez
EINA, Centre Universitari de Disseny i Art de Barcelona
Passeig de Santa Eulàlia, 25
08017 Barcelona
34 93 203 09 23
[email protected]

ABSTRACT incumplimientos del principio de accesibilidad universal son

Objetivos: Explorar las posibilidades ofrecidas por las diferentes considerados por esta ley como supuestos de discriminación y de
plataformas de cartografía digital, así como de otros lenguajes, violación del derecho a la igualdad de oportunidades.
librerías y estándares actuales del Web, para crear una aplicación Diversos autores (Nielsen, 2000; Romero, 2001; Arch y
accesible para personas ciegas que ofrezca servicios relacionados Letourneau, 2002)[1][2][3] han destacado la importancia de la
con información de carácter geográfico. accesibilidad no sólo como cuestión ética, sino también como
factor de competitividad en el entorno empresarial. Teniendo en
Metodología: Desarrollo informático de una prueba de concepto
cuenta tanto el importante porcentaje que representan las
consistente en un mapa digital accesible para personas ciegas
personas con algún tipo de discapacidad entre los usuarios o
basado en la API de Google Maps, a partir de los requerimientos
clientes potenciales de cualquier web o aplicación, como el
derivados del estudio de las características del colectivo objetivo,
elevado grado de exigencia existente en el entorno web fruto de
del análisis de las directrices para la accesibilidad del contenido
una competitividad cada vez mayor, la integración de los
web (WCAG 2.0) y de la especificación WAI-ARIA 1.0.
estándares de accesibilidad en el desarrollo de estos productos
Resultados: Se ha obtenido un primer prototipo que permite puede resultar un rasgo diferencial frente a la competencia.
realizar búsquedas sobre la base de datos de Google Places y Además, la accesibilidad también resulta de utilidad en otros
obtener rutas entre dos puntos a partir del servicio de rutas de ámbitos, como en el SEO (Dolson, 2012; Nielsen, 2012;
Google Maps, en el que destaca el acceso mayoritario a las Carreras, 2013)[4][5][6] o en la mejora de las versiones móviles
funcionalidades de la aplicación mediante una interfaz de de los portales web, entre otros. El proyecto que nos ocupa
teclado, así como la aplicación de la ontología de roles, estados y beneficia principalmente a las personas ciegas, pero también es
propiedades de WAI-ARIA para anunciar los cambios en la aplicable o útil para usuarios con otros tipos de discapacidades.
interfaz provocados por la interacción del usuario con la Se trata, además, de mejorar un tipo de contenido que
aplicación. prácticamente todas las administraciones públicas ya han
incorporado en sus respectivos portales en Internet, generalmente
de una manera no accesible. Algunos ejemplos sobre los que se
Keywords podría aplicar son los callejeros digitales de las ciudades, los
Accesibilidad web, WCAG 2.0, WAI-ARIA, Google Maps API
servicios de rutas en transporte público o, incluso, en mapas que
v3, Servicios basados en la localización, Personas ciegas.
se han creado en otros ámbitos públicos o privados, como los
directorios de empresas o los portales de búsqueda de hoteles,
1. INTRODUCCIÓN Y JUSTIFICACIÓN entre otros.
El campo de la información y la comunicación constituye un
elemento esencial en la vida social, política, cultural y económica 2. LAS RIA, AJAX Y GOOGLE MAPS
de las personas. Los avances producidos dentro del ámbito de las Las RIA (Rich Internet Applications) son aplicaciones web
tecnologías de la información y la comunicación han abierto ejecutadas desde un navegador, capaces de incorporar la
nuevas posibilidades de acceso a estos aspectos, pero al mismo interactividad y dinamismo característico de las aplicaciones de
tiempo también han propiciado, de manera indirecta, la creación escritorio. En este tipo de aplicaciones, el contenido se añade,
de nuevas barreras para las personas con discapacidad. Por lo modifica o elimina sin intervención del usuario, o como
que respecta al disfrute y uso de todo tipo de bienes, servicios y consecuencia de una acción determinada. Una de las tecnologías
tecnologías, el marco regulador debería abarcar todas las protagonistas en el desarrollo de las RIA es AJAX
situaciones en las cuales se ponen a disposición del público en (Asynchronous JavaScript And XML). La naturaleza dinámica de
general. Según lo dicho, resulta poco relevante quién proporcione AJAX es a la vez su gran virtud y su gran inconveniente, si estas
el bien o preste el servicio, ya sea el propio Estado o una aplicaciones no se diseñan desde el punto de vista del usuario
empresa privada. Si el bien o servicio se pone a disposición del con discapacidad. La creación de zonas dinámicas en la interfaz
público en general, resulta lógico que deba adaptarse a unos que se actualizan constantemente o a petición del usuario, sin
requisitos de accesibilidad determinados, como recoge la necesidad de actualizar la página, y el uso de todo tipo de
disposición final sexta de la Ley 51/2003, de 2 de diciembre, de controles y botones similares a los que podemos encontrar en
igualdad de oportunidades, no discriminación y accesibilidad cualquier aplicación de escritorio, plantean desafíos adicionales
universal de las personas con discapacidad. Los de accesibilidad. Google Maps se enmarca dentro de lo que

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 496 |
 Análisis de requerimientos y prototipado de una aplicación accesible para personas ciegas basada en la API de Google Maps
R. Alcaraz Martínez
hemos definido como RIA y se basa en AJAX para ofrecer una para mejorar la accesibilidad de los contenidos y aplicaciones
buena parte de sus funcionalidades. web enriquecidas.
Las áreas de una página web que ven modificados sus contenidos
3. LAS PERSONAS CIEGAS Y EL ACCESO de forma automática se conocen con el nombre de live regions
A LA INFORMACIÓN EN LA WEB (zonas vivas). Estos nuevos contenidos pueden pasar
Conocer las características de los diferentes perfiles de desapercibidos para las personas ciegas o con baja visión si no
discapacidad, así como las estrategias y ayudas técnicas que son marcados adecuadamente, al no ser capaz el lector de
utilizan al interactuar con el contenido, es imprescindible para pantalla de anunciar los cambios producidos. WAI-ARIA propone
poder crear productos accesibles para todos ellos. tres atributos (aria-live, aria-atomic y aria-relevant) que una vez
incorporados a nuestro código fuente, nos permiten identificar las
El usuario ciego accede al contenido web de una manera muy
zonas vivas y especificar cómo y cuándo queremos que se
diferente a la que lo hacen el resto de usuarios. Mientras que los
anuncien al usuario. WAI-ARIA se puede utilizar además para
usuarios videntes exploran y toman decisiones en base a la
especificar el rol de los elementos de la interfaz, determinando
organización visual del contenido, el ciego que accede mediante
su función en la página. También permite codificar propiedades y
un lector de pantalla no es capaz de analizar la totalidad de la
estados, por ejemplo, indicando que el elemento de un formulario
página con tanta inmediatez. Frente a contenidos organizados
es obligatorio o que un elemento se encuentra marcado o
jerárquicamente o al uso de zonas destacadas cromáticamente
desmarcado. Información a la que los lectores de pantalla podrán
para atraer la atención hacia los elementos más importantes de la
acceder para informar al usuario ciego.
página, el contenido al que acceden los discapacitados visuales
es lineal y basado en texto. El posicionamiento en la página o el
diseño gráfico ni ayudan, ni dificultan de manera inherente el 6. TEST DE USUARIOS
acceso de este colectivo a la información que contiene el sitio Las soluciones técnicas comentadas en el punto anterior se
web. Simplemente, resultan irrelevantes. Son otros factores, pusieron a prueba en un prototipo funcional. A lo largo del
como el orden por programación, una sintaxis correcta o la proceso de desarrollo de este prototipo, se realizaron diferentes
posibilidad de saltar bloques de información, los que realmente test con un usuario ciego al que se le propusieron diferentes
suponen una óptima experiencia de usuario para este colectivo. tareas a realizar. Fruto de estos test, se logró comprender mejor
las necesidades de este colectivo, y se corrigieron los diferentes
4. PROBLEMAS DE ACCESIBILIDAD problemas de accesibilidad detectados.
Las principales barreras de accesibilidad de las aplicaciones
desarrolladas a partir de la API de Google Maps las encontramos 7. CONCLUSIONES
en la imposibilidad de acceso a los controles de la interfaz Los test de usuario realizados demostraron la capacidad del
(botones para moverse por el mapa, cambiar de tipo de mapa, usuario ciego para interactuar con los diferentes controles y
pulsar sobre un marcador...) mediante el teclado, y en la formularios del prototipo, acceder a los resultados de búsqueda y
posibilidad de que una buena parte de los contenidos (resultados a las rutas recomendadas por la aplicación. Más allá de algunas
de una búsqueda, indicaciones de una ruta solicitada, errores...), funcionalidades pendientes de implementar, uno de los
pasen desapercibidos para el usuario ciego al actualizarse de principales problemas de accesibilidad pendientes son las
manera dinámica mediante AJAX. indicaciones proporcionadas por el servicio como resultado de las
peticiones de rutas en transporte público. Se trata de un tipo de
indicaciones que incluyen tramos a pie y otros en metro, bus, etc.
5. SOLUCIONES TÉCNICAS Mientras que las indicaciones de rutas a pie son suficientemente
La API de Google Maps genera de manera automática los
descriptivas para la mayoría de usuarios ciegos, las indicaciones
diferentes controles de la interfaz como elementos <div>
mixtas de las rutas en transporte público resumen demasiado la
incapaces de recibir el foco y, por lo tanto, no son operables a
información del trayecto hasta la estación de origen, y el de la
través del teclado. Para solucionar este problema existen varias
estación de destino hasta el punto final del trayecto,
alternativas. La primera de ellas consiste en sacar fuera del mapa
imposibilitando la autonomía del usuario ciego.
los diferentes controles de la interfaz, creando nuevos elementos
interactivos que sí puedan captar el foco. Otra de las opciones
pasa por manipular el DOM de la página mediante JavaScript 8. RECURSOS
para insertar en el código HTML generado por Google Maps, es El texto completo del trabajo se puede consultar en
decir, dentro del mismo mapa, elementos que sí capten el foco en http://hdl.handle.net/2445/58427.
lugar de los <div> creados por defecto. La creación de controles
accesibles no sólo beneficia a los usuarios ciegos, sino que 9. AGRADECIMIENTOS
también resulta de gran ayuda para otros tipos de discapacidades Esta comunicación tiene su origen en el trabajo de final de
como las motrices o cualquier otra que impida a las personas que máster homónimo defendido durante el curso 2013/2014 en el
la padecen usar el ratón. Máster de Gestión de Contenidos Digitales de la Universidad de
En relación a los contenidos que se actualizan automáticamente Barcelona/Universidad Pompeu Fabra. Agradezco toda la ayuda,
en la interfaz, la propuesta de solución pasa por utilizar WAI- consejos y horas de dedicación que, en el transcurso de su
ARIA, una especificación del W3C que proporciona una elaboración, mi tutora Mireia Ribera me concedió.
ontología de roles, estados y propiedades que se pueden utilizar

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 497 |
 Análisis de requerimientos y prototipado de una aplicación accesible para personas ciegas basada en la API de Google Maps
R. Alcaraz Martínez

10. REFERENCIAS [4] Dolson, J. 2012. "SEO and accessibility". En: Accessible
[1] Nielsen, J. 2000. Usabilidad, diseño de sitios web. Madrid Web design. Blog.
[etc.]: Prentice Hall. <http://www.joedolson.com/articles/2012/01/seo-
accessibility/>. [Consulta: 26/05/2014].
[2] Romero Zúnica, R. 2001. "Usabilidad como ventaja
competitiva". En: Accesibilidad a la Red. [5] Nielsen, J. 2012. "SEO and Usability". En: Nielsen Norman
<http://acceso.uv.es/accesibilidad/artics/01-usab- Group. Articles. <http://www.nngroup.com/articles/seo-and-
ventaja.htm>. [Consulta: 26/05/2014]. usability/>. [Consulta: 26/05/2014].
[3] Arch, A.; Letourneau, C. 2002. Beneficios auxiliares del [6] Carreras Montoto, O. 2013 "Accesibilidad web y SEO". En:
diseño web accesible. Paz, L.; Malumián, V. Pioneros y hacedores: fundamentos y
<http://www.w3.org/2003/11/benefits-es.html>. [Consulta: casos de diseño de interacción con estándares de
26/05/2014]. accesibilidad y usabilidad. Buenos Aires: Gogot.

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 498 |
 Factores importantes para un sistema de recomendacion de una red social educativa
V. del Castillo Carrero | I. Hernán-Losada | E. Martín-Barroso

Factores importantes para un sistema de recomendacion

de una red social educativa
Virginia del Castillo Carrero, Isidoro Hernán-Losada, Estefanía Martín-Barroso
Dept. Lenguajes y Sistemas Informáticos I
Universidad Rey Juan Carlos, URJC
Móstoles, Spain
[email protected], [email protected], [email protected]

ABSTRACT encontremos [3] y por tanto, se debe tener en cuenta distintas

informaciones para poder realizar su tarea con éxito. Este
Este artículo presenta los factores más importantes a tener en trabajo trata de encontrar los factores principales que se han de
cuenta para realizar un sistema de recomendación basado en tener en cuenta para realizar un sistemas de recomendación
contenido para una red social orientada al aprendizaje. Los basado en contenido con un objetivo educativo y marcar
resultados son obtenidos del proceso del diseño e diferencias con respecto a sistemas que se crean con objetivos
implementación de un sistema de recomendación basado en distintos a los de la enseñanza.
contenido, aplicado a una plataforma social para el aprendizaje
a través de la creación de vídeos. El sistema ha sido realizado y 2. CONTEXTO: CLIP-IT
aplicado en una experiencia real para la enseñanza de Clip-It es una red social desarrollada dentro del proyecto
conceptos básicos de interacción persona ordenador a través de europeo Juxtalearn que persigue el objetivo de fomentar el
la creación de videos, dando soporte a las fases de discusión y aprendizaje de thresholds concepts mediante la creación de
difusión de los mismos. En la actividad participaron 43 futuros vídeos [4]. Clip-It está compuesto por diversos plugins, que la
graduados pertenecientes a dos grados diferentes: Ingeniería de dotan de funcionalidad. Esta característica convierte a esta red
Computadores y Comunicación Multimedia. en un sistema altamente configurable. La plataforma permite al
Sistemas de recomendación; aprendizaje basado en docente plantear una actividad, sobre la que tienen que trabajar
vídeos;redes sociales de aprendizaje. los alumnos, produciendo un vídeo sobre la temática planteada.
Clip-It permite el trabajo colaborativo a través de grupos,
1. INTRODUCCIÓN dotando de espacios privados para compartir archivos y foros
para la discusión [5]. Además, admite la subida y reproducción
Los sistemas de recomendación (SR) han cobrado una gran
de vídeos en distintos formatos así como los comentarios y
importancia, experimentando grandes avances, sobre todo en el
valoraciones de los vídeos públicos.
campo comercial. De hecho, disponer de un buen SR puede
marcar la diferencia entre el éxito y el fracaso de una empresa. Una característica muy importante es que permite el etiquetado
Las cantidades ingentes de contenidos a los que los usuarios de todos los elementos, con lo que se puede realizar una
hacen frente diariamente y la limitación de tiempo para buscar clasificación en base a las opiniones de los usuarios.
y encontrar productos que se adapten a sus gustos, tiene como Para facilitar el aprendizaje se ha creado un SR basado en
consecuencia que los usuarios a veces no puedan encontrar lo contenido con el cuál se facilite a los usuarios encontrar
que buscan. Una de las principales motivaciones de los SR es materiales que sean de su interés. De esta forma, los alumnos
justamente ayudar a los usuarios a que encuentren aquello que pueden complementar la información sobre el tema en el que
buscan. Los SR buscan sugerir aquello que pueda resultar están trabajando. El SR ha sido implementado en forma de
relevante para los usuarios, realizando una estimación de lo que plugin, para que el administrador de Clip-It pueda situarlo y
puede gustar, interesar o ser de utilidad para ellos. La usarlo donde y cuando quiera.
definición general de SR [1] busca maximizar el valor de
utilidad de un contenido para un usuario. La plataforma Clip-It ha sido probada por alumnos de
Ingeniería de Computadores y Comunicación Multimedia, los
En general, podemos encontrar tres tipos de SR: basado en cuáles debían realizar vídeos relacionados con conceptos
contenido, filtrado colaborativo y una combinación de los dos básicos de la asignatura de Interacción Persona-Ordenador que
anteriores denominado híbrido [1]. Los sistemas de en ese momento cursaban.
recomendación también han pasado a ser una parte importante
en las redes sociales. Como es el caso de Youtube [2], El número total de alumnos que han usado Clip-It en esta
Facebook, Twitter y un largo etcétera. actividad han sido 43, algunos con conocimientos previos
informáticos (alumnos de Ingeniería de Computadores) y otros
Dentro del ámbito educativo, existe un creciente interés por las sin conocimientos previos (alumnos de Comunicación
redes sociales enfocadas al aprendizaje. Viendo el gran Multimedia). Todos ellos contribuyeron a determinar los
potencial que aportan en otros contextos los SR, estos pueden
ser usados para recomendar materiales docentes a los alumnos. factores más influyentes en el SR a través de sus opiniones
Se debe tener en cuenta que los sistemas de recomendación recogidas mediante encuestas.
tienen diferentes objetivos según el contexto en que nos

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 499 |
 Factores importantes para un sistema de recomendacion de una red social educativa
V. del Castillo Carrero | I. Hernán-Losada | E. Martín-Barroso

3. ESTUDIO DE LAS NECESIDADES Usuario con pocas recomendaciones: el sistema dispone del
Los SR en un contexto educativo tienen objetivos diferentes perfil de gustos del usuario, sin embargo el número de
que los SR comerciales, como ya hemos comentado contenidos que puede sugerir el sistema es escaso. El resultado
previamente [3]. Para realizar la captura de requisitos del SR se que muestra el SR está formado los contenidos que se asemejan
pasó un cuestionario al grupo de 43 alumnos los cuales ya a sus gustos, más una selección de contenidos actuales que
habían interactuado con la plataforma de Clip-It. varían de forma aleatoria.

Las principales ideas extraídas a partir de las opiniones de los Usuario con muchas recomendaciones: este caso es una variante
alumnos son que el SR no debería mostrar más de cinco del anterior, donde el sistema tiene una multitud de contenidos
recomendaciones por pantalla, siendo estas visibles desde la que se adecuan al interés del usuario. Para evitar sobre-
página principal del usuario y más ampliamente en un apartado especialización de contenidos, efecto típico de SR basados en
propio en la plataforma, siempre junto al material que estén contenido, el sistema añade algún contenido actual a las
visualizando en el momento. Las recomendaciones deben recomendaciones realizadas por el usuario y/o modifica
basarse en las exigencias académicas del presente inmediato, periódicamente las recomendaciones para que vayan variando.
dando menos importancia a las asignaturas futuras o incluso
aquellas suspensas en el pasado. Dar más importancia a las
5. EVALUACIÓN Y CONCLUSIONES
El SR expuesto en este trabajo ha sido evaluado mediante un
relaciones de compañeros de clase que a las de amistad (caso
estudio posterior con los mismos alumnos que colaboraron en
contrario en otros tipos de redes sociales). Y tener en cuenta las
las diferentes fases de este proyecto. Los resultados de la
asignaturas y trabajos con los que están interactuando en ese
evaluación han sido muy positivos validando las diferentes
momento. Así como los contenidos multimedia basados en
reglas establecidas a la hora de realizar las recomendaciones.
vídeo tienen una mayor importancia para los alumnos.
En general, los alumnos mostraron una buena aceptación del
4. SISTEMA DE RECOMENDACIÓN SR.
El SR creado es un sistema basado en contenido. Se ha elegido Como principal conclusión, destacar la idea de la necesidad de
esta opción porque Clip-It hace uso del etiquetado social, con lo realizar un estudio previo del entorno de trabajo del SR. De
cual proporciona una clasificación sin necesidad de un análisis esta forma el SR será una herramienta útil y necesaria que sirva
previo del contenido. Es el propio usuario quién clasifica el de apoyo a la interacción entre la plataforma y el usuario,
contenido. El uso del etiquetado social tiene sus ventajas y sus evitando que el SR se convierta en una herramienta ineficiente
inconvenientes [7]. La ventaja principal es que se obtiene una o molesta que impida que los usuarios no aprovechen todas la
gran escalabilidad. La desventaja es la tendencia del usuario a ventajas que ofrece el SR.
utilizar pocas etiquetas (o no etiquetar) para describir
contenidos. 6. REFERENCES
[1] Adomavicius, G., & Tuzhilin, A. (2005). Toward the next
Para realizar las recomendaciones utilizamos las votaciones
generation of recommender systems: a survey of the state-
realizadas por los usuarios y además tenemos en cuenta los
of-the-art and possible extensions. IEEE Transactions on
contenidos que han comentado recientemente, estos datos
Knowledge and Data Engineering , 17 (6), 734-739.
formaran un perfil lógico del usuario. Este perfil será utilizado
por el sistema para encontrar contenidos que se adecuen a los [2] Davidson J, Liebald B, Liu J, Nandy P, Van Vleet T,
intereses del usuario. La similitud entre contenidos es dada por Gargi U, Gupta S, He Y, Lambert M, Livingston B,
el número de etiquetas que tienen en común. La comparación Sampath D. The YouTube video recommendation system.
entre etiquetas tiene una previa lematización dejando Proceedings; Fourth ACM Conference on Recommender
únicamente la raíz de la palabra, antes de realizar las Systems; Sept 26-30, 2010; Barcelona, Spain. 2010.
comparaciones entre contenidos. [3] Buder, J., & Christina, S. (2012). Learning with
El sistema puede diferenciar entre tres tipos potenciales de personalizaed recommender systems: A psycological view.
usuario, son los siguientes: Computer is Human Behavior , 28 (1), 207-216.
Usuario nuevo: el sistema no tiene información suficiente [4] A. Adams, Y. Rogers, T. Coughlan, J. Van-der-Linden, G.
acerca de su intereses, es decir, no tiene un perfil de los gustos Clough, E. Martin, P. Haya, T. Collins. “Teenager needs
del usuario, ya sea porque no ha interactuado con la plataforma in technology enhanced learning”. Workshop on Methods
o sus gustos son muy particulares y el sistema no logra of Working with Teenagers in Interaction Design, CHI
captarlos. El sistema hará una recomendación de cinco 2013, Paris, France, 2013.
contenidos aleatorios de los últimos contenidos actualizados o [5] J. Urquiza, J. Castellanos, I. Hernán, E. Martín, P. A.
subidos a Clip-It. Haya. 2013. “Plataforma social para el aprendizaje basado
Si el sistema puede crear un perfil de usuario con sus gustos, en vídeos creados por los estudiantes”. Proc. of XIV
las recomendaciones se crean teniendo en cuenta estos gustos y Congreso Internacional de Interacción Persona-Ordenador
se da una prioridad mayor a los contenidos que comparten 2013, Madrid, Spain. September, 2013.
varios intereses. De estos contenidos, los vídeos tienen mayor [6] Jianming He , Wesley W. Chu. (2010). A social network-
peso que el resto de materiales, como los alumnos indicaron en based recommender system (snrs), Ann. Inf. Syst. Spec.
el estudio previo. En esta situación el sistema se encuentra con Issue Data Min. Soc. Netw. Data. 12, 47-74.
los dos casos

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 500 |
 Factores importantes para un sistema de recomendacion de una red social educativa
V. del Castillo Carrero | I. Hernán-Losada | E. Martín-Barroso
[7] Balby Marinho, L., Nanopoulos, A., Schmidt-Thieme, L., Ricci, L. Rokach, B. Shapira, & P. B. Kantor,
Jäschke, R., Hotho, A., Stumme, G., y otros. (2011). Recommender Systems Handbook (págs. 616-642).
Chapter 19. Social Tagging Recommender Systems. En F. Springer.

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 501 |
 Case Study on Mobile Applications UX: Effect of the Usage of a Cross-Platform Development Framework
E. Angulo

Case Study on Mobile Applications UX: Effect of the

Usage of a Cross-Platform Development Framework
Esteban Angulo
[email protected]

ABSTRACT and the development of mobile applications is supporting their

Cross-platform development frameworks for mobile applications daily life activities. The market offers a great variation of mobile
promise important advantages in cost cuttings and easy devices and platforms, and the challenge for development teams
maintenance, posing as a very good option for organizations is being able to create mobile applications that can be used and
interested in the design of mobile applications for several run seamlessly on different devices and platforms. In order to
platforms. Given that platform conventions are especially cover all these potential markets, specific versions of the
important for the User eXperience (UX) of mobile applications, application need to be developed for each target platform.
the usage of a framework where the same code defines the Cross-platform development frameworks offer a way of saving
behavior of the app in different platforms could have a negative resources in the aim of covering different platforms. These tools
impact in the UX and in the way the mobile application and user provide developers the flexibility to create an app than runs
interact. across several mobile platforms based on the ideal principle of
This work describes a study where two different versions of a “write-once-run-everywhere”.
mobile application were designed and developed, one using a UX and Interaction Design must follow specific conventions in
framework that generates Android and iOS versions order to be consistent with standard or typical interaction
automatically, and another team using native tools. The strategies for each platform. User eXperience (UX) is the key
alternative versions for each platform have been evaluated with differentiator aspect in mobile application acceptance.
37 users with a combination of a laboratory usability test and a
longitudinal study. The results show that differences are minimal The rest of this document is structured as follows: next section
in the Android platform, but in iOS, even if a reasonably good explores the State of the art on the topic. Section 3 is dedicated
UX can be obtained with the usage of this framework by an UX- to UX considerations for cross-platform development and the
conscious design team, a higher level of UX can be obtained usage of cross-platform development frameworks and the
directly developing with a native tool. selection of one of these frameworks. Section 4 describes the
methodological approach for the study carried out, and section 5
details the results and conclusions.
Categories and Subject Descriptors
D.2.6 [Software Engineering]: Integrated environments. 2. STATE OF THE ART
H.5.2 [Information Interfaces & Presentation]: User-centered The development of mobile application using cross-platform
design. frameworks is very popular in these days and several studies or
researches performed on this subject looking for the most
General Terms suitable cross-platform framework to develop an application as
Design, Experimentation, Human Factors. native as possible but usually from the developer’s point of view.
However, the information about a comparison of the usability of
a mobile application developed using native technologies against
Keywords
one developed with cross-platform frameworks is scarce.
Mobile user experience, cross-platform development, interaction
design, mobile applications. Most of the documentation and bibliographic research found on
the topic, evaluates cross-platform frameworks from the point of
1. INTRODUCTION view of development companies or software development
The objective of this study is comparing the cross-platform and easiness. In this case we were looking for another approach: UX.
the native approach for being able to determine if the selected
development approach has any impact on the users in terms of 3. CROSS-PLATFORM FRAMEWORK
UX. To be able to set a base line under this subject, a study on SELECTION
cross-platform frameworks was performed to select the most Our starting point for studying the possible limitations of cross-
appropriate one from a UX point of view. Among several options platform development frameworks in terms of UX of the
which were evaluated under an established criterion, only one developed apps was to select the best one from a mobile UX
cross-platform framework was selected for developing a point of view.
prototype of the mobile application which was used in the study.
At the beginning we found 19 options of cross-platform
In recent years, the growth and usage of smartphones and mobile framework. We have discarded frameworks for specialized
devices has changed the communication habits of the population

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 502 |
 Case Study on Mobile Applications UX: Effect of the Usage of a Cross-Platform Development Framework
E. Angulo
domains, like gaming. We studied the information available for the cross-platform one. To avoid adaptation times, the
each framework, considering the following criteria: participants used their own smartphones where the app was
installed.
• To offer support for Android and iOS native app
development, to be able to compete with directly- We choose to perform the study this way in order to have
developed native apps. different kinds of experience and try to analyze the impact of the
• To be based in a well-known programming language, to first impression.
ease adoption by developers. When both prototypes have been tested, a final evaluation is
• Free availability. completed on which the participants compared their experience
• To be compatible with widespread IDEs, to ease with both versions.
development.
A group of usability attributes were selected as the variables to
• To offer access to device APIs, to be able to offer the
be assessed in this work. The selected variables are:
same level of functionality than native code.
effectiveness, efficiency and satisfaction. We used different
• To have a high use rate among developers.
usability evaluation methods for each study in order to obtain the
After analyzing the characteristics and properties of all the necessary information to measure the different values.
candidates, only the three tools complied with all the
For the laboratory test, the methods chosen are: satisfaction
requirements. These three tools were evaluated with a case
questionnaires, performance measurement and interviews. For
study, consisting on an app to show the bus timetable for a
the longitudinal study, the methods selected are: remote data
university campus.
collection and satisfaction questionnaires.
Based on other evaluation process and the easiness for the
For the purposes of this study, three different questionnaires
development, the best option for this case study was Titanium
were applied to gather the participants’ impressions about each
Appcelerator.
version of the application. The questionnaires used were: System
Usability Scale (SUS); a customized and reduced version of the
4. EVALUATION METHODOLOGY User Experience Questionnaire (UEQ) to focus on the only
The study designed for evaluating the impact of using a aspects that were applicable to the app context of use; and an ad-
framework into the UX was based in having two design teams hoc questionnaires where participants were asked to compare the
working independently: one team designing the Titanium version version of the app tested with the look and behavior of a typical
(which generates Android an iOS versions automatically), and app in their platform.
the other one working on the two native versions.
Another source of gathering information during the 5-day usage
Both sets of versions (cross-platform and native) were used on period of the longitudinal study was Google Analytics. This
the usability study of this work. information helped to understand and control de usage of the
application during this exploration period and get insights about
4.1 Study design the troubles and difficulties that users had during their
The participants of this study were 37 students of the School of interaction with the applications.
Computer Science at UPM. These students have evaluated the
alternative versions of the app for each platform. 5. CONCLUSIONS
The study consisted in the combination of 2 different usability A case study has been presented for evaluating how the decision
evaluations for each version of the prototype: to develop a mobile app using use a cross-platform framework
impacts the UX of the resulting app, against developing with
• Laboratory study: It was a first-impression study in
native code for each platform considered.
which the participants had to complete a set of tasks
while the evaluators were watching their reactions and The advantages of this kind of frameworks from a development
interaction with the application; once the participant point of view are well known, but this time it has been evaluated
was ready with the tasks, he/she had to complete a in terms of the UX of the produced app with a longitudinal study,
usability evaluation survey which addressed different with an app with enough functionality to be considered a full-
topics and was focused on evaluating the prototype scale app, and with a number of test participants to provide a
from a UX point of view. minimum coverage of the two main mobile platforms: Android
• Longitudinal study: It was performed once the and iOS.
participants were ready with the laboratory study, they The results show that a good level of UX can be obtained if the
had five days to use the application in their own cross-platform development framework is chosen carefully in
context and explore it as much as they want. When the terms of providing adapted interaction styles for each platform,
five days were over, the participant had to complete and the development team has UX expertise. The differences are
again the set of tasks (now having more experience minimal in the Android version, but in iOS, even if a reasonably
with the app) and complete the usability evaluation good UX can be obtained with the usage of this framework by an
survey. UX-conscious design team, a higher level of UX can be obtained
The same process was followed with both prototypes (native and directly developing in native code.
cross-platform) in a random way. This means, that some users Further research is needed to tackle the specific interaction
begin the experiment with the native application and others with design issues that will be difficult to cover in the framework-

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 503 |
 Case Study on Mobile Applications UX: Effect of the Usage of a Cross-Platform Development Framework
E. Angulo
generated apps and offer the software engineers and development of adaption the native or the cross-platform development
teams another way to evaluate the advantages and disadvantages approach on terms of UX.

XVI Congreso Internacional de Interacción Persona-Ordenador | 1º Concurso 2015 AIPO de TFG/TFM

| 504 |
INDEX OF
AUTHORS
 Index of Authors

del Castillo Carrero, Virginia, 499

A Deussen, Oliver, 152
Diaz-Estrella, Antonio, 229
Abascal, Julio, 198
Duque Medina, Rafael, 374
Acosta Escalante, Francisco, 40, 276
Durango, Iván, 351
Ahuja, Dishaan, 255
Aknine, Samir, 271
Albertos Marco, Félix, 189 E
Alcaraz Martínez, Rubén, 496
Escudero, David, 445
Allueva, Ana, 290
España, Sergio, 77
Al-Thani, Dena, 234, 545
Espino Espino, Elisenda Eva, 307
Álvarez Rodríguez, Francisco J, 40, 112, 121, 276
Amin, Navya, 217
Angulo, Esteban, 502 F
Arapakis, Ioannis, 392
Arciniegas, José Luis, 362 Falgueras, J., 202
Arellano, Diana, 152 Fardoun, Habib M., 362
Arenas, Rubén, 63 Feliu, Joel, 312
Arroyo Prieto, Lidia, 294 Feng, Feng, 234
Arrue, Myriam, 198 Fidalgo Blanco, Ángel, 438
Flores, Julián, 63
B Flores, Valle, 445
Fuentes García, Noemí Marta, 171, 337
Baena, Antonio Miguel, 25
Baldassarri, Sandra, 31, 341
Bernal, Guillermo, 255 G
Bessa, Maximino, 70, 264
Bonillo Fernández, Clara, 31, 493 Gallud, José A., 189, 351
Borrás-Gené, Oriol, 438 García Arenas, Maribel, 319
Borschbach, Markus, 217 García, Roberto, 487
Botella, Federico, 401 García-Peñalvo, Francisco J., 438
Bryan-Kinns, Nick, 234 Garcia-Sanjuan, Fernando, 133
Buendía, Félix, 103 Garrido, Manel, 26
Bustos Amador, Viviana, 157 Gavin, Ferran, 392
Gil Iranzo, Rosa, 319
Gil-Juárez, Adriana, 312
C Gómez, Javier, 281
González González, Carina, 137, 141, 166, 319
Cabral, Luciana, 264
González Sánchez, J. L., 328
Camilo Cerón, Juan, 362
González, Cesar, 445
Cano, Sandra, 157
González, Pascual, 94
Capa-Arnao, Rodrigo, 181
González-Calleros, Juan Manuel, 112
Cardona Reyes, Héctor, 276
Götzelmann, Timo, 243
Carrapatoso, Eurico, 70
Granollers, Antoni, 368, 387, 487
Carrascosa, Alicia, 351
Gross, Thomas, 217
Carrillo, A. L., 202
Guerrero-García, Josefina, 121
Carvalho, Diana, 70
Gutiérrez Vela, Francisco Luis, 171, 328, 337
Casalegno, Federico, 255
Gutiérrez, Yurena, 445
Castillo, Vanessa, 449
Catala, Alejandro, 131, 133
Català, Andreu, 248 H
Cavallaro, Andrea, 248
Cerezo, Eva, 31, 290, 341 Helzle, Volker, 152
Céspedes-Hernández, David, 112 Hernán-Losada, Isidoro, 499
Collazos, César A., 157, 362 , 368, 387 Hurtado, Julio, 368
Conesa, Ester, 312 Hurtienne, Jörn, 271
Contreras, D., 383
Corradini, Andrea, 345 I
Corrales, Mario, 445
Cruz-Benito, Juan, 438 Iniesto, Francisco, 463

D J
de la Guía, Elena, 86 Jaen, Javier, 49, 131 , 133, 135
de la Rubia, Ernesto, 229

XVI Congreso Internacional de Interacción Persona-Ordenador

| 504 |
 Index of Authors

K Penichet, Víctor M. R., 86, 189, 351

Peñalver, Antonio, 401
Khemaja, Maha, 103 Pérez, J. Eduardo, 198
Knoche, Hendrik, 54 Pérez-López, Carlos, 248
Pérez-Medina, Jorge Luis, 112, 121
Plaza, José Antonio, 28
L Pons, Patricia, 131
López Arcos, Rafael, 171 Puerta Cruz, Yuliana, 387
López, Vicente, 86 Puig, A., 383
López-Arcos, José Rafael, 171
López-Jaquero, Víctor, 94 , 429 R
López-Varquiel, Francisco, 225
Lozano, María D., 86 Rauh, Reinhold, 152
Lozano, María Teresa, 290 Realpe, Paulo C., 368
Revelles-Benavente, Beatriz, 294
M Revert, Ricardo, 478
Ribas-Xirgo, Lluís, 225
Macías, José A., 29, 420 Ribera, Mireia, 469
Magalhães, Luís, 70, 264 Rocha, Tânia, 264
Manresa-Yee, Cristina, 54, 181 Roda, Cristina, 429
Marco, Javier, 31 Rodrigo, Covadonga, 463
Marcos, Mari-Carmen, 392 Rodríguez, Arturo C., 94
Markopoulos, Panos, 24 Rodríguez, I., 383
Martín-Barroso, Estefanía, 499 Rodríguez-Martín, Daniel, 248
Martínez, Paloma, 478 Rodríguez-Vizzuett, Liliana, 121
Mas-Sansó, Ramon, 181 Roig, Jordi, 469
Mehta, Manish, 345 Rojas, Luis A., 420
Méndez, Roi, 63 Rommes, Els, 27
Mendoza G., Alfredo, 40 Rosenthal, Susanne, 217
Mendoza G., Ricardo, 40, 166 Ruiz, Jenny, 207
Menéndez, Nyder, 449
Montero, Francisco, 94, 429 S
Montoro, Germán, 281
Morales Hernández, Dulce, 166 Salamó, M., 383
Moreno, Giankaris, 449 Samà, Albert, 248
Moreno, Juan P., 401 Schaller, Ulrich Max, 152
Moreno, Lourdes, 478 Sedrakyan, Gayane, 207
Morrison, Ann, 54 Segura-Bedmar, Isabel, 478
Muñoz Zavala, Ángel Eduardo, 276 Serna, Audrey, 271
Muñoz Arteaga, Jaime, 40, 112, 121, 157, 166 Snoeck, Monique, 207
Solano, Andrés, 362
N Soriano Marín, A. J., 328
Spicker, Marc, 152
Nacher, Vicente, 49, 135 Stockman, Tony, 234, 454
Navarro, Elena, 94
Navarro-Adelantado, Vicente, 141 T
Ngu Nguyen, Le Nguyen, 248
Nieto-Reyes, Alicia, 374 Talavera, Marina, 29
Teruel, Miguel A., 94
O Therón, Roberto, 438
Tombros, Anastasios, 454
Oliva, Marta, 487 Torrado, Juan C., 281
Orozco, Luis, 86 Torres-Carrión, Pablo, 137
Ortigosa, Eva M., 319 Trillo-Lado, Raque, 290
Tscharn, Robert, 271
Tung Ly, Nam, 271
P
Pabón, María Constanza, 410 V
Paderewski Rodríguez, Patricia, 171, 319, 337
Padilla Zea, Natalia, 171, 319, 337 Valencia, Xabier, 198
Palacios, Alfons, 487 Vall-llovera, Montse, 312
Parra González, Otto, 77 Vázquez, Pere-Pau, 243
Pastor, Oscar, 77 Vergés Bosch, Núria, 294, 298
XVI Congreso Internacional de Interacción Persona-Ordenador
| 505 |
 Index of Authors

Villarroya-Gaudó, María, 290

W
Williams, Kaiser, 449

Y
Yañez, A., 383

XVI Congreso Internacional de Interacción Persona-Ordenador

| 506 |