Computer Graphics

Assignment 3

Techniques in Augmented reality

Group Members:
Azhar Ali Siddiqui [43745]
Hanzala Abdul Basit [43808]
Abstract:
Augmented Reality (AR) research has been conducted for several decades, although until
recently most AR applications had simple interaction methods using traditional input devices.
AR tracking, display technology and software has progressed to the point where commercial
applications can be developed. However there are opportunities to provide new advanced
interaction techniques for AR applications. In this paper we describe several interaction
methods that can be used to provide a better user experience, including tangible
user interaction, multimodal input and mobile interaction

Introduction:
Although Augmented Reality (AR) has been in studied for over forty years it has only been
recently that researchers have begun to formally evaluate AR applications. Most of the
published AR research has been on enabling technologies (tracking or displays, etc.) or on
experimental prototype applications, but there has been little user evaluation of AR interfaces
[43]. In 2004 Swann et al. produced a literature survey reviewing a total of 1104 articles from
the leading journals and conferences. Of the 266 AR-related publications identified, only 38
(14%) addressed some aspect of human computer interaction, and only 21 (8%) described a
formal user evaluation. They conclude that user-based studies have been under utilized in AR
research. One reason for the lack of user evaluations in AR could be a lack of education on how
to evaluate AR experiences, how to properly design experiments, choose the appropriate
methods, apply empirical methods, and analyses the results. There also seems to be a lack of
understanding of the need of doing studies or sometimes the incorrect motivation for doing
them. If user evaluations are conducted out of incorrect motivation or if empirical methods are
not properly applied, the reported results and findings are of limited value or can even be
misleading. It is worthwhile to collect knowledge on user evaluations gathered in other disciples
and to bring it into AR settings. For example studying people’s behavior with various methods is
very common in general Human Computer Interaction (HCI) or Psychology. Various tools have
been developed and tested that can be applied in AR research. Although the specifics of AR
interfaces are different to more traditional interfaces, the basic tools to evaluate user
behaviour or perception are quite similar. This report aims to provide a resource that can be
used by the AR research community to design user evaluations. We first describe the role of
user evaluation studies, and then review previous usability surveys of AR interfaces.

Background:
When a new interface technology is developed it often passes through the following
stages:
1. Prototype Demonstration
2. Adoption of Interaction techniques from other interface metaphors
3. Development of new interface metaphors appropriate to the medium
4. Development of formal theoretical models for user interactions

For example, the earliest immersive Virtual Reality (VR) systems were just used to
view virtual scenes

1) Tangible Augmented Reality Interfaces:

By considering the intimate connection between the physical world and overlaid AR
content, we believe that a promising new AR interface metaphor can arise from combining
the enhanced display possibilities of Augmented Reality with the intuitive physical
manipulation of Tangible User Interfaces. We call this combination Tangible Augmented
Reality [32].
Tangible AR interfaces are extremely intuitive to use because physical object
manipulations are mapped one-to-one to virtual object operations. There are a number
of good tangible design principles can be used to create effective AR applications. Some of
these principles include:
 The use of physical controllers for manipulating virtual content.
 Support for spatial 3D interaction techniques (such as using object proximity).
 Support for multi-handed interaction.
 Matching the physical constraints of the object to the task requirements.
 The ability to support parallel activity with multiple objects.
 Collaboration between multiple participants.

2) Multimodal Interfaces:
Current AR interfaces use a single input modality to interact with the virtual content.
However Tangible AR interfaces have some limitations, such as only allowing the user to
interact with the virtual content that they can see. To overcome these limitations we have
been exploring speech

and gesture interaction in AR environments.

Our example multimodal system is a modified version of the VOMAR application for
supporting tangible manipulation of virtual furniture in an AR setting using a handheld
paddle. VOMAR is a Tangible AR interface that allows people to rapidly put together
interior designs by arranging virtual furniture in empty rooms. Originally objects were
manipulated using paddle gesture input alone and the AR Application is based on the AR
Toolkit library and the VOMAR paddle gesture library.
To create a multimodal interface we added the Ariadne spoken dialog system to allow
people to issue spoken commands to the system using the Microsoft Speech API as the
speech recognition engine. Ariadne and the AR Application communicate with each other
using the middleware ICE.
 A Microsoft Access database is used to store the object descriptions. This database is used
by Ariadne to facilitate rapid prototyping of speech grammar.

3) Mobile AR Interfaces:
There is a need for new interaction techniques for mobile AR experiences. There are a
number of important differences between using a mobile phone AR interface and a
traditional desktop interface, including:
 limited input options (no mouse/keyboard)
 limited screen resolution
 little graphics support
 reduced processing power

Computational Methodology:

In our research we have been seeking to move beyond this and explore new interaction
methods. Unlike most other desktop interface and virtual reality systems, in an AR experience
there is an intimate relationship between 3D virtual models and physical objects these
models are associated with. This suggests that one promising research direction may arise
from taking advantage of the immediacy and familiarity of everyday physical objects for
effective manipulation of virtual objects.

Conclusion:

In order for Augmented Reality technology to become more mainstream there is a need for
new interaction techniques to be developed that allow people to interact with AR content in a
much more intuitive way. In this paper we review several advanced interaction techniques
based on the tangible AR metaphor which combines tangible user interface input techniques
with AR output.
In the future more evaluation studies need to be performed to validate these techniques. User
centered design approaches could also be applied to transfer these research ideas into
commercial applications that meet the needs of a variety of application domains. Finally,
formal theoretical models could be developed to predict user performance with a variety of
tangible AR methods.

References:

T. Akinbiyi, C. E. Reiley, S. Saha, D. Burschka, C. J. Hasser, D. D. Yuh, and A. M. Okamura,

"Dynamic Augmented Reality for Sensory Substitution in Robot-Assisted Surgical Systems,
presented at Engineering in Medicine and Biology Society, 2006. EMBS '06. 28th Annual
International Conference of the IEEE, 2006.
1. D. Aliakseyeu, S. Subramanian, J.-B. Martens, and M. Rauterberg, "Interaction
techniques for navigation through and manipulation of 2D and 3D data," in Proceedings
of the workshop on Virtual environments 2002. Barcelona, Spain: Eurographics
Association, 2002.
2. M. Anastassova, C. Megard, and J.-M. Burkhardt, "Prototype Evaluation and UserNeeds
Analysis in the Early Design of Emerging Technologies," presented at HumanComputer
Interaction. Interaction Design and Usability, 12th International Conference, HCI
International 2007., Beijing, China, 2007.
3. K. Asai, H. Kobayashi, and T. Kondo, "Augmented instructions - a fusion of augmented
reality and printed learning materials," presented at Advanced Learning Technologies,
2005. ICALT 2005. Fifth IEEE International Conference on, 2005.
4. R. Azuma and C. Furmanski, "Evaluating Label Placement for Augmented Reality View
Management," in Proceedings of the 2nd IEEE/ACM International Symposium on Mixed
and Augmented Reality: IEEE Computer Society, 2003.