G u e s t E d i t o r s ’ I n t r o d u c t i o n

Ambient Intelligence—
the Next Step for
Artificial Intelligence
Carlos Ramos, Polytechnic of Porto • Juan Carlos Augusto, University of Ulster
Daniel Shapiro, Institute for the Study of Learning and Expertise

T
he European Commission’s Information Society Technologies Advisory Group

(Istag) introduced the concept of ambient intelligence.1–3 Basically, AmI refers

to a digital environment that proactively, but sensibly, supports people in their daily

lives.4 IEEE Intelligent Systems was one of the first publications to emphasize AmI’s

importance, with Nigel Shadbolt’s editorial in the So, artificial intelligence is important for AmI,
July/August 2003 issue.5 Other concepts such as but why is AmI important for AI? We claim that
ubiquitous computing, pervasive computing, con­ AmI is a new challenge for AI and is the next step
text awareness, and embedded systems overlap with in AI’s evolution.
AmI, but there are distinctive differences.6
The Istag reports define AmI at a concep­ AI’s evolution
tual level and identify important technologies for Figure 1 illustrates AI’s evolution. In the begin­
achieving it. In Ambient Intelligence: From Vision ning, researchers applied AI to hardware, such as
to Reality, Istag refers to these AmI components: Marvin Minsky and Dean Edmonds’ Snarc (Sto­
smart materials, microelectromechanical systems chastic Neural Analog Reinforcement Computer).
and sensor technologies, embedded systems, ubiq­ Neural nets were one of the technologies imple­
uitous communications, I/O device technology, and mented on such systems. The Mycin expert sys­
adaptive software.3 The report also mentions these tem is a good example from AI’s second phase,
intelligence components: media management and where AI centered on computers. The third phase
handling, natural interaction, computational intel­ focused on networks; a landmark application
ligence, context awareness, and emotional com­ here was American Express’s Authorizer’s As­
puting. In our opinion, achieving AmI will require sistant. During the ’90s, the Web boom produced
borrowing much more from artificial intelligence. several search engines and recommender systems

March/April 2008 1541-1672/08/$25.00 © 2008 IEEE 15

Published by the IEEE Computer Society
G u e s t E d i t o r s ’ I n t r o d u c t i o n

sensors, RFID, and GPS) can also benefit

Knowledge-based Ambient from AI techniques.
Neural nets systems Agents Ontologies intelligence

Hardware Computer Network Web Environment

Representing the information
and knowledge associated
Walter’s Turtle MYCIN Authorizer’s Recommender systems with the environment
SNARC STRIPS assistant Search engines AmI environments involve real-world
problems, which are characterized by in­
Figure 1. The evolution of artificial intelligence. completeness and uncertainty. Generally,
we deal with information; some part of it
using intelligent agents and, more re­ Interpreting might be correct, some part might be incor­
cently, ontologies. the environment’s state rect, and some part might be missing. The
So, what comes next? Current trends point The technologies for this task involve an­ question is how to proceed with an elabo­
to incorporating intelligence into our envi­ alyzing various sensing inputs. rated reasoning process dealing with these
ronments. AmI is the way to achieve this. Humans most often interact through writ­ information problems. To handle this situ­
ten or spoken language. So, it’s clear that ation, researchers have used many tech­
Combining AmI and AI they will also expect this kind of interaction niques, such as Bayesian networks, fuzzy
Today, some systems treat AmI like a buzz­ with AmI environments. Speech recognition logic, and rough sets.
word, incorporating only a limited amount and natural language processing are differ­ Knowledge representation is one of the
of intelligence. Some researchers are build­ ent and complementary problems, using dif­ most important areas in AI. Expert sys­
ing AmI systems without AI, concentrating ferent techniques. tems have achieved tremendous success in
on the operational technologies, such as sen­ Speech recognition obtains an electric areas such as medicine, industry, and busi­
sors, actuators, communications, and ubiq­ signal from a microphone. The first step ness. During the ’90s, with the strong de­
uitous computing. However, sooner or later, is identifying phonemes in this signal, velopment of the Internet and the birth of
that low level of intelligence will be a clear which involves signal processing and pat­ the Web, humans faced a critical problem.
drawback. AmI’s acceptability will result tern recognition. The next step is joining The amount of information became huge,
from a balanced combination of operational phonemes and identifying words. Several and the mapping between information and
technologies and AI. speech recognition systems are available knowledge became urgent. The AI commu­
Figure 2 shows our vision of AmI, high­ and are more or less successful, depending nity started paying attention to information
lighting AI’s importance. AmI environments on how the user speaks. retrieval, text mining, ontologies, and the
might be very diverse—for example, your Natural language input is a written se­ Semantic Web. Early experience in intelli­
home, car, or office, or a museum you’re quence, resulting from a speech recogni­ gent systems development shows us that in­
visiting. AmI systems are inserted in these tion system or obtained from a keyboard or telligence isn’t possible without knowledge;
environments, receiving information, inter­ even a written document. Natural language this is also true for AmI.
acting with users, performing elaborate rea­ processing aims to understand this input.
soning, and ordering actions on the environ­ The first step is syntax analysis, followed Modeling, simulating,
ment. Sensing captures information, through by semantic analysis. Knowledge represen­ and representing entities
humans using their senses or through auto­ tation plays an important role in NLP. Au­ in the environment
matic systems such as ultrasonic devices, tomatic-translation systems are one of the People expect agents to support features
cameras, and microphones. Action on these most studied areas of NLP, using statistical such as sensing capabilities, autonomy, reac­
environments occurs through human deci­ and knowledge-based approaches. tive and proactive reasoning, social abilities,
sions and actions and through automatic sys­ Vision is humans’ richest sensorial input. and learning. Multiagent systems emphasize
tems such as robots and agents. In addition, So, the ability to automate vision is impor­ social abilities, such as communication, co­
persons or agents not directly interacting tant. Basically, computer vision is a geo­ operation, conflict resolution, negotiation, ar­
with the system might change the environ­ metric-reasoning problem. Computer vision gumentation, and emotion. Multiagent sys­
ment, and unexpected events might occur. comprises many areas, such as image acqui­ tems rapidly became the main paradigm in
To deal with all this, AmI systems employ sition, image processing, object recognition AI. After the Web boom, agents received
the operational technologies we mentioned (2D and 3D), scene analysis, and image- even more attention.
previously (the operational layer). And, if in­ flow analysis. Computer vision can be used Multiagent systems are especially good
telligence is to be more than just a buzzword, in different situations in AmI. For example, at modeling real-world and social systems,
these systems will incorporate AI methods intelligent transportation systems can use it where problems can be solved in a concur­
and techniques (the intelligent layer). to identify traffic problems, traffic patterns, rent and cooperative way without needing
or approaching vehicles. Computer vision optimal solutions (for example, in traffic or
AI methods can also identify either human gestures to manufacturing).
and techniques in AmI control equipment or human facial expres­ In AmI environments, agents are a good
In AmI environments and scenarios, AI sions to identify emotional states. way to model, simulate, and represent
methods and techniques can help accom­ The processing of data acquired by many meaningful entities such as rooms, cars, or
plish the following important tasks. other sensorial sources (for example, raw even persons.

16 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Planning decisions or actions
Planning assists problem solving by pro­ Operational layer
ducing a plan of action to achieve a par­ Ambient-
intelligence Hardware, operating systems, communications, databases,
ticular goal. AI planning deals with all the computer graphics, ubiquitous computing, ...
system
aspects of general planning. Plans can be
established before they execute (offline) or Intelligent layer
while they execute (online). They can be
Knowledge representation, logic, ontologies, speech recognition,
deliberative (planning and executing what natural language processsing, information retrieval, text mining, expert systems,
was planned without considering unex­ machine learning, planning, computational intelligence, incompleteness and
pected events), reactive (reacting to stim­ uncertainty, computer vision, optimization, decision support systems,
ulus in a much more basic way), or hybrid multiagent systems, intelligent tutoring, emotion, argumentation, ...
(combining the best of deliberative and re­
active policies). Intelligent interaction
Planning is particularly linked with intel­ Automatic sensing
ligence. Convincing someone that a system is Raw sensors
Automatic action
intelligent is difficult if that system can’t plan RFID
Automation
how to solve problems. Consequently, AmI Sensor nets
Intelligent robots
environments must support planning to give GPS
Human sensing Intelligent agents
Audio and speech
intelligent advice to users. A clear example and action ...
Image
is in intelligent transportation systems—both ... Ambient-
inside vehicles, where intelligent driving sys­ intelligence
tems will help drivers, and on the road, where environment
Home, factory, office,
route planning will consider constraints re­ meeting room, classroom, External events
lated to traffic, time, and cost. vehicle, hospital, Unexpected events
Most often, planning is associated with cultural-heritage site, Others’ actions
tourist attraction, ...
some kind of optimization. Here, combining
AI and operations research makes sense.
Some computational-intelligence and bio-in­
spired methods such as genetic algorithms, Figure 2. The ambient-intelligence vision from an artificial intelligence perspective.
ant colonies, particle swarm intelligence,
taboo search, and simulated annealing are understand user commands, but they’re not tant for incorporating such capabilities into
useful. intelligent enough to avoid doing things that AmI systems.
the user doesn’t want. Basic machine learn­
Learning about the environment ing methods will enable AmI systems to Acting on the environment
and associated aspects learn by observing users, thus making these As we mentioned before, automated de­
Machine learning has received attention systems more acceptable to them. vices such as robots could perform ac­
from the AI community from the begin­ tions. Cognitive-robotics research can pro­
ning. Since the ’70s, neural networks have Interacting with humans vide benefits for AmI environments such as
had great success, being applied in many AmI systems should be able to interact smart homes. This is especially true when
real-world problems such as classification. intelligently with humans. Such interaction persons live alone, are elderly, or have health
Techniques that use more high-level de­ requires context awareness. In AmI sys­ problems. The creation of intelligent robots
scriptions—for example, inductive learning, tems, context awareness will involve such that can perform several tasks or just act as
case-based reasoning, and decision-tree- factors as mixed-initiative interfaces, adapt­ companions is important. However, in the
based methods—have also seen success. ing to users and situations, learning by ob­ current state of the art, we can create robots
During the ’80s, the term “data mining” serving users, consciousness of the current that operate well only for specific tasks. Cre­
started appearing. Many database research­ situation, and scalable intelligence. We’ve ating robots with the flexibility to do differ­
ers have used this term to refer to machine already discussed interaction through natu­ ent tasks, as humans can do, is too complex.
learning techniques (together with some sta­ ral language and gestures. This limitation is due primarily to physical
tistics methods such as k-means) employed Because AmI systems deal with humans, constraints.
in knowledge discovery. Data mining con­ they will need to consider all pertinent so­
stitutes one phase of knowledge discovery cial and emotional factors. For example, a
(selection, cleaning, and preprocessing are person might not be interested in watching
phases before data mining, while interpreta­
tion and evaluation come after data mining).
Nowadays, machine learning is widely
his or her favorite TV program, a soccer
game, because friends who don’t like soc­
cer are visiting (a social aspect) or because
A mI can’t be achieved without AI. So,
AmI environments provide the next
stimulating challenge for the AI commu­
used, so AmI will likely also need to handle he or she is in a bad mood (an emotional nity. Here we’ve mentioned many AI meth­
this technology. One requirement for AmI is aspect). Current AI research on affective ods and techniques useful for AmI. Our
to learn by observing users. Several systems computing and social computing is impor­ discussion complements previous attempts

March/April 2008 www.computer.org/intelligent 17

G u e s t E d i t o r s ’ I n t r o d u c t i o n

T h e A u t h o r s
Carlos Ramos is the director of Gecad (the Knowledge Engineering
and Decision Support Research Centre) and coordinator professor at the
Polytechnic of Porto’s Institute of Engineering. His main areas of inter­
est are ambient intelligence, knowledge-based systems, decision sup­
port systems, multiagent systems, and planning. He received his PhD in
electrical and computer engineering from the University of Porto. He’s a
member of the IEEE. Contact him at ISEP, Rua Dr. António Bernardino
de Almeida, 431, 4200-072 Porto, Portugal; [email protected].

Juan Carlos Augusto is a lecturer at the University of Ulster’s School

of Computing and Mathematics. His research interests are ambient in­
telligence and smart environments. He’s the editor in chief of the book
series Ambient Intelligence and Smart Environments (IOS Press). He re­
ceived his PhD in computer science from the Universidad Nacional del
Sur. He’s a member of the AAAI and ACM. Contact him at the School of
Computing and Mathematics, Univ. of Ulster and CSRI, Newtownabbey,
BT37 0QB, UK; [email protected].

Special Issue Daniel Shapiro is the executive director of the Institute for the Study of

on Activity-
Learning and Expertise (ISLE) and the president of Applied Reactivity
(ARi). He works on cognitive-agent architectures at ISLE and on appli­
cations of reactive-control and discrete-logic-control systems that learn

Based at ARi. He’s an affiliate of the Computational Learning Laboratory and

formerly a senior researcher at the Center for the Study of Language and
Information, both at Stanford University. He received his PhD in man­

Computing agement science and engineering from Stanford University. Contact him
at ISLE, 2164 Staunton Ct., Palo Alto, CA 94306; [email protected].

IEEE Pervasive to highlight AI’s importance to AmI.7–10 In­telligent Systems, vol. 18, no. 4, 2003, pp.
2–3.
Computing’s April–June The other articles in this special issue
6. C. Ramos, “Ambient Intelligence Environ­
present some of the AI community’s cur­ ments,” to be published in the Encyclope­
issue will feature peer- rent research in AmI prototypes and sys­ dia of Artificial Intelligence, J. Rabuñal, J.
reviewed articles on tems. This research involves AI methods Dorado, and A. Sierra, eds., Information
Science Reference, 2008.
and techniques such as multiagent systems,
case-based planning, fuzzy systems, logic 7. J.C. Augusto and C.D. Nugent, eds., De­
signing Smart Homes: The Role of Artificial
• activity recognition programming, hidden Markov models, and Intelligence, LNAI 4008, Springer, 2006.
ontologies. The target environments include 8. J.C. Augusto and D. Shapiro, eds., Proc.
in real-world hospitals, geriatric residences, homes, work­ 1st Workshop Artificial Intelligence Tech­
places, cultural-heritage sites, and tourist niques for Ambient Intelligence (AITAmI
applications 06), 2006.
attractions.
• rapid prototyping 9. J.C. Augusto and D. Shapiro, eds., Proc.
2nd Workshop Artificial Intelligence Tech­
of context and activity- niques for Ambient Intelligence (AITAmI
07), 2007.
recognition applications 10. C. Ramos, “Ambient Intelligence—a State
References of the Art from Artificial Intelligence Per­-
• the evolution and 1. IST Advisory Group, Scenarios for Ambi­ spective,” Proc. 13th Portuguese Conf.
ent Intelligence in 2010, European Com­ Ar­tificial Intelligence Workshops, LNAI
design of a mobile mission, 2001. 4874, Springer, 2007, pp. 285–295.
sensing platform 2. IST Advisory Group, Strategic Orienta­
tions & Priorities for IST in FP6, Euro­
for activity recognition pean Commission, 2002.
3. IST Advisory Group, Ambient Intelligence:
From Vision to Reality, European Com­
mission, 2003.
4. J.C. Augusto and P. McCullagh, “Ambient
www.computer.org/ Intelligence: Concepts and Applications,”
Int’l J. Computer Science and Information
pervasive Systems, vol. 4, no. 1, 2007, pp. 1–28.
For more information on this or any other com­
puting topic, please visit our Digital Library at
5. N. Shadbolt, “Ambient Intelligence,” IEEE www.computer.org/csdl.

18 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS