Pervasive Computing

Pervasive Computing
1. What is Pervasive Computing
2. User Interactions
Context-Aware Decision Engine
3. Distributed Services
Jini,UPnP,Osgi,Bluetooth SDP
4. Infrastructures
5. MIT Oxygen Project
6. HP Cooltown Project
7. References
Ubiquitous Computing (1)
Ubiquitous, Invisible Computing (Mark Weiser [1]) :
Computers disappear into the background.

Computers embedded in walls, in tabletops, and in every day
objects
A person might interact with hundreds of computers at a
time, each invisibly embedded in the environment and
wirelessly communicating with each other.
Ubiquitous Computing (2)
PARCTAB System
- Doors open only to the right badge wearer.
- Rooms greet people by name.
- Receptionists actually know where people are.
- Telephone calls can be automatically forwarded.
- Computer terminal retrieve the preference of whoever is
sitting at them.
Technologies for Ubiquitous Computing :
Display, network, and ubiquitous software system
The Personal Motor
Popular at turn of 20th Century
One motor and lots of peripherals: washing
machine, woodworking tools, sewing
machine, grain milling
System of belts and pulleys to distribute
motive power
Similar to the PC situation today
Intel, Microsoft like it this way
The Invisible Computer
Embed the capability where it is needed
rather than centralize it
You know youve succeeded when the
technology becomes a fashion item
Wristwatches: definitely
Mobile phone: almost
Laptop PC: nope
Pervasive Computing (1)
A computing environment that seamlessly and ubiquitously supports users
in accomplishing their tasks and that renders the actual computing devices
and technology largely invisible [5].

Environments created when computing power and network connectivity are
embedded in virtually every device humans use. Anytime/anywhere, any
device, any network, any data [4].

Computation is freely available everywhere, like batteries and power
sockets. Anonymous devices, either handheld or embedded in the
environment, will bring computation to us, no matter where we are or in
what circumstances. These devices will personalize themselves in our
presence by finding whatever information and software we need [18].
Pervasive Computing (2)
Device as portal / Application as task / Physical surroundings
as computing Environment [2]
- A device is a portal into an application/data space.
- An application is a means by which a user perform a task,
not a piece of software to exploit a devices capability.
- The computing environment is the users information-
enhanced physical surroundings, not a virtual space to
store and run software.

Pervasive Computing (3)
It is all about access to your information, anytime, anywhere,
from any device.
Four major aspects [6]
- Computing is spread throughout the environment.
- Users are mobile.
- Information appliances are becoming increasingly
available.
- Communication is made easier -- between individuals,
between individuals and things, and between things.

Challenges [1-6]
User Interactions
- Different interface modalities and form factors
- Multiple user interfaces that rely upon user intent
Distributed Services
- Adaptable to users, to the environment, to change, to failure
- Dynamic service discovery, adaptation and composition
- Horizontally-layered network-based services
Infrastructures
- Dynamically configured proximity network of sensors, actuators, and
appliances.
- Limited network connectivity, intermittent & unpredictable
User Interactions (1)
Heterogeneous devices: different user interfaces, ranging from window-
and keyboard-based to pen-based to voice-based interaction.
Data from a variety of location and motion sensors, identification tags, and
on-line services.
Maintaining task-oriented consistency across physical devices while
managing the multiple interfaces in a coherent manner.
- e.g. newspaper service to home display and PDA

User Interactions (2)
New and novel interaction methods such as vocal, handwritten, video,
gestural, or olfactory interface
The application front-end should be device-neutral: no assumption
about the screen size or device capabilities, even no screen.
Allows mobile clients to discover a service interface suited to the
clients size, shape, abilities, or resource limitation.
- Berkeley IDL(Interface Definition Language) or a XML schema
- Motorola VoxML: integration of speech interfaces with web content
User Interactions (3)
From explicit commands and towards interfaces that implicitly take their
direction from peoples behavior/intent.
Make the UI fit so well into the environment that it becomes invisible
(Invisible interface and intuitive user interface)
Context aware computing attempts to coalesce knowledge of the users
task, emotions, location, and attention with other available data such as
the time and knowledge about other users.
- Georgia Techs CyberDesk
- Spatial location work at AT&T Laboratories Cambridge
- Xerox PARC
User Interactions (4)
Heterogeneous devices vs. Universal personal appliances

Universal personal appliances [18]
In response to user requests, they can reconfigure themselves through
software into many useful appliances such as a two-way radios, cell
phones, geographical positioning systems, and PDAs.


Context-Aware Decision Engine
University of Hong Kong, Lum and Lau [19]
Problem: how to render content for a variety
of mobile devices?
Bad solution: store multiple representations of
the content on the servers
Better solution: proxy server, transcoding
Challenge: flexibility to handle wide variety of
evolving devices?
Context-Aware Decision Engine(2)
WAP phones are widespread but suffer
limitations
Limited bandwidth (9600 baud 2G, 56KB 2.5G,
114Kbps Verizon east-coast CMDA1x)
Display size (92x74 = 5 Korean characters)
Conflicting Standards: HTML vs. WML, JPEG
vs. WBMP
Context-Aware Decision Engine(3)
What is the context?
User preferences
Device capabilities
Quality of service
Context-aware Decision Engine (4)
How to determine device capabilities
J2ME uses configuration (e.g. CLDC) and
profile (e.g. MIDP)
Composite Capability/Preference Profile: user
prefs within user agents set of capabilities,
retrieved via http over the Internet rather
than from the mobile device
Context-Aware Decision Engine(5)
Decision engine evaluates each possible mode
of presenting content
Mode characterized by quantized user
preferences, device capabilities
Search algorithm (various ones) finds a mode
that meets user preferences and device
capabilities
Mode scores for each user and device
precalculated
Distributed Services (1)
Building applications using composable, distributed component
services, rather than static application partitioning used for multi-tier
computing.
Service should not be explicitly named, but rather specified in an
abstract manner (intent-based access not through explicit
location/name, but characteristics)
Specify an abstract service description language: a means to express
the expected function of a service.
Create a task-based model for application structure:
- Tasks in a PIM calendaring application could be user authentication,
browsing the appointment, and entering a new appointment.
Distributed Services (2)
System must be able to dynamically discover and compose
the services that are available in the local vicinity.
The system needs to support dynamic selection of an
appropriate interface, based on the devices resources and
form-factor.
The system needs to seamlessly integrate the applications and
service found in the environment (e.g. a discovered map
application should be able to use a discovered GPS service.)
Distributed Services (3)
At load-time, a device need to negotiate application
apportioning with a server (adaptation to the device resource
available).
Support handoff of task context from one environment(e.g.
newspaper service in office) to another (e.g. car): adaptation
to detect changes in environment and re-apportioning.
Service deployment, dynamic upgrade, customization, and
consumer choice.
Distributed Services (4)
Mobile Agent technology. And active network allowing the
flexible integration of applets and servlets.
Persistent storage: an object store to 1) preserve the structure
of application data, 2) provide associative access to this data,
and 3) support atomic operations.
Service Framework [7 - 13] (1)
Dynamic discovery of functionality based on context.
Self-managing, as devices can join and leave as necessary.
OSGi, Jini (Sun) , UPnP (Microsoft), Bluetooth SDP, and HAVi
- Local service database vs. ARP-style request (announce-listen model)
- Query model: Java object, XML, or URI
- A resource name vs. an object written in Java or Otcl

Discovery Protocols (1)
Spontaneous discovery and configuration of network
devices and services
Selection of specific types of service
Low (preferably no) human administrative requirements
Automatically adaptation to mobile and sporadic availability
Interoperability across manufactures and platforms
Jini




Bound to Java RMI
Multicast request/Multicast announcement/Unicast discovery
Service matching by service type and attributes
Lease enables robust system without manual intervention.
Remote event enables the run time composition of services.
What Jini is not
Jini is not a name server: lookup service is
similar, but does more than provide references
to objects
Jini is not Java Beans: beans communication
based on direct method invocation, not
remote. Also, beans do not automatically
become aware of each other as do Jini objects
Strong typing is important
In Java, an RMI defines the interface, while in
CORBA, an IDL description is a wrapper
around the interface
Polymorphism works in RMI -- you can define
subinterfaces; you can pass an ojbect of a
subtype to remote server and the subtype
methods are appropriately used (doesnt work
in CORBA).
Discovery
Services=Devices or Software
Services collect themselves into communities
The resources available to a community are kept
track of by a lookup service
No one-to-one mapping between communities and
lookup services
Discovery=finding available lookup services

Lookup
Downloadable proxy provided with each
service item
When any entity wants to use your service it
downloads the proxy and runs it -- the proxy
handles sending the remote server the
necessary pieces
Proxy can be used as a secure, network-aware
device driver
Lookup contd
Lookup returns a service item when queried
successfully.
Service item contains proxy object (local stub
for remote object) and attributes that
describe the service
Lookup, contd
Lookup service
Service item
Proxy
attribute
attribute
Service item
Proxy scenarios
Proxy may perform the service by itself,
without any remote method invocation
Proxy may be an RMI stub that knows how to
talk to a remote service (e.g. IMAP mail
server)
Proxy uses private communication protocol to
talk to legacy (non-Java) systems.
Using a Jini Service
Lookup service
Downloads proxy
Service
Client
Client communicates
with service via
proxy
Universal Plug and Play (UPnP)
Microsoft-backed
UPnP is multistage protocol
- Discovery/Description/Control/Eventing/Presentation
Devices, Services, and Control Points
Based on Web technologies: IP/HTTP/XML/GENA/SOAP
SSDP(Simple Service Discovery Protocol)
- URI style query (similar to SLP) returns URL which points to an XML file
with an elaborate description of the service.
Similar functionality to Jini plus automatic IP configuration (AutoIP)
www.upnp.org

SSDP
ssdp:alive (like Jini registration with lookup
svc)
ssdp:discover (like Jini discovery)
Can work with/without control points (lookup
server) in contrast to Jini

UPnP Services
Description is stored as XML file
Control via SOAP messages: SOAP developed
for web service
Most every language/platform has SOAP/XML
libraries
Event notification with XML in General Event
Notification Architecture
Presentation URL can be supplied by device
Open Services Gateway Initiative
(OSGi)
Open, standards based, language/OS neutral
Consists of framework in which bundles of
services that register with a registry can run
On top of J2RE
OSGi Service Specifications
Three version 1 services
Logging service
Web server
Device access
Version two services
added:
Configuration svc
Preferences svc
User admin svc
Permission admin svc
Pkg admin svc
Framework
Allows the downloading, execution, and removal of
bundles
M anages bundle installation and updates
dynamically (scalably?)
Consistent programming model for bundle
developers
Bundles can select an available implementation at
runtime through the framework registry
Bundles
JAR files
May include other resources (html, help files,
icons, etc)
Contains manifest
States dependencies on other resources
Bundle activator class
Bundles, contd
Framework itself is a bundle (system bundle)
Management bundles implement policies (e.g. from what
location is a bundle to be loaded)
Each bundle has a classloader with it, providing bundle with
own namespace, permitting sharing of packages between
bundles
Time for the soapbox
Jini has seen little market acceptance -- no one wants
to be tied to Java, worried Sun will drop it
UPnP complies with standards++, but almost gets it
right
Why fool with all this -- you can write your own
without much fuss?
Because XML is the only way of doing RPC that
anyone can agree on
But wait and see what happens with OSGi
Infrastructures (1)
Environment populated by sensors, actuators, and appliances.
People control and monitor their environments by taking advantage of
freely-available computation throughout that environment.
Environment provides mobile devices with computational power,
because they are not limited neither by size nor by power source.
- For example, in order to improve speech recognition, use video
imaging of lip movement, or take into account an image-based
understanding of a persons gaze, expression, and intent.

Infrastructures (2)
Global computing infrastructure, which spans different
organizations and smaller networks
- Autonomous administration.
Self-managing, ad-hoc, proximity network of people (mobile
devices) and things (environment).
- Bluetooth, HomeRF, USB, IEEE 1394
Location discovery
- Location beacon embedded in the walls
- GPS for outdoors
Infrastructures (3)
Intermittent connectivity due to power, cost, bandwidth, latency and
congestion limitations.
Bluetooth and HomeRF addresses small-proximity network, intermittent
connection environment, but not complex issues such as hand-off and
signal strength analysis.
Vertical handoffs to provide seamless and power efficient connectivity
across a wide range of areas
Active network infrastructure

Infrastructures (4)
Ubiquitous persistent storage
- Coda file system designed specifically for mobile clients
- The Bayou project using a distributed database approach
Disconnection: automatic migration and disconnected operation
Disconnection should be distinguished from failures.
Adapting traditional checkpoint strategies.

MIT Oxygen Project [18] (1)
Vision: In the future, computation will be freely available everywhere,
like batteries and power sockets, or oxygen in the air we breathe.

Goals:
- pervasive: it must be everywhere, with every portal reaching into the same
information base.
- embedded: it must live in our world, sensing and affecting it.
- nomadic: its users and computations must be free to move around according to
their needs.
- eternal: it must never shut down or reboot; components may come and go in
response to demand, errors, and upgrades, but Oxygen as a whole must be non-
stop and forever.
MIT Oxygen Project (2)
Foundation technologies
MIT Oxygen Project (3)
Knowledge access technologies offer improved access to information,
customized to the need of people, applications, and software systems.
They allow users to access their own knowledge bases, the knowledge
bases of friends and associates, and those on the web.

Automation technologies offer natural, easy-to-use, customizable, and
adaptive mechanisms for automating and tuning repetitive
information and control tasks. For example, they allow users to create
scripts that control devices such as doors or heating systems
according to their tastes.
MIT Oxygen Project (4)
Collaboration technologies enable the formation of spontaneous
collaborative regions that accommodate the needs of highly mobile
people and computations. They also provide support for archiving
speech/video fragments from meeting, and for linking these
fragments to issues, summaries, keywords, and annotations.

Speech and Vision technologies provides the main modes of
interaction in Oxygen. Multimodal integration increases the
effectiveness of these perceptual technologies, for example, by using
vision to augment speech understanding by recognizing facial
expressions, lip movement, and gaze.
MIT Oxygen Project (5)
Software environment is build to support change(adaptable). Change is
occasioned by anonymous devices customizing to users, by explicit user
requests, by the needs of applications, by current operating conditions,
by the availability of new software and upgrades, by failures, or by
other causes.

Networks, called N21s, connect dynamically changing configurations of
self-identifying mobile and stationary devices to form collaborative
regions. N21s support multiple communication protocols for low-power
point-to-point, building-wide, and campus-wide communication.
MIT Oxygen Project (6)
Intelligent Spaces are created by stationary devices, called
E21s, embedded in offices, buildings, homes, and vehicles.
E21s provide large amounts of embedded computation, as
well as interface to camera and microphone arrays, large area
displays, and other devices.

Mobile(Handheld) Devices, called H21s, provide mobile
points of interaction for users. H21s accept speech and visual
input, and they can reconfigure themselves to support
multiple communication protocols or to perform a wide
variety of useful functions (e.g., to serve as cellular phones,
beepers, radios, televisions, GPSs, cameras or PDAs). H21s
can conserve power by offloading communication and
computation onto nearby E21s.
HP CoolTown
Tim Kindberg, John Barton, HP labs (Palo Alto)
[19,20]
Link the physical presence of
people/places/things in the real world to the
logical presence of those people/places/things
on the web
You can click on things in the world
Example applications
Aim your mobile phone at a billboard and click on it
to get info about the product (actually this is done
with a BREW phone by KDDI in Korea, but the spirit
is similar to Cooltown), also similar to U.Florida
iGrocer
Get information on paintings with your PDA as you
explore an art museum (and pay to get a high quality
reproduction printed on an HP color large-format
printer at the gift shop :-)
Elders, children/education, repair/maintenance, etc
HTTP is everywhere
Use http/web servers/post strings to handle
the system -- anything can run a web server
Users are familiar with the idea of a browser
(and HP sells PDAs with browsers)
Developers are familiar with developing web
applications, so leverage that

Infrastructure
Somehow must add at least tags to things in
the world
Examples: RFID, barcode, infrared
The web server for a thing doesnt have to be
on the thing, but the URL does
No configuration for users
Somehow must be able to infer the users
context
References
[1] Mark Weiser. The computer for the 21st century. Scientific American, 265(3):94--104,
September 1991.
[2] Guruduth Banavar, James Beck, Eugene Gluzberg, Jonathan Munson, Jeremy Sussman,
and Deborra Zukowski, "Challenges: An Application Model for Pervasive Computing,"
MobiCom 00, August 2000, pp. 266 - 274
[3] M. Esler, J. Hightower, T. Anderson, and G. Borriello, Next century challenges:data-
centric networking for invisible computing. the portolano project at the university of
washington, MobiCom 99, August 1999.
[4] Thomas Agoston, Tatsuro Ueda, and Yukari Nisimura, Pervasive computing in a
networked world, In Proc. of INET 2000.
[5] Robert Grimm, Tom Anderson, Brian Bershad, and David Wetherall, A System
Architecture for Pervasive Computing, In Proceedings of the 9th ACM SIGOPS
European Workshop, pages 177-182, Kolding, Denmark, September 2000.
[6] Brent A. Miller, Bluetooth Applications in Pervasive Computing,
http://www.ibm.com/pvc.
[7] Robert E. McGrath, Discovery and Its Discontents: Discovery Protocols for Ubiquitous
Computing, UIUC 2K project, April 2000.
[8] Steven E. Czerwinski, Ben Y. Zhao, Todd D. Hodes, Anthony D. Joseph, and Randy H. Katz,
An Architecture for a Secure Service Discovery Service, MobiCom '99, August, 1999,
pages 24 - 35.
[ 9] Erik Guttman, Service Location Protocol: Automatic Discovery of IP Network Services, IEEE Internet
Computing, vol. 3, no. 4, July/August 1999.
[10] E. Guttman and C. Perkins, Service location protocol version 2, IETF RFC 2608, June 1999.
[11] Service Location Protocol Home Page, http://www.srvloc.org.
[12] Universal Plug and Play Device Architecture, http://www.upnp.org/resources.htm
[13] Jini specificaations v 1.1, http://www.sun.com/jini/specs
[14] Dey, A.K. et al, The Conference Assistant: Combining context-awareness with wearable computing,
Proceedings of the 3rd International Symposium on Wearable Computers (ISWC '99), 1999, pp. 21-
28.
[15] Dey, A.K., Salber, D., & Abowd, G.D, A Context-based infrastructure for smart environments, In
Proceedings of the International Workshop on Managing Interactions in Smart Environments (MANSE
`99), pp. 114-128.
[16] S. D. Gribble and et al, The ninja architecture for robust internet-scale systems and services, Special
Issue of IEEE Computer Networks on Pervasive Computing, 2000.
[17] S. D. Gribble, M. Welsh, E. A. Brewer, and D. Culler, The MultiSpace: an Evolutionary Platform for
Infrastructure Services, In Proceedings of the Usenix Annual Technical Conference, June 1999.
[18] MIT Project Oxygen, http://oxygen.lcs.mit.edu
[19] Lum, Wai Yip, and Francis C.M. Lau, A Context-
Aware Decision Engine for Content Adaptation, Pervasive
Computing, IEEE Press, Vol. 1, No. 3, pp. 41-49.

[20] People, Places, Things: Web Presence for the Real
World, Tim Kindberg, John Barton, Jeff Morgan,
Gene Becker, Ilja Bedner, Debbie Caswell, Phillipe
Debaty, Gita Gopal, Marcos Frid, Venky Krishnan,
Howard Morris, Celine Pering, John Schettino, Bill Serra
and M. Spasojevic. In proceedings WMCSA2000. To appear in
MONET Vol. 7, No. 5 (October 2002).

[21] A Web-Based Nomadic Computing System.
Tim Kindberg & John Barton. In Computer Networks,
Elsevier, vol 35, no. 4, March 2001, pp. 443-456.