95.

420 Co-op Work Term IV Report

BitFlash, Inc.

Jabber Whiteboard

by
Sunir Shah (228439)
supervised by

Haras Mykytyn, Research Team Leader

ABSTRACT
BitFlash, Inc. (BitFlash) specializes in putting graphics on small handheld devices like a Palm

Pilot. Recently, there has been a great boom in messaging between handhelds, reaching 15 billion

in the month of December 2000 alone. Consequently, BitFlash has been interested in putting

graphics into those messages.

Jabber is an open source instant messaging/distributed platform that uses XML natively on its

protocol layer. Its architecture is extremely flexible.

BitFlash asked me to investigate embedding images described by the W3C Scalable Vector

Graphics (SVG) specification into the Jabber instant messaging stream. After looking into the

problem, I came up with a more interesting application: whiteboarding.

This report describes the design and principles behind the Jabber SVG Whiteboard that I

developed during my term at BitFlash.

 ACKNOWLEDGEMENTS

I would like to acknowledge Rick Graham for developing the SdVG specification and Haras

Mykytyn for supervising the project and providing the crucial insight that lead to the solution of the

desynchronization problem.

I would also like to acknowledge the Jabber.org and Jabber.com teams for their assistance in

developing the specification and the reference implementation, especially David Waite, Jeremie Miller,

Thomas Muldowney, and Peter Saint-Andre.

TABLE OF CONTENTS

BACKGROUND.............................................................................................................................................2
COMPANY OVERVIEW...................................................................................................................................2
JABBER..........................................................................................................................................................3
CONCEPTS....................................................................................................................................................7
INTRODUCTION..............................................................................................................................................7
CONCERNS.....................................................................................................................................................8
USER INTERFACE.....................................................................................................................................10
FUNDAMENTALS..........................................................................................................................................10
THE PARABLE OF THE BOOK......................................................................................................................10
INPUT..........................................................................................................................................................12
DESIGN........................................................................................................................................................12
FUTURE ENHANCEMENTS............................................................................................................................14
PROTOCOL.................................................................................................................................................16
THE DISTRIBUTED MODEL..........................................................................................................................16
PHILOSOPHY................................................................................................................................................17
DATA FORMAT............................................................................................................................................18
FUTURE DIRECTIONS.............................................................................................................................19

CONCLUSION.............................................................................................................................................20

APPENDIX A. JABBER SCALABLE VECTOR GRAPHICS (SVG) WHITEBOARD PROTOCOL

BASIC DRAFT V. 0.1..................................................................................................................................21

References......................................................................................................................................................29
 2

BACKGROUND

Company Overview

BitFlash Graphics, Inc. (BitFlash) is a Gloucester, ON based company that focuses on providing

graphics on handheld devices, such as a Palm Pilot. BitFlash aims to solve many of the constraints that are

fundamental to displaying graphics the handheld environment, such as bandwidth limitations, memory

limitations, processing power limitations, and small and low-resolution displays. They achieve this through

a mixture of three technologies that are part of the BitFlash Mobility Suite:

 Optimized server. Customers use the BitFlash Server as a gateway between their content

(e.g. a web page, PowerPoint files, images) and the wireless handheld device. Based on

the device, the server will send only the information necessary to render the image.

 Optimized client. On the handheld device, BitFlash places its highly optimized and low

footprint viewer to display the content stream sent to it by the server.

 Optimized content. Instead of sending a complete bitmap, BitFlash employs use of the

W3C candidate recommendation, Scalable Vector Graphics (SVG) [SVG, 2000]. Vector

graphics are a much more compact representation of an image than a bitmap.

The lattermost point is the most important to this report. BitFlash is a heavy supporter of vector

graphics. It has a member on the SVG working group. We believe strongly that vector graphics have an

important role to play on handhelds, not only because they are more efficient representations of some

images but because of the limited display capabilities of those handhelds. With vectors, it’s possible to

zoom into an image for more detail, or pan the image to see parts that don’t fit onto the tiny display.

Over the last year, BitFlash has more than quintupled in size. In order to better support non-

product tasks, the development group has spawned a separate research group. It is the research group that is
 3

responsible for representing the company at the W3C, especially on the SVG working group. It is also

responsible for demonstrating uses of the core technology and demonstrating potential new ways of using

graphics on handhelds. In particular, research is interested in finding and demonstrating uses for vector

graphics. This is important in a world of print magazines and graphic editing, where bitmapped graphics

the norm and vector graphics aren’t well understood. Additionally, the team is also responsible for looking

into new technologies not directly related to the main development stream at the company.

One technology of interest was instant messaging. According to GSM World, “a record 15 billion

SMS (Short Message Service) Text messages were sent over the world’s GSM (Global System for Mobile

communications) wireless networks during December 2000.” Further, as Rob Conway, CEO of the GSM

Association says, “The GSM Association is already anticipating that by December 2001, we will be seeing

monthly global SMS volumes achieve the 25 billion mark. And over 200 Billion in total for 2001.” [GMS,

2001] A large number of those messages are interactions between users, or “instant messaging.”

In recent developments in instant messaging on the desktop, AOL Instant Messenger (AIM), MSN

Messenger and Yahoo!Messenger have added images to the message stream. They translate “emoticons”

into their representative images. For example, :-) translates into . However, those images aren’t actually

sent over the Internet. When a client receives the text “:-)” it just displays . One improvement that

research was enlisted to make was to embed images in normal text messages.

However, since SVG is an Extensible Markup Language (XML) [XML, 2000] dialect, it would be

ideal to embed SVG images inside of an XML document. XML is structured for this, as evidenced by the

XHTML efforts. [XHTML, 2000] None of the instant messaging protocols listed above support XML

directly. Moreover, since all of them are proprietary protocols, they are difficult to extend with our own

technology. The answer came with Jabber.

Jabber
 4

Jabber [Jabber, 2000] is an open sourced protocol for instant messaging and presence notification

that uses XML at the protocol level. This makes the protocol easily extended by third parties (such as

BitFlash) without breaking existing Jabber clients. Jabber’s initial goal was to be the glue between all the

proprietary instant messaging networks so an end user wouldn’t be locked into using one or the other. With

Jabber, the end user could use all the networks at once from the same client.

But Jabber is an instant messaging network on its own too. It too is designed as a star network,

with the central server connecting the outlying clients. [Saint-Andre 1, 2001] Clients communicate with

each other by sending messages through the server that relays them to the destinations like a giant

switchboard. See Figure 1.

Client

Server

Client Client

Figure 1. Common instant messaging architecture. A central server connects many clients.

Unlike the others, Jabber extends this by allowing many servers to communicate with each other

as well. This way, networks that are hosted by different sites will still be able to connect to and interact

with each other. See Figure 2.

 5

Client

Server

Client Client

Client Client

Server Server

Client Client Client Client

Figure 2. Jabber allows different networks to talk to each other.

Furthermore, each server is really a network of subservers (called transports). See Figure 3. This

allows each Jabber network to communicate not only with other Jabber networks, but also with networks

that are completely different, such as MSN Messenger. Each transport translates the foreign networks into

the Jabber protocol for use on a Jabber network and vice versa. This then allows Jabber to be the glue

between those networks.

The transport architecture also allows services to be put on the network, like a chat server for

instance. Clients send messages to the chat server that then broadcasts the messages to all clients subscribed

to that server. This architecture is extremely flexible. Any number of additional services may be put on the

network. For instance, a server to acquire play-by-play sports scores could easily be written just by writing

another transport.
 6

Jabber Server
Client Chat transport Client

Client Client

Client Jabber transport Client

MSN transport Foreign Jabber transport

MSN Foreign Jabber transport

…
Jabber Server

Client Client Client Client Client Client

Figure 3. Internally, a Jabber server is a network of subservers. Each subserver performs a unique
task, like connecting to foreign instant messaging networks, or by implementing services like chat.

This flexible architecture coupled with the flexibility of XML makes Jabber very adaptable. This

makes it possible to create applications on top of Jabber that aren’t even tied to instant messaging. Indeed,

from the start, Jabber was designed as a distributed application platform, not just an instant messaging

platform. It just happens to do instant messaging really well.

Building on this, after some discussion, it was agreed that I wouldn’t try to merely embed SVG

graphics directly in the messages, but I would develop a way of making SVG graphics collaboratively over

Jabber. That is, I would create a whiteboard application. This report describes the design of the whiteboard

application.
 7

CONCEPTS

Introduction

A very common application in groupware environments is some sort of collaborative drawing

system. This is normally called a "whiteboard." Common examples include the one in Microsoft

NetMeeting.

Most whiteboards are bitmap based. For instance, the one in Microsoft NetMeeting looks and

behaves very much like Microsoft Paintbrush, the default bitmap editor that comes with Windows 95 and

higher. Jabber itself has a draft protocol for bitmapped whiteboarding. [Eatmon, R., et. al., 2001] For

simple scribbling this is sufficient and it's very quick to implement. However, it is not very good for

collaboratively working on actual images or structured images like architectural drawings or diagrams. The

ultimate reason is that the bitmap loses the semantics of the image, so the whiteboard no longer knows that

a rectangle is really meant to be a rectangle.

SVG is designed to maintain more of the semantics of an image, as it is an abstract representation

of the drawing shapes. For instance, it knows that a rectangle is a rectangle because it explicitly stores the

geometric description of that rectangle.

Thus, for more structured applications, like engineering applications for instance, an object-based

whiteboard is necessary. For now, an SVG whiteboard will be an important first step as it maintains enough

information to manipulate the image collaboratively. For example, individuals can rearrange pieces of the

flowchart easily by just moving the shapes in the document.

Jabber is the ideal platform for constructing a collaborative whiteboard (or object sandbox)

because it maintains the communication between the peers and it uses the flexible XML language as its

transmission language. This allows end applications to embed object descriptions in the message packets,

like for instance the geometric description of a rectangle.

 8

In the end, the required end product is a collaborative whiteboard on handheld devices. An

example user story might be:

Dick and Jane both work for a software consulting company that has been contracted to

a corporation to solve their e-commerce infrastructure problem. Jane has been sent into

the field to determine the existing infrastructure of their client and to represent her

company in negotiations. In order to state informed opinions during negotiations, she

needs to get the input of the engineering core back at her home office. Thus, while

interviewing engineers at her client, she draws flowcharts, data flow diagrams, and other

visual representations on her Palm Pilot that Dick simultaneously watches on his desk.

After reviewing the situation with his engineering team, he makes annotations and

changes to Jane's diagram while talking to her on the phone to visually demonstrate the

company’s recommendations. This additional communication channel greatly improves

Jane’s ability to understand Dick’s presentation.

Concerns

This project has a few pressures that affect its development. While the end goal is to create a

handheld and portable representation of the whiteboard, much work must be done in advance. For instance,

it will not be useful to create a whiteboard protocol that no other clients use. Thus, one necessary item is to

get the protocol accepted by the Jabber client implementers.

Furthermore, if the implementers adopt the protocol, the whiteboard will be used on the much

more powerful desktop. Therefore, some forethought must be given towards allowing higher quality

documents to be sent over the protocol whilst allowing the much underpowered handhelds to do something

useful. Also, in order to gain wide adoption, the protocol must be simple to implement, preferably with a

reference implementation (ideally written in the platform-neutral Java).

 9

In a similar vein, the application won't be used if the implementations are too complicated,

especially our own. A usable initial interface will likely do more to sell the whiteboard than anything else

because people will be able to immediately and tangibly see the benefits. Also, if the whiteboard

fundamentally requires (or even merely reflects) a complicated user interface, it will be difficult or even

impossible to implement on a handheld.

Finally, the application is a distributed protocol. Distributed computing is notoriously difficult to

do correctly. An incorrect protocol will either be a market failure or a market headache. Doing a good job

necessarily means doing a correct job.

 10

USER INTERFACE

Fundamentals

There are competing forces at work. First, due to the choice of simple Java as the reference

implementation language, the poverty of Java's AWT, especially java.awt.Graphics limits what can be

done. At best, the GUI will be half mocked up. Secondly, while the system is being developed on a desktop

and thus has all the capabilities of a full workstation (filtered through AWT), the end goal is to put it onto a

handheld, so the interface has be simple. Also, the aim is not to create a full-blown drawing application, but

a very simple scribble pad.

In any other circumstance but the user interface, it's normally best to do whatever is simplest and

easiest, ignoring any future applications of the system. For example, if the application is running on a

desktop, don't worry about handhelds. However, the user interface makes the first impression; it’s also the

hardest to get right or change after the fact, so I chose to start doing a good job earlier rather than later.

To this end, I chose to aim for a "ridiculously" simple interface, with a "ridiculously" simple input

system. By targeting for an abstract stylus system, which is available on all systems, I was able to develop

an interface that would work on handhelds, yet could be prototyped on a desktop using the mouse.

Additionally, Jef Raskin further suggests avoiding modes in the system. [Raskin, 2000] I aimed to

keep the system a fluid as possible, hopefully nearing the simplicity of interaction in the real world.

The Parable of the Book

In the real world, when two people reach for the same book on a table, grabbing it at the same

time, conflicts are resolved through feedback--not locking and synchronization. Consider that if they both

tug on the book, one person will let go first, allowing the other to keep it. Problems of course arise when

both want the book at the same time, but communication normally resolves this (between adults acting in
 11

good faith; interfaces for children or attackers may have different requirements). For example, "Oh, sorry,

could I have the book for a minute. I just wanted to look something up quickly." "Sure. No problem."

In online environments, it is often sufficient to indicate that the objects in contention are in

contention. An additional, secondary communication channel (like a chat session), should provide all the

conflict resolution required. This will result in a simpler protocol and a simpler user interface.

Some might respond that it would be better to just prevent interacting with an object once it's in

use by someone else. Note that this would require a very annoying interface, as one might start using an

object before one receives the conflict notification. That is, as both clients have sent notifications, should

both clients lock out the object (resulting in a dead object)? Should they both drop it (unilaterally aborting

both users' actions)?

A better move might be to demonstrate that, firstly, the object you are controlling is contended for

and, secondly, the current status of the object to other people (say, where it currently is).

Another case is when one user grabs an object before she receives notification that the foreign

client has destroyed it. In this case, it’s best to restore the object, even if the first user intends to destroy it

as well.

Notice that the only way people will both grab for the book at the same time in the real world is if

they aren't paying attention to each other. With groupware, often the case is that they can't even see each

other. Therefore, one should demonstrate to the foreign client what the local client is doing, say by

notifying the other client of current actions. Thus, two users can "get out of each other's way."

A warning, however. Unlike a book, if the users can both simultaneously change the state of an

object independently of each other, then the object isn't analogous to a book. This is the case with the

objects on the whiteboard. In this case, you can enforce dependencies, allow desynchronization, or lock the

object.
 12

The best choice is to constructively allow desynchronization by transparently duplicating the

resource once desynchronization has been detected. That is, give each user her own book. This way, neither

user loses data and their actions are not interrupted.

The above has been adapted from an earlier publication. [Shah, 2001]

Input

For the input, as mentioned above, the system abstractly uses a stylus that can either be

implemented with a desktop's mouse or a Palm Pilot's stylus. There are important distinctions between the

two. For instance, with a mouse, the system always knows where the cursor is (even if the user doesn't).

The user cannot lift the cursor off the screen like he could with a stylus. Thus, all movements of the mouse

can be tracked and represented. One cannot simply jump from point A to point B--one has to move the

cursor through some trajectory. However, this is unlike a stylus system where the user knows exactly

where the stylus is at all times, even if the system does not if the stylus is off the screen.

Consequently, to abstract this sufficiently, the system limits the classes of input it recognizes.

Really, this interface is designed with a Palm stylus in mind. For our purposes, the mouse is capable of

simulating a stylus completely. Thus herein dropping the stylus is equivalent to pressing the action button

on the mouse (i.e. usually the left button), lifting the stylus is equivalent to raising the action button, and

tapping the stylus is equivalent to clicking the action button. Dragging the stylus means moving the stylus

with the tip on the screen. This is akin to moving the mouse while the action button is down (dragging the

mouse).

Design

The Parable of the Book neatly summarizes the fundamental design philosophy of the interface

(and parts of the protocol). The input metaphor will be one of grabbing an object and acting on it. In order

to provide feedback on what other users are doing, objects that are selected on foreign whiteboards are
 13

drawn in red so you know others are working with them. In order to resolve conflicts, the whiteboard has a

chat window associated with it.

The interface uses a simple vertical toolbar (in the traditional sense) to switch between drawing

shapes. The available shapes are line, rectangle, circle/ellipse, and polyline. See Figure 4. Choosing

between the current shapes is as modal as the interface gets. Everything else is done by top-level

interactions.

Figure 4. Screenshot. On the left half is the chat interface. On the right is the whiteboard.

As an example of that, the interface includes an element known as the Red Dot. The Red Dot is

special in that it has no defined role, but it provides a range of actions. All those actions can be loosely

placed in a "negative" class:

 Deleting objects.
 Deleting vertices from a polyline.
 Canceling a draw operation.
 Unselecting everything.

To activate the Red Dot, the user merely needs to drag objects into it or tap it. It's always live,

always functional.
 14

Drawing actions are also available as top-level interactions. Creating a new object is as simple as

dropping the stylus where you want to begin drawing and dragging a handle to define the shape. To select

an object, just tap it. It will remain selected. You can select multiple objects by just tapping the ones you

want. To reconfigure a shape, drag one of its handles. To move an object, drag the object itself.

Grouping, ungrouping, or changing the z-order of objects is a simple matter of tapping the

appropriate button above the Red Dot. When hit, the button will perform its respective action on the

currently selected objects.

All precision operations will allow some (small) tolerance for error because it is very difficult to

hit a line one pixel thick with a stylus. Usually, if the distance to the shape is less than some threshold,

currently four pixels, you will succeed in activating that shape.

Future enhancements

The current interface has usability problems. For instance, the current scheme for selecting

multiple objects is too complicated. Often, one will want to select a group of nearby objects even if they

aren't grouped together in the document. Modern drawing programs have a selection tool known as a lasso

to do this easily. The user selects the lasso tool and then outlines the shapes she wants to select. This could

easily be done. This was planned for in the preliminary design, as demonstrated in Figure 5.
 15

Figure 5. The preliminary design called for a lasso tool to make selecting multiple objects easier.

Other useful but missing features are copy and paste. The traditional means of doing this are

through quasimodes [Raskin, 2000, p.55] like hitting CTRL-C and CTRL-V. Unfortunately, these

capabilities are unavailable to the abstract input system described above. One solution would allow users to

create "cookie cutter" templates instead. In this system, the toolbar would include a tray to drag templates

into after which the templates could be used just like the primitive shapes. That is, select one, then draw

with it. The preliminary design also called for this, as shown in Figure 6. This interface has the advantage

of being useful for schematic drawing, probably one of the best uses of the whiteboard.

Figure 6. The "cookie cutter" interface. Grouped objects can be made into drawing templates.

Finally, some means of just drawing freehand needs to be developed. No good answer has been

developed yet because it is very difficult to translate free information into abstract geometric information.

One solution might be smooth the free curve into lines using fitting algorithms, but that’s computationally

expensive. A cheaper solution might be to lower the sampling rate of the stylus input so that the points are

farther away from each, and then draw lines between the broken line. However, this means a short drawing

may not have enough information to represent it faithfully.

 16

PROTOCOL

The Distributed Model

The Jabber network is a distributed system. Consequently, any user application that runs on top of

it is bound by the properties of its distributed model. Jabber's model has the following salient properties:

 First in, first out message channel.

 Partial reliability. (i.e. messages will arrive)
 Network error detection.

Jabber's distributed model is not strong enough to resolve synchronization. Indeed, it's been

shown by the Two General's Problem [Gray, 1978] that there is no way to perfectly synchronize data

between egalitarian nodes without outside influence.

One advantage of Jabber's architecture is that it allows new components to be plugged into the

server. One option to solve race conditions is to impose total message ordering by forcing all messages to

go through a transport on the server. The transport would simply stamp each message going through it with

a counter (i.e. 1, 2, 3, …). Then, in the even of a race condition, the lowest message wins.

However, this solution requires extra functionality on the server. This is undesirable for many

reasons. First, it requires explicit support from both the server and the clients, which means it will be more

difficult to adopt. Second, the Jabber transport interface changes semi-frequently. A transport would

require maintenance above and beyond a soft protocol that was only implemented in clients. Finally, a

transport is more difficult to install than a Jabber client.

Therefore, as discussed in The Parable of the Book above, duplication of data in the event of

desynchronization was chosen instead. So, if two users change the same object, that object is replicated so

each user has her own copy to modify. This allows the protocol to be implemented entirely in clients,

which means it can be used anywhere there is a Jabber server without explicit support from the server host.
 17

Philosophy

The overarching principle of the protocol is that each peer node is responsible for enforcing its

version of the model. So, if you are in the middle of a moving a line and another user deletes that line, you

are responsible for restoring that line.

There are two means of accomplishing this. The first is to simply ignore any unacceptable changes

that come in. If another user's client invalidates her own whiteboard, that's her problem. One example

would be moving an element inside of a group independently of the rest of the group. This policy prevents

rogue or defective clients from destroying the common document.

The second means is by forking objects, as mentioned above. In the event that two or more users

are simultaneously making changes to the same object, each user makes a copy of that object for their own

uses. They all also delete the object in contention because it is obsolete.

In order to determine whether the foreign clients are aware of your current state, all updates to the

state of the document (except for deletions) require acknowledgments from all the foreign clients. Because

the messages are first in, first out, changes to that object arrive before an acknowledgment implies a race

condition.

In order to know when all the peer nodes have acknowledged your action, you need to know who

is listening. Therefore, peers that want to use the whiteboard need to explicitly subscribe to it. They should

also unsubscribe from it when they are done. In the event a peer node loses its connection, Jabber's network

error detection is powerful enough to detect the failure. That peer node will automatically be removed.

Clearly this system of each node maintaining its own state can desynchronize the whiteboards,

especially if they disagree fundamentally on what's right and what's wrong. Moreover, a node joining an

existing whiteboarding session will need to acquire the current state before it can meaningfully participate.

Thus, nodes can request synchronization at any time. All nodes will then send their current states. Any
 18

differences between the local state and the foreign state will either by replaced by the foreign state or fork

the model.

Finally, running on top of this simple system is the feedback necessary in the Parable of the Book.

Each node can flag objects that it is currently using. This is merely informational, and has no other purpose

in the protocol except as notification to other users.

Version 0.1 of the protocol is attached as Appendix A.

Data Format

The protocol itself must reside inside of the Jabber message stream. In order to maximize the

number of different types of streams the protocol could run on (e.g. private messages, private chat, group

chat), the protocol packets existing entirely within the extension mechanism (the <x/> tag). The Jabber

protocol allows these extensions to be carried along with any message packet (<message/>). [Saint-Andre

2, 2001]

Also, as described above, the objects are described in SVG. However, eventually styling (like

colours) needs to be added. SVG currently uses Cascading Style Sheets (CSS) [CSS, 1999] to describe

style information. However, the Jabber protocol currently has no CSS in it at all. This means that clients

may not have CSS parsers available. They definitely have normal XML parsers though.

Taking this into account, as well as the overall complexity of the complete SVG specification, the

whiteboard will use a limited subset of SVG without CSS. The most suitable replacement is the Small

Device Vector Graphics (SdVG) specification currently under development at BitFlash. It is currently

pending publication.

SdVG defines only the basic primitives, plus a basic set of styles. The styles are all available as

first-class XML attributes, not CSS attributes. SdVG is sufficiently powerful to be able to describe both
 19

complex images as well as engineering level diagrams, but sufficiently simple as to be implementable on

small devices and by the Jabber client developers.

FUTURE DIRECTIONS
The current published version of the whiteboard, v. 0.1, is not sufficiently powerful to

collaboratively create SVG or SdVG documents. It does not have the ability to create a hierarchical

document or to maintain z-order. The next version will address this and it is currently under development.

The major hurdle has been defining a simple means of resolving desynchronization. Complicated solutions

abound, but they aren’t good enough to be published.

After the next version of the protocol is developed, the protocol will be ready to move to the next

phase, an SMS implementation. SMS is a much weaker protocol than Jabber. It loses the first in, first out

capabilities that are necessary for the protocol correctness. Consequently, the next development effort will

be in investigating a control layer for SMS. Once one is found or developed, the whiteboard will be able to

move onto an actual handheld device.

Meanwhile, there is some interest amongst the Jabber developers in getting this protocol done. I

will continue to work with them on that front. However, a limitation of the Jabber architecture is preventing

the whiteboard from moving forward. By default, Jabber servers have transfer rate limitations to prevent

abuses of their bandwidth. Unfortunately, the whiteboard protocol defined here quickly exceeds this

limitation. The only real solution is to develop another protocol to send messages directly from client to

client, thus avoiding the server.

If the protocol becomes dependent on SdVG, advancing the SdVG specification will become a

priority. Work on that front continues at a rapid pace.

 20

CONCLUSION

Judging from the reactions of other Jabber developers, a vector graphics whiteboard has a lot of

potential if done well. I believe my work so far is sufficiently simple yet powerful to succeed, providing I

can complete the protocol.

The additional advantage of this protocol is that it too is extensible. There are no fundamental

reasons why it is limited to SVG or SdVG. I hope it will be used as the basis to other object sandboxes in

the future, or perhaps the whiteboard itself may grow into a fundamental underlying synchronization

technology on the Jabber platform.

 21

APPENDIX A. JABBER SCALABLE VECTOR GRAPHICS

(SVG) WHITEBOARD PROTOCOL BASIC DRAFT V. 0.1
Jabber Scalable Vector Graphics (SVG) Whiteboard Protocol
Basic Draft v. 0.1

Sunir Shah (mailto:[email protected]; JID:[email protected]),

Research,
BitFlash Inc.

-----------------------------------------------------------------

There are two halves to the protocol: what is maintained locally

by each peer, and what is maintained collectively by all peers.
Maintenance of the local model is simple, and is left to each
client to implement as appropriate. The distributed model is
where the protocol comes into play.

Basic model
-----------

The whiteboard is really a sandbox. Each shape on the whiteboard

is a separate object. Each object has several properties relevant
to the protocol in addition to its geometry (e.g.
line(0,10,42,13)):

* A unique identifier
* Whether or not it is selected by the local user
* Which foreign clients have selected it
* Which foreign clients have yet to acknowledge changes on
the object

Essentially, the network is a set of peer nodes communicating

through the Jabber protocol. Each node has a unique identifier.
All messages are broadcast to all peers (as this is the easiest
to implement).

Additionally, the following data structures will be maintained by

each peer:

foreignClients -- SET. The set of known foreign clients

whiteboardObjects -- ORDERED SET. The objects on the whiteboard.

Protocol Message Schema

-----------------------

All messages sent over the wire have the following structure:

<message ...>
...
<x
xmlns="com:bitflash:jabber:whiteboard"
 22

action=:ACTION:
[to=:jabberID:]
>
[<svg>
...
</svg>]
</x>
</message>

The outer <message/> tag is whatever is native to the message

stream. For instance, [email protected] was working on a private
whiteboard with [email protected], it might look like:

<message
type="chat"
to="[email protected]"
from="[email protected]"
>
<thread>1001</thread>
<x
xmlns="com:bitflash:jabber:whiteboard"
...
>
...
</x>
</message>

:ACTION: is any one of the messages (see below, Messages).

Some protocol messages need to be targeted to specific

individuals, hence the "to" attribute. The Jabber ID (JID) should
be the JID used by the other person in the message stream. For
instance, if Alice and Bob were talking in the MostlyHarmless
groupchat on conference.example.com, the message might look like:

<message
type="groupchat"
to="[email protected]"
from="[email protected]"
>
<x
xmlns="com:bitflash:jabber:whiteboard"
to="[email protected]/Alice"
...
>
...
</x>
</message>

If a message is addressed to a particular client like so, only

the client with the given JID should parse it.

The <svg/> subelement is optional because some messages (e.g.

SUBSCRIBE) don't require it. For those that do need to transmit
whiteboard objects, the SVG used here is limited. Although it is
the responsibility of the receiving client to extract what it can
display from the incoming SVG, it is expected that competing
 23

implementations will settle on a basic subset of SVG between

themselves.

The elements contained inside the <svg/> represent the whiteboard

objects. All such objects must minimally have IDs. If they are
missing IDs, they must be ignored. Other information that is not
necessary to the message need not be included. For instance, to
delete the <line/> called "[email protected]/4", only the
following needs to be sent:

<message
type="chat"
to="[email protected]"
from="[email protected]"
>
<thread>1001</thread>
<x
xmlns="com:bitflash:jabber:whiteboard"
action="delete"
>
<svg>
<line id="[email protected]/4"/>
</svg>
</x>
</message>

Whereas, to create the line called "[email protected]/3", the

full geometry needs to be sent, ala:

<message
type="chat"
to="[email protected]"
from="[email protected]"
>
<thread>1001</thread>
<x
xmlns="com:bitflash:jabber:whiteboard"
action="update"
>
<svg>
<line
id="[email protected]/3"
x1="30"
y1="42"
x2="98"
y2="12"
/>
</svg>
</x>
</message>

NOTE: The order of the elements in the <svg/> are important, if

not now, in the future. The SVG specification defines the
z-order by the order the objects appear in the list.

Finally, messages that are delayed (i.e. those with

<x xmlns="jabber:x:delay"/> subelements) must be ignored.
 24

Local Actions
-------------

Each local node can do several actions to its local model. The
results of these actions are passed onto the peer nodes by the
protocol messages as described below. In actuality, many of the
operations below can be done on more than one object at once for
efficiency, but the basic principle is the same.

Herein, only the actions that a client must do will be described.

A client will probably want to do more. Commands in all capitals
are message sends. See below, Messages.

create( localObject )
1. Assign a new ID to localObject.
2. UPDATE( localObject ).

select( localObject )
1. SELECT( localObject ).

unselect( localObject )
2. UNSELECT( localObject ).

delete( localObject )
1. DELETE( localObject ).

The client should remove localObject from

whiteboardObjects as well.

update( localObject )
An update happens whenever an object is reconfigured
locally, say by changing its position or size.

1. Add all foreignClients to the pending acknowledgement

list of localObject.
2. UPDATE( localObject ).

Messages
--------

There are seven basic messages. They are the :ACTION:s in the
message schema. Listed here are the :ACTION: strings followed by
the message response. The definition here goes beyond what is
strictly necessary by the protocol to state some of the expected
behaviour. For instance, it's not strictly necessary that a
client fork() or even detect conflicts. However, this is what
most clients should do. Client implementations may want to do
more, however.

All messages are aware of who sent them. This will be universally
known as "SENDER".

Finally, only the case where one object is sent with a message is
considered below, but in actuality any number of objects may be
sent. The principle is the same, though. Simply apply the
procedure below to EACH object sent along with the message.
 25

NOTE: Description of "send", "reply", "fork()" is below.

"subscribe"
SUBSCRIBE()
Inform others on the whiteboard that this client
a) Can see the whiteboard,
b) Wishes to participate in the whiteboard,
c) Has to acknowledge changes to the whiteboard.

All clients MUST send this message first before any other
messages are sent to the whiteboard.

1. UNSUBSCRIBE().
This will clean up a bad private chat session.

2. Add SENDER to foreignClients.

3. reply ACKNOWLEDGE().

"unsubscribe"
UNSUBSCRIBE()
Inform others on the whiteboard that this client no
longer wants to be notified of whiteboard events and
that it no longer has to acknowledge changes on the
whiteboard.

1. Remove SENDER from foreignClients.

2. For each localObject in whiteboardObjects
2.1. Remove ALL pending acknowledgments from SENDER.
2.2. Remove SENDER from the set of foreign clients
who have selected localObject.

"synchronize"
SYNCHRONIZE()
This is the second message that a client should should
send out upon joining a whiteboard. It will retrieve the
the best known state of the whiteboard for the client.

1. reply UPDATE( whiteboardObjects ).

2. reply SELECT(
all selected objects in whiteboardObjects
)

"select"
SELECT( remoteObject )
Let localObject be the object in whiteboardObjects with
the same ID as remoteObject, or null if no such object
exists.

1. If a localObject is not null, add SENDER to the set of

foreign clients who have selected localObject.

"unselect"
UNSELECT( remoteObject )
Let localObject be the object in whiteboardObjects with
the same ID as remoteObject, or null if no such object
 26

exists.

1. If a localObject is not null, remove SENDER from the

set of foreign clients who have selected localObject.

"update"
UPDATE( object )
Let localObject be the object in whiteboardObjects with
the same ID as remoteObject, or null if no such object
exists.

1. If localObject is null
1.1. Add remoteObject to whiteboardObjects.
[RECOMMENDED] 1.2. Update our last ID counter if
appropriate. (See Appendix A.)

2. Else if the localObject has acknowledgments pending

from foreign clients
2.1. fork( localObject )

3. Else, update the geometry of localObject with the

geometry of remoteObject.

4. reply ACKNOWLEDGE( remoteObject )

NOTE: For efficiency, if many objects were sent with
the UPDATE() message, it would be best to
acknowledge them all together at once instead
of each individually.

"delete"
DELETE( remoteObject )
Let localObject be the object in whiteboardObjects with
the same ID as remoteObject, or null if no such object
exists.

1. If localObject is null, do nothing.

2. If localObject has acknowledgments pending from

foreign clients [RECOMMENDED: or if localObject is
LOCALLY selected]
2.1. fork( localObject )

3. Else remove localObject from whiteboardObjects.

"acknowledge"
ACKNOWLEDGE()
This happens in response to a SUBSCRIBE().

1. Add SENDER to foreignClients.

ACKNOWLEDGE( remoteObject )
This happens in response to an UPDATE().

Let localObject be the object in whiteboardObjects with

the same ID as remoteObject, or null if no such object
exists.
 27

1. If localObject is not null, remove one instance of

SENDER from the pending foreign acknowledgment list of
localObject.

Helpers
-------

send
This creates a new <message/> as described above in
Protocol Message Schema.

reply
This creates a new <message/> as described above in
Protocol Message Schema, except that the "to" attribute
of <x> is set to SENDER.

So, if the incoming message was

<message
type="groupchat"
to="[email protected]"
from="[email protected]/bob"
>
<x
xmlns="com:bitflash:jabber:whiteboard"
action="update"
>
<svg>
<line
id="[email protected]/bob/3"
x1="30"
y1="42"
x2="98"
y2="12"
/>
</svg>
</x>
</message>

the reply would be

<message
type="groupchat"
to="[email protected]"
from="[email protected]"
>
<x
xmlns="com:bitflash:jabber:whiteboard"
action="acknowledge"
***NOTE*** to="[email protected]/bob"
>
<svg>
<line id="[email protected]/bob/3"/>
</svg>
</x>
</message>
 28

fork( localObject )
Forking is the heart of the synchronization. When a
conflict is detected by a client, it is responsible for
protecting its local user's data. Consequently, it does
the following:

1. create()s a new object, forkedObject, which has the

same geometry of localObject (but nothing else is the
same).

2. delete( localObject ).

3. If localObject was selected, select( forkedObject ).

APPENDIX A. ID naming convention

--------------------------------

It is strongly recommended to use the format <jabberID>/<count>

for the object IDs. The Jabber ID should be the one used in the
message stream. So, if Alice was talking on

[email protected]

her JID would be

[email protected]/Alice

The count is simply a monotonically increasing counter starting

at 1 (i.e. 1, 2, 3, ... ).

Note that as the client may rejoin a session it was a member of

previously, care must be taken not to reuse IDs already on the
whiteboard. For instance, if Alice left the whiteboard and
rejoined it later on, she shouldn't start counting from 1 again.
Instead, note that when she SYNCHRONIZE()s, she will discover all
the existing objects on the whiteboard, including the ones she
created. She can easily determine from this what the next ID
should be (say, by taking the highest ID and adding 1).
 29

REFERENCES

[CSS, 1999] Lie, H.W. and Bos, B. "Cascading Style Sheets, level 1," World Wide Web

Consortium. http://www.w3.org/TR/REC-CSS1 (1999)

[Eatmon, R. et. al, 2001] Eatmon, R, et al. “Collaborative Imaging (Whiteboarding via Streaming

XPM),” Jabber.org. http://docs.jabber.org/draft-proto/html/sxpm.html. (2001)

[GMS, 2001] GSM Association. "More Than 200 Billion GSM text messages forecast for full year

2001" GSM World. http://www.gsmworld.com/news/press_2001/press_releases_4.html (2001)

[Gray, 1978] J. N. Gray, "Notes on Data Base Operating Systems." Operating Systems: An

Advanced Course, Springer-Verlag Lecture Notes in Computer Science 60:393-481 (1978).

[Jabber, 2000] "Jabber- Open Source XML-Based Interoperable Instant Messaging and Presence

Platform." Jabber.org. http://jabber.org/ (2000)

[Johnson, et. al, 1993] Johnson, E. and Reichard, K. Power Programming... Motif (2nd Ed). MIS

Press. (1993)

[Raskin, 2000] Raskin, Jef. The Humane Interface, Addison-Wesley. (2000)

[Saint-Andre 1, 2001] Saint-Andre, Peter. “Jabber Technology Overview,” Jabber.com.

http://www.jabber.com/pdf/overview.pdf (2001)

[Saint-Andre 2, 2001] Saint-Andre, Peter. "Jabber Protocol Overview," Jabber.com.

http://www.jabber.com/pdf/protocol.pdf (2001)

[Shah, 2001] Shah, Sunir. (editor) “ParableOfTheBook,” MeatballWiki.

http://www.usemod.com/cgi-bin/mb.pl?ParableOfTheBook. (2001)
 30

[SVG, 2000] Ferraiolo, J (editor). "Scalable Vector Graphics (SVG) 1.0 Specification," World

Wide Web Consortium. http://www.w3.org/TR/2000/CR-SVG-20001102/ (2000)

[XHTML, 2000] Pemberton, S. (chair) "XHTML™ 1.0: The Extensible HyperText Markup

Language," World Wide Web Consortium. http://www.w3.org/TR/xhtml1/ (2000)

[XML, 2000] Bray, T., et. al (editors) "Extensible Markup Language (XML) 1.0 (Second

Edition)," World Wide Web Consortium. http://www.w3.org/TR/2000/REC-xml-20001006