TM

LiveMove Pro
Director’s Cut

TM
AiLive
www.AiLive.net
Machine Learning for Games
 LiveMove Pro solves the technical problem of motion recognition.

LiveMove Pro lets you concentrate on creating instead of engineering.

The path from a great idea to its realization in a successful game just got a lot shorter.

c 2007 AiLive Inc.

All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means
without the written consent of AiLive Inc.

AiLive, LiveMove and LiveCombat are either registered trademarks or trademarks of AiLive Inc. in the
United States and/or other countries. Other product and company names mentioned herein may be the
trademarks of their respective owners.
 5

Contents
1 What is LiveMove Pro? 6
1.1 Basic capabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Major new features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 The components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Why Use LiveMove Pro? 8

2.1 The gameplay advantage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 The development advantage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Game design and production with LiveMove Pro 10

3.1 Key concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Game production with LiveMove . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.3 The (current) limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Licensing 14

5 Conclusion 15
6 1 WHAT IS LIVEMOVE PRO?

1 What is LiveMove Pro?

LiveMove Pro is a major new release of the LiveMove brand of products. LiveMove Pro replaces LiveMove as
the Gold Standard for motion recognition software, and can do everything LiveMove did, but better and faster.
LiveMove Pro provides major new functionality on top of the basic LiveMove capabilities.
Movies of LiveMove Pro can be found at www.ailive.net/liveMovePro.html.

1.1 Basic capabilities

With LiveMove, developers built motion recognizers by showing examples instead of coding.

• LiveMove games can recognize any motions performed with a motion sensing device.
• LiveMove automatically recognizes the motions you define.
• The recognition is accurate across different users.

1.2 Major new features

LiveMove Pro offers significant new functionality.

• Zero-lag recognition
Accurate recognition, or “classification” is now available within just a few frames. More precisely,
we can tell two moves apart within just a few frames after the motion actually diverges. For example,
when recognizing numbers 1-9, you can tell when a player is doing a “2” or a “3” instead of the other
numbers within 1/10th of a second. You will be able to tell them apart completely within frames after
the second loop on the “3” should start. Our Zero-lag recognition is so responsive that the game will
often know what you are doing as soon as you start moving!
1.2 Major new features 7

The LiveMove Pro movie uses this ability to fire animations almost as soon as the player
starts moving the Wii controller.

An experimental version of this capability was introduced in LiveMove version 1.1. It was unstable and
you had no way of knowing how good the classification was. The version included in LiveMove Pro is
complete and powerful: classification are available at any time, accurate answers come much more
quickly, and we tell you how accurate that answer will be.

• No more buttons!
LiveMove Pro does not need to be told when a player’s motion starts and ends. Instead, LiveMove Pro can
tell that to the game. So with a LiveMove Pro-enabled game, the player can simply pick up the Wii Re-
mote and start moving – without the need to press buttons.

• Player synchronization
LiveMove Pro provides several tools to help synchronize the player’s physical motion with the on-
screen action.

– Motion progress.
LiveMove Pro provides a measure of ‘how far’ the player is through the performance of the move.
So the player can drive in-game animations. For example, a ‘turn’ or ‘twisting’ move can control
the rotation of a door handle.

The first clip of the movie shows how motion progress helps closely link anima-
tions to physical motions. The player make shapes in the air, and we then draw
pre-rendered animations in time with the player. Animation speed is dynamically
changed to match the player’s drawing speed. The volleyball clip shows a another
use of this capability to give the player the sense of complete control over their
in-game volleyball stroke.
– Motion score.
LiveMove Pro can tell you how well, or how poorly a move was performed by the player. For
example you will be able to detect when a novice player is performing an uppercut, but just
barely, as compared to the experienced player that gets it nearly perfect every time.
– Move tree.
The LiveMove Pro “move tree” is a new asset that is a very powerful development time and run-
time design aide. At development time, it will tell you what kinds of animation will synchronize
with your motion controls to give immediate and correct feedback to the player. At runtime, it
tells you when it is safe to start animations while the player is performing a move. Move Trees
are “made to order” per classifier, and you can create more than one if you wish!. For example,
build a low accuracy tree to decide when to quickly fire off general-purpose animations, and build
a high-accuracy tree to determine when to commit to your more distinctive animations.

• Ease of Use
We’ve streamlined the data collection process and made it much easier to collect examples from differ-
ent people. Classifiers can be tested immediately both on the Wii and in lmMaker . LiveMove Pro is
distributed with both statically and dynamically linked runtime libraries, to meet a wider range of
needs. Debugging is enhanced. For example, the motions that are being used to build your classi-
fiers in lmMaker can be visualized with just a mouse click. LiveMove Pro is over 50% faster than
LiveMove on similar tasks. And there is much, much more.
8 2 WHY USE LIVEMOVE PRO?

1.3 The components

LiveMove Pro has foure major components:

• libLM, a run-time library optimized for the Wii that recognizes motions in real-time.
The libLM code interface is small and conceptually simple. To integrate libLM into your game
requires only a small amount of coding, at most 1 programmer for a day or so.

• lmMaker, a development-time application for the PC that builds classifiers from motion examples.
The lmMaker tool provides a Graphical User Interface that lets anyone collect examples and build
classifiers. No coding skills are needed. For example, game designers can directly author moves.

• An extensive User Manual with introductory tutorial, some sample code, and some additional applica-
tions.

• AiLive support!

2 Why Use LiveMove Pro?

Motion sensitive controllers, such as Nintendo’s Wii Remote, make it possible to refresh existing franchises
and create new and exciting entertainment experiences.
LiveMove Pro makes it possible to take advantage of all of the novelty of these devices, without suffering
through the significant technical and time-related hurdles. It doesn’t matter if you’re a large studio or a small
one. The limiting factor for motion control on the Wii is now driven by the creativity of individuals.
Anyone can begin exploring new motion controls in minutes. Literally.

2.1 The gameplay advantage

LiveMove Pro can recognize moves on the Wii Remote, Nunchuk or both together. It can recognize simple or
comlex moves, fast or slow, many or few.. LiveMove Pro cracks open the entire motion control design space
for you.

• Faster iterative cycles make for better games

LiveMove Pro ONLY: Simply put, there is no better fast prototyping tool for motion design.

• Immerse the player

– LiveMove Pro ONLY: With zero-lag response, you can give your players immediate feedback.
– LiveMove Pro ONLY: With move trees, your players get the most relevant feedback possible.
– LiveMove Pro ONLY: With motion progress, speed or slow your animations to match your player.
– LiveMove Pro ONLY: With no buttons, your players need not remember they are holding a con-
troller instead of an actual racket, or sword, or steering wheel.

In a volleyball game with 50 different moves, your player might prepare for a spike as soon as the ball
crosses the net...
She can ..wait .. wait ... Wait.. SMASH!
Or, she could just lightly spike it crosscourt.
And so can your game, and in lock-step with the player.
2.2 The development advantage 9

• Captivate players that want more

Use the complete design flexibility for motion control to differentiate your games!
Don’t just give your players whacks and wiggles, when they could just as easily have letters and num-
bers, spinning drop shots, secret magic spells, karate forms, Hawaiian dancing, – or just about anything.
LiveMove Pro ONLY: Unmatched control over recognition speed and accuracy is at your disposal.
LiveMove Pro can recognize any motion performed on the Wii Remote, and can support dozens of dif-
ferent moves in the same recognizer from the simple to the complex, the short to the long.
Imagine a Harry Houdini simulator, in which the game recognizes loosening different kinds of knots,
removing hoods and handcuffs, flicking open catches, picking locks, and finally pushing open any chest
you happen to be trapped in.

• Attract players normally intimidated by joysticks. Keep players that need more intimidation
Get celebrities like golf professionals, boxing champions, tennis stars, etc. to create motion data for
LiveMove Pro. Get regular folks like your engineers or your focus groups to create the rest of your the
data.
Combine the two, and you have a game that can be fun for the young and the old alike, and can provide
a continuous challenge to those who want to perform like their favorite star!
LiveMove Pro ONLY: Use motion scoring to reward players as they move along the path to performing
like a celebrity.

• Help players understand what they are doing wrong

Create moves that are common mistakes, and when they fire, give feedback accordingly.
Even when the right move is being recognized properly, you can still coach the player.
LiveMove Pro ONLY: Use motion scoring to realize when your player could use some help, and motion
progress to understand what type of help he could use. For example, was the player’s rendition of the
“create flowers” spell too fast to be effective?
Compare a boxing game that coaches players to uppercut like the Heavyweight Champion to one that
doesn’t. Which game will your consumers prefer?

• Invigorate your Wii ports

Porting an existing title to the Wii? LiveMove Pro makes it simple to add Wii-specific motion controls
to your game. Refresh an existing franchise with new motion recognition features.

2.2 The development advantage

LiveMove Pro solves the hard technical problems you face when interpreting the motion sensor data of the
Wii controllers. LiveMove Pro takes a machine learning approach to motion recognition. Motion recognizers
are learned, not coded, which has many advantages.

• Lower development costs, better results.

Writing code to robustly recognize a set of moves in-house could easily employ one or more developers
full-time over the duration of a project. Without LiveMove Pro , evaluating new ideas can takes weeks,
assuming you can get over the fear of changing any of that magic code. One of the many developers
who’ve tried LiveMove tells it best:

”The code we currently have is messy, full of magic numbers, ridiculously long, and VERY
hard to understand and modify. Changing even the slightest thing could break everything in
unexpected ways.”
10 3 GAME DESIGN AND PRODUCTION WITH LIVEMOVE PRO

The design cycle for initial motion control exploration is so much simpler and quicker with LiveMove Pro,
it is revolutionary. You’ll be able to explore a much wider set of motion control ideas, discarding bad
ones earlier and finding great ones quicker.

• Eliminate the technical risk of motion control

Want to make an action game with dozens of special moves – all of which can be performed on the
Wii Remote and the Nunchuk? Or a sports game with hundreds of victory celebrations? You can. Your
designs can be as ambitious as you want.
LiveMove Pro can accurately recognize any moves that can be performed on the Wii Remote, including
any variations of those moves. Recognition is robust across all users.
Unsure which moves feel right together? Once you’ve collected examples, you can try out different
subsets of moves with LiveMove Pro within minutes.
LiveMove Pro tools even let you test your ideas before any coding for a game begins.
Building a motion recognizer for your game will never be on the critical path.

• Fast, scalable and efficient.

LiveMove Pro ONLY: is 50% faster than LiveMove on the same tasks. With 5% of your CPU on a given
frame, LiveMove Pro can drive recognition for dozens of different motions in your game with 4 players
simultaneously classifying on 4 Wii Remotes and 4 Nunchuk controllers at 60 frames per second. With
one player and one controller, there is practically no limit. Total memory use is typically below 700k.
If you need fewer moves, your classifiers will shrink accordingly.

• Build reusable assets along with games

LiveMove Pro is data-driven, and the assets it generates are data-driven as well. So for example, adding
a new move to a game is done by adding motion examples and rebuilding a classifier. So you can
refresh titles by downloading new motion recognizers.
LiveMove Pro generates data assets, such as motion examples, project files, motion recognizers and
move trees. These assets are independent from your game code, and can be directly re-used in sequels,
or other games with similar motion control schemes.

• Quickly see the impact of motion design on animation requirements

LiveMove Pro ONLY: The move tree gives you the ability to frontload even more of your motion design
work. Now you can see exactly how different collections of moves will impact the animation work on
your plate.
A move tree will tell you the complete set of animations you may need to show off the moves. It tells
you which animations are likely to blend into which others. And it will tell you about where in the
move you should expect that to happen.
For example, for the numbers 0-4, one piece of information it will tell you is that you need a “2,3”
animation, as well as a “2” and a “3”. Furthermore, you will may need to blend “2,3” into “2” roughly
55% of the way through the move.

We can’t wait to see how you will use LiveMove Pro in your games!

3 Game design and production with LiveMove Pro

3.1 Key concepts
For starters, what are motions, motion sets, moves, and classifiers?
A motion is any movement of the Wii Remote (normally half a second to five seconds). A motion set is a
collection of motions of the same type. But they are not identical. See Figure 1. In this this example the
3.1 Key concepts 11

motion is a counter-clockwise full circle with the Wii Remote pointing up. Practically speaking, a move is a
player-level concept, like a sword thrust or cross-court forehand. Finally a classifier is synonymous with a
motion recognizer. Feed it a motion and it tells you what that move is.

Figure 1: This motion set is a collection of four motions counter-clockwise circles.

circle

Figure 2: A move is a labeled motion set. In this example, the move is ‘circle’. You collect example motions
from different people to define a single move.

Classifier

What is it? It’s a circle!

Runtime
Library

Wii,
NDEV

Figure 3: You define moves and LiveMove Pro automatically builds a classifier to recognize them.
LiveMove Pro classifiers recognize motions. They are motion recognizers.

To put these phrases into context, imagine you are designing a ‘Zombie Cowboy’ game and you wish to add
a cowboy lasso move.
You decide that the player must whirl the rope 3 times in the air before snapping to throw forwards (to capture
fleeing zombies). Simply collect examples of people performing this motion. Collect the examples together
12 3 GAME DESIGN AND PRODUCTION WITH LIVEMOVE PRO

Figure 4: Wield a rope as a weapon in the new motion sensitive game ‘Zombie Cowboy’

into a motion set. Label the examples ‘lasso’. Then LiveMove Pro builds a classifier to recognize the move
‘lasso’.
You decide to include a second type of lasso move. The player must whirl the rope 2 times pointing at the
ground before snapping to throw backwards (to snag a cactus for a quick turn while riding a horse). Again,
simply collect examples of this motion, label the examples ‘lasso backwards’ and LiveMove Pro does the rest.
The same classifier will easily distinguish between the two types of lasso move.

3.2 Game production with LiveMove

"Normal"
Design Code Test

LiveMove Pro Collect Build

Design Examples Classifier Test

Figure 5: Motion design cycle without (on the top), and with (on the bottom) LiveMove Pro (the highly
iterative nature of this cycle is not shown).

LiveMove Pro will change how your game production works, at least with respect to creating motion control.

Design

Up front design with LiveMove Pro is different. This is primarily because you can start trying out ideas in
LiveMove Pro immediately, before any game code has been written. You can start cutting away at game
designs based on practical experience before the first line of code has been checked in.
Once there is game code, you will need to integrate the libLM run-time library before being able to test your
classifiers in game. Classifying player motions is a simple matter of:
3.2 Game production with LiveMove 13

• Loading a classifier object into console memory.

• Passing accelerometer data from Nintendo’s WPAD or KPAD libraries to the LiveMove Pro interface.
• Asking LiveMove Pro for a classification label.

Lacing up these runtime calls is the work of about a day.

Collecting Examples

You can build a classifier to recognize the numbers 1,2,3,4 drawn in the air in under 5 minutes without writing
any code by simply providing a few of your own examples to lmMaker.
More examples, from different people, are usually required for production-quality recognition. Consider the
‘Zombie Cowboy’ game that has 30 different moves, including “lasso”, “whip”, “any square”, and “chicken
dance” (poor zombie cowboy got confused with square dance).
How long would that take? This is a very rough estimate, of course. In our experience, once you become
familiar with LiveMove Pro,

You can build a prototype classifier that recognizes 30 different moves in one day.

Providing examples is a very straightforward and quick process. For example, giving 25 different examples of
a lateral pass in football will just take a few minutes. For ’Zombie Cowboy’, we’d expect collection sessions
of about 40 minutes with each person who provides motion examples.
General moves, such as “any square”, are built from examples of all kinds of squares (large, small, pointing
at floor or ceiling, performed clockwise or anti-clockwise etc.) In contrast, specific moves, such as “small
clockwise square pointing at the floor”, do not require varied examples. Normally more general moves require
more examples.
A very specific move, which is intentionally difficult to perform, such as a pro-tennis serve, requires very few
examples – perhaps as few as 5 to 10.
So the amount of data you need to collect depends on the number and generality of the moves in your game.
You’d generally need to collect data from 5-10 people before reaching a classifier that is robust enough to test
in your game.
Once you have collected this data, it is a permanent asset. You need not collect it again.

Building Classifiers

LiveMove Pro does all the legwork here. lmMaker ’s classifier construction process is very simple. Once
you understand the interface, the actual construction step takes just a few minutes.
The project files, classifiers and move trees you generate in this step are data-driven assets that you could use
in other games.

Testing

LiveMove Pro does not have an impact on your testing. You could, of course, turn your testers into data
generation machines, in which case they will be generating assets at the same time as they are testing your
game for you.

Finalizing

LiveMove Pro is a fantastic fast prototyping tool. It is also a complete motion design aid that can take you all
the way through to shipping a game. Once you are ready to finalize your classifier for production, you will
14 4 LICENSING

find LiveMove Pro has a complete set of tools that will help make it as easy as possible. These tools include
a wide range of options for getting information on the current intermediate classification (is it “safe” to use
the it, what moves might it be confused with, etc), the current motion of a player, and even runtime masking
of certain moves from the classifier.
LiveMove Pro also supports a process called tuning. Tuning adapts the classifier you built at development
time to a particular game player in their home.
Game players come in all shapes and sizes with different movement styles. So even with play testing there
remains the chance that an individual will experience initial difficulty performing a move. Tuning guarantees
that a classifier will work “in the wild” for players that move differently from those seen during game testing.
If you add tuning to your game then some design decisions need to be made. For example, you may decide
to present the tuning process as an explicit training step (e.g., the Zombie Cowboy introduces the moves to
the player with guided feedback) rather than hide the tuning process (e.g., the game collects examples from
players without them knowing).

3.3 The (current) limitations

But there are some limitations that you should keep in mind when designing games that use LiveMove Pro.

• LiveMove Pro operates on motion sensor data alone. Think of this as the force applied to the controller
over time.
This is not locational data. For example, LiveMove Pro ignores the player’s height, orientation and
distance from the TV. LiveMove Pro does not know where your hand actually is in real physical space,
at any point in time.
• LiveMove Pro does not directly communicate to the player how to perform a move.
For example, in ‘Zombie Cowboy’ the developer must display an animation of the protagonist lassoing
a cactus before players try performing the move.
• Animations and moves must be pre-defined by the developer.
The player can not yet create moves of their own using LiveMove Pro .
• LiveMove Pro can sometimes fail to distinguish between visually distinct moves that generate identical
motion sensor data.
Wii motion sensor data has ‘blind spots’ in which the data looks the same, but the player’s movement
is very different. In these cases, moves need to be redefined to avoid the problem.
• The duration of a move is capped to about 5 seconds.
But of course, moves may be chained.

We are always eager to hear new feature requests from developers.

4 Licensing
LiveMove Pro’s licensing is much simpler than that of LiveMove version 1.
LiveMove development licenses were $2500 per developer/PC pair. If you had 10 active developers, or 10 PCs
with LiveMove installed, you needed 10 development licenses. LiveMove also had a separate runtime license.
Each game that ships with the LiveMove runtime libraries cost $10,000.
LiveMove Pro has done away with all that. There is just one license, and it is site-wide for a given Studio.
There is no extra cost for shipping games that use LiveMove Pro’s runtime libraries and classifiers. And that’s
it. See the AiLive website, or mail [email protected] for details.
 15

5 Conclusion
LiveMove Pro lets you focus directly on creative work without the burden of difficult coding. You can quickly
realize the potential of the motion-sensitive control. The only limitation is your imagination.
With LiveMove Pro you will create motion recognizers at a pace that can run circles around the modern game
development cycle of today. With LiveMove Pro, anyone on the game development team can create motion
recognizers by simply providing example motions. From rapid prototyping, all the way through to building
fast and reliable motion recognizers for the shipped game, LiveMove Pro provides the complete solution.