3D Animation Coding: its History and Framework

Euee S. Jang
Samsung AIT
P. 0. Box 111
Suwon, Kyunglu 440-600, KOREA
[email protected]

ABSTRACT in the history of multimedia standardization. MPEG-4 is the

first standard that handles not only frames of video, but also
In this paper, a framework on 3D animation coding is proposed. natural and synthetic objects!
The current MPEG-4 standards provide means to compress
synthetic objects efficiently. The SNHC tools (e.g., , face and MPEG-4 visual includes not only the conventional video object,
body animation (FBA), still texture coding, and 3D mesh but also still texture object, face and body animation (FBA)
coding tools), combined with BIFS, can represent interactive object, and 3D mesh object [lo]. FBA and 3D mesh coding
3D scenes efficiently. Although MPEG-4 SNHC and BIFS are tools are to handle graphics objects such as animation of virtual
designed to cope with VRML 2.0 and to yield a reasonably high human face and body as well as compression of 3D polygonal
compression, 3D animation is not supported efficiently. A mesh.
framework to support 3D animation coding within the MPEG-4 In a similar way to VRML, MPEG-4 can represent a virtual
structure is proposed. world. MPEG-4 has a tool called binary format for scene (BlFS)
which is an extension to VRML 2.0. MPEG-4 is only better in
1. INTRODUCTION that it provides compression means to VRML. For example, a
1MB VRML 3D object (with IndexedFaceSet) can be
Computer graphics, started from the 195Os, is one major source compressed to 20-KB file through 3D mesh compression.
of multimedia at the end of the 20th century. Movie industries
MPEG-4 opened up a new way to converge computer graphics
have used computer graphics from the 1970s. At the present
and audio-visual signal processing. As far as 3D graphics and
time, i t is hard to find a movie that contains no computer
animation are concerned, MPEG-4 is currently bounded by
graphics at all. Breathtaking special effects in the blockbuster
VRML2.0, which means that there is enough room for further
movies are often from computer graphics.
improvements.
Despite the outburst of computer graphics in multimedia, the
Nevertheless, MPEG-4 remains a better framework for the
ordinary viewer was only allowed to experience computer
standardization of 3D animation. This is grounded on the
graphics in a predefined world: a sequence of video frames.
assumption that efficient compression has been a stronghold of
Generated in a proprietary 3D format, 3D objects had to be
MPEG, and that network bandwidth is going to be a critical
captured in frames for application. For example, Pixar’s movie
issue while hardware performance in terminallclient side is
A Bug’s Life was composed of 138,000 frames and stored in 3
accelerating according to the Moore’s law.
TB [3]. Yet, it is no more than a conventional 2D movie in that
i t provides neither 3D navigation nor interactivity. In the following section, I will provide some background on the
history of computer animation. The structure and framework of
As a file format to represent the as-is 3D world, the Virtual
MPEG-4 are explained in section 3. In section 4, the proposed
Reality Modeling Language (VRML) has been provided as an
framework is detailed. Finally, I summarize the paper in the
International Standard from IS0 in 1997 [7]. Even today,
conclusion.
however, we still have many different 3D file formats. VRML
does not seem to be as successful as it was expected. In fact,
VRML was a good start, but the entire computer graphics and 2. Graphics File Formats and VRML
animation are too heavy to be covered by it, yet. 2.1 3D Graphics File Formats
Independently of computer graphics, audio, image, and video as
part of signal processing have evolved into multimedia. Different computer platforms resulted in different file formats in
Especially, the introduction of multimedia coding standards like 3D graphics and animation. Although tens of different file
MPEG and JPEG has been a great stimulus to the quantum leap formats are available, none is successfully playing a role as an
of multimedia communication era. industry standard. About 8 different file formats are detailed in
[l]: 3DS, COB, DXF, NFF-WTK, OBJ, POV, RAW, and
MPEG has been successful in introducing MPEG-1 and MPEG- VRML. In this paper, I concentrate on VRML. Each file format
2 into the commercial fields. In the next decade, it is expected has its own pros and cons. VRML, among them, is the first open
that the digital TV will replace the analog TV that dominated international standard of its kind.
over the last 70 years. MPEG-4 is going to be another landmark

0-7803-6536-4/00/$10.00 (c) 2000 IEEE 1119

 2.2 VRML[4][7][11] bit rate (c64 Kbps) transmission [12]. Then, the scope was
expanded to object based coding. The major features of MPEG-
The standardization of VRML started from the World Wide 4 are following:
Web conference in Geneve in 1994. And the second version of LARGE
VRML was finalized as international standard by IS0 in 1997
and it has the following characteristics:
0 Scene Graph in which each entity is a node. The scene
graph contains the 3D world. Group nodes are
containers for children nodes. Leaf nodes describe the
components that make up a 3-D world.
0 Interaction with a 3D world is possible by moving
around its viewpoints or by navigation. Sensors will
detect the user interaction to initiate behaviors or
animation.
SMALL
0 Animation is supported in VRML 2.0. For simple
linear animation, key-frame animation is suitable. For
more complex animation, scripting is also needed. LOW HIGH

2.3 Obstacles of 3D Animation in VRML 2.0 Figure 1. Relationship between network load vs.
Notwithstanding the inherent strong capabilities-web-based, 3- terminal computing power.
0, easy-to-use, VRML 2.0 is reported to have some noticeable Object-based Processing. MPEG-4 handles video or
weak points [ 111: audio signals as frames. And the notion of object is
More than one browser required extended to synthetic objects.
Unacceptable file download time Composition. Now the decoder can compose different
Slow rendering of complex scenes audio, video, or synthetic objects at its convenience. In
0 Time to author complex shapes and behaviors other words, MPEG-4 becomes an integrated
0 Wide color variation across platforms framework of multimedia
0 Awkward 3-D navigation Compression. After all, MPEG-4 is a coding standard.
Among the listed weak points, each item seems trivial except It provides efficient compression on audio and visual
objects.
Unacceptable file download time and Slow rendering of complex
scenes. The former determines the critical service time, whereas Regarding the synthetic objects the tools are provided as shown
the latter determines the quality of service. In other words, two in Table 1. From the table, you can see that most MPEG-4 tools
major issues in 3D graphics and animation lie in transmission provide not only the functionality, but also the compression
time and computing complexity. efficiency.

What was possible in highly sophisticated super computers 3.1 BIFS[5][6]

about 20 years ago is now possible with PCs at home. As the
computer power increased, so did the complexity and realism of MPEG-4 Binary Format for Scene (BIFS) is the MPEG-4 scene
3D animation [9]. Unlike the speedy evolution of hardware description protocol. It was designed to compose MPEG-4
performance, the user data rate does not seem to obey the objects, to describe interaction with MPEG-4 objects, and to
Moore’s law. Hence, if a 3D animation file is to be transmitted, animate MPEG-4 objects. BIFS was designed to be compliant to
a major bottle neck is going to be the channel bandwidth. As can VRML 2.0. Furthermore it provides superior compression
be seen from Fig. 1, the increasing computing power at the efficiency to VRML 2.0.
terminal side can lower the burden of heavy transmission. It is
possible with the assumption that the terminal will be equipped BlFS is composed of 4 elements:
with high computing power. Nodes: nodes represent the attributes of media objects,
the structure of the scene graph which contains these
Compact representation and efficient compression of 3D media objects, the animation of the content, or the
graphics and animation will help not only to avoid bandwidth
interactivity with the individual objects.
shortage, but also to fully utilize the improved decoder
Binary Syntax: The binary syntax is provided for the
computing power. MPEG-4 SNHC and BIFS provide some
compression of the scenes.
efficient means of compression for 3D graphics and animation,
BIFS-Update: A protocol to stream scene changes, to
which is detailed in the next section.
insert new scenes or objects, to delete objects, etc.
BIFS-Anim: A protocol to stream animation of the
3. MPEG-4 media objects.
MPEG-4 started in 1993 when MPEG-2, a digital TV standard, BIFS has its advantages in integration of 2D/3D synthetic media
was being finalized. MPEG-4 was originally started to cover low together, streaming environment, and compression. 3D

0-7803-6536-4/00/$10.00 (c) 2000 IEEE 1120

 Incremental rendering: It allows the user to view the
decoded 3D model while bits are transmitted, not after
the entire bit stream is downloaded.
Error resilient coding:This is useful in an error prone
environment when some packets get lost during
transmission. Even with losses of packets the decoder
is still able to reconstruct the 3D model from the intact
packets.
Hierarchical 3D mesh coding: It provides the quality
Feature Related MPEG-4 Advantage scalability to the user, such that level of details (LOD)
Standards can be transmitted hierarchically.
1 I I I
Scene 3DMC provides the competitive compression ratio of 40-50: 1
description Efficiency over the IndexedFaceSet node in VRML. 3DMC was tried for
3D animation coding in [15]. It should be noted that 3DMC is
Human
designed for static model compression, not for 3D animation.
Efficiency
I I I I

2D graphics SystemsIBIFS 3.5 Why MPEG-4?

3D animation is an integration of multimedia comprised of
speech, sound, image, video, 3D attributes, and animation. In
Efficiency this regard, MPEG-4 can play a very important role in 3D
AudiofITS animation. MPEG-4 provides audio-visual coding with 2D/3D
media integration. And it can be a good basis that 3D animation
coding can launch.
Sound
Synthesis red Audio Functionality 4. Framework of 3D Animation Coding
4.1 Objective
3.2 SNHC
The objective of 3D animation coding is to provide necessary
The major activities in SNHC have been still texture coding, 2D means to enable animation of 3D objects on top of the existing
mesh coding, text-to-speech ('ITS), structured audio, face and MPEG-4 tools. MPEG-4 SNHC subgroup started discussion of
body animation (FBA) and 3D mesh coding (3DMC). An 3D animation coding in the beginning of 1999. In the group, the
abundance of details on SNHC can be found in [13]. In the details on 3D animation are being discussed. And the contents
following, FBA and 3DMC are described more in detail. of the framework in this paper are reflected from the discussion.

3.3 FBA 4.2 System Structure

A 3D animation coding system can be divided into the encoder
Face animation coding took place first in SNHC, then body and the decoder. The encoder, as shown in Fig. 2, will generate
animation coding. Both face and body animation is parametric the bit stream (or file) that contains the following:
coding. Face animation combined with 'ITS was intended to Compressed model data. Compact representation and
replace low bit rate audiohide0 coding. Body animation added a high compression of 3D models with different
way to virtual human animation [14]. FBA is done through modalities (polygonal mesh, curved surfaces,
parameters called: Face/Body definition parameters (FDPIBDP) subdivision surfaces, etc.). MPEG-4 supports 3D mesh
and FacelBody animation parameters (FAPIBAP). Coding of the coding. Compact representation and compression on
parameters and other details can be found in [14]. Nevertheless, curved surfaces and subdivision surfaces are also
i t is noted that FBA was designed for low bit rate applications, necessary.
not necessarily for high quality (hence complex) animation. Compressed animation data. Key-frame animation
with (inverse) kinematics or free form deformation
3.4 3DMC[14] shall be supported through efficient compression of
animation parameters.
In VRML, an IndexedFaceSet node represents a 3D polygonal Compressed attributes. Lighting, camera motions, and
model. 3D mesh coding (3DMC) provides a nice compression texture mapping and other special effects can be
on the IndexedFaceSet node. A 3D model (represented by represented and compressed for fast and reasonably
triangular meshes) is comprised of connectivity, geometry, and high quality rendering.
property (like normals, color, texture coordinates, etc) Compressed video/texture. An integration of existing
information. 3DMC can support polygonalltriangular, imagehdeo coding technologies is anticipated.
manifoldhon-manifold models. On top of that, it provides
Compressed Sound. Synthetic speech, audio, and 3D
following functionalities:
sound effect will be supported by the existing standards.

0-7803-6536-4/00/$10.00 (c) 2000 IEEE 1121

 considered, but worth the consideration. The bottom line is that
the current animation data coding may also need a comparable
13D ANIMATION 1 compression rate to 3D mesh coding to envisage the commercial
realities of animation coding.
0
m ’: 5. SUMMARY
z m
0 In this paper, I proposed a framework of 3D animation coding
0
that will nicely integrate what we have achieved VRML and
0 0 MPEG-4 as a common format to carry 3D graphics and
0 0 animation. Interaction and streaming are still the issues to be
m m solved in the future. I summarized the technologies critically
R
related to the two issues.
R
6. REFERENCES
[l] Rule, K . 3 0 Graphics File Formats: a Programmer’s
__x
Reference. Addison-Wesley Developers Press, Reading,
MA, 1996.
Figure 2. 3D animation file format
[2] Requirements Group “MPEG-4 Requirements, version 1 1 ”,
From Fig.2, you can see that the 3D animation file format is a ISO/EC JTCl/SC29/WGll N2992, Melbourne, 1999.
multimedia format that contains most aspects of multimedia [3] Hamit, F. “Bug’s Life Management: Pixar’s Massive Asset
from audiolvideo to graphics. MPEG-4 and other multimedia Storage Solution for Animation Features”. Advanced
coding standards provide abundance tools in coding audio and Imaging, February 1999.
video. The remaining items for high compression are detailed in [4] Carry, R. and Bell, G. The Annotated VRML 2.0 Reference
the following sections. Manual. Addison-Wesley Developers Press, Reading, MA,
1997.
While this paper was being prepared, MPEG has issued a call [5] Signes, J. “MPEG-4 Binary Format for Scene: BIFS”. in
for proposals for generic animation framework as a new work Vancouver MPEG-4 Seminar. Vancouver, July 1999.
item. [I81 According to the call, it is expected to finish the [6] Dreier, R. and Rump, N. “MPEG Systems FAQ, Version
standardization by 2002. 7.0a”. ISO/IEC JTCI/SC29/WGI 1 N2527, Oct, 1998.
[7] VRML 2.0 specification, ISO/EC 14772-1:1997,
4.3 Compact representation and compression of www.
3D model modality [8] Signes J. “BIFS: Combining MPEG-4 media to build rich
multimedia services”.
3D models can be designed in many different ways [16][17]. [9] Greenberg, D. P. “A Framework for Realistic Image
Regardless of the origination of models, you have three popular Synthesis”. Communications of the ACM, Vo1.42, No.8,
ways to represent generic 3D models: Polygonal meshes, curved August 1999.
surface, and subdivision surfaces. [IO] Koenen, R. “MPEG-4: Multimedia for Our Time”. IEEE
The representation of a 3D model through 3D polygonal meshes Spectrum, February 1999.
is already supported in MPEG-4. In curved surfaces, we have 113 Burton, N. D., Kilgour, A.C., and Taylor H. “Chapter. 15 A
NURBS representation as an example. The curved surfaces are Case Study in the Use of VRML 2.0 for Marketing a
effective where a smooth curvature should be represented. If Product”. in Virtual Worlds on the Internet. IEEE
represented by curved surfaces, one can manipulate the accuracy Computer Society, 1988.
of tessellation more easily. 121 Chiariglione, L. “MPEG-4, why use it?”.
http ://drogo.csel t .i t/ufv/leonardo/paper/mpeg-4/mpeg-4. h t m.
Subdivision surfaces provides a way to combine polygonal 131 Special Issue on Synthetic/Natural Hybrid Coding, IEEE
models and curved surfaces in a hierarchical way. This method Trans. on CSVT, Vol. 9, No. 2, March 1999.
can solve many problems that polygonal meshes and curved [14] Jang, E., Nakaya, Y., and Visual FDAM Editors.
surfaces pose on their own. This advanced technology has been “Preliminary Version of ISO/IEC 14496-2 FDAMI”.
gven less attention in compression, since it was majorly used in ISO/IEC JTCl/SC29/WGI 1 M5311, Maui, HI, 1999.
the authoring stage. [15] Lengyel J. “Compression of Time-dependent Geometry”. in
4.4 Compact representation and compression of Symposium on Interactive 3 0 Graphics, Atlanta, GA, 1999.
[16] Kerlow, I. The Art of 3 - 0 Computer Graphics and Imaging,
animation data John Wiley & Sons, 1996.
If a half of an animation file is occupied by model descriptions, [I71 Fitzmaurice, G. W., Balakrishnan, R., and Kurtenbach, G.
the other half would be animation data that make the models be “Sampling, Synthesis, and Input Devices”,
animated. In VRML and MPEG-4, we have interpolators, BIFS Communications of the ACM, Vol. 42, No. 8, August, 1999.
update, and BIFS Anim. The aforementioned techniques are [18] The MPEG committee, Call for Proposals for a generic
rather concentrated on processing key-frame animations. A animation framework of synthetic objects, ISO/EC
simple motion trajectory type of representation is not yet JTCl/SC29/WGll N3341, March 2000.

0-7803-6536-4/00/$10.00 (c) 2000 IEEE 1122