Appendix A.

Digital Compositing
Software: Tools and Features

The purpose of the list given in this appendix is twofold. First, if you are al‐
ready working with compositing software, it may help you to identify and un‐
derstand tools within that package which you were unfamiliar with. We have
tried to list features that are common to a variety of different packages, and to
give a brief description of each. A more detailed description of certain features
and operators can be found throughout the body of this book.

Second, this list can be used as a guide for evaluating and comparing software
that you have never used but possibly intend to purchase or recommend.
Given the wide and expanding variety of digital compositing tools that are
available, it can be difficult to make a reasonable comparison without having
a common baseline to reference. Of course, not all compositing packages will
have all the features listed here. Many packages may have additional features
that are not listed here. Please understand that the presence or absence of cer‐
tain features does not necessarily determine whether the package will be use‐
ful. Only by evaluating your specific needs can you determine whether a par‐
ticular package is suitable to the task. You may even find that a combination of
several different tools will be needed to cover all the possible scenarios that
you need to address. In general, you shouldn’t hesitate to pass over a poorly
designed “all-in-one” package in favor of a group of well-designed packages
that can be used in conjunction with one another effectively.

This appendix will try not only to list a number of different potential features,
but also to give an idea about what a reasonably good implementation of that
feature might include. Don’t use this as a “yes or no” checklist, but instead try
to examine the features of the package in question to understand how com‐
plete the implementation might be.

Finally, remember the importance of the overall design of the package’s user
interface. This is probably the most difficult component of a package to evalu‐
ate, since it requires spending time with the product to understand the partic‐
ular paradigm. It is something that cannot be dealt with in a simple features
list, but absolutely should not be ignored. Chapter 9 goes into greater detail
about certain features and methodologies that are relevant to this topic.

The categorization of the features in this list is rather arbitrary, as is the nam‐
ing of the tools themselves. Some of the features described below are named
fairly consistently throughout the industry but others vary wildly. Even (per‐
haps especially) the most common terms, such as “Brightness,” will have a dif‐
ferent underlying algorithm depending on the software used. If you are using
this list to help evaluate the functionality of a particular piece of software, you
may need to explore a little bit before you can accurately determine whether,
and to what extent, the software in question supports a specific feature.

Remember too that many operators can be built as a combination of other op‐
erators. The list here consists primarily of tools that either cannot be built
from other tools or are common enough to warrant inclusion. Most composit‐
ing systems will consolidate some of the operators that are mentioned here
into larger operators that can perform multiple functions instead of providing
them as individual “atomic” tools.

This chapter is also available online at www.digitalcompositing.com as a

user-modifiable document. Feel free to contribute to the discussion on how
your favorite compositing package implements specific tools and features.

Color Corrections

Although we will often refer to operations as taking place on each pixel in an

image, in many cases the operation will actually be applied appropriately to
each channel of that pixel.

On many systems certain color corrections can be applied not only to the en‐
tire image, but can also be limited so that they affect only to certain ranges of
image brightness, generally broken into the categories of Shadows, Midtones,
and Highlights.
In addition to the specific operations listed below, a good compositing system
would ideally include the ability to use arbitrary mathematical expressions to
manipulate color via a robust expression language.

Add: Add a constant value to each pixel in an image. Also known as “Lift.”
Sometimes known as “Brightness.”

Brightness: Multiply the RGB channels of each pixel in an image by an equal,

specific amount. Also known as “Exposure.” Sometimes, however, “Brightness”
refers to adding a specific value instead of multiplying.

Clamp: Clamp the value of each pixel in an image to be between a specific

range. Values outside of this range will be forced to the limits of the new range.

Color-Space Conversions: The ability to convert images between various rep‐

resentations of color. These could include RGB, HSV, HLS, CMY, CMYK, etc.

Compress: Map a range of color values in an image into a new, smaller range.

Contrast: Modify the range/ratio of brightness values in an image. The mid‐

point of this contrast should be something that the user can control.

Divide: Divide the value of each pixel in an image by a constant value.

Expand: Map a range of color values in an image into a new, larger range.

Fade: Multiply the RGBA values of a four-channel image by an equal, specific

amount.

Gamma: Modify the apparent brightness of an image by raising or lowering

the midrange tones of the image, typically by raising the value of every pixel to
the power of 1 divided by the gamma value.

Histogram Equalization: Stretch the range of values in an image to fit as

completely as possible between a given range, usually 0–1.

HSV Modifications: Modify an image based on its hue, saturation, and value.

Hue Rotation: Rotate the hue of an image through the color spectrum.
Invert: Invert the value of each pixel in an image, producing a negative of the
image.

Look-up Table: Apply a user-defined table that specifies a particular mapping

of input pixel values to output values.

Matte Divide: Divide the RGB channels of an image by its alpha channel.

Matte Multiply: Multiply the RGB channels of an image by its alpha channel.

Multiply: Multiply the value of each pixel in an image by a constant value.

Reorder: Shuffle the order of the channels in an image.

Saturation: Modify the saturation of an image.

Set: Set any channel or set of channels to a specific constant value.

Filters

Blur: Apply some type of blurring algorithm. The user should be able to
choose different horizontal and vertical blur values.

Convolve: Convolve an image with an arbitrary filter. The size of the kernel
should be user-definable as well.

Dilate: Increase the coverage of bright areas in the image, while decreasing
the dark areas. Generally used primarily for mask manipulations.

Emboss: Create a new image that appears to have varying height based on the
luminance values in the source image.

Erode: Decrease the coverage of bright areas in the image, while increasing
the dark areas. Generally used primarily for mask manipulations.

Grain: Add simulated film grain to an image. You should either be able to
specify a variety of parameters that describe grain characteristics (such as
grain size, density, luminance and chrominance variance, etc.), to choose from
a list of different film stocks that you wish to emulate, or to sample and mimic
grain from an existing image.

Median: Replace the value of each pixel in an image by the median value of
the neighboring pixels.

Radial Blur: Blur or smear an image radially about a certain point, ideally
with control over the fall-off as a function of distance from the center point.

Sharpen: Apply some type of sharpening algorithm to enhance apparent edge

detail. The user should be able to choose different horizontal and vertical
sharpening values.

Smear: Smear an image in a certain direction by a certain amount. Often used

to simulate motion blur.

Geometric Transformations and Warps

The system should support transformation in both a 2D and a 3D space and

there should be support for a functional camera model within any 3D space.

Complex hierarchies of transformations should concatenate so as to ensure

the highest quality results.

For any transformation, portions of an image that are moved out of frame
should not be permanently cropped. Instead, the data should be available for
use by subsequent transformations. Optionally, you should also be able to
specify whether you wish any portion of the image that moves out of frame to
wrap around and reenter the frame on the opposite side.

The user should be able to choose the type of filtering (resampling) that is used
for many transformations. There are times when one may wish to choose dif‐
ferent filtering along the vertical and horizontal axes.

Finally, for any animated geometric transformation, you should be able to

choose whether or not the result is rendered with motion blur, and be able to
control the extent of the motion blur that is applied.
In addition to the specific operations listed below, a good compositing system
would ideally include the ability to use arbitrary mathematical expressions to
manipulate transformations and warps via a robust expression language.

Crop: Remove a portion of an image that lies outside a specific boundary, ei‐
ther by specifying two corners that define a rectangular region or by specify‐
ing a single point and the resolution of the desired result.

Fit: Scale an image to fit within a given resolution, usually with the option of
maintaining the aspect ratio of the original image.

Lens Distortions: Create or undo the distortion caused by a camera’s lens.

Ideally includes the ability to analyze the distortion of a lens by examining the
curvature of objects that are believed to be straight.

Pan: Reposition an image within a given frame or in terms of its distance from
camera.

Pin, Cornerpin: Distort an image by moving four arbitrary points within the
image to four new positions.

Resize, Scale: Change an image’s size by specifying a new resolution or a mul‐

tiplier for the x- and y-axes.

Rotate: Rotate an image by specifying the center of rotation and the amount
of rotation in any of the x-, y-, and z-axes.

Shear: Shear an image around a given point, specifying shear amounts for
both the x- and y-axes.

Twirl: Distort an image by twirling it around a certain point.

Warp: Warp an image based on a control grid or a set of control splines.

Flip: Mirror the image around the x-axis.

Flop: Mirror the image around the y-axis.

Image Combination
The image-combination tools are described using the same conventions that
were discussed in Chapter 5. Thus, A and B are considered the two source im‐
ages. If there is a need to specify certain channels in an image explicitly, they
will be indicated with subscripts; otherwise it is assumed that the operation
will be performed on all four channels of an image equally. All images are as‐
sumed to be four-channel premulti-plied images.

Add: Add two images together on a pixel-by-pixel basis (A + B).

Mix: Combine two images by using a weighted pixel average (MV × A) + [(1 —
MV) × B], where MV is the mixing value.

Atop: Place the foreground over the background, but only inside the back‐
ground alpha. Thus, A atop B is really the same as (A in BA) over B.

Reorder: Move specific channels between two different images.

Displace: Warp an image by using a second image to control the amount of

warp at any given location.

In: Retain the foreground only within the background’s matte. Thus, A in B is
the same as A × BA.

Luminosity: A tool to modify the matte channel of a four-channel premulti‐

plied image. Decreasing the matte channel gives the image a luminous look
when it is placed over a background. Also known as the Opacity operator.

Max: Use the maximum value of the equivalent pixel location in the two
source images.

Min: Use the minimum value of the equivalent pixel location in the two source
images.

Multiply: Multiply one image by another image (A × B).

Out: Retain the foreground only outside of the background’s matte. Thus, A
out B (or “A held out by B”) is the same as A × (1 — BA).

Opacity: See Luminosity.

Over: Place the foreground over the background using the foreground alpha.
A over B is the same as A + [B × (1 — AA)].

Screen: Invert both images, multiply them together, and then invert the result.
1 — [(1 — A) × (1 — B)].

Subtract: Subtract one image from another image (A — B). It is useful to be

able to choose to have the result of negative numbers in the resulting pixel be‐
come the absolute value of that result instead.

Under: Place the background under the foreground using the foreground al‐
pha. Thus, A under B is the same as B over A.

Xor: Retain both source images only where their mattes do not overlap. A × (1
— BA) + B × (1 − AA).

Z-Compositing: Choose the resulting pixel from either source image by com‐
paring the Z-depth values for each image.

Field Controls

Interlace: Merge two images into a new image by choosing alternating lines
from each image.

Deinterlace: Separate a single image into two new images, one containing
only the even-numbered lines from the original image and one containing
only the odd-numbered lines.

Swapfields: Switch the even- and odd-numbered lines in an image.

Matte Generation

Lumakey: Create a matte based on the luminance values in an image.

Chromakey: Create a matte based on a specific thresholded color in an image.

Difference Matting: Create a matte based on the differences between two

images.
Specialized Keying Tools: Any of a number of proprietary tools for creating a
matte for an object. These tools may rely on the fact that the object was shot in
front of a uniformly colored background or may use other techniques, such as
motion analysis, to determine the matte’s location and transparency.

Timing and Animation

Keyframing: The ability to set specific “key” frames for a set of parameters
and have the system create values for the in-between frames based on a vari‐
ety of interpolation methods.

Procedural Animation: Animation that is driven by some sort of mathemati‐

cal formula instead of specifically indicated by the user. This may be some‐
thing as simple as a standard sine wave or as complex as a physically modeled
wind simulation.

In addition to tools for dealing with the timing of animation parameters, there
also needs to be tools for retiming actual image sequences. Typically these
would include the ability to drop or duplicate frames, to average frames to‐
gether, or to create completely new frames via optical flow techniques. There
should also be the ability to perform the standard film/video conversions
known as Pullup and Pulldown as described in Chapter 10.

Image Generation

Gradient: Create an image that contains horizontal, vertical, diagonal, or radi‐

al gradients between different colors.

Paint Tools: A broad category of possible features. An ideal group of paint

tools would allow for procedural painting that can be applied to a sequence of
images instead of just the one in question.

Shape Drawing: Create an image that contains user-defined shapes created

via curves or polygons. The user should be able to control the color and trans‐
parency of the shapes and the softness of the edges.
Text: Create text with a user-specified font, color, size, orientation, and
placement.

Texture/Patterns: An open-ended category that can include just about any‐

thing that might be used as imagery within a composite. Typically this could
include things like a fractal cloud generator or a random noise image.

Particle system: Many compositing systems now include the ability to simu‐
late a limited set of natural phenomenon (smoke, dust, rain, fire) via a set of
procedural effects tools.

Tracking

Multiple Point: The ability to track several points at the same time.

Integration: Tracking information should be easily applicable to most geo‐

metric transformation tools, including Pan, Rotate, Scale, Pin, and Warp.

Subpixel Accuracy: The ability to accurately track features at greater than

whole-pixel resolutions.

Stabilization: Diminish or remove camera movement in a plate. This can ei‐

ther be feature-based (choose a specific point and stabilize the plate based on
the assumption that the point in question is unmoving) or through a more so‐
phisticated analysis of the entire frame as it changes over time.

Camera Reconstruction: Analyze a sequence of images and derive the origi‐

nal camera move in 3D space that produced these images.

Control

You should be able to control and limit how a particular operator will be ap‐
plied to an image. Various control methods include:

The ability to restrict the operation so that it is only applied to specific

channels of the image.
The ability to restrict the operation so that it is only applied to the even-
numbered or the odd-numbered scan lines of the image.
 The ability to specify a rectangular region of interest (ROI) outside of which
the effect will not be applied.
The ability to control the amount of the effect that will be applied, based on
the grayscale values in an additional (separate or integrated) matte/mask
image.
The ability to limit the effect so that it will only be applied if it changes a
given pixel by a user-specified threshold.
You should also be able to have some control over how the software uses the
resources of the system on which it is running. This includes the ability to limit
the amount of memory that the software will use, as well as having control
over what disk space is used for any temporary image storage.

Other

Finally, there are a number of additional features that most compositing pack‐
ages should support. These include the following:

Resolution Independence: The ability to translate compositing parameters

that have been created to work on a sequence of images at a certain resolution
to work with a sequence at a different resolution.

File Format Independence: Support for reading and writing a wide variety of
image file formats, such as those listed in Appendix C.

Platform Independence: The ability of a given piece of software to run on a

variety of different hardware platforms, often from different manufacturers.

Extensibility: The ability to add functionality to the package via the use of
macros, plug-ins, and so forth.

Optimization: Automated tools that can analyze and optimize complex com‐
positing scripts.

Scalability: The ability to create complex scripts that define large hierarchical
compositing processes while still maintaining reasonable user interactivity.

Distributed Rendering: The ability to process a given compositing tree on

multiple machines simultaneously. The work may be divided up so that differ‐
ent frames of the composite are sent to different machines or may even allow
a specific frame to be broken into multiple segments, distributed, and then
reassembled.

Internal Accuracy: The ability to process data at higher bit depths. Minimally
16 bits per channel but really there should be full support for a floating-point
workflow that can maintain pixel values outside the range 0–1.

HDR support: Tools should be aware of high dynamic range images and be‐
have appropriately when working with these images.

Proxy Support: The ability to use low-resolution stand-ins for the image/se‐
quence in question, and the ability to prototype effects or scripts based on
these proxies.

Graphics Hardware Acceleration: Many graphics subsystems can now per‐

form a variety of compositing operations using onboard processing that is ded‐
icated to specific imaging-related tasks. This sort of thing can significantly in‐
crease the interactivity of a compositing system.

Undo/Redo: A robust system for erasing or negating any changes that have re‐
cently been applied and then a way to reapply those changes if necessary. The
best systems will maintain a complete list of recent actions, making it easier to
see exactly what has occurred and allowing the user to undo multiple opera‐
tions in a single step.

Script Organizational Tools: As compositing hierarchies grow more com‐

plex, it will become desirable to apply some sort of organization to the tree
that is being shown to the user. Tools for this would include the ability of the
user to select collapse segments of a script into a simpler representation, to
hide portions of a script, to annotate areas in the tree and tools that help to
layout the script in a visually comprehensible fashion.

Interoperability: As your compositing work will probably be done in an envi‐

ronment where there are a variety of software tools being used for other parts
of the production process, any ability to transfer data more efficiently to and
from these systems will be beneficial. This could include shared file types, data
structures, and even plug-ins. Systems you may need to interact with would in‐
clude editing tools, paint packages, 3D animation, lighting and rendering soft‐
ware, audio applications, etc.

Outside Resources: Finally, when choosing a compositing system, never un‐

derestimate the importance of having good documentation, both directly from
the software’s manufacturer and from independent third party publishers.
Check on the availability of online forums or discussion groups. And generally
an active community of users who are skilled in a particular application can
prove invaluable, both when learning how to use a new piece of software and
as an aid to finding employment with your new expertise.