Mlaa
Mlaa
Alexander Reshetov
Intel Labs
Figure1.FairyForestmodel:morphologicalantialiasingimprovesthequalityoftherenderedimagewithouthavinganoticeableimpactonperformance.
ABSTRACT
We present a new algorithm that creates plausibly antialiased
images by looking for certain patterns in an original image and
then blending colors in the neighborhood of these patterns
according to a set of simple rules. We construct these rules to work
as a post-processing step in ray tracing applications, allowing
approximate, yet fast and robust antialiasing. The algorithm works
for any rendering technique and scene complexity. It does not
require casting any additional rays and handles all possible effects,
including reflections and refractions.
CRCategories:I.4.3[ComputerGraphics]:ImageProcessingAnd
ComputerVisionEnhancement
Keywords:antialiasing,imageenhancement,morphologicalanalysis,
rendering
1 INTRODUCTION
A human eye has a discrete number of light receptors, yet we do
not discern any pixels, even in peripheral vision. What is even
more amazing is that the number of color-sensitive cones in the
human retina differs dramatically among people by up to 40
times [Hofer et al. 2005]. In spite of this, people appear to perceive
colors the same way we essentially see with our brains. The
human vision system also has an ability to ascertain alignment of
objects at a fraction of a cone width (hyperacuity). This explains
why spatial aliasing artifacts are more noticeable than color errors.
Realizing this fact, graphics hardware vendors put significant
efforts in compensating for aliasing artifacts by trading color
accuracy for spatial continuity. Multiple techniques are supported
in hardware, based on mixing weighted color samples, similar to
the integrating property of digital cameras [ATI; NVIDIA].
email: [email protected]
implementation. By using 8 bits for color channel, each RGBA
color requires 32 bits, so 4 RGBA pixels will fit into one SSE
register. The next C++ code fragment shows our implementation
of ssedif() function, which compares 4 RGBA values using SSE2
instruction set:
#define most_significant_color_bits char(0xff ^ ((1<<4) - 1))
unsigned short int ssedif(const __m128i& c0, const __m128i& c1) {
// return 16 bits set to 1 for each channel where c0,c1 are different.
__m128i hibits = _mm_set1_epi8(most_significant_color_bits);
__m128i d = _mm_sub_epi8(_mm_max_epu8(c0, c1),
_mm_min_epu8 (c0, c1));
d = _mm_and_si128(d, hibits);
d = _mm_cmpeq_epi8(d, _mm_setzero_si128());
return 0xffff ^ _mm_movemask_epi8(d);
}
The implementation is pretty straightforward, allowing simul-
taneous processing of sixteen 8-bit values. We compute the
absolute differences of 4 RGBA colors (variable d), keep only the
most significant bits, and compare the result with 0. Note that we
cannot compare the variable d directly with a threshold value since
there are no comparison operations for unsigned 8-bit values in
SSE
set. If an arbitrary threshold is required, this function could
be easily re-implemented with available instructions, albeit less
efficiently.
We use this function for finding discontinuities between each two
adjacent rows and each two columns in the original image. Since
pixels are stored in a framebuffer row-wise, accessing 4 colors in a
row (sixteen 8-bit channels) can be achieved with one
_mm_load_si128 instruction. For columns, we use
_MM_TRANSPOSE4_PS macro [Intel] to convert data into SSE-
friendly format.
We did not try to implement other MLAA steps using SSE
ins-
tructions (though it might be possible to do it using SSE4
operations), opting instead for preserving the universal nature of
the algorithm. The upcoming Larrabee chip [Seiler et al. 2008], as
well as modern GPU cards, are capable of handling 8-bit data
extremely efficiently, so our algorithm will benefit from porting to
these architectures.
Figure5.Amodelconsistingof100elongatedquads(notexturing).
3 MLAA LIMITATIONS
Arguably the biggest MLAA limitation is its inability to handle
pixel-size features. This problem is illustrated in Figure 5, which
shows 100 elongated quads at a resolution of 1024x512 pixels.
MLAA takes the aliased image shown on the top and produces the
one in the middle. Despite being very narrow, quads, which are
closer to the camera, span multiple pixels. For these quads, MLAA
is working just fine, removing higher frequencies from the image.
Closer to the top of the image though, there are no identifiable
features, resulting in moir pattern, which is a typical example of
geometrical aliasing. Similarly, if presented with unfiltered
texture, MLAA will try to remove higher frequencies where
possible and then give up. Of course, textures could be reliably
filtered with mipmapping.
Geometrical aliasing could happen not only for very small or thin
objects, but also when just few rays penetrate space between
bigger objects. Another type of aliasing, which is characteristic for
ray tracing applications, may happen when shading normals are
used. Indeed, when an angle between incoming rays and
geometrical normal is close to /2, for shading normals the angle
could become bigger than /2, causing reflected rays to go in
wrong direction (inside the object). This usually happens just for
few pixels resulting in an aliasing effect.
Isolated pixels are naturally blended with neighboring ones after
MLAA pass, since these pixels belong to multiple shapes. It is
even possible to identify all such pixels or small groups of pixels
and then apply additional blending to them. However, this will not
help for certain moir patterns, for which aliased images appear to
have some structure (caused by interference of multiple
frequencies).
A somewhat less noticeable drawback of MLAA is its handling of
border pixels, for which no full information about neighborhood is
available. This is similar to all adaptive techniques that rely on a
single sample per pixel. Another type of problem is specific for
MLAA. Because MLAA uses only image data for the
reconstruction, very slowly changing scenes might produce non-
smooth transition between frames. Images at different time steps
might have identical sets of morphological shapes if camera or
objects are moving at sub-pixel speed (see discussion related to
Figure 7). In practice, it is not perceptible if the output resolution
is higher than 200x200 pixels. For color images, even if two
images have identical sets of shapes, MLAA output might be
different, since interpolation coefficients depend on color values,
to certain degree allowing reconstruction of sub-pixel motion.
MLAA processing of text could result in additional distortion,
since MLAA does not distinguish between foreground and
background. It is especially noticeable if original font was already
antialiased (see Figure 6).
Figure 6. Top row: bitmap font; second row: TrueType font (antialiased),
thirdrow:MLAAprocessingofbitmapfont;fourthrow:MLAAprocessing
ofTrueTypefont.Fontsizefromlefttoright:24,12,8,and6.
4 COMPARISON WITH SUPERSAMPLING
To understand similarities and differences between MLAA and
other antialiasing techniques, we created a very simple example by
rendering a model consisting of a single half-plane. Figure 7 shows
six pixels separating white (top) and black (bottom) areas.
We compute pixel coverage analytically (black line on the top
chart) and compare it with MLAA (blue) and two supersampling
approaches, using 4 and 16 samples per pixel. All techniques
estimate area with 15% accuracy, while MLAA provides a linear
estimation, without any unwarranted frequencies. At the same
time, MLAA estimation will be the same for all images in which
separation line intersects two black arrow-ended intervals on the
bottom chart. MLAA sacrifices accuracy of local estimation in
favor of maintaining smoother global silhouette approximation,
stitching it at different morphological shapes.
Figure7.Sampling6pixelsseparatingblackandwhiteareas(2x2samples
perpixelareshown).Bottomimage:Zshapeisboundedbythegreenline,
while red line shows MLAA reconstruction of true silhouette edge.
MLAA reconstruction will be the same for all true curves intersecting
twoblackarrowendedintervals.
On Figure 8, the half-plane silhouette is almost horizontal,
straddling 24 pixels per Z-shape. For this view, 2x2 sampling
results in significant errors in area estimation. For MLAA, actually
the opposite is true since a longer Z-shape allows for smaller
variability in the true image. Accordingly, to match MLAA
quality, 8x8 sampling is required. These examples also illustrate
the well-known fact that uniform sampling is not the best approach
to reduce variance [Laine and Aila 2006], and supersampling or
adaptive antialiasing techniques would benefit from using different
sampling patterns. While very common in global illumination
algorithms, low variance sampling patterns are rarely used for
antialiasing in ray tracing applications (though it is a common
practice in hardware-accelerated rasterization). One early example
of using stochastic sampling, particularly for ray tracing, is given
in [Dippe and Wold 1985].
Figure8.Sampling24pixels.
5 SUMMARY AND FUTURE WORK
We have proposed a new approach to antialiasing based on
searching for certain shapes in images and processing pixels
adjacent to these shapes according to a set of simple rules. This
removes artificial high frequencies due to a discrete sampling. The
new approach has unique characteristics that, we hope, will make
it worthwhile considering, along with other algorithms:
MLAA can be used for any image processing task and does
not use any data besides color values. It is very simple and
does not require any modifications to the rendering pipeline.
It does not need casting any additional rays.
The algorithm is embarrassingly parallel and can be executed
concurrently with rendering threads (using double buffering),
allowing for better processor utilization.
The algorithm achieves reasonable performance, in many
cases without any noticeable impact.
The quality is comparable with supersampling. MLAA has
very different approximation characteristics, achieving better
results in situations in which uniform sampling suffers from
significant errors and undesired frequencies.
In order to illustrate our technique's universal nature, we have also
applied it to images from Call of Duty