Modern Game Engine - Theory and Practice

Lecture 07

Rendering on Game Engine

Render Pipeline, Post-process and Everything

WANG XI GAMES 104 2022

Ambient Occlusion
AO Off
AO On
 Modern Game Engine - Theory and Practice

Ambient Occlusion
• Approximation of attenuation of ambient light due to occlusion

Visibility Along W Normal Weight

 Modern Game Engine - Theory and Practice

Precomputed AO
Using ray tracing to compute the AO offline and store the result into texture, which is widely
used in object modeling process
• Extra storage cost
• Only apply to static object
 Modern Game Engine - Theory and Practice

Screen Space Ambient Occlusion (SSAO)

• Generate N random samples in a sphere around each

pixel p in view space

• Test sample occlusions by comparing depth against

View depth buffer
Direction
Depth • Average visibility of sample points to approximate AO
 Modern Game Engine - Theory and Practice

SSAO+
• Recall the AO equation is acutally done on the normal-oriented hemisphere
SSAO+ Off
SSAO+ ON
 Modern Game Engine - Theory and Practice

HBAO - Horizon-based Ambient Occlusion

• Use the depth buffer as a heightfield on 2D surface
• Rays that below the horizon angle are occluded

Occluded Area

attenuation function
Slice
N
 Modern Game Engine - Theory and Practice

HBAO Implementation
• Use the depth buffer as a heightfield on 2D surface
• Trace rays directly in 2D and approximate AO from horizon angle

Depth Image Ray Marching

P
Randomly jitter the step size and randomly Find the max horizon angle
rotate the directions per pixel
 Modern Game Engine - Theory and Practice

GTAO - Ground Truth-based Ambient Occlusion

GTAO introduces the missing cosine factor, removes the attenuation function, and add a fast
approximation of multi bounce

cosine factor
 Modern Game Engine - Theory and Practice

GTAO - Ground Truth-based Ambient Occlusion

Add multiple bounces by fitting a cubic polynomial per albedo
 Modern Game Engine - Theory and Practice

Ray-Tracing Ambient Occlusion

• Casting rays from each screen pixel using RTT hardware
• 1 spp(sample per-pixel) works well for far-field occlusion
• With 2-4 spp, can recover detailed occlusion in contact region
 Modern Game Engine - Theory and Practice

Fog Everything
 Modern Game Engine - Theory and Practice

Depth Fog

Linear fog:
factor = (end-z)/(end-start)
Exp fog:
factor = exp(- density*z)
Exp Squared fog:
factor = exp(- (density*z)^2)

Linear Exp Exp Squared

 Modern Game Engine - Theory and Practice

Height Fog
• Height Fog integration along view diretion

• Fog color after transmission

 Modern Game Engine - Theory and Practice

Voxel-based Volumetric Fog

Anti-aliasing
 Modern Game Engine - Theory and Practice

Reason of Aliasing
• Aliasing is a series of rendering artifact which is caused by high-frequency signal vs.
insufficient sampling of limited rendering resolutions

Edge Sampling Texture Sampling Specular Sampling

 Modern Game Engine - Theory and Practice

Anti-aliasing
• The general strategy of screen-based antialiasing schemes is using a sampling pattern to
get more samples and then weight and sum samples to produce a pixel color
 Modern Game Engine - Theory and Practice

Super-sample AA (SSAA) and Multi-sample AA (MSAA)

• Super sampling is the most straightforward solution to solve AA

SSAA, 4x rendering resolution MSAA, only multi-sampling necessary pixels

4x z-buffer and framebuffer 4x z-buffer and framebuffer
4x rasterization and pixel shading 4x rasterization and 1+ x pixel shading
 Modern Game Engine - Theory and Practice

FXAA (Fast Approximate Anti-aliasing)

Anti-aliasing based on 1x rendered image
• Find edge pixels by luminance
• Compute offset for every edge pixel
• Re-sample edge pixel by its offset to blend with a neighbor

M: Luminance of middle pixel

(L = 0.299 * R + 0.587 * G + 0.114 * B)

#define _MinThreshold 0.05

float MaxLuma = max(N, E, W, S, M);

float MinLuma = min(N, E, W, S, M);
float Contrast = MaxLuma - MinLuma;
if(Contrast >= _MinThreshold)
...
 Modern Game Engine - Theory and Practice

Compute Offset Direction

Horizontal = abs(1*0.25+1*1-2*0.1)
+abs(2*0.15+2*1-4*0.2)
+abs(1*0.35+1*1-2*0.3) = 3.3

abs(1 - 0.2) >

3.3 > 0.3
abs(0.15 - 0.2)
Direction is horizontal

Vertical = abs(1*0.25+1*0.35-2*0.15)
+abs(2*0.1+2*0.3-4*0.2)
+abs(1*1+1*1-2*1) = 0.3
 Modern Game Engine - Theory and Practice

Edge Searching Algorithm

• Find aliasing edge that the pixel is in
• Record constrast luminance and average
luminance of current pixel and offset pixel

• Search along the 2 perpendicular direction

and calculate the average luminance

• Until abs( - ) > 0.25

abs( - ) > 0.25
 Modern Game Engine - Theory and Practice

Calculate Blend Coefficient

• Compute blender coefficient
targetP is the nearer edge end of CurrentP

if(( - )*( - ) > 0)

magnitude = 0;
else
magnitude = abs(0.5 - dst / edgeLength);
 Modern Game Engine - Theory and Practice

Blend Nearby Pixels

• Compute blender coefficient

PixelNewColor = Texture(CurrentP_UV + offset_direction * offset_magnitude )

 Modern Game Engine - Theory and Practice

FXAA Result
Origin FXAA
 Modern Game Engine - Theory and Practice

TAA (Temporal Anti-aliasing)

Utilize spatial-temporal filtering methods to improve AA stability in motion

history
history N-1 blend history N
rectification

neighborhood
reproject with
color
motion vector
boundingbox

input frame N
(jittered) output
 Modern Game Engine - Theory and Practice

TAA (Temporal Anti-aliasing)

Motion Vector Blend Ratio Blend Result

Modern Game Engine - Theory and Practice

TAA On/Off
 Modern Game Engine - Theory and Practice

But, the real magic in Post-process...

 Modern Game Engine - Theory and Practice

Post-process
Post-process in 3D Graphics refers to any algorithm that will be applied to the
final image. It can be done for stylistic reasons (color correction, contrast, etc.)
or for realistic reasons (tone mapping, depth of field, etc.)

Bloom Tone Mapping Color Grading

Bloom Effect
 Modern Game Engine - Theory and Practice

What is Bloom
• The physical basis of bloom is that, in the
real world, lenses can never focus
perfectly
• Even a perfect lens will convolve the
incoming image with an Airy disk

Airy Disk
 Modern Game Engine - Theory and Practice

Detect Bright Area by Threshold

Threshold

Find Luminance (Y) apply the standard

coefficients for sRGB:
 Modern Game Engine - Theory and Practice

Gaussian Blur

Blur

Gaussian distribution A classic gaussian Linearly separable

kernel 5+5(10) samples per pixel
5*5(25) samples per pixel
 Modern Game Engine - Theory and Practice

Pyramid Guassian Blur

Down
Down
Down
+
Blur
+ Up
+ =
Blur
Up
=
Blur
Up
=

We can't do all that filtering at high resolution, so we need

a way to downsample and upsample the image
Need a weight coefficient to tweak final effect
 Modern Game Engine - Theory and Practice

Bloom Composite
Tone Mapping
 Modern Game Engine - Theory and Practice

Tone Mapping
• No way to directly display HDR image in a SDR device
• The purpose of the Tone Mapping function is to map the wide range of high dynamic
range (HDR) colors into standard dynamic range (SDR) that a display can output
 Modern Game Engine - Theory and Practice

Tone Mapping Curve

Get a filmic look without making

renders dirty
Give images proper contrast and
nicely roll off any pixels over 1
 Modern Game Engine - Theory and Practice

ACES
• Academy Color Encoding System
• Primarily for Film & Animation
• Interesting paradigms and transformations

• The useful bits

• Applying Color Grading in HDR is good
• The idea of a fixed pipeline up to the final OTD
transforms stage is good
• Separates artistic intent from the mechanics of
supporting different devices
 Modern Game Engine - Theory and Practice

HDR and SDR Pipeline

• Visual consistency between HDR / SDR
• Similar SDR results to previous SDR color pipeline
• High quality
• High performance
• Minimal disruption to art teams
• Simple transition from current color pipeline
• Minimal additional overhead for mastering HDR and SDR
 Modern Game Engine - Theory and Practice

Tone Mapping Curve Comparison

Color Grading
 Modern Game Engine - Theory and Practice

Lookup Table (LUT)

• LUT is used to remap the input

color values of source pixels to
new output values based on data
contained within the LUT

• A LUT can be considered as a kind

of color preset that can be applied
to image or footage
 Modern Game Engine - Theory and Practice

LUT 3D or 2D

Sliced by Blue Axis

3D 2D Slices
 Modern Game Engine - Theory and Practice

Artist Friendly Tools

Modern Game Engine - Theory and Practice

Color grading is the most cost-effective feature of game rendering

Rendering Pipeline
 Modern Game Engine - Theory and Practice

One Equation for Everything

 Modern Game Engine - Theory and Practice

What We Learned about Rendering (1/4)

Rendering objects with meshes, texture and shaders Culling

 Modern Game Engine - Theory and Practice

What We Learned about Rendering (2/4)

Lighting, Shadow and Global Illumination PBR Materials

 Modern Game Engine - Theory and Practice

What We Learned about Rendering (3/4)

Terrain Sky and Cloud

 Modern Game Engine - Theory and Practice

What We Learned about Rendering (4/4)

Ambient Occlusion Fog Anti-aliasing

Bloom Tone Mapping Color Grading

 Modern Game Engine - Theory and Practice

Rendering Pipeline
• Rendering pipeline is the management order of all rendering operation execution and
resource allocation

ShadowPass Shading Post-process

drawCall skybox

drawCall
 Modern Game Engine - Theory and Practice

Forward Rendering
for n meshes
for m lights
color += shading(mesh, light)
 Modern Game Engine - Theory and Practice

Sort and Render Transparent after Opaque Objects

Transparent Order Render from far to near

 Modern Game Engine - Theory and Practice

Forward Rendering

Just Cause 1 2006 Heavy Rain 2010

 Modern Game Engine - Theory and Practice

Rendering with Many Lights

 Modern Game Engine - Theory and Practice

Deferred Rendering

Pass 1

Pass 2

Pass 1: Rendering G-Buffer

Pass 2: Deferred Shading

 Modern Game Engine - Theory and Practice

Deferred Rendering
Pros
• Lighting is only computed for visible fragments
• The data from the G-Buffer can be used for post-
processing
Cons
• High memory and bandwidth cost
• Not supporting transparent object
• Not friendly to MSAA

Scene with Many Lights

G-Buffer Size:1920*1080, 32bit*1920*1080*4 = 63MB

 Modern Game Engine - Theory and Practice

Pilot Engine Deferred Rendering

 Modern Game Engine - Theory and Practice

Tile-based Rendering
 Modern Game Engine - Theory and Practice

Light Culling by Tiles

Light List in a Screen Tile

 Modern Game Engine - Theory and Practice

Depth Range Optimization

• Get Min/Max depth per tile from Pre-z pass
• Test depth bounds for each light
 Modern Game Engine - Theory and Practice

Tile-based Deferred Rendering

Battlefield 4 Ryse
 Modern Game Engine - Theory and Practice

Forward+ (Tile-based Forward) Rendering

• Depth prepass (prevent overdraw / provide tile depth bounds)
• Tiled light culling (output: light list per tile)
• Shading per object (PS: Iterate through light list calculated in light culling)

DIRT GRID
 Modern Game Engine - Theory and Practice

Cluster-based Rendering

Doom 2016
 Modern Game Engine - Theory and Practice

Visibility Buffer
G-Buffer V-Buffer
Real Rendering Pipeline
 Modern Game Engine - Theory and Practice

Challenges
• Complex parallel work needs to synchronize with complex resource dependency
• Large amount of transient resource whose lifetime is shorter than one frame
• Complex resource state management
• Exploit newly exposed GPU features without extensive user low level knowledge
 Modern Game Engine - Theory and Practice

Frame Graph
A Directed Acyclic Graph (DAG) of pass and
resource dependency in a frame, not a real
visual graph
 Modern Game Engine - Theory and Practice

Render to Monitor
 Modern Game Engine - Theory and Practice

Screen Tearing
 Modern Game Engine - Theory and Practice

Screen Tearing

In most games your GPU frame rate will be highly volatile

When new GPU frame updates in the middle of last screen frame, screen tearing occurrs
 Modern Game Engine - Theory and Practice

V-Sync Technology
Synchronizing buffer swaps with the Vertical refresh is called V-sync
V-Sync can be used to prevent tearing but framerates are reduced, the mouse is lagging & stuttering
ruins gameplay
 Modern Game Engine - Theory and Practice

Variable Refresh Rate

 Modern Game Engine - Theory and Practice

Homework 2
• You are supposed to...
• Implement ColorGrading shader code
• Generate own style ColorGrading result
• Add a new post-process pass that you want (advanced)
• Write a report document that contains screenshots of
your results

• Download at
• Course-site:
https://games104.boomingtech.com/sc/course-list

• Github:
https://github.com/BoomingTech/Pilot/tree/games104/homewor
k02-rendering
 Modern Game Engine - Theory and Practice

Pilot Engine V0.0.3 Releasing – April 26

New Features
• Deferred shading pipeline
• Configurable global rendering resource
• Motor system with accelerations
• Character-following camera blending

Bugfixes
• Fixed image layout transition in “pick” pass
• Fixed overlapped button and cursor twinkling

Optimizations
• Optimized display of rotation as Euler angles
• Optimized AMD and NVIDIA graphic device race when initializing
Vulkan
• Optimized editor camera controlling

Contributors
 Modern Game Engine - Theory and Practice

Pilot Engine Learning

• The first version of the engine architecture

document will be uploaded to Github Wiki
and official website on April 30

• Videos of Pilot Engine source code

walkthrough will be released in the near
future
 Modern Game Engine - Theory and Practice

Labor Day Holiday Arrangement

• Lecture 08 on May 2 will be postponed to May 9
• All subsequent classes will be postponed
Modern Game Engine - Theory and Practice

Q&A
 Modern Game Engine - Theory and Practice

Lecture 07 Contributor

• 一将 • 爵爷 • 沛楠 • QIUU
• 光哥 • Jason • Leon • C佬
• 炯哥 • 坤哥 • 虎哥 • 阿乐
• 玉林 • BOOK • Shine • 阿熊
• 小老弟 • MANDY • 晨晨 • CC
• 建辉 • 婷姐 • Judy • 大喷
Modern Game Engine - Theory and Practice

Enjoy ;)
Coding

Course Wechat

Please follow us for

further information
Please note that all videos and images and other media
are cited from the Internet for demonstration only.