Realtime 3D Computer Graphics Virtual Reality

Graphics

Computer graphics
3D-Computer graphics (3D-CG) currently used for Simulators, VR, Games (real-time) Design (CAD) Entertainment (Movies), Art Education
tennis for two pong spacewar (from l. to r.)

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Real-time 3D-CG
Goals Generating 3D worlds visualize application domain specific data in >=3D (maybe using 3D metaphors). Implement user interaction with the virtual world. Operate in real-time, minimize lag, maximize simulation rate. Methods Computer graphics Implement depth cues discussed earlier, e.g., mapping 3D world to 2D pictures to simulate user-eye-specific views in interactive rates (stereoscopy, immersion). CG methods differ in primitives used and processed (e.g., vertex-, volume-, pixel-based, etc.). lighting model and calculation used (local or global models). direction of lighting application, shading (scanline shader, raytracer, etc.).

Mathematics (Trigonometry, linear algebra, analysis).

By 2005, the majority of VR-systems produces 2D images based on surface (vertex) models and (image) textures using a local Phong lighting model processed by a pipeline architecture (changing significantly just now from fixed to programmable pipeline).
Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

The 3D-pipeline
Application Geometry Rasterizer

Model & View Transform

Lighting

Projection

Clipping

Screen Mapping

application stage geometry stage

model and view transform lighting (shading in rasterizer) projection clipping screen mapping visibility (z-buffer) shading texturing

parallelization on different stages

rasterizer stage

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

The 3D-pipeline
Model Object object space Scale, Rotate Translate, Object Specify View Calculate Normal Illumination world Backface world (eye) (eye) space space Remove hidden surfaces Apply Normalizing Transformation, Clip normalized eye space Perspective Transformation / Projection

Map to Viewport/ Draw to Screen Spaces:

Shade, Texture

image space

1. 2. 3. 4. 5.

Object Space: Local objects frames of reference (aka Modeling Space). World Space: Where the scene and viewing specification is made. Eyespace (Normalized Viewing Space): Eye point (COP) is at the origin looking down the Z axis. 3D Image Space: A 3D perspected space. Dimensions: -1:1 in x & y, 0:1 in Z. Where Image space hidden surface algorithms work. Screen Space (2D): Coordinates 0:width, 0:height

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Steps in the 3D pipeline

3. Triangle Setup 1. Application/Scene Back-face Culling (or can be Scene/Geometry database traversal done in view space before Movement of objects, and aiming and movement of view lighting) camera Slope/Delta Calculations Animated movement of object models Scan-Line Conversion Description of the contents of the 3D world 4. Rendering / Rasterization Object Visibility Check incl. possible Occlusion Culling Shading Select Level of Detail (LOD) Texturing Fog 2. Geometry Alpha Translucency Tests Transforms (rotation, translation, scaling) Depth Buffering Transform from Model Space to World Space (Direct3D) Antialiasing (optional) Transform from World Space to View Space Display View Projection Specify View Scale, Apply Calculate Normal Model Rotate Trivial Accept/Reject Culling Normalizing Illumination Object Translate, Transformation, Backface Object Clip world world object Back-Face Culling (can also be done space space space eye later in Screen Space), Lighting space image space Map to Perspective Divide - Transform to Clip Space Perspective Remove Viewport Shade, Transformation hidden / Texture / surfaces Draw to Clipping Projection Screen Transform to Screen Space
Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

The 3D-pipeline
Example Image Space Viewing Algorithm Steps: 1. Create World Space Scene: Position each object (scale, rotate, translate) Set up view parameters 2. Calculate Backface, illumination, etc.: Can Perform Backface removal here 3. Calculate and Perform World to Eyespace transformation 4. Calculate and Perform Perspective Transformation 5. Clip Objects 6. Perspective Divide Can Perform Backface removal here 7. Map to 3D Screen Space: Viewport Transformation 8. Perform Hidden Surface Removal 9. Draw to Screen (Scan Convert): Shade, texture
Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Implementations of the 3D-pipeline

Vertex data sources control
Polynomial Evaluator Display List

Vertex Processing control

Primitive Rasterization control

Pixel Processing control

Frame Buffer data

Video Scanout control

DirectX

data CPU

Application

Per Vertex Operations & Primitive Assembly

OpenGL

Rasterization

Per Fragment Operations

Frame Buffer

Texture Memory Pixel Operations

Hardware (NVidia)

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

VR-development tools
Several software packages and APIs support the real-time 3D-CG relevant processing stages. Complex tools support additional simulation tasks (auditory, tactile feedback, ). For mature simulations, the overall task is separated into
modeling the world and designing the interaction. (procedural)

Direct rendering, gfx and simulation packages programming

OpenGL, Direct3D, GKS (3D), VCollide, OpenAL, ODE, Vortex, HAVOC,

Scene graph based tools and APIs

VRML, X3D, OpenGL Performer, OpenGL Optimizer, Open Inventor, OpenSG, PHIGS+,

VR modeling toolkits
AVANGO, World toolkit, Masive1-3, Dive, Lightning, game engines,
(declarative) modeling design

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Scene Graphs
are data structures for 3D-CG:
DAG Directed Acyclic Graph or Tree (no multiple instancing).

View

organize and control the rendering of its constituent objects. consist of

a root node (base frame of reference). intermediate nodes (grouping, transformation, sensors, engines, materials). leaf nodes (geometry, lights, views). Mater

T1

T8

T2

T4

View

group objects spatially to support

proximity detection. collision detection. view frustum culling. occlusion culling.

S1

T3

T5

L1

Geo

T6

T7

may support local or global state inheritance. may support different types of intermediate nodes

Mater.

Geo

Geo

Geo

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Application Graphs
are data structures for 3D-CG:
Can be either DAG or DCG (Directed Cyclic Graph).
6DOF sens T1

T8

organize and control data flow. constitute a data-flow network:

Node components (variables) are represented as fields. Nodes are represented as Mater1 field containers. Fields (of equal types) can be interconnected by routes. Changing field-values generates events processed
once per field change. once per frame.

T2

Interp.

T4

View

S1

T3

Time

T5

L1

Geo

6DOF sens

T6

T7

can support multiple Field Fan-out/Fan-in for their fields.

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik

Geo

Geo

Geo

A Scene Graph Language with Field Routing: VRML/X3D

#VRML V2.0 utf8 Transform { translation -3 0 0 children Shape { geometry Box { } appearance Appearance { material Material { diffuseColor .8 .2 .2 } } } } Transform { translation 3 0 0 children Shape { geometry Cone { } appearance Appearance { material Material { diffuseColor .2 .2 .8 } } } }

More VRML later in this course!

Realtime 3D Computer Graphics / Virtual Reality WS 2006/2007 Marc Erich Latoschik