CHAPTER 1: INTRODUCTION.

Game Era Evolution (Games and Datewise) - First commercial video game was Computer Space (1971) BUFFERING SINGLE BUFFERING: computer writes to framebuder displays framebuder to screen with scanline.Start filling in the
by the founder of Atari – No RAM needed, all logic implemented in hardware. A famous game in 1978 – Space Invader – fight with an pixels here for the next frame What happens if we draw ahead of the scanline or that the drawing is too slow and is overtaken by the
alien army.Atari Era (1977-1985)- games written in assembly language. (Super) Nintendo Entertainment System Era (1983-1995). scanline? -> Screen Tearing Happens: When the display shows part of two diderent frames at the same time, we have screen tearing.
Representative games:Super Mario Bros (1985), The Legend of Zelda (1986).Handheld Game Console (1980s)- One game on one Framebuder: A framebuder is some physical memory (RAM) used to store the entire image to be displayed on a screen. Each element
console.Handheld Game Console (2nd Generation) (1990s) - Multiple games on one console. (eg. Tetris). Digital Pet (1996- in a framebuder is called a pixel (picture element). The screen size (or window size) determines the resolution of the framebuder. How
Tamagotchi). Ps/Ps2 era (1995-2005) – games written in C.Xbox 360, PS3, and Wii Era (2005 – present). Now- games by independent to avoid screen tearing if we use only single budering? Writing to framebuder can only be done during the VBLANK interval. Problem:
developers, Xbox Live Arcade, Playstation network, steam, App store and Gplay. 2D games (now) on mobile. (Angry Birds- 100K USD not enough time to render everything to framebuder during the limited time. DOUBLE BUFFERING: It maintains two framebuders: –
Budget by 4 programmers in 8 months). Pokemon Go (2016). Game Genres: (1)Traditional (eg. Card games, monopoly, tiktaktoe). (2) Front buder: screen shows what is inside this buder – Back buder: where we are current drawing to. Process- computer writes to back
Adventure – No action, text adventure (zork/dungeon) or graphical adventure(kings quest). (3) Action- Space Invaders, Pong, etc. (4) buder. Front buder displays to the screen. Once the content of the front buder is fully displayed to the screen, the two buders will be
Platformer – Character Running andn playing field(super mario bros). (5) RPG- Role playing games (final fantasy, diablo). (6) MMORPG- swapped. Buder swap takes place once the back buder is filled. Can attain framerate at the rendering speed Problem: Screen tearing
massing multiplayer online RPG (pubg/cod). (7) Racing. (8) Education. (9) Others- FPS, Survival Games, Simulations, Battle Royal will still happen. What if double budering takes place during VBLAKN? Buder swap takes place during VBLANK. No screen tearing
(Multi-Online Battle Arena MOBA), TBS, RTS, Puzzle. MOST COMMON GENRES (in order) – Action, Action Adventure, Adventure, Problem: Framerate will be capped at refresh rate of the monitor, and Framerate can even be halved. TRIPLE BUFFERING: Triple-
Casual. Game Development methodology: (1) Code and Fix- v little planning, do implementation and problems when they come up – budering maintains one front buder and two back buders Rendering is done to the two back buders alternately Swapping from the
aka firefighting., small team, short time. (2) Waterfall Method – Sequencial, well planned, accurate schedule and milestones. (3) Agile most recently completed back buders to front buder during VBLANK and monitor displays from front buder. Process: Computer can
Methodology. render as fast as possible, no need to sit idle waiting for VBLANK. computer draws to two back buder alternately. swap from most
recently done back buder to front buder. front buder displays to monitor. No screen tearing, since monitor always display from a
CHAPTER 2: GAME DESIGN: Concepts- Story, Rules, Rewards, Punishments, Look n feel, timing and pacing, risk taking, experience. complete frame. Back buder 1 and back buder 2 swap once back buder 1 is filled. Back buder 2 and front buder swap during VBLANK
A game designer creates an experience, how to understand- Psychology, anthropology (mental, physical, cultural aspects), design. only. WHY NO SCREEN TEARING in triple budering? Screen tearing will not happen because the contents of front buder will not change
Diderent Lenses By Schell- (1) Fun to play. (2) Surprise element (3) curiousity. The 4 basic elements- (1) Aesthetic (look n feel), (2) between any two VBLANKs. SPRITES: A sprite is a 2D visual object in a game world that can be drawn using a single image on any
Mechanics (procedures and rules), (3) Technology, (4) Story. (ALL are equally important). Bartle’s Taxonomy of Player types- (1) frame Used for representing characters, dynamic objects or background Major element of a 2D game, and contributes largely to the
achievers- like achieving goals, (2) explorers- enjoy making discoveries, (3) socializers(like to link up with people and (4) killers (like to download size. Image file format is a concern, since it adects the file size, image quality and ediciency in rendering PNG / JPG / GIF:
complete and defeat). Understanding the Player, Flow - a feeling of complete and energized focus in an activity, with a high level of less space, but most hardware cannot natively draw PNGs / JPGs / GIFs, need to be converted when loaded and hence less edicient
enjoyment and fulfilment. (More challenging and less skills – makes frustration, more skill and less challenging makes boredom). Other TGA (Truevision Graphics Adapter): can be drawn directly and can support truecolor or high-color display, but takes up much space
. Design and Development Issues – Game Theme, character design, world/level design, visual design, playtesting, profit. Playtesting: Most game developing frameworks such as Unity, XNA, cocos2d have built-in functions to load images of diderent file formats.
Playtesting is to get people to really play your game and to check if they experience exactly what you have design, people like your PAINTER’S ALGORITHM: Sprites can be arranged in layers. The Painter's Algorithm solves the visibility problem of 2D sprites in a scene
game, that's great! The five key questions of playtesting that you should ask yourself: Why, Who, Where, What and How? by sorting all sprites from back to front, and then rendering the sprites in back-to-front order Bottom-most (farthest away from viewer)
sprites are rendered first (e.g., background) Top-most (closest to the viewer) sprites are rendered last (e.g., character for player).
CHAPTER 3: GAME ENGINE AND UNITY: Game Engine- A software framework designed for the creation and development of video Sprites within each layer can be ordered as well.
games. Developers use them to create games for consoles, mobile devices, and personal computers. Components of Game Engine:
(1)Main game program: The actual game logic being implemented using algorithms. (2) Rendering engine: How to display or project PART 2: ANIMATIONS; a minimum of 24 FPS is required for perceiving a smooth continuous motion. Animating Sprites: In each game
2D or 3D graphics onto the screen? (3) Audio engine: Componentry consisting of any algorithm related to sound (4) Physics engine: frame, need to determine which image in the animating sequence is to be shown. Some older libraries require that an image for a sprite
Responsible for giving the application a realistic sense of the laws of physics in the application (5) Artificial intelligence: Usually to have dimensions in powers of two, e.g., 16x16 or 64x64. If an image is not of such dimensions, it needs to be padded with
outsourced from the main game program into a special module to be designed and written by software engineers with specialist background pixels and hence a waste of space. Also, it is more edicient to load a large image into the texture memory on the GPU
knowledge. FRAME: All updates are made in a buder first and then contents of the buder are copied to the screen. As a game usually (graphics processing unit) than to load a number of smaller images. Therefore, a SPRITE SHEET is commonly used for holding multiple
involves the update of several components, this approach can help to eliminate flickering edect. The default frame rate in Unity is 50 sprites in a single image file. Size of a sprite sheet is limited by the maximum texture size, e.g., 2048 x 2048 for iOS. Implementation in
frames per second (fps). That is, the Update() method will be called every 1 / 50 = 0.02 sec. Unity Frame: The default frame rate in Unity Unity: To tell Unity it is a sprite sheet, set the “Sprite Mode” as “Multiple” in the sprite’s Inspector window. Next open “Sprite Editor” to
can be updated by “Edit → Project Settings → Time → Fixed Timestep”. Guidelines: Frame rate > 10 fps (i.e., Timestep < 0.1): Human eyes slice the sprite sheet into multiple sprites. Choose “Slice” tab and press the “Slice” button in the pop-up window. The sprites will be
will interpret a sequence of images as a video. Frame rate > 100 fps (i.e., Timestep < 0.01): Too much processing power required. Fixed extracted from the sprite sheet automatically. You can click on diderent sprites and name them independently (default is “<sprite
Timestep: A framerate-independent interval that dictates when physics calculations and FixedUpdate() events are performed. name>_<index>”). Drag the sprite collection file into the Hierarchy view. Open the Animation view by choosing Window\Animation.
Maximum Allowed Timestep: The fixed timestep keeps the physical simulation accurate in real time but it can cause problems in cases You can also add this view to your layout by choosing Animation from the Add Tab menu connected to any other tab. You will be asked
where the game makes heavy use of physics and the gameplay frame rate has also become low (due to a large number of objects in to save the animation concerned. An Animator object will appear in the Inspector window. 2D Texturing: A texture is simply an image
play, say). The main frame update processing must be “squeezed” in between the regular physics updates. The Maximum Allowed that is used to modify the appearance of a surface. Texture mapping is the process of "gluing" a texture onto a surface by giving each
Timestep setting puts a limit on the amount of time Unity will spend processing physics during a given frame update. Maximum Particle point of the surface an appropriate texture coordinate. Types of Texture Settings: (1) Tile (repeat, wrap): The texture image repeats itself
Timestep: A framerate-independent interval that controls the accuracy of the particle simulation, at the expense of performance. across the surface. Equivalently, we drop the integer part of the texture coordinates, e.g., (1.5, 1.5) → (0.5, 0.5). (2) Mirror: The image
(Particles are less noticeable than other objects and so they can be treated diderently.). SlowDown Game: You can slow down the repeats itself across the surface,but is mirrored on every other repetition. The overall texture is continuous along the edges. (3) Clamp:
game time by updating Time.timeScale in your program: Time.timeScale = 1.0F As fast as the real-time Time.timeScale = 0.5F 50% Texture coordinates outside the range [0,1) are clamped to this range. Results in repetition of the edges of the texture. (4) Border:
slower than the real-time Time.timeScale = 0.0F The game is freezed. GAME LOOP: A collection of frames form a game loop. Or each Texture coordinates outside the range [0,1) are rendered with a specific border color. TILE MAPS: 2D game worlds/levels often contains
iteration of the game loop is a frame. A game keeps doing a series of actions over and over again until the user quits. The game loop is repetitive elements. Creating a unique texture for each world/level is not memory edicient. Instead, we partition the world into squares
the overall flow control for the entire game program. Real-time games mostly update at 30 and 60 FPS (frames per second). Typical (or other polygons) of equal size. Each square references the sprite (or tile) that should appear in that location. A tile set contains all
Game Loop – While Loop - game is running, process inputs, update game world and generate oututs. GAME OBJECTS: A game object the tiles that the game will use and all tiles are placed into a single sprite sheet. Square tile maps are good for game with a top- down
is anything in the game world that needs to be updated, drawn, or both updated and drawn on every frame. Game objects that need to or straight-on-the-side view Isometric view is used to achieve a greater sense of depth. Isometric tile maps comprises diamond-
be both updated and drawn are those that are visible and movable on screen e.g., character, creature, movable objects. Game objects shaped and hexagonal tiles. Tile Editor: A computer graphics software which generates a larger graphic from re-using number of
that are only drawn but not updated are the static objects e.g., buildings, backgrounds Game objects that are updated but not drawn smaller graphics to save RAM and increase real-time rendering performance.
are mostly game controls e.g., camera, trigger (detects the player action and triggers some events). 3 Important Unity
Syntax/Functions: Update() This function will be invoked in each frame. LateUpdate() This function will be invoked after Update() PART 3: SCROLLING: Single Axis Scrolling: In single-axis scrolling, the game scrolls only in the x or y direction Depending on the current
function is invoked. It is used in occasion like the camera moves after the character moves. FixedUpdate() Similar to Update()but this viewing location, the appropriate portion of the background images is rendered. If the background image is too large to reside entirely
function is used to handle rigidbody issues. Mathf.Repeat(t, length) → [0.0, length) Loops the value t, so that it is never larger than in the texture memory, it is split into smaller images and be loaded whenever needed. In general, we fix the camera (i.e., the view) and
length and never smaller than 0. This is similar to the modulo operator but it works with floating point numbers. Example 1: t = 3.0 & scroll the background to achieve the edect. We may use the background as a sprite and move it in the game world. Move background
length = 2.5 ➔ result = 0.5 Example 2: t = 5 & length = 2.5 ➔ result = 0.0. sprite to the left; essentially we are changing the position of the sprite. Or we may use the background as a texture on a polygon and
shift the texture on the polygon. Move the texture to the left, while fixing the polygon and game view. Essentially, we are changing the
CHAPTER 4: AUDIO AND VIDEO: AUDIO 2 Types – Mono/ Stereo. Pulse Code Modulation (PCM) Extention: .wav example: 44,100
texture coordinates of the polygon. Infinite Scrolling: Infinite scrolling is when the game continues until the player losses. We need
samples per second, 16 bits per sample, stereo (44,100 x 16 x 2 = 1,411,200 bit/s = 1,411.2 kbit/s = 176.4 kbyte/s) A 5-minute song:
repeating images or seamless textures to preserve continuity along edges of two images. Key Ideas: Draw the first image with the left-
52.92 Mbytes. Mp3: .mp3- lossy compression by reducing audio parts not sensitive to human. 32, 96, 128, 192 (commonly used), 320
top corner touch the point (0, 0). Draw the second image with the left-top corner touching the point (first image’s width, 0). Repeat for
kbit/s (highest level supported). Variable bit rates (VBR): Audio parts in MP3 can be divided into frames, each has its own bit rate all remaining images. Define a timer, continuously change the position of each image towards the left. What if the first image is moving
according to the complexity of audio. ID3 tag: Additional information such as artist, album, genre, lyrics, album art. .*.WMA- (Windows
out of the screen? Append it to the right of the last image. Repeat the same thing for all other images. Parallax Scrolling: The background
Media Audio) A lossy audio codec. Audio signals that are deemed to be imperceptible to the human ear are encoded with reduced
is broken up into multiple layers at diderent depths. Each layer scrolls at a diderent speed, which gives an illusion of depth. The
resolution during the compression process. Can encode audio signals at up to 48 kHz with up to two discrete channels (stereo). WMA
background is broken up into multiple layers at diderent depths. Each layer scrolls at a diderent speed, which gives an illusion of
9 was introduced in 2003. The result is a bit-for-bit duplicate of the original audio file; in other words, the audio quality on the CD will
depth.with parallax; clouds are in a diderent layer that scrolls slower than the foreground, giving the impression that they are farther
be the same as the file when played back. It supports 6 discrete channels and up to 24-bit/96 kHz lossless audio. Same file extension:
away. Four Way Scrolling: In four-way scrolling, the game world can scroll both horizontally and vertically. Can be done in the same
*.wma. MIDI: Musical Instrument Digital Interface: For describing musical events, such as notes, dynamics, volume, and vibratons.
way as single-axis scrolling, i.e., by moving background sprite or by moving textures.
Polyphony of 32 notes: play at least 32 notes at the same time Multi-timbral: producing two or more diderent instrument sounds
simultaneously. Preparing Audio Edect in Unity: Software such as Audacity, drag .p3 files in project window. Finally, choose “File” →
CHAPTER 9: CAMERA. Two types of projections. (1) Orthographic projection: representing a 3D object in 2D. and (2) Perspective
“Export as WAV” or “Export as MP3” to obtain your sound track.GRAPHICS/ IMAGE: Bit Map: Raw data with header, simple and direct
projection: objects in the distance appear smaller than objects close by. FIELD OF VIEW: One of the parameters for a perspective
operation Header with information: XL - # of pixels in x-direction YL - # of pixels in y-direction bpp - bit per pixel, etc Raw data, for width
projection is field of view (FOV), which specifies the amount of the world that is visible using an angle subtended at the camera. Human
of XL pixels and heigth of YL pixels, N bpp, need XL*YL*N/8 bytes + header size For 16 color (4 bits to represent), 1 byte may represent
eyes has a nearly 180 field of view ( about 120 for binocular view) An incorrect field of view can lead to eye strain, or worse, motion
2 pixels, or 4 byte for 8 pixels. May have compression, RLE (running length encoding) AAABBCCCC ➔ 3A2B4C. Digital Camera Raw sickness for some players. PC games usually has a FOV of 90 to 100 degress. FISHEYE EFFECT: Increasing FOV increases what can be
Images: Original image pixel data For professional digital photographers (preserve an image’s fidelity) Preferred to a digital negative by seen in the game world, which means that more objects will be rendered Too large an FOV leads to the fisheye edect, in which objects
many photographers, who feel it preserves the subtle color and details possible Can adjust brightness, contrast, sharpen, white near the edge of the screen are distorted. CAMERA POSITIONS. (1) FIXED CAMERA: Fixed (Non-player controlled) cameras Camera
balance and can apply noise reduction, sensor dust removal, etc. using proper software. Canon introduced S-RAW and M-RAW remains in the same position at all times May goes to a predefined location depending on where the player character is located. (2)
Provides all the advantages of a RAW file, but in a smaller file size (so more images can be captured to a media card). S-RAW: FIRST PERSON CAMERAS: A first-person camera is one that is designed to give the perspective of the player character moving through
approximately 25% the pixel count and approximately half the file size of a RAW image. M-RAW: approximately 55-60% of the pixel the world. 10 Can give an immersive sensation to the player. Very popular in first- person shooter games First-person camera can be
count and approximately two thirds of the file size of a RAW image. S-RAW and M-RAW are suitable for wedding photographers who implemented by placing the camera at eye level of the character avatar (an image that represents you in the game). 11 Problem: When
don’t need full resolution for wedding candidates, but who do want the post- production control that RAW oders. S-RAW and M-RAW player looks straight forward, he cannot see his hands… Problem: If a regular character model is drawn for the player, may get the
could also be used by sports/action photographers who will get an increase in the number of frames when shooting in bursts due to artifact of the camera looking from within the model. Solution: Draw only a special model for the player character, which only has the
using a smaller file size (cameras can process smaller files faster). However, photo prints cannot be larger than A3 size. Graphical arms (and legs) being drawn at an anatomically incorrect position. (3) FOLLOW CAMERA: A follow camera is one that follows behind a
Interchange Format (GIFs): Graphics Interchange Format (GIF) relies on LZW compression, which is lossless. Takes advantage of target object in various ways. Examples: racing game, third-person action game Many diderent implementations: may rigidly follow a
repetition of data Encoding – defines codes to build up dictionary Decoding – recovers codes based on the dictionary LZW was target, or with some degrees of springiness; may also allow rotation of camera or to look behind. (4) CUTSCENE CAMERA: A cutscene
patented and cause problem for GIF usage PNG (a format specifically designed to succeed GIF and to avoid patent which was expired is a non-interactive scene sequence that cut away from the actual gameplay in order to advance to the story of the game, e.g.,
in 2003). Good for graphics such as cartoon, line art, etc Not good for photos! Maximum 256 color (i.e., 8 bits per pixel), needs color conversations between characters. 14 For some cutscenes, we need to interpolate camera positions and orientations for compositing
table (some colors may be missing for photos) GIF contains block for color table, data, extension… Block “Graphic Control Extension” a smooth camera sequence. Camera Interpolation for Cut-Scene Cameras: For cutscene cameras, very often we need to determine
contains delay time (to allow animation edect) and transparency color. PNG (Portable Network Graphics): Lossless data compression a smooth transition of the camera over a given set of camera positions and look at directions. The problem is to compute the up vectors
DEFLATE (in zlib) • No patent license • Support transparency • Not limited to 256 colors (palette, grayscale or RGB) • PNG does not at each camera positions and the goal is often to minimize the rotation between two consecutive frames. Z-FIGHTING: The
support animation. MNG is an extension to PNG with animation. • 8-byte signature followed by chunks: – PLTE for palette; list of colors phenomenon in which two fragments from diderent objects that are close to each other and far from the camera will flicker back and
– IDAT for image data – IEND for marking the image end. JPEG( Joint Photographic Experts Group): Introduced in 1992 Good for natural forth on alternating frames when being rendered. Two or more fragments have similar or identical values in the z-buder (buder for
image and photo (lots of color levels and removing some of them cannot be easily noticed) Digital camera photo native format For 640 storing depth information). It is particularly prevalent with coplanar polygons, where two faces occupy essentially the same space.
x 480 24-bit image BMP: need 900 kB JPEG may become 30 kB Can set diderent compression rate for diderent qualities (usually high Problem alleviation: Use of a higher resolution depth buder Moving the polygons further apart. Post transformation to eliminate
quality, medium quality and low quality). Advantages: Smaller file sizes (more images can be stored on a CF or SD card) Images are overlap.
easy to view, mail and print than RAW files Disadvantages: Reduced post-processing flexibility Reduced color depth and resolution
Need to get everything correct in-camera (only limited computer processing is possible). . Problem with art line Much computation CHAPTER 12: U.I. TYPES OF TOUCH SCREENS. (1) TOUCH SCREEN: In 1984, Bell Labs engineered a touch screen for changing
power EXIF (Exchangeable Image File format) for information, such as thumbnail image, time, date, orientation, aperture, shutter images with hands. Applied to PDA, watch, phone. (2) MULTI-TOUCH: In 2005, Apple acquired Fingerworks, a company run by
speed, focal length of the camera. Motion JPEG (M-JPEG): Multiple JPEG files to simulate a movie Motion JPEG (M-JPEG) encode each University of Delaware. In 2007, iPhone was launched with multi-touch UI. In 2007, Microsoft Surface was launched, a multi- touch
video frame or interlaced field of a digital video sequence by JPEG images Mainly used in digital camera, IP-CAM Sound is often table-top computer. And Windows 7 supports multi-touch UI also. (3) RESISTIVE TOUCH SCREEN: In some Android phones Two
uncompressed PCM or low- compressed ADPCM Content was typically stored in the popular Windows AVI or Apple Quicktime MOV. flexible sheets coated with a resistive material and separated by an air gap A uniform, unidirectional voltage gradient is applied to the
MPEG (Moving Picture Experts Group) : High compression by exploiting both intra-frame and inter- frame correlation: I (Intra, or Index) first sheet. When the two sheets are pressed together, the second sheet measures the voltage as distance along the first sheet,
pic: only intra-frame coded. P (Predicted) pic: with reference to a previous picture (I or P). Same quality as I picture. B (Bidirection) pic: providing the X coordinate. When this contact coordinate has been acquired, the voltage gradient is applied to the second sheet to
high compression based on interpolation, need both a past and a future picture. MPEG1 in VCD • MPEG2 in DVD • MPEG4 in mobile ascertain the Y coordinate. Contact can be made with a finger or any other pointing device. (4) CAPACITIVE TOUCH SCREEN: In iPhone
phone / camera / internet • MP3 is audio for MPEG1 Layer 3. JPEG 2000: JPEG successor with improved compression / quality / iPad and some other Android phones Consists of an insulator such as glass, coated with a transparent conductor. As the human
Developed from 1997 to 2000 by a Joint Photographic Experts Group committee chaired by Touradj Ebrahimi (later the JPEG president). body is also an electrical conductor, touching the surface of the screen results in a distortion of the screen's electrostatic field,
A discrete wavelet transform to decompose the image pixels into subbands (for diderent frequency components of the image) Each measurable as a change in capacitance. Diderent technologies may be used to determine the location of the touch. The location is
subband is then partitioned into relatively small blocks known as code blocks, which are then coded independently, into embedded then sent to the controller for processing. Contact must be made with a finger. (5) FORCE-TOUCH: For Force click, Pressure Sensitive
block bit streams. Unfortunately, it is not as popular as JPEG 1992 up to today. BITMAP FOR DIFFERENT SCREENS – SCANNING Gesture. Smart Glasses: Augmented reality head-mounted display Hands-free displaying of information Examples Sony Glass Xiaomi
SYSTEM: Progressive scanning (p), redraws an image frame (all of its lines) when refreshing each image. Interlaced scanning (i), draws Smart Glass. VIRTUAL REALITY: An artificial, computer-generated simulation or recreation of a real-life environment or situation
the image field ("odd numbered" lines) during the first image refresh, and then draws the remaining "even numbered" lines during a Immerses the user by making them feel like they are experiencing the simulated reality first-hand, primarily by stimulating their vision
second refreshing. Interlaced yields a loss of up to half of the resolution when the subject is moving. Bitmap Scaling: To make a bitmap and hearing Made possible through a coding language known as VRML (Virtual Reality Modelling Language) which can be used to
smaller, say from 10x10 to 8x8 To make a bitmap larger, say from 8x8 to 10x10 Line drawing algorithm for horizontal line draw line: x for create a series of images and specify what types of interactions are possible for them. AUGMENTED REALITY: A technology that layers
source width, y for destination width Enlarge: copy some pixels Shrink: delete some pixels Vertical handling copy some lines (for computer-generated enhancements atop an existing reality in order to make it more meaningful through the ability to interact with it
enlarge) delete some lines (for shrink) Better scaling: Sub sampling. Anti- Aliasing and Transformation: Transforming a rectangular Developed into apps and used on mobile devices to blend digital components into the real world in such a way that they enhance one
bitmap into any quadrilateral. • Calculate color area covered inside a pixel and become finalized color value. Fooling the eye into another but can also be told apart easily. VUFORIA: Vuforia enables Unity developers to create engaging AR experiences and reach the
thinking a jagged edge on an image is smooth. Font smoothing Bitmap re-sampling and scaling Scalable graphics, e.g., Flash graphics broadest possible audience Vuforia uses the device’s camera feed combined with accelerometer and gyroscope data to examine the
Disintegrating textures. SUBPIXEL RENDERING: It is a way to increase the apparent resolution of a computer's liquid crystal display world. Vuforia uses computer vision to compute the camera’s view, to build a model on top of the environment. After setting up the
(LCD) or organic light-emitting diode (OLED) display by rendering pixels to take into account the screen type's physical properties. It environment, the device is able to augment the model in the environment, by roughly identifying its coordinates. MIXED REALITY:
takes advantage of the fact that each pixel on a color LCD is actually composed of individual red, green, and blue or other color Merging of real and virtual worlds to produce new environments and visualizations Physical and digital objects co- exist and interact
subpixels to anti-alias text with greater detail or to increase the resolution of all image types on layouts which are specifically designed in real time. Encompassing both augmented reality and augmented virtuality via immersive technology. One of the most notable early
to be compatible with subpixel rendering. Improves display resolution by as much as 300% LCD are arrayed in vertical stripes with 3 mixed reality apparatuses: Microsoft’s HoloLens. METAVERSE: Metaverse is a hypothetical iteration of the Internet as a single,
sub-pixels of Red, Green, Blue Subpixel rendering technique to improve the readability of text Used in Microsoft ClearType, announced universal, and immersive virtual world that is facilitated using virtual reality (VR) and augmented reality (AR) headsets. Metaverse is
in 1998 Already used in Apple II (1976). Preparing Graphics: Cartoons, removing backgrounds, making bg transparent since character also a network of 3D virtual worlds focused on social connection. MOTION DETECTION BY MEMS: Micro-Electro-Mechanical Systems
moves on top of another bg image. (MEMS) accelerometer for detection of acceleration Basic principle of operation: Displacement of a small proof mass etched into the
silicon surface of the integrated circuit and suspended by small beams. The support beams act as a spring, and the fluid (usually air)
CHAPTER 5: 2D GRAPHICS: PART 1: SCREEN RENDERING: REFRESH RATE: The number of times a screen can be redrawn in one trapped inside the IC acts as a damper, resulting in a second order lumped physical system. MOTION DETECTION BY CAMERA: A low-
second is called the refresh rate. you may set your monitor to refresh at a rate of 60Hz, 80Hz, depending on the hardware capability of cost camera for detecting light pattern Example: TikTokTech Limited, obtained HK Digital Entertainment Award 2007 A camera-
your monitor. The most common monitor refresh rate for a monitor is 60 Hz. FRAME RATE: Frame rate (also called Frame Per Second, interactive TV game system Camera-based pattern recognition tracking technology, with wireless Magic Wand. Players can draw freely
FPS) is the rate that your game (or GPU) can output a frame (a complete image in the framebuder) in one second. Our eyes have in the air and interact intuitively. Technological Originality and Content. MOTION DETECTION BY KINECT: infrared technology for depth
persistence of vision (i.e., the ability to retain an image for a very short time even after it disappears) which enables us to perceive info- Azure Kinect Developer Kit. (Force Feedback)-Steering Wheel/Joystick.
motion naturally. The minimum FPS is about 25-30 in order for us to perceive smooth animation.
CHAPTER 6: 3D Graphics PART 1: TRANSFORMATIONS Left Handed Cartesian – Unity, Right handed cartesian – 3D Studio Max. In
the Unity GUI and GUILayout classes, the y-axis is also flipped (i.e., y-coordinates increase downwards). Vectors and Points can be
Linear interpolation uses weighted average for finding intermediate positions. Unity Functions for interpolation of points/vectors:
Mathf.Lerp() Linearly interpolates between a and b by t. The parameter t is clamped to the range [0, 1]. When t = 0 returns a. When t =
1 return b. When t = 0.5 returns the midpoint of a and b. If an object moves with constant speed, we can make use of Mathf.Lerp()
function. What if an object moves with acceleration? Assuming you want an object to go from 0-10 in 5 seconds. A = final velocity - Bounding Spheres: 1. Compute the center of mass for all vertices, and use it as the center of sphere C. 2. Compute the largest distance
current velocity / time A = 10 - 0 / 5 = 2 units/second to speed up. Matrix Transformations: We may apply diderent transformations to from all vertices to C, and take it as the sphere radius r. Collision tests between bounding spheres are trivial: Two bounding spheres
alter the position of a point, and thus the shape of an object. TRANSFORMATION example of a point: (Example 1) : Rotation by an intersect if and only if: (C1 – C2).(C1 – C2) < (r1 + r2)2. Axis-aligned bounding box (AABB) is a rectangular prism where every side of the
angle THETA prism is parallel to one of the coordinate axis planes in the game world. Oriented bounding box (OBB) also uses a rectangular prism to
bound an object, but the sides of the prism are not required to align with the coordinates planes. Capsule: is a cylinder with two
hemispheres attached to the top and bottom caps. It can also be thought of as a sphere-swept line (SSL) (or a line segment with a
radius). How to perform collision detection between two capsules? Compute d (using above formula), the shortest distance between
two line segments. If d <= r1 + r2, then the two capsules collide. Separating Axis Theorem: Two convex shapes do not intersect if and
(Example 2 – Scaling by S) and (Example 3- Translation by a vector v) only if there is an axis along which the projection of the two shapes do not overlap. The axis is called the separating axis of the two
shapes. The theorem does not apply to concave shapes: the shapes do not intersect even though there is no separating axis found.
Bounding Volume Hierarchy (BVH) is to maintain multiple levels of bounding volumes with increasing bounding tightness. Continuous
collision detection (CCD) is to detect if there is any collision between two moving objects within a time span.

CHAPTER 8: Network: Peer-to-Peer Connection: Clients broadcast messages to each other, Easy to code, But need more bandwidth
Functions in Unity – (1 ) transform.Translate: public void Translate(float x, float y, float z, Transform relativeTo); Moves the transform for each client (especially when graphics are involved), Limited number of players, Fault tolerant, High security risk. Client-Server
by x along the x axis, y along the y axis, and z along the z axis. The movement is applied relative to relativeTo's local coordinate system. Connection: One server serves many clients, More complicated coding, Can support many players, The server needs more bandwidth,
If relativeTo is null, the movement is applied relative to the world coordinate system. (2) transform.Rotate public void Rotate(Vector3 Expensive, Improved security, Not fault tolerant, Dedicated server. Networking Problems and Solutions: 1. Limited network bandwidth:
eulerAngles, Space relativeTo = Space.Self); Applies a rotation of eulerAngles.z degrees around the z axis, eulerAngles.x degrees Data packet size is important; Solutions: Data compression, Lower update frame rate. 2. Low update frame rate: Synchronization is
around the x axis, and eulerAngles.y degrees around the y axis (in that order). If relativeTo is left out or set to Space.Self the rotation is the key; Solutions: Position Extrapolation, Position Prediction, Position Chasing. Position Extrapolation: Use Position (i-1) & Position (i)
applied around the transform's local axes. If relativeTo is Space.World the rotation is applied around the world x, y, z axes. to get the next possible Position (i+1), Use current facing direction to predict next position. Position Chasing: Always being late for one
Transformation Rule (Easy): Rotate and Scalar is multiplied, Translate is added To apply multiple transformations to a point, transform step as the master, Not good for moving fast objects. Zone Concept: A zone is logically a region of game happening on servers. A zone
in the order of right to left. Example 1. Suppose we want to first rotate a point by R1, then scale by S, then rotate again by R2, we have, is not really a physical hardware server. A zone is usually mapped to a scene in the gameplay. A software region that the players
then P’ = R2 .S. R1p = M (matrix). Example 2 : first rotate a point by R1, then translate by v, then rotate again by R2, we have: p’ = R2 . (v communicate directly (in the same memory block). Dynamic Zoning: A zone is divided into several groups. A group is a collection of
+ R1.p) => p’ = R2.v + R2.R1.p. players in the scene (population). All players in the same group run the same process. Players from diderent groups communicate
Homogenous Coordinates: Definition: Any point in the projective plane is represented by a triple (X, Y, Z), called homogeneous between processes (Inter-process communication (IPC)). Population: When a player logins a zone, he/she should be assigned into
coordinates or projective coordinates of the point, where X, Y and Z are not all 0. We represent 2D transformations using 3x3 matrices, one available population group. A player should not move between population groups when he/she is staying in this zone. Groups can
and 3D transformations using 4x4 matrices A 2D point (px, py ) is represented as (px, py, 1) A 3D point (px, py, pz) is represented as (px, be created in, deleted from or moved across servers for load balancing purpose. Unity Networking: Peer-to-peer connections can be
py, pz, 1). Converting Homogenous to Non-Homogenous Coordinates: A point (x, y, w) in homogeneous coordinates can be converted established using NetworkManager. Client-to-server connections can rely on Photon Unity Networking (PUN), which is a Unity
into non-homogeneous coordinates in 2D by dividing the x, y components by the w component: Example: (x,y,W) is non homo. To package for multiplayer games. Always connects to a dedicated server. Provides authentication options, matchmaking and fast,
convert to Homo, divide all by W. New homo point becomes (x/W, y/W). Two points (wx, wy, w) and (ux, uy, u) in homogeneous reliable in-game communication through the Photon backend. Behind the scenes, PUN uses more than one server. Several "Game
coordinates are the same. (Question can be asked using actual values). Divtej’s Example-> Is (9,6,3) and (12,8,4) homogenous? YES. Servers" run the actual rooms (matches) while a "Master Server" keeps track of rooms and match players.
VECTORS Transformations NOTE: Translation has no edect on vectors. In homogenous coordinates, a vector (vx, vy) is
represented as (vx, vy, 0). (if we divide by by 0, it indicates that vector’s direction).VERY IMPORTANT DIAGRAM; 3D/ 2D Transformations CHAPTER 9: AI for Computer Games: Artificial Neural Networks (ANNs): A family of models inspired by biological neural networks
(the central nervous systems of animals, in particular the brain). Used to estimate or approximate functions that can depend on
a large number of inputs and are generally unknown. Game Tree: Objective: Tells the probability of winning (or losing) for each possible
next move. In game theory, a game tree is a directed graph whose nodes are positions (stages) in a game and whose edges are moves.
The complete game tree for a game is the game tree starting at the initial position and containing all possible moves from each position.
The number of leaf nodes in the complete game tree is the number of possible diderent ways the game can be played. E.g., the game
tree for tic-tac-toe has 255,168 leaf nodes. Alpha-Beta Pruning:

INVERSE Transformations: if p’ = (M) . p; then p = (M^-1). p’; take inverse of M and multiply with p’ to bring back p. Example: Matrix
Representing a 2D rotation of 45* about a point (1, 1)?. M = M1(translation by 1,1) * M2(rotation by 45) * M3(trans by -1,-1) .(m3 for
bringing to origin.). MORE ON ROTATIONS: Euler Angles: We may create any rotation in 3D by specifying a sequence of three rotations
about the x-, y-, z-axis. The 3 angles of rotations are called the Euler angles. There is no correct definition of the order of rotations and
the most commonly used definition for describing a rotation is using Euler angles (phi, theta, gamma). (aircraft case-> yaw, pitch and
roll).Question: What is the problem representing rotations as matrices? And how do quaternions solve it?: Problems: Need 16 floating-
point values to represent a rotation (in homogeneous coordinate system) Rotating a vector requires vector-matrix multiplication, which
involves 4 dot products (= 16 multiplications and 12 additions). Unit Quaternions: Mathematical object developed by Sir William Rowan
Hamilton in 1843 as an extension to the complex numbers A quaternion is represented as a 4-dimensional vector, but it behaves
diaerently. Unit-length quaternions represent 3D rotations. q=cos(θ/2)+sin(θ/2)(ix+jy+kz); FORWARD AND INVERSE KINEMATICS:
Forward Kinematics The calculation of the position and orientation of an end edector / target using the variables (a, b, L) of the joints
and linkages connecting to the end edector. Given the current positions, angles, and orientation of the joints and linkages, forward
kinematics can be used to calculate the position and orientation of the end edector. Inverse Kinematics The calculation of the
variables (a, b, L) of the set of joints and linkages connected to an end edector / target. Given the position and orientation of the end
edector, inverse kinematics can be used to calculate the variables regarding those joints and linkages including position, angle, and
orientation.
Chap6 PART 2: RENDERING, LIGHTS, SURFACES, REFLECTIONS: Steps to render a 3D Scene: Basic steps: 1. Setting up a virtual Search space: All locations in a game world that a non-player character may assess constitute a search space, i.e., a space where a
scene 2. Positioning a virtual camera 3. Defining light sources 4. Defining the visual properties of surfaces 5. Computing the colour solution to the pathfinding problem can be sought. A graph is a common abstract representation of a search space (represented as a
and intensity of the light rays "shining" on a screen pixel. Question: MODEL SPACE AND WORLD SPACE: Model Space: The points of set of adjacency lists, in which each node points to a list of adjacent nodes.) Path Nodes: A path node is placed manually by level
an object in a game world is positioned relatively to its own local coordinate system called model space, local space or object space. designer at locations that the AI can reach. Edges are created either manually or automatically as long as they do not cross any
The origin of model space is either the centre of an object or at some convenient location. World Space: The game world has a obstacles. Tradeod between performance and accuracy: • Too many nodes: takes longer time for pathfinding • Too few nodes: resulting
coordinate system common to all objects, and this is called the world space. Each object has a position and orientation relative to the in an unnatural path. Heuristic Algorithms: Greedy Best-First Algorithm: Explore all neighboring nodes of the current node. Pick one
world origin. Apply a model-to-world transformation to transform all points on an object to the world space. CAMERA PLACEMENT and which has the lowest cost towards the destination. Dijkstra's Algorithm: Explore all neighboring nodes of the current node. Point each
PROJECTIONS: Question: PERSPECTIVE AND ORTHOGONAL PROJECTIONS: Determines how a 3D scene is rendered onto the 2D node to a parent which has the lowest cost towards the source (i.e., process each node). A* Algorithm: Explore all neighboring nodes
display. 9 Perspective projection Objects further away looks smaller than objects closer to the camera. Orthographic projection of the current node. Pick one which has the lowest cost towards both the destination and the source.
Objects further away have the same size as objects closer to the camera. Diderences: Perspective projection gives a sense of depth,
while orthographic projection is better if measurements or comparisons of lengths are needed. Most 3D games use perspective CHAPTER 10: Optimization: Shifting: For some platforms, multiplication and division are slower than operations like shifting / addition
transformation. Pure 2D games can be thought of as using an orthographic projection. Some games like Diablo use isometric / subtraction by 10 to 50 times. Left shifting: Symbol: <<; N << 2 = N * 4; E.g. N = 2 = 102; N << 2 = 10002 = 8; Thus, N << i = N * 2i. Right
projection (orthographic projection at an angle) to convey 3D information. LIGHT SOURCES: LIST and Diderences: (1) AMBIENT LIGHT; shifting: Symbol: >>; N >> 2 = N / 4; E.g. N = 8 = 10002; N >> 2 = 102 = 2; Thus, N >> i = N / 2i. 24-bit color representation: each of the 3
uniform amount of light is applied to every object in the scene Independent of viewing angle and has no specific direction. (2) POINT basic colors (channels) red (R), green (G) and blue (B) is represented using an 8-bit number. Assume the 3 colors are now R = 10101010,
LIGHT/ Omni LIGHT: Point light (omni light) – Emits from a specific point and casts light in all direction – Can illuminate only one side G = 01010101, B = 11110000. The 24-bit color representation can be obtained by: R << 16 = 10101010 00000000 00000000; G << 8 =
of an object – Has attenuation, i.e., a fallod radius that determines how much light decreases as the distance from the light increases. 00000000 01010101 00000000; R << 16 + G << 8 + B = 10101010 01010101 11110000.
(3) SPOTLIGHT: use of spotlight: - Lock the direction of spotlight on a moving object - Attach the spotlight to a moving object such as
the player (like holding a torch). Has its position and attenuation Can focus the emitted light in a cone parameter: orientation
(direction), angle. (4) Directional light Without position but is emitted from a specific direction Parallel light from infinity Best for
emulating sunlight. (5) Area light Simulate light coming from a large surface area, e.g., large rectangular ceiling light boxes Can cast
soft shadow that add realism to a scene Computationally intensive, cannot be done in real-time Can be simulated using several point
lights or spotlights. VISUAL PROPERTIES OF SURFACES: Types of Objects; Objects in a scene might be Opaque: light cannot pass
through its volume Transparent: light passes through without scattered Translucent: light passes through and is scattered in all
directions. Question: OBJECT PROPERTIES/ DIFFERENCES: (ALPHA VALUE): Opaque objects can be rendered by considering only
their surfaces. For transparent or translucent object, we need to model how light is reflected, refracted, scattered, and absorbed as it
passes through the object's volume. We usually use an alpha value to describe the opacity (or transparency) of a surface and render a
transparent/translucent object almost the same way as an opaque object. Hence, games primarily concern about rendering surfaces.
TYPES OF SURFACES: (1) Spline, (2) Subdivided, (3) Triangle Mesh: Spline surfaces: defines a surface as patches using a small number
of control polygons. Subdivision surfaces: like spline surfaces, a surface is represented by a mesh of control polygons. Subdivision
rules (e.g., Catmull-Clark algorithm) can be applied to iteratively subdivide a polygon and thereby refining a surface. Subdivision level
can be up to pixel resolution. Triangle Meshes: Game developers mostly modelled their surfaces using triangle meshes, which are
piecewise linear approximation to a surface. Triangles are the polygon of choice for real-time rendering because simplest type of
polygon a triangle is always planar remains a triangle under most kinds of transformation virtually all commercial graphics-
acceleration hardware is designed around triangle rasterization. Coarse decimation may result in bad visual quality, but fine
decimation entails a large storage space Use meshes at diderent levels of detail (LOD), depending on the distance between object
and camera. VERTEX ATTRIBUTES IN TRIANGLE MESH: Each vertex in a mesh is attached with some attributes which determine the
visual properties of the surface, e.g., Position: defines the shape Texture coordinates: for texture mapping Surface normal: used for
shading computation. SHADING: 3 TYPES – (1) flat shading, (2) Gourand Shading, (3) Phong Shading. Why Need shading: Shading
specifies how the surface of a triangle is filled in for every pixel that it occupies on screen. To determine how a triangle should look on
screen, it needs to undergo some lighting computations (to account for how lights and surfaces interact in a scene).(1) Flat shading is
to carry out light calculation once for each triangle, and the entire triangle will be filled with the same color. (2) Gourand SHading:
Lighting calculation is done once for each vertex, which may then have diderent colors The rest of the triangle is then filled by bilinearly
interpolating the colors at the 3 vertices. (3) Phong Shading Lighting calculation is carried out for every pixel covered by a triangle.
Surface normal at a pixel is estimated using bilinear interpolation. LOCAL ILLUMINATION MODELS account for direct lighting in which
light is emitted, bounces od a single object in the scene, and then proceeds to the image plane of the camera The most common
local illumination model is the Phong reflection model, which models light reflected from a surface as a sum of three terms: 1 -
Ambient, 2-Diduse, 3- Specular. (1) Ambient An approximation of the overall illumination of the scene Regions in shadow will not
appear totally black Independent of the location of lights and the camera. (2) Diduse Accounts for lights that is reflected uniformly in
all directions from each direct light source (directional, point, or spotlights) adecting an object Depends on N (surface normal) and L
(vector from surface to light); independent of view direction. (3) Specular Represents shiny highlights on the surface Occurs when
viewing angle is closely aligned with the direction of the reflected light R. Phong Diiusion Model = Ambient + Diiuse + Specular.
GLOBAL ILLUMINATION achieves true photorealism by accounting for indirect lighting, where light bounces multiple times od many
surfaces before reaching the virtual camera.

CHAPTER 7: COLLISIONS: Equation of a line : (y-y1) = m(x-x1) point slope form. M = y2-y1/x2-x1. Collision detection determines
whether or not two game objects intersect with each other. Why is collision detection based on triangle mesh comparison inedicient?
Checking whether two polygons or two triangles collide is easy. A complex character in a modern 3D game can be composed of up to
40,000 triangles. To check whether any two triangles collide, how many comparisons are required? 40,000 x 40,000 = 1,600,000,000
➔ highly inedicient!. Alternate/Solution: Use simple collision geometry to approximate a game object. These primitives are not drawn
on screen, but are used only for edicient collision tests. (1) Spheres (2) Boxes (3) Capsule (4) Convex polygons / polyhedrons.