Scribd
Upload
52 views

Image Processing: Thinhuv - Ios Team

This document provides an overview of basic image processing concepts and operations in iOS, including: - Images are 2D arrays of pixels that can be represented as bitmaps using individual color/transparency values or vectors using mathematical formulas. - Image processing involves applying mathematical operations like filters to images to modify properties and output a result. Core Image provides built-in filters and the ability to chain multiple filters. - Advanced operations can utilize the GPU through APIs like OpenGL, Metal, and shaders to perform image processing much faster than on the CPU. Optimization techniques include using singletons for CIContext and clearing caches.

Uploaded by

Lê Phương Tiến
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
52 views

Image Processing: Thinhuv - Ios Team

This document provides an overview of basic image processing concepts and operations in iOS, including: - Images are 2D arrays of pixels that can be represented as bitmaps using individual color/transparency values or vectors using mathematical formulas. - Image processing involves applying mathematical operations like filters to images to modify properties and output a result. Core Image provides built-in filters and the ability to chain multiple filters. - Advanced operations can utilize the GPU through APIs like OpenGL, Metal, and shaders to perform image processing much faster than on the CPU. Optimization techniques include using singletons for CIContext and clearing caches.

Uploaded by

Lê Phương Tiến
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 22

Image Processing

ThinhUV - iOS Team


Contents
1. Basic concepts
2. Advanced Operations
3. Optimization
4. Demo
5. Q&A
Basic concepts
• What’s Image?

Image is a 2-D array of pixels


- Pixels are samples
- Image has limited resolution
Basic concepts
• Bitmap
- A set of pixels with color and transparency
-> Break into pieces when zoom in
• Vector
- Use methematical fomulas to draw lines and curves
Basic concepts
● Color space
- RGB
- Grayscale
- HSV

● Blending
Basic concepts
• What’s Image Processing?
=> Input + mathematical operation -> output
Input: image, filter, video, output of another one
Mathematical operation: fill color, detect egde, detect face
algorithm, color lookup, Gaussian algorithm …
Output: like input :D
Basic concepts
● Graphic Context: a global state object that holds all the
infomation for drawing.
○ Store resource, state
○ Generate result …
Basic concepts
• Filter: allows us to apply various effects on image, able
chaining
• Capture mode: still and real-time
Basic concepts
What are Apple definitions?

● Core Image
- Greate framework built-in on Apple systems.
- Operate data types from Core Graphics, Core Video and
Image I/O frameworks using either a GPU or CPU rendering
path.
Basic concepts
● Context: CIContext, CGContextRef
● Input and output representation: CIImage, CGImage,
UIImage, Video(Data)
● Filter: CIFilter
Basic concepts
• CIContext
An evaluation context for rendering image processing
results and performing image analysis.

import CoreImage

let context = CIContext(options: nil)


ciContext = CIContext(options: nil)

// Advanced options will be present later


Basic concepts
● CIFilter
- A representation of an image to be processed or produced
by Core Image filters
- Apply for video
- Apply for Game Scene
Basic concepts
● CIFilter
import CoreImage

let filter = CIFilter(name: “CIBoxBlur”)

More 170 built-in filters (Referrence)


Basic concepts
● CIFilter : chain

Input -> filter 1 -> filter 2 -> filter 3 … -> ouput

* Quickly and efficiently


Advanced 0perations
● GPU and CPU
OpenGL and OpenGL ES
import CoreImage

let eaglContext = EAGLContext(api: .openGLES3) ?? EAGLContext(api: .openGLES2) // 1


let ciContext = CIContext(eaglContext: eagl, options: nil) // 2
Advanced 0perations
● Metal: Enabling high performance for graphics rendering
and parallel compute workflows (GPU)

import Metal
let device = MTLCreateSystemDefaultDevice()
let ciContext = CIContext(mtlDevice: device)
Advanced operations
● Shading: use GPU to perform image and video
manipulation much faster than could be done in CPU-
bound routine
● Example: gamma filter
-> 40x faster than Core Image
-> 180x faster than CPU-bound
Advanced operations
● Shader: actual complex to build but very powerful
● Suggested open frameworks: GPUImage

void main()
{
gl_Position = transformMatrix * vec4(position.xyz, 1.0) * orthographicMatrix;
textureCoordinate = inputTextureCoordinate.xy;
}
)
Advanced operations
● Kernel: actual complex to build but very powerful
● Suggested open frameworks: Vivid

kernel vec4 filterKernel(__sample image, __sample blurredImage) {


return vec4(vec3(image.rgb - blurredImage.rgb + vec3(0.5,0.5,0.5)), image.a);
}
Optimization
• Create singleton for CIContext (very important)
• Clear cache if possible
• If the filters chain rendering is slow then try to reorder
filters
• Use autoreleasepool if possible
DEMO

You might also like