This document provides an overview of mathematics in visual data processing. It discusses topics like digital image processing, analysis, and computer vision. Digital image processing involves tasks like restoration, enhancement, compression. Image analysis includes segmentation and object recognition. Computer vision aims to recover 3D structure from 2D images. The document outlines related mathematical topics that have applications in these areas, including spectral analysis, wavelets, snakes, level sets, and regularization. It provides examples of applications such as deblurring, denoising, sampling, compression, registration and segmentation.
This document provides an overview of mathematics in visual data processing. It discusses topics like digital image processing, analysis, and computer vision. Digital image processing involves tasks like restoration, enhancement, compression. Image analysis includes segmentation and object recognition. Computer vision aims to recover 3D structure from 2D images. The document outlines related mathematical topics that have applications in these areas, including spectral analysis, wavelets, snakes, level sets, and regularization. It provides examples of applications such as deblurring, denoising, sampling, compression, registration and segmentation.
This document provides an overview of mathematics in visual data processing. It discusses topics like digital image processing, analysis, and computer vision. Digital image processing involves tasks like restoration, enhancement, compression. Image analysis includes segmentation and object recognition. Computer vision aims to recover 3D structure from 2D images. The document outlines related mathematical topics that have applications in these areas, including spectral analysis, wavelets, snakes, level sets, and regularization. It provides examples of applications such as deblurring, denoising, sampling, compression, registration and segmentation.
This document provides an overview of mathematics in visual data processing. It discusses topics like digital image processing, analysis, and computer vision. Digital image processing involves tasks like restoration, enhancement, compression. Image analysis includes segmentation and object recognition. Computer vision aims to recover 3D structure from 2D images. The document outlines related mathematical topics that have applications in these areas, including spectral analysis, wavelets, snakes, level sets, and regularization. It provides examples of applications such as deblurring, denoising, sampling, compression, registration and segmentation.
The set of lecture notes are from the following references:
1. G. Aubert and P. Kornprobst, Mathematical problems in image processing, Applied Mathematical Sciences, Vol 147, Springer-Verlag, 2006. 2. S. Mallat, A wavelet tour of signal processing, AP Professional, London, 1997. 3. S. Osher and R. Fedkiw, Level set methods and Dynamic implicit surfaces, Spring Verlag, New York, 2003 4. I. Pitas, Digital image processing algorithm and Applications, John Wiley & Sons, New York, 2000.
Chapter 1
Background 1.1
Overview
Human vision is one of the fundamental perception mechanisms by providing
visual information needed for many tasks. As the Chinese proverb says One picture is worth a thousand words, it is actually very surprising to realize the tremendous amount of information contained in a single picture. With the rapid advances in digital image acquisition devices and the fast growth in computational capability, a new discipline known as image processing and computer vision emerged since the sixties in last century. Since then, it has exhibited a tremendous growth in industial applications and scientific research. Image processing concerns the transformation of an image to a digital format and its processing by digital computer. Computer vision is more related to the description and recognition of the digital image content. One classification of image processing and computer vision has three distinct classes:
Figure 1.1: The human visual system can interpret the subtle variation in transparency, shading in this image, foreground flower and blurred background
CHAPTER 1. BACKGROUND 1. low-level vision are essentially digital image processing. Its input and output are both digital images. 2. middle-level vision has the digital images as the input and low-level symbolic representation of image content as the output, e.g., representation of object contour. 3. high-level vision take either digital images or low-level symbolic content as the input and high-level symbolic representation as the output, which tries to simulate the high level of human visual perception, e.g., 3D structure of the world, object recognition.
1.2
Digital image processing
This section is devoted to a brief overview of the various topics of digital
image processing. Digital image formation is the first step in any digital image processing. In typical digital imaging system, the optical signal is first captured to an electrical signal by a sensing device, followed by a digitizer to be converted to a digital one. Thus, current digital imaging systems always introduce some degree of deformations and degradations to the captured digital images, e.g., geometrical distortion, noise. Image restoration concerns the reduction of the deformations and degradations introduced during the digital image capturing. The goal of image restoration is to reconstruct or to recover a better version of the digital image. Image enhancement concerns the improvement of the visual quality of the digital image. The typical image enhancement techniques usually include contrast enhancement, digital image sharpening and noise reduction. One important concept which plays the essential role in image restoration and enhancement is image frequency content. Digital image transforms are used to obtain the image frequency content (or related measurements). Thus, transform theory is an integral part through most digital image processes. Digital images requires a large amount of memory for their storage. A color image of size 1024 758 pixels easily needs 2+ MB disk space without any optimization. Thus, the reduction of the storage is important to many applications. digital image compression take advantages of massive redundancy existing in most real life images to aggressively compress it.
1.2. DIGITAL IMAGE PROCESSING
(a) Noisy image
(b) de-noised image
Figure 1.2: Removing noise from images
(a) Blurred image
(b) De-blurred image
Figure 1.3: Removing motion blurring from images
CHAPTER 1. BACKGROUND
(a) Bio-mage of cells
(b) Cell segmentation
Figure 1.4: Image segmentation
1.3
Digital image analysis
The first step in digital image analysis is often the detection of image features, e.g., edges, lines, feature points. These low-level image features should be robust to image noise and also they should be robust to various environmental changes, dependent on the subsequent image analysis tasks. One fundamental problem in image analysis is the segmentation. The goal of image segmentation algorithms is to partition an image into multiple segmentations such that the image can be represented in a more meaningful way. Image segmentation typically is used to locate objects and their boundaries. The result of image segmentation is a set of segments and pixels in each segment are similar with respect to some characteristic property, e.g., color, texture. Object recognition is another important topic in digital image analysis. The goal of object recognition is to find a given object in an image or video sequence. Human visual system can recognize objects in image with ease, despite the fact that the look of the same object in different images could vary somewhat in different views, in different sizes or even partially obstructed from the view. The main challenges in object recognition is to develop representation scheme for images to match the reliability of human perception.
1.4. COMPUTER VISION
Figure 1.5: Recognizing horses
(a)
(b)
Figure 1.6: 3D reconstruction from multiple images
1.4
Computer vision
human visual system perceive the 3D structure of the world around us from 2D imagery. In parallel, computer vision focus on how to recover the 3D shape and appearance of objects from 2D images. Computer vision has been used in a wide range of applications, which includes: machine inspection, 3D model building, medical imaging, motion capture in animation, surveillance, etc. While all these applications are extremely important, they mostly pertains to fairly specified imagery and narrow domain. The current vision techniques are far away from the dream of having a machinery which interpret an image at the same level as human beings. The main reason why why vision is so difficult is because it is an inverse
CHAPTER 1. BACKGROUND
(a)
(b)
Figure 1.7: Some optical illusions and what they might tell us about the human visual system. problem which is to recover some unknowns given in-sufficient information to fully specify the solution. The solver to this inverse problem therefore need accurate model to disambiguate between solutions. However, the model of the visual world is overwhelmingly difficult considering the rich complexity of the real world.
1.5
Related mathematical topics
Image processing has traditionally been an area in applied mathematics research as image transforms play an essential role in most image processing tasks. Most image transforms used in the theoretical and implementational tools are liner ones, notably in image filtering, restoration, encoding and analysis. The basic transforms are Fourier transform with a long history and wavelet transform which have been popular in last decades. Also, Variational principle and PDE-based approaches see their widely applications in-developed to solve many image processing tasks, e.g., image segmentation. Many algorithms in image processing and computer vision are tackling inverse problems to revert image formation processes, which are often closely related to the optimization techniques. This note is to give a rigorous treatment of various important mathematical topics which have wide applications. The main math topics will be covered in this note include: 1. Spectral analysis and digital filtering 2. Gabor transform and wavelets
1.5. RELATED MATHEMATICAL TOPICS
3. Snake and level set method 4. Basics in Regularization The following applications will be touched lightly in this module: 1. Image deblurring and denoising 2. Image sampling and compression 3. Image registration and image segmentation
