Image convolution is probably the most important operation in image processing. I'll start the talk about convolution from image noise. There are several types of image noise but I will consider only one of them, additive Gaussian noise. In this case noise is added to the true signal. Intensity of the noise is drawn from a Gaussian normal distribution. Each pixel is corrupted randomly and independently from all other pixels. Given a camera in a static scene, what are the easiest way to remove the noise? Because noise is random and independent, the simplest way is to capture several images and average them. Because noise is random, averaging will reduce it. But usually has only one noisy image and cannot average several images. So we need to devise another way for reducing the noise in image. Instead of averaging pixels in several images, we can replace each pixel X, with weighted average of its local neighborhood. Weights are jointly named as filter kernel. The simplest case is equal weights. This particular filter is called the box filter. Sum of all weights in the Filter Kernel should equal to one so that applying the filter does not change overall brightness of image. For clarity, we can write the common factor separately. The process of filtering the image with weighted averaging is called convolution. Let F be the image, and G be the kernel. The output of convolving F with G is denoted F star G. Each pixel is coordinates M, N. In the result of convolution, can be computed as sum of all K and L, of products of F sub M minus K, N minus L by G sub K, G. We can visualize convolution as a scanian of image F is filter kernel on G from top to bottom. The simplest filter is Identity Filter, where only one weight in the center of filter kernel equals to one, and all other weights equals to zero. Applying this filter kernel for image convolution will result in no change in filter of images. But, if you move the only one from center of the kernel to the other place, then the image after convolution will change. It will be shifted. In this case it will be shifted left by one pixel. Generally, any filter kernel is all positive weights and the sum equal to one will be the image smoothing filter. Here's the example of box filter now, imaging with a lot of fine details. You can see that a lot of spurious lines in the right image appear after smoothing. We believe that smoothing operation should look like a image from a defocused lens because it introduced blurred to the image. And we can see that smoothing with average doesn't compare too well with defocused lens. If we try to apply averaging to a single point of light, we will get a little square. If we see the same point light in a defocused lens, it will look like a fuzzy blob. We want to device a filter kernel that will produce the same effect as defocused lens when applying to the point of light. To reach these results, to eliminate edge effects, weight contribution of neighborhood pixels should be according to the closeness to the center, like this image of fuzzy blob. We can set filter weight according to two dimensional Gaussian distribution, centered at the filter center with arbitrary sigma. In this particular example, sigma is set to one and filter size is set five by five. Filters with weights distributed according to Gaussian distribution are called Gaussian filters. And they're probably the most used filters in image processing and image recognition. Gaussian filters have infinite support, but discrete filters use finite kernels. For the same sigma, we can build filters of different sizes. Here for the sigma equals five, two filters with sizes 10 by 10 and 30 by 30 are given. Notice that for the second filter, the weight near the boundaries of the filter are close to zero. As a general rule of thumb, we usually set Gaussian filter half-width to about three sigma. So the total size of Gaussian filter equals to six sigma. In this case as demonstrated in this light, all weights near the edges of the filter are close to zero. Here's the comparison of smoothing this box filter and this Gaussian filter. Please know that spurious lines in the left image are absent in the right image. It can even be proved that Gaussian filter is the only filter that doesn't add any additional information to the image. So no additional signals of spurious lines can appear when applying Gaussian filter. This is one of the reasons why Gaussian filter are so extensively used in algorithms. Applying Gaussian filter can reduce noise in image, but it also blurs the image. The larger the intensity of noise, a larger filter kernel should be used to remove the noise. But the larger the filter, the stronger is the blur. So it's a compromise between smoothing of the image and reducing the additive noise. Image convolution can be used for many tasks. For example, we can use image convolution to reduce image blur and make edges more pronounced. The idea of this process is very simple. Let's try to apply blurring and subtract blurred image from original image. What does blurring take away? It is information regarding sharp edges in images. We can add this information to the original image with some scale factor. And this will make edges more pronounced, more sharp. So this operation is called image sharpening. You can combine all operations into single convolution and approximate it with negative Laplacian of Gaussian. Such filter is called unsharp mask. On this slide, you can see an example of image sharpening with this specific kernel. You can see that edges in the image are more pronounced and the image look sharper. And the left image in the slide look a bit blurrier than the image of the right. Sharpening produce visually pleasing results usually. So it's oftenly used in consumer cameras to improve the quality of images. And you can only turn it off if you tried to capture the raw images with no post-processing.
