In this class you will learn the basic principles and tools used to process images and videos, and how to apply them in solving practical problems of commercial and scientific interests. Digital images and videos are everywhere these days – in thousands of scientific (e.g., astronomical, bio-medical), consumer, industrial, and artistic applications. Moreover they come in a wide range of the electromagnetic spectrum - from visible light and infrared to gamma rays and beyond. The ability to process image and video signals is therefore an incredibly important skill to master for engineering/science students, software developers, and practicing scientists. Digital image and video processing continues to enable the multimedia technology revolution we are experiencing today. Some important examples of image and video processing include the removal of degradations images suffer during acquisition (e.g., removing blur from a picture of a fast moving car), and the compression and transmission of images and videos (if you watch videos online, or share photos via a social media website, you use this everyday!), for economical storage and efficient transmission. This course will cover the fundamentals of image and video processing. We will provide a mathematical framework to describe and analyze images and videos as two- and three-dimensional signals in the spatial, spatio-temporal, and frequency domains. In this class not only will you learn the theory behind fundamental processing tasks including image/video enhancement, recovery, and compression - but you will also learn how to perform these key processing tasks in practice using state-of-the-art techniques and tools. We will introduce and use a wide variety of such tools – from optimization toolboxes to statistical techniques. Emphasis on the special role sparsity plays in modern image and video processing will also be given. In all cases, example images and videos pertaining to specific application domains will be utilized.
Inverse Filtering
Loading...
Del curso dictado por Northwestern University
Fundamentals of Digital Image and Video Processing
657 calificaciones
De la lección
Image Recovery: Part 1
In this module we study the problem of image and video recovery. Topics include: introduction to image and video recovery, image restoration, matrix-vector notation for images, inverse filtering, constrained least squares (CLS), set-theoretic restoration approaches, iterative restoration algorithms, and spatially adaptive algorithms.
Conoce a los instructores
Aggelos K. KatsaggelosJoseph Cummings Professor
Department of Electrical Engineering and Computer Science
We discuss in this segment the simplest restoration
filter one can derive, the so-called inverse filter.
The inverse filter has traditionally been developed for an LSI system,
and it can therefore be implemented in the discrete frequency domain.
In this case, you just invert the degradation
operator, at least for the frequencies it is invertible.
We present actually the Inverse filter as
the solution to a least squares problem which
results in the generalized Inverse filter even when
that the degradation is not linear especially invariant.
We demonstrate the performance of the filter in
restoring an image which is blurred due to motion.
We're going to use the same example
for the material presented during this and next
week so as to be able to
make direct comparisons among the various restoration feeders.
The main drove of Inverse filter is that
it amplifies the noise present in the data.
So even if the noise is not visible in the
observed image, it is greatly amplified in the restored image.
Due to this main drawback, an improved
version of the filter will be presented next.
The simplest possible restoration filter one can devise is the Inverse filter.
Here's the degradation equation we're using, y the observed image.
H, the known degradation, and f, the original
image we try to find, and the additive noise.
With a model like this, the only constraint
during possible degradation, is that it is linear.
[SOUND] So given the degradation model, the simplest possible solution
approach is to minimize this squared error between the observation and hf.
A necessary condition for this functional to have a
minimum, is that it's gradient is equal to 0.
And by the way, this is the same gradient that I denoted like this earlier on.
So we do know that the norm can be expanded as follows.
[BLANK_AUDIO]
[SOUND] So this term is independent on F.
It can go away, and then as I showed earlier,
the gradient of this term is minus 2, H transposed Y.
Y's the gradient of the quadratic term is 2 times H transpose H f.
So this should be equal to 0,
and therefore we obtain the so-called normal equations.
So the solution f is equal to the generalized inverse.
This is this plus here.
So generalized Inverse.
If H transpose H is invertible, then that's the Regular Inverse.
So this solution, again, is for just any matrix with no particular structure.
Now, if the degradation system is linearly special in variant, we then know that H
is a block circulant matrix and we
furthermore know that they can take the solution.
Equation to the discrete frequency domain, and if
we do so we end up with this solution.
So according to this the restored image had frequency discreet frequencies UV is
equal to 0 when the frequency parts of the system as those frequencies is equal to 0.
Otherwise, it's given by this expression here.
This expression actually can be rather simplified because I have H transposed,
I'm sorry, H cong, conjugated UV, YUV, and then the
magnitude in the denominator, is H complex conjugate, times H.
So these guys cancel out and this is just the observation of the
frequency domain over the frequency response
of the system in the frequency domain.
And clearly you can obtain this expression
by just look at the convolution equation, and
taking it to the discrete frequency domain,
and dividing by this since convolution becomes multiplication.
Clearly the main disadvantage of the inverse filter is the fact that the noise
is ignored as mentioned, and if we look at this expression here right, y is equal to
h u v f u v divide by eight uv.
So this will give me the through solution plus the noise, then, divided by Huv.
Now, the noises are assumed to be broadband
and, therefore for those frequencies that Huv is small.
I have, course divided by something very small.
So, this gives me a large number that's the noise of litigation.
I
can try to control the noise amplification by introducing a threshold here.
So I'm not looking for exact 0s of
the frequency response here, but for values that
are smaller than a threshold t, otherwise for
values greater than t I use the other expression.
So let's see how this inverse filter performs through some examples.
So here's an example would be using, kind
of, throughout this presentation to compare different restoration approaches.
There's the original image, cameraman.
It's blurred by a system H here which introduces
motion blur over eight pixels in the horizontal direction.
And then noise is added so that the blurred signal
to noise ratio is 20 dB, so this is clearly blurred.
The noise is hardly visible, but through the
restoration, when it's amplified it will be really visible and bothersome.
So with this setting given this image here this is our observation again y with no H.
This is the description of H and we try to get an estimate of the original limit.
Well let's at the impulse frequency response of the degradation system.
It's introducing one dimensional motion, blur along the horizontal direction.
So, this is a point spread function, the impulse response.
It's normalized therefore the height of each of these samples is one over eight.
So in in two dimensions if from left to right h n one and two it's equal to
h n one which is this one if I rename it times delta at m two.
So it take if I take the fully transformed of
this signal here we do know that the sink function.
And so this is the discriptfully transform.
Let's call this the u domain.
So I show the magnitude of H u,v discrete frequencies for any v.
So this is independent of v so it will be the same for every v.
So it centered.
So this a 56 point DFT.
This is therefore 128.
The height is 1, since this is normalized.
And then it was chosen carefully, this motion blur over eight
pixels because the 0s are at integer multiples of 256 over 8.
Let's say times l.
So this is equal to 32l.L So the space in between the zero is
32, therefore this is at 128 plus 32, so this is at 160.
The next one is at 192, and the last one is a 224.
Right and similar in the other side this is zero at 96, 0 at 64 and a 0 at 32.
So there exact 0s at these discrete frequencies.
So if I look at the two dimensional frequency response, the magnitude squared.
It looks like this, it's this sync function so this is U, this is V, axis.
So, the shape changes along U but it does not change along V.
It's independent of V, alright?
So this is the, the frequency response of the system that is introducing
the 1 dimensional motion blur and this is what we want to invert.
That this is all about, right?
The restoration.
I want to, to invert in the frequency domain
this shape and this will give me the inverse filter.
We see here two results.
That are obtained by this thresholded
inverse filter for the case under consideration.
Motion blur over eight pixels plus 20 dB blur signals [UNKNOWN] ration.
So, two different thresholds are utilized here 10 to the minus 16 and .01.
The main observation is that both restorations, they look
very similar actually, and they're both buried in noise.
So this is not an acceptable result.
We show here also one slice of the frequency response of the restoration
filter, which is equal to one over
the frequency response of the degradation system.
We see that the at the locations, frequency locations of
exact 0, the generalized inverse is also equal to 0.
Without do, using this generalized in this idea, one of the 0s should blow up.
They should be infinity values.
At 0 the frequency is one.
Right, it's normalized while another one is one.
And then at higher frequencies, because the frequency
response becomes, of the degradation system is smaller.
When I invert it, I see larger values
of the restoration filter at high, higher frequencies.
Let's see now how we can control the
result by modifying the threshold using a higher threshold.
We show here two additional restorations, by this thresholder
inverse filterer for the case on the course of iteration.
So the thresholds are increased to this value and this value.
And we see now that there is some distant control of the noise amplification.
As a mat, as a matter of fact, this might be a.
Acceptable restoration result.
We also show the frequency response of the corresponding restoration filters.
Since there are small, smaller values in the
frequency response of the gradation system at higher frequencies
these are the ones that are fresh [UNKNOWN]
when I look at the inverse filter down here.
The main characteristic is that the high frequencies are tapered off.
The values are, are increase, decreased,
with high frequencies in both these, restorations.
I've kept the vertical axis intentionally at, the same kind of
length here, so that it's clear it's easy to compare the values.
Restoration filters, so while at low
frequencies the inverse filter is, is faithful.
At high frequencies the Inverse filter is, is altered due to this
threshold, but this is a desirable
effect, since the noise simplification is controlled.
That way.
We increase the values of the threshold even further to these values.
And we see that as T increases, the noise is further controlled,
so we see less noise at the output of the restored image.
The frequency response of the filter is out, out of considerably.
I feel for this threshold here that you see it's only at low frequency and these
mid frequencies that the that there restoration filter
has some values, and everything else is 0.
We see some of the so called rinding artifacts
here that we'll talk a little bit about later.
So these are artifacts, undesirable effects in the image.
So maybe this is also an acceptable restoration.
So we do see here that the inverse filter by itself
is not doing an acceptable job because primarily it amplifies noise.
Through the introduction of this threshold, we're able to
control the noise amplification in an ad hoc fashion.
We don't know.
What the proper value of T is.
However, one might experiment around, as we did here,
and choose a restoration that is suitable for their purposes.
Explora nuestro catálogo
Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.
Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.
© 2018 Coursera Inc. Todos los derechos reservados.
