Hello, and welcome to week three of our class on image and video processing.
What you're seeing right now is me in a relatively dark room.
You can probably see the Duke Chapel behind me but you can't see details, for
example, the color of the shirt I'm wearing.
Or, if you were to look at my desk, it's basically very dark, and you can't see
much. What happen when we take an image like
this? What happen when we have a video that is
very dark? That happens us all the time, we take a
picture at night. Doesn't have the lighting that we want.
It's too dark or too bright, sometimes too noisy.
What we are going to do during this week is to learn how to improve those images.
How to enhance those images. We're going to start with very simple
manipulations. That will basically help us to improve
the contrast, remove the noise. And we're going to go towards the end of
the week to more advanced techniques. Which are basically leading to state of
the art approaches today. And we're going to come back to some
these towards the end of the class to basically week seven, week eight, to
really, really advance techniques for image enhancement.
During this week and during the following weeks we're also going to do online and
real time demos so I'm going to basically teach you some of the material and then
we're going to run some different software platforms to basically do some
of these operations. Right now on my PC, and you are going to
see them, in practice, how they are working.
We're also going to introduce some online video quizzes so, basically I'm going to
pause for a second and you're going to be able to think about a question that I
will post to you. That won't basically affect your, your
certificate at the end but it's just to have you think for a while before I give
you the answer to the question, like in any regular class that I will basically
pause to have the students think for a minute.
Before we continue let me turn on the lights so I can continue this and write.
And explain to you the basic principles of image enhancement and contrast
enhancement and image you're now seeing. So bear with me for a second.
I'm going to turn on my lights and we will be right back.
So now I'm turning the lights on. So you can see.
A bit of the details. you can't see much in the background
anymore but you can see what's in the room.
And what I'm going to show you is how we turn the lights on, digitally, in the
computer. So you have taken your image.
You can now go back to SIM. I want to turn the lights on with image
processing and video processing software so let's start doing that.
And we basically are going to start from very simple operations, which are
histogram modifications, or pointwise operations.
So, the basic idea is illustrated in this figure.
Every pixel of the image, it's going to change it's own value,
following certain function. So.
The pixel here will move to a new image where basically it changed its value but
it will change its value only based on its own value right now.
It's not going to be influenced by anything that's happening in any other
part of the image. That's the first part of this week three.
We're going to see later how we basically influence the pixel value by other things
that are happening in the image. But to start, very simple, what's called
point-wise operations. There's a certain pixel value in, in the
input, a certain pixel value in the output.
Similarly here. For example, what's happening here is
that everybody up to pixel value K. Becomes zero, everybody above K becomes
one or the maximum value. In the image.
So this is basically a thresh holding operation.
So, what we need to specify, is it one dimensional function, that translates the
input to the output. And, let's just see, a few examples of
that. So here we see a few illustrations of
this. Several of these operations, inverse log.
Of course lets start from the identity. The input is identical to the output.
So basically this is the identity. If we basically mark the output as S.
An entity is S = R. The output is identical to the input.
We have done nothing to the image. We also have a log operation.
For example the output is some constant times the log of the input, and we
normally are going to add let's say 1. Because if the input is 0, we cannot take
the logarithmic of zero. So we do we this.
So that's an hour operation. Whatever is the grade value of the input
we transform it to the output. There is also the inverse, which is
written here. And I want to, I want for you to think
for a second, what will be basically the function that will represent the inverse.
Considering that our image goes from 0 to L.
So the minimal value is 0, the maximum value is L.
While would be the function, basically I want to write S.
As a function of r. Why would be that function that will
basically do the negation of the image. The maximal value of the input will
become zero. And the minimal value of the input
meaning zero would become L. So let's just think for a second and it's
a very simple exercise just to illustrate how simple.
And as we are going to see in a second, how powerful at the same time, these type
of operations are. So, we know that, basically, I ask you
how to, basically, construct this function?
And it's very simple. We want the output to be the inverse.
of the input, so S equal L, the maximum value minus R.
So when R is L, S becomes zero because its L minus L.
When R is zero, then basically S become L, which is the maximum.
So, a very, very simple operation that just involves our great values.
And note, that all what occur in all of these operations, is the actual great
value. I'm not asking where's the pixel located
in the image? I'm just asking what's the pixel value?
What's the great value of that image? So, here is one example of this.
Basically this function that we just wrote S equal L minus R is basically
taking every pixel and doing the inversion so it's the negative image and
some of the things that we see is that is basically easier for us to.
See just visually some of the things in the negative image than it is.
In that original image. And why is this?
Remember early on we talk about the properties of the visual system that
there are certain contrast that we understand better.
That we perceive better at different levels of the background.
And we saw that there are some contrasts that are perceived better when we are in
the dark regions. Some contrasts are perceived differently
when we are in the light regions. So, this inversion, basically, might help
us to perceive things that are difficult to notice in the original image.
Very simple illustration. One other important thing of these type
of operations is, I can go back. So, if I start from here, I have not
destroyed the information. I can simply go back and reverse this
operation, that's not always possible as we're going to see later this week.
But, for this particular operation that is doable.
Another extremely important operation is basically an exponential function, and
I'm going to write it here S, the output is going to be a constant, the input, to
some exponent, gamma, and the basic idea here is that depending on, gamma in
particular we get different functions, that transforms the input, to the output,
and this is very often called gamma correction.
And this actually is very popular. In monitors, because of some properties
of the monitor, if you don't do a gamma correction to your image, your image is
going to look relatively dark. So, if this is.
My image its going to look dark, and it's going be hard to perceive exactly the
details. And that's because of the properties of
the monitor. So sometimes before you send the image to
the display to the monitor you do what's called a gamma correction.
So you gamma correct the image, and then you basically send it to the display.
And what's basically the gamma correction is doing is inverting this.
Darkening operation that happens due to some properties in the monitor.
So it's a very useful, very common operation.
Basically shows up. In virtually every monitor there is a
gamma correction before the image is basically display.
But gamma correction or basically this exponential function is very useful in
general to basically kind of stretch if you need the pixel values.
Remember we have instagrams and. Which are the distributions of the
pixels. And, for example, if we look at this
image it looks like most of the pixels are concentrated on the bright values.
So, if we were to draw, and we're going to do.
This. This event.
Very often, basically starting, in the next in the next video, but allowed in
this way. So, probably, if we were to plug the
distribution of the pixel values of this image, there would occupy a lot, this
region. Because we look like the image is
basically too much of the high values, very bright.
So, with gamma corrections, we can manage to actually stretch this histogram and
this are just different values, but I think the simplest is to just look at
this one. And we see that we have basically a nicer
image it looks better than this image first of all.
But it also helps us to better understand some of the details and I hope that you
can still in your own monitor that it's doing its own gamma correction.
Then you can actually perceive better, details here, details here, and in
different regions in the image. And sometimes, it might become too much,
so you might do a different type of gamma.
So you might say oh, this region actually, I'm not.
... Perceiving a lot of the details.
So maybe I should a different value of gamma as illustrated here.
This are researches of different values of gamma.
And you could actually do some of these operations not in the whole image but in
some regions of the image. So you can invert only some region of the
image. You can do a gamma correction or an
exponential function in only some regions of the image.
So you don't have to do them every place. You can do them locally.
So that's. A very simple operation, that immediately
gives us a much nicer perception of the image.
Remember the video, as we started this video, was very dark.
That means that the distribution, basically peaks of values.
Were around here. If this is the histogram distribution and
this kind of operation help us to stretch to basically use better the resources
that we have in the display to use the whole spectrum of values from 0 to 255
instead of just using the dark part of the values or only the bright part of the
values. Again a very, very simple operation.
here there is a lot of art in designing of these operations but with experience
we understand what we need to do. Here is yet in our example of the
stretching of the pixel values. So once again this is the input, this is
R, this is S, this is the output. So if your pixel values are between this
point and this point not a lot is happening.
Okay. So this is going to be the original
image. Pixels values that are relatively dark,
not a lot is going to happen to it, then. Pixels that are relatively bright.
Not a lot is going to happen to them but pickus value in between are going to be
stretch basically I'm going to, this has slope much larger than one.
If the slope was one, nothing will happen to them.
If the slope is less than one, it would be basic contracted.
If the slope is larger than one, it would be stretched.
Okay? So if we have slope larger than one, zero
stays zero. But one basically goes up to the slope
and gets stretched. And then we see what happened.
This is the retaliament because the mid-range of the pixel values gets
stretched. All of a sudden, I can see details that I
couldn't see before. Look at the points here are extremely
difficult to notice, but here I can see that actually there were spots in this
image. And then if you wish you can threshold
that image and that's whats happened here so we can distinguish the different
regions. But notice the difference of going from
here to here, very hard to understand. What's happening here?
Much nicer looking image here, not only much nicer looking, but I can also see
the details in the image, and that's very important for image analysis, and I can
do simple things like thresholding. So everything that's happened here is
extremely, extremely simple operations. By the way, this is this.
Going from here to here, basically, is also.
A very simple operation. We actually saw it before.
It's an operation of this tie. Okay?
So, this is S, this is R. The sum of the pixel values go to zero if
they are below this particular value. And if they are above, they go to the
maximal value that they can display in the image.
So very simple operation can do drastic transformations in the image.
And very useful transformations in the image.
A couple of additional examples, here. We have, it's kind of a thresh hold, but
in the mid range. So, basically we say, okay, anybody that
goes from zero to A, I want it to be very low.
Everybody that is above B, I wand it to be very low, and everybody in between, I
want it to be very high. And, this is another example.
Basically everybody between zero and eight, don't touch.
Everybody above B, don't touch. Everybody between A and B make it very,
very bright, or I could say, make it very, very dark, okay?
So let's assume that you have information that the pixel values of your region of
interest are between A and B and you want them to be highlighted in the image, then
you do this type of operation or this type of operation.
Let's just see the effects of that in the next slide.
This is the original image here. And this is the first operation, where we
basically take a wrench of the image, and that wrench of the image we'll make very
brighter. Very bright, everything else, we'll make
very dark. And this is another example.
This is the second example where we take a region of the image.
And we only change that region of the image.
Everything else basically stays the same. Of course, you need information.
What's the region that I care of. And that comes, as I say, from
experience. Offer information about the sin that
you're capturing. In this case.
Basically, information about the biology and what kind of pixel values.
This region of the image appears. And that's a region that I'm going to
basically say that I need to make it brighter or I to make it darker and
similarly here. The, the goal of this is to illustrate
how very, very simple operations can get us basically what we want.
Can get us to improve the visual appearance of the images, going from very
dark or very bright to very uniform and nice looking images or can help us to
basically highlight the region of interest.
If you want to do the analysis of this region in the image, maybe it's nice to
basically make everything as dark. So I can concentrate my attention here.
But maybe you say, yes. But I lost details, because you made
everything white. You say, okay.
Then let's do something else. Let's just take the whole.
Background, basically, and make it zero. But let's not change this range of
values. And this is the type of operation that
appears here. We make everything that we don't like
just very dark or very bright. But everything we care, we leave
untouched. And then we can actually.
Concentrate our visual observation with concentrate only on that region.
So these are very simple operations. In the next video we're going to start
talking about histogram equalization but before that, let's just make a short
video that basically run some of these operations in real time so we can see how
simple they are and we can actually observe how they are working on our
images. So see you on the next video.
Thank you.
