Vectors and Functions (by Rich Pang)

Loading...

Del curso dictado por University of Washington

Computational Neuroscience

433 calificaciones

University of Washington

Computational Neuroscience

433 calificaciones

This course provides an introduction to basic computational methods for understanding what nervous systems do and for determining how they function. We will explore the computational principles governing various aspects of vision, sensory-motor control, learning, and memory. Specific topics that will be covered include representation of information by spiking neurons, processing of information in neural networks, and algorithms for adaptation and learning. We will make use of Matlab/Octave/Python demonstrations and exercises to gain a deeper understanding of concepts and methods introduced in the course. The course is primarily aimed at third- or fourth-year undergraduates and beginning graduate students, as well as professionals and distance learners interested in learning how the brain processes information.

De la lección

What do Neurons Encode? Neural Encoding Models (Adrienne Fairhall)

This module introduces you to the captivating world of neural information coding. You will learn about the technologies that are used to record brain activity. We will then develop some mathematical formulations that allow us to characterize spikes from neurons as a code, at increasing levels of detail. Finally we investigate variability and noise in the brain, and how our models can accommodate them.

Vectors and Functions (by Rich Pang)30:01

Convolutions and Linear Systems (by Rich Pang)16:15

Change of Basis and PCA (by Rich Pang)18:56

Welcome to the Eigenworld! (by Rich Pang)24:13

Conoce a los instructores

Rajesh P. N. Rao
Professor
Computer Science & Engineering
Adrienne Fairhall
Associate Professor
Physiology and Biophysics

Hi, guys.

Rich here, one of your friendly Coursera TA's at the University of Washington.

And today I'm here to talk to you about math.

As the course progresses, I'll be going through a couple little miniature

mathematical tutorials regarding the topics that come up during the lectures

for those of you who could maybe use a refresher on the mathematics.

I won't be going through detailed proofs and derivations,

which I'm sure some of you are very thankful for.

Instead, I'll be focusing more on the intuition and how we can start to think

about different mathematical objects in the terms of computational neuroscience.

I don't have my adventure hat on, but

you'll have to trust that I'm wearing my adventure pants.

Let's start with something that will come up over and over again in this course.

We are going to discuss the topic of vectors.

Vectors and functions.

And at the end of this tutorial hopefully you will start to understand that they

are really quite similar.

And the things we can do with vectors, we can do with functions as well.

So, some of you may have heard about vectors in a physics or engineering class,

for example.

Things that have a direction and a magnitude or whatever, whatever.

But today, while we won't be inconsistent with that understanding,

we are going to start to think about vectors in a more abstract way

that will be useful for us as computational neuroscientists.

So, what is the definition of a vector?

We're not going to say that it's just a length and a direction, or a direction and

a magnitude, whatever buzz words you may have heard before.

Instead we're going to call a vector a list of numbers.

Nothing more, nothing less.

Just a list of numbers.

What's one example?

What are several examples?

Here's an example, (3,1).

That's a list of two numbers.

Here's another example (-10,8), another list of two numbers.

want to get crazy?

Let's add another number.

Let's say (3, 6, -5).

That's a list of three numbers.

Let's make a list of five numbers,

(-1, 2, 8, 6, -5).

I hope this is blowing your mind.

The dimensionality of a vector is

the number of elements in the vector.

Hopefully that doesn't confuse you too much.

So, what would be the dimensionality of this guy?

Well, how many elements are there?

Two.

So this is a 2D vector.

What about this guy?

How many elements are there?

Two again.

And this guy?

This is a 3D vector.

And what about this guy?

This one is a 5D vector.

Now, there's something very special about 2 and

3D vectors that is not quite the case with higher dimensional vectors.

And that is that, for 2 and 3D vectors you can draw them as arrows or

points in the plane or in some three-dimensional space.

How would we draw the vector 3,1?

First we would start with our 2D plane.

And let's label the axis x1 and x2, and

we want to plot the vector 3,1 in our plane.

How do we do that?

First we look up the x1 coordinates.

Well, that's just the first number, that's 3.

So we going to go over 1, 2, 3.

Now we look up the x2 coordinates.

That's 1.

So we've been over 3,

since the x2 coordinate is 1 we go up 1 and we put a point there.

Pretty crazy, right?

I know, I thought so too.

Sometimes, maybe in physics and engineering,

you'll see the vectors drawn as arrows.

But often in neuroscience you'll just see them as dots.

So for now we'll just leave it as a dot.

Sound good?

So that's 3,1.

What if we wanted to plot the vector?

-2, 4.

Well we go over to over -2, and up 4.

1, 2, 3, 4.

So we can plot multiple vectors in the plane as points.

You can do the same thing with 3D vectors if you're good at drawing 3D.

I am not, unfortunately.

So, I wont give you an example there.

However, when you get to four dimensional and

five dimensional vectors it becomes awfully hard to draw pictures.

Yet, all the mathematical wonders we can do with 2 and

3D vectors we can do with 5D vectors as well.

So now we're going to talk about the basic operations you can do with vectors.

And the reason we introduce them with vectors

is because we want to understand their applications to functions as well.

What is a very simple operation?

This first simple operation is just called the sum operation.

Let's say we have the vector x equals, you can put numbers, but I'm just

going to leave them as variables for now so that they can mean anything you want.

And I'm going to have them be arbitrarily dimensioned.

So, what is the dimensionality of this vector?

This is just ND vector, a list of N numbers.

How do we take the sum?

Well, it's very simple.

All you do is add all of the elements together.

Not too hard, right?

So, with the sum operation.

What goes in?

A vector goes in, that was a list of numbers.

And what comes out?

Another vector?

No, a scalar, which is just a single number.

And the reason the output is a scalar is because each of the elements

in the vector x was a scalar.

So when we add a bunch of scalars together, we get another scalar.

All right, next and most important operation is the dot product.

This will come up over and over again in your life.

And if you understand it, you will be a better person.

So how does the dot product work?

First of all, how do we write the dot product?

So let's say vector x = (x1,

x2,..., xn,

and the vector y = y1,

y2, ..., yn.

Then the dot product is just written as x.y.

Pretty crazy.

In order to compute the dot product, all we do is multiply together

the elements of x and y, element ys and then add the whole thing up.

We'll start with x1 and y1, multiply those together.

And to that we add the product of x2y2, and to that we add the product of x3y3.

And we keep going and going and

going until we get to the end, where we add the product xnyn.

So let's do an example.

Let's say x, we'll just use four d vectors for now.

Let's say x is minus 1,0, 5 and

2 and y is equal to 0, 2 minus 3, 6.

What is x.y equal for?

How do we do it?

Let's start with the first element,

is minus 1 times 0, 0 times 2,

then 5 times minus 3, added to 2 times 6.

Did I do that right?

I think so.

So what's that look like?

0 plus 0 minus 15 plus 12 equals minus 3.

Not too hard, right?

Hopefully that made sense.

For your confusion, I'm going to erase it, so that I have a little bit more space.

And so we'll say that the dot product was, what did we say?

Minus 3.

So how does that dot product work in our little machine?

So what is the input to the dot product operation?

The input is two vectors.

And what's the output?

The output, here let's draw arrows on these things so they make more sense.

So the input to the dot product is two vectors and the output is a scalar.

An important thing to not is that you can take the sum and

dot products regardless of the dimensionality

of the vector or vectors in the case of the dot product.

This will come in handy later.

Okay, but returning to the dot product, why is the dot product useful?

Some of you may enjoy multiplying pairs of numbers together and

adding them up on a Friday night.

But I promise there's actually a reason to do this too, besides mere boredom.

Let's return to our geometric understanding of vectors.

So this is x1, that's

x2 and let's take the dot product between two vectors,

where we draw them out and see what things look like geometrically.

So, let's call x1, I'll put the index up there is equal to 3,

3, and x2, our second vector,

this doesn't mean square, this is just an index saying that it is a second vector,

is equal to let's say minus 1,5.

Okay, let's draw those out.

X1 will go over 1, 2, 3, up 1, 2, 3, that's x1.

So let's draw an arrow now to make this clear.

And what's x2?

So that's x1, x2 you go over 1 up 5, that's x2.

Oops, let's stay consistent and leave the 1 up there.

What are we actually doing when we are computing the dot product?

What the dot product does is it tells us how aligned two vectors are.

So, how it works, and this is actually one of the reasons I sometimes call it

a dot product projection, a term you will be hearing a lot throughout our lectures.

Is that we imagine a light shining.

So we are going to draw an axis perpendicular to x1 and

we are going to imagine a light, a big bright light,

maybe the sun, shining down on x2 and on x1.

And this light will cast a shadow of x2 over x1,

or if you like it projects x2 onto x1.

Crazy, right?

Now we can ask, what is the product of the magnitude of x1 and

the component of x2 that is lined up with x1.

Or what is the product of the magnitude of this blue vector and

the magnitude of this red vector that was x1?

Well, it is the dot product.

All we do is calculate the dot product.

So x1.x2 equals 3 times

minus 1 plus 3 times 5

which is equal to 12.

So, that means that the length of the blue vector, or the magnitude of the blue

vector, multiplied by the magnitude of the red vector x1 is equal to 12.

I'm not going to prove to you why that is the case, but

if you're interested it's a fun exercise for the reader.

But, anyhow,

that's a good way of thinking about what the dot product does geometrically.

It's kind of hard to picture for things like five or ten or

20 dimensional vectors, but it works in generally the same.

It calculates the amount that the two vectors are lined up.

So, just as a quick thing, let's introduce, what color can we use?

Cyan.

Vector x3 which is equal to 4 minus 4.

Where would we draw that?

1, 2, 3, 4, up 4 so that's x3.

What is the dot product of x3 and x1?

X1.x3, the dot product of x1 and

x3 is equal to, you can do the math,

that's 12 minus 12 is equal to 0.

So the dot product is 0, and what does that mean geometrically?

Geometrically, x3 was perpendicular to x1, so

when the light shone down on it, it cast no shadow.

So in general, if the dot product between two vectors is 0,

the two vectors are perpendicular

or the fancy word for perpendicular is orthogonal.

Sound good?

Alright, so that is everything you need to know about vectors for now.

Now we are going to introduce functions.

And the idea is that what you have learned about vectors

will help you understand what you can do with functions.

So what's a function?

Just a refresher.

Here's an example of a function f of x equals 3x minus 2.

A function takes an input or argument x and

it spits out a function value f of x.

So if x were equal to 2, we would say f of 2 is equal to 3 times 2 is 6 minus 2 is 4.

Not too hard, right?

And what does this function look like?

Well, first of all it's just a line and it's got a slope of three and

y-intercept of, -2.

So we start at -2 and for every one we go over we go up 3, about like that and

it extends forever in to the infinite beyond in both directions.

Okay, so that's one way to think of a function and

a way that a lot of you have probably encountered before.

Today, we are going to try to think of a function as a vector.

That might sound crazy, but

I promise there are crazier things in math than thinking of a function as a vector.

So let's pick a new example and

I'll try to tell you what the heck I'm talking about.

Let's pick the example f(x), we'll call it f1(x),

just in case we need to introduce more functions, is equal to x/2.

And we're going to say that x is between 0 and 1, we'll make that inclusive.

Sound good?

So how do we draw this out?

So here we have x and f(x).

Let's zoom in.

So let's say, that's 1.

All right.

We know this is a line.

How do we draw the line?

I want x as 0, f1 as 0.

So i'll put one dot there.

And when x is 1, f should be one half,

that's one half.

So we'll draw our line like that.

So that is our function.

I claim that we can think of this function as a vector.

And how does that work?

Well, well, let's do this.

Let's pick a few points on our function.

Let's say we'll pick 0 because everybody loves 0.

We'll pick 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 and so forth.

And now what we're going to do is we're going to write down the function

values at each of those numbers.

So this was 0, 0.1, 0.2, 0.3,

these are the x values, 0.4, and so on and so forth.

So what is at f(0)?

f1(0)?

That's just 0.

What is f1(0.1)?

That's 0.1 over 2, which is point 0.05.

What is half of 0.02?

Well that's just 0.02 over 2, 0.1.

So we can continue to fill this out and

hopefully this is starting to look a little familiar to you.

We have sort of described f1 by listing a bunch of numbers.

So f1 in fact is starting to look like a vector.

Now we didn't have to choose 0.1 as our increment.

We could have chose 0.01 in which case

our vector would look like 0.005,

0.01, 0.015 and so on and so forth.

And if we wanted to not only 0.01 as our increment for

x, but 0.001 or 0.0001.

And in fact, you can keep making your increments smaller and smaller and

smaller until you have an infinite list of numbers.

But, the key thing to remember is that

the function is still just a list of numbers.

And I claim this is true because if you had all the numbers for

every increment that would specify your function completely.

Therefore, the function is a list of numbers or,

as we like to call it, a vector.

Is that crazy?

Yeah, it's pretty crazy.

And I'm not going to go into the math to actually prove that, but

hopefully the intuition makes some sense.

Okay.

Now that we can think of functions as vectors.

And if that doesn't quite sit comfortably with you spend a few minutes.

Pause the video and just really let yourself that this is true,

that functions are vectors.

Because they are nothing more than a really,

really, really detailed list of numbers.

Okay, that makes sense.

So let's move on to operations, but

now performed not with vectors, but with functions.

And we actually only talked about two operations of vectors, and

those will be the two operations we talk about with functions.

So the first one in the vector world it was the sum.

And for those of you who have taken calculus, what happens to sums when we

move our elements infinitesimally closer together and

let there be an infinite number of them?

Well, that becomes an integral.

And the sum becomes an integral because that is how you would most naturally add

up an infinite list of numbers, which was our function.

Okay, how does the integral work?

The integral, if we have some arbitrary function, half of x versus x,

and we'll again have it justified under the domain from 0 to 1.

The integral is just the area under the curve.

That's not too hard really.

There are lots of good calculus textbooks that tell you exactly how to compute

the area under the curve.

But what we're interested in is just the fact that it is this integral.

So here's another reason why the sum and the integral are very similar.

What did the sum take in as its input?

It took in a vector.

And what was its output?

A number, a scalar.

What does the integral take in as its input?

It takes in a function.

And what is output?

Output is just a number.

So the integral is the function version of a sum.

Sound good?

Okay.

Now in the vector world we had dot products, or projections.

And in the function world, luckily we can still call these projections.

The name of things doesn't change too much.

So what did we do to take a dot product in the vector world?

Well, we took two vectors, right?

Those were our inputs.

And we multiplied all of the elements together pair-wise and

added them together.

What is the equivalent way of doing that with functions?

Where our functions are these crazy vectors that

have a ton of different elements in them.

An infinite number in fact, and they're infinitely close together.

Well, when we deal with infinities we go to calculus.

So that sum over all of the pair-wise products

becomes an integral over the pair-wise products.

So what does that look like?

The projection of one function onto another.

Let's say the projection of, and

sometimes we write this in the same way as a dot product.

We say f1 of x dotted with f2

of x is equal to the integral.

And let's just work from 0 to 1 now.

You can pick other bounds if you like, but for now since we've been using 0 and

1 as our bounds, we'll stick with those.

The dot product of the two functions is the integral of the products

of the functions.

Crazy.

Does that make sense?

I hope so.

And the intuition is that this is just the natural generalization

of dot products to the world of functions.

Sums become integrals.

So for our sum operation, we were adding together a bunch of elements.

That becomes an integral.

In the dot products operation, we were adding together a bunch of these products.

But remember each product was just a number.

So it was a sum of a bunch of numbers, therefore,

in the function world we're just doing an integral of these products.

Hopefully that makes sense.

Okay, so let's do a quick example of how we would compute the dot product of two

functions, or the projection of one function onto another.

So, remember that is the integral.

Let's make it general, let's just say from a to b,

f1(x) times f2(x), dx.

So let's say that f1(x), we'll draw that in green,

is something like a line, so f1(x).

Always label your axes.

And let's say f2(x) is something that looks like that.

And let's say that our a and b are there,

so a is 0, b is just some number.

How do we calculate the dot product?

Well, first we have to multiply the two functions together.

So that was f2(x), we would like to multiply these two functions together.

That might look something like that.

And I'm not very good at doing this by hand, but

maybe it would come down like that.

That's not too bad, right?

I think that's okay.

So that's the product of f1(x) and f2(x).

And then the integral is just the area under that curve.

So the integral of f1(x) and f2(x) from

a to b, dx is just the shaded area.

Cool?

Cool.

So what is the dot product between two functions actually telling you?

It's basically telling you how much the functions overlap.

So when the functions overlap a lot,

they will contribute very positively to this integral.

But when the functions don't overlap very much,

they will contribute very little to the integral.

What if two functions don't overlap at all?

If two functions don't overlap at all, then their dot product will be.

That's the integral from a to b,

f1(x) f2(x), dx.

The dot product will be 0 if the functions don't overlap.

So that might be a little abstract still, so where can we find an example of this?

Well, here's a very simple example.

Let's say that f1, what did we use for f1, green?

f1 looks like that.

So let's say f1(x) is this parabola like thing,

and f2(x), some sort of sinusoid like that.

That's f2(x), and so this is a and b.

What is the dot product of f1(x) and f2(x)?

How do we figure this out?

Well, we multiply f1(x) and

f2(x) together.

So in this part, it's negative because f1(x) is positive, but f2(x) is negative.

And in this part, it's positive because f1(x) and f2(x) are both positive.

And now we take the integral.

So, if you were to take the integral of that blue curve,

which is the product of f1(x) and f2(x) what would you get?

And the answer is 0.

Why is the answer zero?

Well, the first lobe is under the x-axis, so that's a negative area.

And the second lobe is above the x-axis, so that's a positive area.

And since the functions were at least supposed to be kind of symmetric,

the area underneath the x-axis cancels out the area above the x-axis,

which means that the integral is 0.

So you would say that these two functions are orthogonal.

So we would write that f1(x) and

f2(x) are orthogonal.

And so all orthogonal means in this case is that they don't overlap at all.

Or, rather, their overlaps cancel each other out.

That's about it for right now.

I hope that made some sense.

I hope that made things not more confusing, but

at least a little less confusing.

The idea Idea of projecting one function on to another, will come up over and

over again, especially in convolutions in Linux systems.

So it's a good thing to make sure you understand at least basic idea of.

Okay, that's it, see you next time.

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2018 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play