Variance via matrix multiplication

Loading...

Del curso dictado por Johns Hopkins University

Advanced Linear Models for Data Science 1: Least Squares

88 calificaciones

Explore Related Videos

At Coursera, you will find the best lectures in the world. Here are some of our personalized recommendations for you

Johns Hopkins University

Advanced Linear Models for Data Science 1: Least Squares

88 calificaciones

Welcome to the Advanced Linear Models for Data Science Class 1: Least Squares. This class is an introduction to least squares from a linear algebraic and mathematical perspective. Before beginning the class make sure that you have the following: - A basic understanding of linear algebra and multivariate calculus. - A basic understanding of statistics and regression models. - At least a little familiarity with proof based mathematics. - Basic knowledge of the R programming language. After taking this course, students will have a firm foundation in a linear algebraic treatment of regression modeling. This will greatly augment applied data scientists' general understanding of regression models.

De la lección

Background

We cover some basic matrix algebra results that we will need throughout the class. This includes some basic vector derivatives. In addition, we cover some some basic uses of matrices to create summary statistics from data. This includes calculating and subtracting means from observations (centering) as well as calculating the variance.

Variance via matrix multiplication6:38

Coding example2:10

Conoce a los instructores

Brian Caffo, PhD
Professor, Biostatistics
Bloomberg School of Public Health

So, our variance of our vector y is equal to, let's call it s squared.

That's equal to the summation of yi minus y bar quantity squared over n minus 1.

So that's exactly equal to the norm,

y minus y bar times jn squared divided by n minus 1.

So that is exactly equal to 1 over n minus 1,

let's say y tilde transpose.

Y tilde, where y tilde is the standard version of y.

So that could be written as I minus jn, jn transpose,

jn inverse, jn transpose, times y.

Now it's interesting to note some things about computing the variance this way.

First, consider and let's just ignore the 1 over n minus 1 and

just focus on the y tilde transpose times y tilde part.

So that's equal to y times I minus jn, jn transpose,

jn inverse, jn transpose, that repeated again.

Okay, and there should be a transpose in front of that y and

then I would put a transpose up here, but notice this matrix is symmetric.

The i is symmetric and

then this matrix right here is symmetric if you stare at it carefully.

And then another thing that's interesting to note

is that if I were to multiply this out, I times I is just I.

I times this guy is just, yet again,

I times this matrix which I like to call a hat matrix.

So I times the hat matrix is just itself and then I times the hat matrix,

this way, is just itself, so we get minus 2 hat matrices.

And then, the hat matrix times itself, you'll notice is idempotent.

In other words, if you multiply it times itself, you get itself back.

So we get negative 2 hat matrices and then a plus 1 hat matrix,

so we get I minus the hat matrix again.

So and if you don't see all of the details, then just work this out.

I know you all know linear algebra really well, so just work this out for yourself.

And notice that this matrix is in fact idempotent.

Both interior matrices, the I and the hat matrix are idempotent,

but their subtraction is also idempotent.

So y transpose times i minus jn, jn transpose,

jn inverse, jn transpose, times y.

Is exactly the sum of the squared deviations around the mean.

We don't actually have to multiply that matrix out twice

to obtain these deviations.

And so that's another interesting little shortcut or it just, I would say,

interesting little fact about creating variances.

Is that all we need to do is sort of sandwich in between the two vectors,

this matrix, to create the collection of squared deviations.

So let me give you another example that is quite useful.

Let's suppose you have a matrix x, which is n by p, okay?

And you took x transpose times I minus jn,

jn transpose, jn inverse,

jn transpose, times x.

Well again, remember, if we pre-multiply x,

if we pre-multiply x by that matrix, it's going to center all the columns.

So, and if we then pre-multiply,

post-multiply by it, multiply on the right by it.

This is gonna center all the rows of x transpose,

which is centering all the columns of x.

Again, so this is say, x tilde transpose times x tilde.

And again, because it's idempotent,

it's having one in the middle there, it's just like having two.

So because of that, this equation right here

is exactly x tilde transpose times x tilde.

And if you look at that, if you look at this matrix, okay?

The result is nothing other than a matrix where every diagonal element.

Is the squared deviations around that particular column of x.

And every off diagonal element is the cross deviations of that

particular column minus its mean, times another particular column minus its mean.

And so this matrix, if we then multiply it times 1 over n minus 1,

is the so called variance-covariance matrix.

Down the diagonal are the variances of the columns,

while the off-diagonals are the covariances.

Between the ith and jth column in the ij of diagonl entry.

So you can see that with this matrix manipulation,

it's actually quite easy to arithmetically arrive at a covariance matrix.

Without actually having to write any loops if you have matrix arithmetic defined.

So why don't we go through a coding example just to show this.

And then we'll be done with some of our basic matrix algebra prerequisites.

Hopefully this will have gotten us into the mode and the mindset for

using matrices and block partition manipulations of matrices.

That we're gonna need a lot throughout the class.

And if you're getting a little bit lost now,

we're gonna keep going over these concepts over and over and over again.

So you'll get very, very much so familiar with them later on in the class.

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2019 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play