The feature selection task

Loading...

Del curso dictado por University of Washington

Machine Learning: Regression

3700 calificaciones

University of Washington

Machine Learning: Regression

3700 calificaciones

Curso 2 de 4 en SpecializationMachine Learning

Case Study - Predicting Housing Prices In our first case study, predicting house prices, you will create models that predict a continuous value (price) from input features (square footage, number of bedrooms and bathrooms,...). This is just one of the many places where regression can be applied. Other applications range from predicting health outcomes in medicine, stock prices in finance, and power usage in high-performance computing, to analyzing which regulators are important for gene expression. In this course, you will explore regularized linear regression models for the task of prediction and feature selection. You will be able to handle very large sets of features and select between models of various complexity. You will also analyze the impact of aspects of your data -- such as outliers -- on your selected models and predictions. To fit these models, you will implement optimization algorithms that scale to large datasets. Learning Outcomes: By the end of this course, you will be able to: -Describe the input and output of a regression model. -Compare and contrast bias and variance when modeling data. -Estimate model parameters using optimization algorithms. -Tune parameters with cross validation. -Analyze the performance of the model. -Describe the notion of sparsity and how LASSO leads to sparse solutions. -Deploy methods to select between models. -Exploit the model to form predictions. -Build a regression model to predict prices using a housing dataset. -Implement these techniques in Python.

De la lección

Feature Selection & Lasso

A fundamental machine learning task is to select amongst a set of features to include in a model. In this module, you will explore this idea in the context of multiple regression, and describe how such feature selection is important for both interpretability and efficiency of forming predictions. <p> To start, you will examine methods that search over an enumeration of models including different subsets of features. You will analyze both exhaustive search and greedy algorithms. Then, instead of an explicit enumeration, we turn to Lasso regression, which implicitly performs feature selection in a manner akin to ridge regression: A complex model is fit based on a measure of fit to the training data plus a measure of overfitting different than that used in ridge. This lasso method has had impact in numerous applied domains, and the ideas behind the method have fundamentally changed machine learning and statistics. You will also implement a coordinate descent algorithm for fitting a Lasso model. <p>Coordinate descent is another, general, optimization technique, which is useful in many areas of machine learning.

The feature selection task3:45

All subsets6:15

Complexity of all subsets3:11

Greedy algorithms7:29

Complexity of the greedy forward stepwise algorithm2:39

Conoce a los instructores

Emily Fox
Amazon Professor of Machine Learning
Statistics
Carlos Guestrin
Amazon Professor of Machine Learning
Computer Science and Engineering

[MUSIC]

In this module,

we're gonna discuss how to select amongst a set of features to include in our model.

And to do this, we're first gonna start by describing a way to explicitly search over

all possible models.

And then what we're gonna do is describe a way to implicitly do a feature selection

using regularized regression,

akin to the types of ideas we were talking about when we discussed ridge regression.

So let's start by motivating this feature selection task.

So question is, why might you want to select amongst your set of features?

One is for efficiency.

So let's say you have a problem which has 100 billion features.

That might sound like a lot.

It is actually a lot, but there are many applications out there these days where

we're faced with this many features.

Well, every time we go to do prediction with 100 billion features,

that multiplication we have to do between our feature vector and

the weights on all these features is really computationally intensive.

In contrast, if we assume that the weights on our features are sparse,

and what I mean by that is that there are many zeros,

then things can be done much more efficiently.

Because when we're going to form our prediction, all we need to do is

sum over all of the features whose weights are not zero.

But another reason for

wanting to do this feature selection and the one that perhaps might be more common,

at least classically, is for the reason of interpretability, where we wanna

understand what are the features relevant for, for example, a prediction task.

So, for example, in our housing application, we might have a really long

list of possible features associated with every house.

This is actually an example of a set of features that were listed for

a house that was listed on Zillow.

And there are lots and lots of detailed things, including what roof type the house

has, and whether it includes a microwave or not when it's getting sold.

And the question is,

are all these features really relevant to assessing the value of the house, or

at least how much somebody will go and pay for the house?

They probably wouldn't mind if they're spending a couple $100,000

to go buy a microwave.

That's probably not factoring in very significantly in making their decision

about the value of the house.

So when we're faced with all these features,

we might wanna select a subset that are really representative or

relevant to our task of predicting the value of a house.

So here I've shown perhaps a reasonable subset of features that we might use for

assessing the value.

And one question we're gonna go through in this module is,

how do we think about choosing this subset?

And another application we talked about a couple modules ago was this reading

your mind task, where we get a scan of your brain, and for our sake,

we can just think of this as an image.

And then we'd like to predict whether you are happy or

sad in response to whatever you were shown.

So when we went to take the scan of your brain, you're shown either a word, or

an image, or something like this.

And we wanna be able to predict how you felt about that just from your brain scan.

So we talked about the fact that we treat our inputs, our features,

as just the voxels.

We can think of them as just pixels in this image, and wanted to relate

those pixel intensities to this output, this response of happiness.

And in many cases, maybe what we'd like to do is find a small subset

of regions in the brain that are relevant to this prediction task.

And so here again is a reason for interpretability,

that we might wanna do this feature selection task.

[MUSIC]

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2018 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play