Explore
For Enterprise
Join for Free
Log In

artes y humanidades
Negocios
Ciencias de la Computación
Ciencia de Datos
Tecnología de información
salud
Matemáticas y Lógica
Desarrollo Personal
Ciencias Físicas e Ingeniería
Ciencias sociales
Aprendizaje de un idioma
Grados
Certificados

Explorar todo lo que Coursera tiene para ofrecer

Explorar
Buscar
For Enterprise
Inicia Sesión
Únete de forma gratuita

Model Evaluation and Refinement

Loading...

Análisis de datos con Python

4.6 (4,811 calificaciones) | 45K estudiantes inscritos

Curso 6 de 9 en Certificado profesional de Ciencia de datos de IBM Programa Especializado

Learn how to analyze data using Python. This course will take you from the basics of Python to exploring many different types of data. You will learn how to prepare data for analysis, perform simple statistical analysis, create meaningful data visualizations, predict future trends from data, and more! Topics covered: 1) Importing Datasets 2) Cleaning the Data 3) Data frame manipulation 4) Summarizing the Data 5) Building machine learning Regression models 6) Building data pipelines Data Analysis with Python will be delivered through lecture, lab, and assignments. It includes following parts: Data Analysis libraries: will learn to use Pandas, Numpy and Scipy libraries to work with a sample dataset. We will introduce you to pandas, an open-source library, and we will use it to load, manipulate, analyze, and visualize cool datasets. Then we will introduce you to another open-source library, scikit-learn, and we will use some of its machine learning algorithms to build smart models and make cool predictions. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge. LIMITED TIME OFFER: Subscription is only $39 USD per month for access to graded materials and a certificate.

Destrezas que aprenderás

Predictive Modelling, Python Programming, Data Analysis, Data Visualization (DataViz), Model Selection

Revisiones

4.6 (4,811 calificaciones)

5 stars
3,497 ratings
4 stars
985 ratings
3 stars
251 ratings
2 stars
49 ratings
1 star
29 ratings

KA

Apr 22, 2019

I am highly impressed with this course !!\n\nThis course is very informative. Regular assignments helped me a lot to learn and to complete my course on time before the deadline.

KS

Mar 22, 2019

Highly recommended course. All i was about data analysis with using Python as a tool. In this course you can learn basic econometric, python, and of course how to analyse data.

De la lección

Model Evaluation

Model Evaluation and Refinement7:30

Overfitting, Underfitting and Model Selection4:20

Ridge Regression4:26

Grid Search4:33

Impartido por:

Joseph Santarcangelo
Ph.D., Data Scientist at IBM

Transcripción

Model evaluation tells us how our model performs in the real world.

In the previous module,

we talked about in-sample evaluation.

In-sample evaluation tells us how well our model fits the data already given to train it.

It does not give us an estimate of how well the train model can predict new data.

The solution is to split our data up,

use the in-sample data or training data to train the model.

The rest of the data, called Test Data,

is used as out-of-sample data.

This data is then used to approximate how the model performs in the real world.

Separating data into training and testing sets is an important part of model evaluation.

We use the test data to get an idea how our model will perform in the real world.

When we split a dataset,

usually the larger portion of data is used for

training and a smaller part is used for testing.

For example, we can use 70 percent of the data for training.

We then use 30 percent for testing.

We use training set to build a model and discover predictive relationships.

We then use a testing set to evaluate model performance.

When we have completed testing our model,

we should use all the data to train the model.

A popular function in the scikit-learn package for

splitting datasets is the train test split function.

This function randomly splits a dataset into training and testing subsets.

From the example code snippet,

this method is imported from sklearn cross-validation.

The input parameters y_data is the target variable.

In the car appraisal example,

it would be the price and x_data,

the list of predictive variables.

In this case, it would be all the other variables in

the car dataset that we are using to try to predict the price.

The output is an array.

x_train and y_train the subsets for training.

x_test and y_test the subsets for testing.

In this case, the test size is a percentage of the data for the testing set.

Here, it is 30 percent.

The random state is a random seed for random data set splitting.

Generalization error is a measure of how well

our data does at predicting previously unseen data.

The error we obtain using our testing data is an approximation of this error.

This figure shows the distribution of the actual values in

red compared to the predicted values from a linear regression in blue.

We see the distributions are somewhat similar.

If we generate the same plot using the test data,

we see the distributions are relatively different.

The difference is due to

a generalization error and represents what we see in the real world.

Using a lot of data for training gives us an accurate means

of determining how well our model will perform in the real world.

But the precision of the performance will be low.

Let's clarify this with an example.

The center of this bull's eye represents the correct generalization error.

Let's say we take a random sample of the data using

90 percent of the data for training and 10 percent for testing.

The first time we experiment,

we get a good estimate of the training data.

If we experiment again training the model with a different combination of samples,

we also get a good result,

but the results will be different relative to the first time we run the experiment.

Repeating the experiment again with

a different combination of training and testing samples,

the results are relatively close to the generalization error,

but distinct from each other.

Repeating the process, we get a good approximation of the generalization error,

but the precision is poor i.e.

all the results are extremely different from one another.

If we use fewer data points to train the model and more to test the model,

the accuracy of the generalization performance will be

less but the model will have good precision.

The figure above demonstrates this.

All our error estimates are relatively close together,

but they are further away from the true generalization performance.

To overcome this problem, we use cross-validation.

One of the most common out of sample evaluation metrics is cross-validation.

In this method, the dataset is split into K equal groups.

Each group is referred to as a fold.

For example, four folds.

Some of the folds can be used as a training set which we use

to train the model and the remaining parts are used as a test set,

which we use to test the model.

For example, we can use three folds for training,

then use one fold for testing.

This is repeated until each partition is used for both training and testing.

At the end, we use the average results as the estimate of out-of-sample error.

The evaluation metric depends on the model,

for example, the r squared.

The simplest way to apply cross-validation is to call the cross_val_score function,

which performs multiple out-of-sample evaluations.

This method is imported from sklearn's model selection package.

We then use the function cross_val_score.

The first input parameters,

the type of model we are using to do the cross-validation.

In this example, we initialize a linear regression model or object

lr which we passed the cross_val_score function.

The other parameters are x_data,

the predictive variable data,

and y_data, the target variable data.

We can manage the number of partitions with the cv parameter.

Here, cv equals three,

which means the data set is split into three equal partitions.

The function returns an array of scores,

one for each partition that was chosen as the testing set.

We can average the result together to estimate out of

sample r squared using the mean function NnumPi.

Let's see an animation,

let's see the result of the score array in the last slide.

First, we split the data into three folds.

We use two folds for training,

the remaining fold for testing.

The model will produce an output.

We will use the output to calculate a score.

In the case of the r squared i.e.

coefficient of determination, we will store that value in an array.

We will repeat the process using two folds for training and one fold for testing.

Save the score, then use

a different combination for training and the remaining fold for testing.

We store the final result.

The cross_val_score function returns

a score value to tell us the cross-validation result.

What if we want a little more information?

What if we want to know the actual predicted values

supplied by our model before the r squared values are calculated?

To do this, we use the cross_ val_predict function.

The input parameters are exactly the same as the cross_val_score function,

but the output is a prediction.

Let's illustrate the process.

First, we split the data into three folds.

We use two folds for training,

the remaining fold for testing.

The model will produce an output and we will store it in an array.

We will repeat the process using two folds for training one for testing.

The model produces an output again.

Finally, we use the last two folds for training,

then we use the testing data.

This final testing fold produces an output.

These predictions are stored in an array.

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2019 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play

Coursera
Acerca de
Liderazgo
Empleo
Catálogo
Certificados
Certificados MasterTrack™
Grados
For Enterprise
For Government

Comunidad
Learners
Partners
Developers
Beta Testers
Translators

Conectar
Blog
Facebook
LinkedIn
Twitter
Instagram
Youtube
Blog de Tecnología

Más
Términos
Privacidad
Ayuda
Accesibilidad
Prensa
Contacto
Directorio
Afiliados