Extrapolating from Small Samples (People Analytics)

Loading...

Del curso dictado por University of Pennsylvania

Business Analytics Capstone

318 calificaciones

University of Pennsylvania

Business Analytics Capstone

318 calificaciones

Curso 5 de 5 en SpecializationBusiness Analytics

The Business Analytics Capstone Project gives you the opportunity to apply what you've learned about how to make data-driven decisions to a real business challenge faced by global technology companies like Yahoo, Google, and Facebook. At the end of this Capstone, you'll be able to ask the right questions of the data, and know how to use data effectively to address business challenges of your own. You’ll understand how cutting-edge businesses use data to optimize marketing, maximize revenue, make operations efficient, and make hiring and management decisions so that you can apply these strategies to your own company or business. Designed with Yahoo to give you invaluable experience in evaluating and creating data-driven decisions, the Business Analytics Capstone Project provides the chance for you to devise a plan of action for optimizing data itself to provide key insights and analysis, and to describe the interaction between key financial and non-financial indicators. Once you complete your analysis, you'll be better prepared to make better data-driven business decisions of your own.

De la lección

Module 3: Your Strategy

In Module 3, you will focus on creating your recommended strategy for GYF to address adblockers. Your strategy does not have to be lengthy, but it must be clear, and it must address the problem. (Hint: if you have a clearly defined problem, your strategy is much more likely to be clearly defined as well). You'll be submitting your strategy for peer review, and then also reviewing the work of at least 3 of your peers. It's OK if reviewing the strategies of other learners in this course gives you further ideas for revising your own strategy. One of the primary benefits of peer review is to expand the range of feedback you can get, and we designed this Module around peer review so that you can get as much feedback as possible before moving on to the next phase of the project. You may find the resources and lectures below helpful in formulating your strategy and considering how data can be leveraged and appropriately understood. You are strongly encouraged to complete and include your response to Application Exercise 2 as part of your Strategy.

Performance Evaluation: the Challenge of Noisy Data (People Analytics)6:46

Finding Persistence: Regression to the Mean (People Analytics)11:56

Extrapolating from Small Samples (People Analytics)5:53

The Wisdom of Crowds: Signal Independence (People Analytics)5:56

Process vs. Outcome (People Analytics)7:47

Hiring 1 (People Analytics)8:50

Hiring 2 (People Analytics)9:07

Conoce a los instructores

Wharton Teaching Staff
Educators
The Wharton School

So the second issue that I want to talk about is sample size, and

I want to to begin with the psychology of sample size, essentially.

And this motivates our use of data to increase samples and

provide some caution on our use of small, intuitive samples.

The first is an example, this is from a research study, classic research study

where the participants were asked the following vignette.

Your firm has two plants, one large and one small,

which mass-produce a standard computer chip, other than the amount they produce,

the two plants are identical in all essential regards.

Both use the same technology to produce the same product.

When properly functioning, this technology produces 1% defective items.

Whenever the number of defective items from one day's production exceeds 2%,

a special note is made in the quality control log to flag the problem.

At the end of the quarter,

which plant would you expect to have more flagged days in the quality control log?

So, two identical plants big and small.

How many fall outside the quality control acceptable limits?

Everything else about the plant's the same,

it's just that one is larger than the other.

Does the small plant have more of these deviations?

Does the large plant have more, or

are they the same because after all they're identical in every other respect?

So if you ask people this, about 22% say the smaller plant,

30% the larger plant, and then about half say the same, because in fact,

all of the technology's the same and how should it be any different.

What's the right answer?

The right answer is the small plant,

of course, because from small samples, you get greater variation.

And what we're talking about here is variation,

these are production outcomes that fall outside the quality control limits,

those are variations from expectation.

Small samples necessarily or more vary and people don't appreciate that and

this is a major problem, for performance evaluation,

because we are apt to draw too strong a conclusion from small samples.

Let's look at this in a little more detail.

Sample means converge to population means as the sample size increases.

If you've had a statistics course,

you might recognize this as the Central Limit Theorem.

So, because this happens as we get larger and

larger samples the converses exactly true.

That the smaller the sample the more likely you see more extreme values.

So, there is a couple example questions when you more like to see a 400 batting

average in baseball?

Which is a very good batting.

Far higher than we typically see.

Do you see it at the beginning of the season, around May 1, or

towards the end of the season, September 1?

Well of course, you're far more likely to see such an extreme batting average from

a small sample, the kind of sample you would have at the beginning of the season

then you would later.

What about hospitals?

Where are you likely to see a dramatically higher percentage of boys than girls or

vice versa, born on any given day?

A small community hospital, which might have four or five births a day?

Or a large city hospital, which might have 100 births a day?

Where are you more likely to see the population mean vary in that daily sample?

The small hospital of course, we'll see large variations.

You might have all girls one day, all boys another day and

a small hospital there's no way you'll have that in a larger hospital.

We emphasize this because it should give us great caution in what we

say from small samples, but importantly it shows that we don't have that caution.

We tend to believe that what we see in the small sample

is representative of the underline population.

We don't appreciate that a small sample can bounce around quite a bit and

not in fact be representative of the underline population.

This means that we should be very careful to not follow our intuition

on what it means for will we have a small sample.

This is a bias acknowledged in the literature,

called in the literature the Law of Small Numbers.

Which is silly of course the is no law of small numbers.

It's not like the law of large numbers.

The small numbers in fact, what makes them interesting is that there is no law,

they bounce around.

People believe small samples closely share the properties

of the underlying population, but in fact they don't.

This means they too readily infer population's properties,

the average, from the samples.

They neglect the role variability plays, chance plays, in these small samples.

What are the implications?

What are the implications in your organization?

This is a fundamental issue in performance evaluation, because often,

maybe even typically, performance evaluation is on small samples.

The smaller the sample,

the more we have to be cautious what kind of inference we draw from it.

So, this is a place where data can make a big difference.

This is a place where, by using more sources,

more history, using data as a opposed to just relying on our intuition.

We had the chance to collect larger samples, but

I don't want to say data alone is sufficient.

In fact, it's interesting to note that in academia,

especially in psychology, but now it's expanding to other disciplines as well.

We are discovering that we have been drawing too strong an inference from even

our academic research.

There's somewhat of a revolution going on

in psychology now being picked up in other disciplines.

That the main theme of which is,

we've been drawing too strong an inference from the samples we've been using.

So, this isn't just academia saying be careful out there you non-academics.

It's even academics that have had to learn,

that we too readily draw inferences from samples that aren't really sufficient.

The lesson is almost always get more data, get bigger samples,

be careful about the inferences you draw from these small numbers, and

know that we have a bias toward jumping to quickly based on the small samples.

Explora nuestro catálogo

Inscríbete de manera gratuita y obtén recomendaciones personalizadas, actualizaciones y ofertas.

Coursera brinda acceso universal a la mejor educación del mundo, al asociarse con las mejores universidades y organizaciones, para ofrecer cursos en línea.

© 2018 Coursera Inc. Todos los derechos reservados.

Descargar en la App Store

Consíguelo en Google Play