So the final sort of model that I'm going to show you,

probabilistic model is called a Markov chain.

Now, a Markov chain is a dynamic model.

It's a probabilistic model and it's discrete.

And what it does is modeled a discrete time state space transition.

Now that's a bit of a mouth full.

So I'm going to immediately give you an example, so

you can understand what we mean by that.

And the example that I'm thinking about is what a public policy person might do when

they're trying to understand an individual's employment status.

So obviously, unemployment and employment are key features of the economy.

We like to understand them.

We can understand them at a point in time by doing a survey.

And asking people whether their employed or not employed.

But we're, also, frequently interested in the dynamics of that process for

how long do people stay unemployed, how liked they are.

They do transition from employment to non-employment, so,

in this particular example, I'm going to treat time not as a continuous variable,

but as a discrete variable.

And I'm going to consider time in six-month blocks, and

I'm going to consider an individual's employment status as being in

one of three possible categories.

First one is that you're employed, you got a job.

The second one is that you're unemployed and looking for a job.

And the third one is that you're unemployed and

you're not looking for a job.

Now, it's quite possible to move from one of those states.

That's what we mean by state.

There are three states here to another.

So you could be employed and you get fired.

So that's going to take to being unemployed,

and maybe you're upset about being fired, and you're going to try and find a job.

And so you're unemployed and you're looking, or

maybe you've said, that's enough, I'm unemployed and I'm not going to look.

So you could go from state one to either state two or state three.

Likewise, you could be unemployed and looking for a job and then get employed.

So you could from one time period to the next go from state two to state one,

but if you're unemployed and not looking, well,

you can't go from state three to state one.

Because even not looking for a job, you're not going to become employed.

So you can see there are some transitions that you can make and

others that you can't between these three states.

Now, what a Markov chain does for you is model

the probability of transitions between those three states.

So if you have a look at the graphic on the right-hand side here,

you can see the three possible states that an individual is in.

So they're employed, they are unemployed and looking,

that's state two, or unemployed and not looking, that's state three.

And I've drawn arrows that show you the possible transitions.

And so, if you are employed, you could certainly move to the unemployed and

looking state, you could also move to the unemployed and not looking state.

It's important to realize that you could stay employed in the next time period,

which is why there's a darker blue arrow from the state back into itself.

You can certainly stay in the same state.