Now let's go out and use PyMC3 to look at an example for linear regression. In this example we have a single output variable given by y here, and two input variables given by x1 and x2. Here, alpha beta 1 and beta 2 are the parameters of this model and sigma is the error associated with this model. Now let's go ahead and generate some data so we can simulate the linear regression model. First, we go out and import all the modules necessary here, then we set the parameters for alpha, sigma and beta. We want to use the data set of size 100, then we go and generate these two data sets corresponding to that size. Finally, we compute y based on the equation here and the values that we have selected here for x1, x2 and the parameters here. Now that we have generated the data, let's go in and set up the model and PyMC3. First, we start off by importing the necessary modules in Python, in this cell here, we're going to set up the model necessary for creating our linear regression model. We start off by explicitly defining a context manager unlike what we did before, so we had a PM dot model, instead, we're doing in this line here and then using that explicitly. And anything to do with model creation happens within this block, within this context. So the first three lines correspond to the creation or the definition of the prayers for parameters, alpha, beta and sigma, alpha and beta are drawn from normal distributions here. Alpha is drawn from a normal distribution with mean given by 0 and standard deviation of 5. Beta is drawn from a normal distribution with a mean of 0, a standard deviation of 5, but you would notice there's an extra parameter here, that is a shape parameter. This is because we're grouping beta 1 and beta 2 into one variable instead of creating separate variables for those terms. Sigma is the error term that we saw here before, and because of the way we're defining it, we want it to be a positive value and therefore we're drawing from a half normal distribution. The expected value of the outcome is nothing but an equation that we had before minus the error term, so that's alpha plus beta, 0 times x1 plus beta 1 times x2. Now, you notice that I have here a command similar to what we had for the expected value, this is because PyMC3 does not store mu as a value in the trace information if we don't pass it to the PM deterministic function. So, if we want to have that listed during a model summary, we need to specify it this way. And finally we have the likelihood information, for that we're specifying a normal likelihood and the mean is nothing but the expected value for outcome here. And the standard deviation is a sigma or the error term that we defined earlier and the last parameter is nothing but our observed data that we're passing as a value to our observed parameter. Now, once that's done, we have predetermined model and you can visualize that model that we just generated by calling PM dot model to graphics, so that plots plate notation, the model that we just created. Plate notation is nothing but a visual representation of the random variables used to define our model and the inter relationships between them. So here you noticed that several plates representing our data, so sigma is drawn from a half normal, alpha is drawn from a normal and beta is drawn from a normal. So you notice this additional plate around beta, this is because we're trying to represent the fact that we're grouping several terms together and the dimensionality of the terms is given here. And you have y observation, which is our likelihood for observations, which depends on these variables, so the direction of the arrows indicate that dependency. And because we have a data corresponding to size which is 100, we represent that here in displayed notation and you will also notice that y observation is shaded to indicate the fact that these are observations. So plate notation is a way to graphically represent variables in a probabilistic framework and even though we don't require a great deal of understanding of plate notation for this course. If you're going to be working in probabilistic modeling, I encourage you to at least go to this link and read up more about it. Now that we have built the model, we can find various estimates using that model. So PyMC3 can be used to compute the map estimate using numerical optimization, and by default it uses the BFGS algorithm. So just as a note, these provide a point estimate which may not be accurate if the mode does not appropriately represent the distribution. So we can do that by calling the fine map method from within PyMC3 bypassing the name of from all and the number of evaluations we want to use for computing our map estimate. And if you print it out, you'll see the map estimates for each of the parameters we had in our model, it's alpha, beta and sigma.
