Welcome back. We left off about to launch into the construction of neural response models. In this next couple of sections, we'll be looking at models of increasing complexity that incorporate more and more features of realistic neural responses. The main work that we'll be doing centers on methods for finding out what components of a stimulus a neural system responds to and the response function that links stimulus to response. For now, we're going to take the response to mean a single spike produced by a chosen neuron. Further, we'll consider how the probability of seeing a single spike at a certain time, or the firing rate, which we'll denote by r of t, depends on the stimulus. We'll talk more later, about the model for how the probability of seeing that spike, explicitly depends on this firing rate. But, for now let's move to a description in which we try to determine the rate r(t) as a function of the input s. So, what's the simplest, possible relationship that one could imagine? That the response at each time, is linearly dependent on the stimulus, at that time. Or perhaps, at some small time in the past. That's a t minus tao. If this were the case, then if we have some stimulus which is varying in time like this, our response, r (t), is going to look just like that stimulus that's scaled by some factor, perhaps delayed by a little bit and in the appropriate units. So what could go wrong with a picture like this? In general we expect that the response depends not just on the stimulus at some particular time in the past but on some combination of recent inputs. If the response depends on some weighted sum of previous inputs like this we can still express that as a linear system but in this more general sense. So what we're seeing here is that by starting at some time, t, we step back over a range of time points in the past here indexed by k. And at each of these time points, we weight or multiply, just like our firing factor in the previous slide. We weight the stimulus at that particular time by some factor, but now the factor depends on how far back you're looking. The weight factors are given by these f (k), which we can write as a function, f, here defined relative to this index k, which is stepping backward. So we take the kth element of our stimulus weighted by the value of f(k), multiply the two together. Move along to the next step. Do the same thing. Add them together. We can also write this in integral form, where now the variable tau is a dummy variable that acts llike the index K. Some of you might recognize what we've written here as a convolution. The set of weights f are equivalent to a linear filter on the input. Note the effects of temporal filtering here. Although this is just a cartoon, the fast variations in S can get smoothed out in producing R. Let's think about some examples. What if the response is proportional to a running average of the input? What would f look like in that case? In this case what we expect is that our response at some given time has to be weighted by some linear function and what's that going to look like as a function of time. Let's say we're averaging over some window with n time steps. So now we'd like to take each of our values of s at those n time steps. And weighted by a factor of 1 over n. That will give us an average over a window of length n. And so now our filter f is simply going to look, look like that. Where the height here is 1 over n. See if you can figure out what a leaky average would look like. What if r depended on s, averaged over time, but with a memory that fades in time? So time points near to time t would be weighted with a large value. But as one goes back from time t, those white, those weight values would get smaller and smaller. So if they're exponentially decreasing, that's something called a leaky integrator. Such a system sums it's inputs, but with a strength that decays exponentially into the past. We'll see a physical implementation of this later. The neuronal cell membrane behaves in exactly this way. So that was a first pass at predicting how neuronal responses might depend on a stimulus that changes in time. What if the neuron is sensitive to patterns of light in space? We can apply the same idea. In fact, you've already met this idea when you were introduced to receptive fields in the first lecture. You might recall from the previous lecture that receptive fields in the retina, have what's called a center surround form. And that V1 neurons are sensitive to oriented edges. Let's think more concretely what that means. So our goal here is to take what we just learned about temporal filtering, and apply it to inputs s, which are now defined, rather than being defined over time, they're defined in space. So we're thinking about spacial receptive fields. We're thinking about the response of a neuron to some particular region of visual space. So lets say this is our, our visual scene, we have a neuron in the retina, its response is centered at some point, x, y, in that space. If the response were truly linear, then, neglecting for now, any time variation, the response of a neuron who's receptive field is centered at that point, x, y, could be computed like this. We would take the stimulus which is now a pattern of light in this two dimensional space, and weight it at each point in the scene by the neuron's receptive field f, now defined over space. And add up all those values to get one single number, the predicted response. So here's our s centered around x and y. We now vary this index, x prime and y prime. At each point, x prime y prime, we look up our filter, f, which is a function of x prime and y prime of this relative location. We multiply those two together and add them up. Now when a retinal ganglion cell is responding to some image, its response is determined by how similar the image is to its receptive field, f. What does similarity mean? It's exactly what we just did, the weighting of the image by the filter defined by that shape. We can also write this again as a convolution. Now let's be a bit more precise about the cartoon that we drew before for our receptive field f. We drew it here in 2D, where the shading roughly represented the value of f at each point in space, or in relative space. Here's a much better picture. Now again defining the filter over the relative co-ordinates, X-prime and Y-prime, lets draw the values or weights of f going into this third dimension. Now you can see that f has large positive values here in the middle, and negative values here in the surround. So lets think about how a receptive field like this interacts with an image. This means, if there's a bright part of the image, around zero. So let's center this now, again, at some particular point, x and y. We plunk this receptive field over the image at that point. We take each of the values in the image, in this coordinate frame, x prime, y prime. We multiply them by the values of f at those points and we sum them together. That means that if there's a bright part of the image at zero, that's going to drive the neuron positively because f there is multiplied by that large bright number. That increases the response r. But there, if there is a bright part of the image that falls into the surround, that's going to contribute a large, that large, bright number, multiplied by a negative weight. And that's going to decrease r. This model for a receptive field is often approximated as a difference of Gaussians. A narrow positive Gaussian blob in the middle. That's the excitatory center, and subtracted from it, a broader and shallower Gaussian to capture the, the suppressive surround. The effect of such a differencing filter is to detect local changes. Such a filter will respond strongly when there's, when there's a bright patch near to a dark patch. This is not just some crazy thing that biology does. Such filters are ones we use all the time in image processing. Look at this example from the gimp software suite. Here's again that photo of the Taj Mahal. And here is the same image in black and white, when a similar filter has been applied to it. Every point in the image on the right is the result of a difference of gaussians filter applied to the image on the left. You should be able to see that the new image consists only of edges because of its differencing property. This filter cancels out regions in the image that have constant intensity and has a large value only when there's a change in light level or contrast. So now we've seen how a linear filter can extract a spatial feature that explains the responses of retinal visual neurons. We started with cases where we just had temporal dependance and thought about how a time varying response might depend on temporal variations in the stimulus. Now let's put them together. Because in general the sensory neurons response will depend both on time and on other properties of the stimulus. In the case of vision, the spatial distribution of light intensity. So what this implies is that we need a filter f that depends on space and on time. So it could be thought about as a little 3D movie. Here, you're just seeing frames of the receptive field f over different points in time. So now let's go back to the simple case of pure time dependence. Imagine, for example, we're driving our retina with a blank screen whose brightness is varying, or that this might be an auditory signal going into a cochlear neuron. While the idea of linear filtering is very powerful and useful, it can't quite be the full picture. So can you see some shortcomings? So for example, can firing rates be negative? Can they increase indefinitely as the input increases? Both of those are a possible result from a linear filtering operation like this. Solving these two problems can be achieved with a clever and simple fix. Without abandoning the power and beauty of the idea of linear filtering, we can ensure that our output does the right thing by imposing an extra step, a nonlinear function applied to the filtered stimulus. For example, the function shown above, this guy here, will have the effect that, I'll call this g, when the filtered stimulus which we represent here as s*f or s convolved with f is small or negative. Go down to here the firing rate goes to zero. When it's very large then the firing rate will saturate to some fixed value. This nonlinear function g could have other behaviors as well. So now that we have these basic components of a coding model, this linear filter, and a nonlinear, static non-linearity that transforms the filtered signal into a, into an estimated firing rate. Let's take a break and in the next section, we'll move onto finding the components of a model like this from data.