The impulse response is the output of a filter when the input is the delta function. A fundamental result states that the impulse response fully characterizes the behavior of an LTI system. Let's see why that is so. Assume that we have a filter and we can measure its impulse response by inputting a delta function, and it turns out that the impulse response is an exponentially decaying sequence, h of n equal to alpha to the power of n, times the unit step. Now, we want to use the same filter to filter an arbitrary sequence x of n that, in this example, is simply a three point sequence that is equal to 2 for n=0, is equal to 3 for n=1, and is equal to 1 for n=2, and is 0 everywhere else. So, we can always write our sequence as a linear combination of delayed delta function. So, in particular for our example, x of n is equal to 2 times delta of n + 3 times delta of n-1 plus delta of n-2. Now, we know the impulse response, so the response to the delta. And by exploiting linearity and time-invariance, we can compute the response of the system to the input sequence x[n] just by knowing the impulse response. Indeed, we apply the linear filter to the linear combination of deltas and by exploiting linearity first, we can split the operation of the filter over the three components of the signal. And by exploiting time and variance, we just sum together appropriately scaled version of the impulse response delayed. We can look at this graphically and we see that, when we filter the first component, 2 times delta of n, we get 2 times h of n. The second component of the signal is 3 times delta of n minus 1, and this gives rise to 3h of n minus 1, which we plot on top of the other response. And finally, the last component is simply delta of n minus 2 which filter gives h of n minus 2. So, now we have the three components, we sum them together in linearity, and we have obtained the response of the system to our arbitrary input. In general, remember that we can always write x of n, a generic discrete time sequence, as the sum for k that goes from minus infinity to plus infinity of a sequence of time delayed deltas scaled by the values of the sequence itself. So, this probably seemed like a futile exercise in module 3.2, but now we see the usefulness of this representation because, by linearity and time invariance, we can express the output as the sum from k that goes from minus infinity to plus infinity of the values of the sequence times the impulse response, time reversed and delayed by n. This sum here is so important in signal processing that it gets its own name and it's called the convolution of sequences x[n] and h[n]. The convolution which represents the output of a filter given its impulse response and an arbitrary input sequence x[n ], is actually an algorithmic formula to compute the output of the filter. The ingredients are a input sequence x[m] and a second sequence, h[m]. And the recipe involves the following steps. First, we time-reverse the impulse response. So, we flipped it in time, if it goes like this. Then, it will look like this. And at each step, from minus infinity to plus infinity, we center the time-reversed impulse response in the current sample n, so we shift the time reversed equal response by minus n. And then we compute the inner product between this shifted replica of the impulse response and the input sequence. Let's look at this graphically using this the same examples that we used before. So, we have an impulse response which is a decay in exponential sequence, and we have a three point simple input sequence. We plot these three actors on a chart like this. We have the input sequence on top. We have the time reversed and the delay impulse response on the second panel. And here, we have the inner product between these two sequences. So, at each step, as I said, we center the time-reversed impulse response on the current sample, so we start as -4 and we compute the inner product. Since the impulse signal in our example is now 0, only between 0 and 2, up to 0, fundamentally, nothing happens and the inner product is always 0. And we can see that it was 0 for values before -4, and it will continue to be 0 until we hit 0. At which point, we start to have an overlap between these two sequences, and the inner product will not be 0, in particular, on the first step, it will be equal to this sample which is equal to 1 times this sample which is equal to 2. So, the sum will be equal to 2. We advance another step and then we see that the overlap involves two points now, here and here. And we compute their products and their sum and we get the second point in our output sequence. Third step, we'll finally involve all three points from the input sequence. We compute the product with the impulse response and the sum and we get our third output sample. And the process continues like so. Now, since the impulse response is an infinite sequence, one side of the infinite sequence, from now on, the inner product will always be non-zero and will have an output that will continue to be non-zero forever and ever. Finally, a few words on the convolution. The convolution is of course linear and time invariant because it describes a linear and time invariant operation. It is also commutative which means that if you have two filters in cascade, you can safely invert their sequence and the result will not change. For absolutely and square summable sequences, the convolution is also associative, which means that if you have a cascaded system, you can lump their effect into a single filter, whose impulse response is the convolution of the individual impulse responses.
