Summation sign rules: identities for simple regression
There are many sources on the Internet. This and this are relatively simple, while this one is pretty advanced. They cover the basics. My purpose is more specific: to show how to obtain a couple of identities in terms of summation signs from general properties of variance and covariance.
Shortcut for covariance. This is a name of the following identity
where on the left we have the definition of and on the right we have an alternative expression (a shortcut) for the same thing. Letting in (1) we get a shortcut for variance:
see the direct proof here. Again, on the left we have the definition of and on the right a shortcut for the same.
In this post I mentioned that
for a discrete uniformly distributed variable with a finite number of elements, the population mean equals the sample mean if the sample is the whole population.
This is what it means. The most useful definition of a discrete random variable is this: it is a table values+probabilities of type
Here are the values and are the probabilities (they sum to one). With this table, it is easy to define the mean of :
A variable like this is called uniformly distributed if all probabilities are the same:
In this case (3) becomes
This explains the statement from my post. Using (4), equations (1) and (2) rewrite as
Try to write this using summation signs. For example, the first identity in (5) becomes
This is crazy and trying to prove this directly would be even crazier.
Remark. Let be a sample from an arbitrary distribution. Regardless of the parent distribution, the artificial uniform distribution from Table 2 can still be applied to the sample. To avoid confusion with the expected value with respect to the parent distribution, instead of (4) we can write
where the subscript stands for "uniform". With that understanding, equations (5) are still true. The power of this approach is that all expressions in (5) are random variables which allows for further application of the expected value with respect to the parent distribution.