TASK 3 - Theoretical exarcise number 3

1.

Consider a random vector \(X \sim \mathcal{N}_2 \left( \mu,\,\Sigma \right)\) for \(\mu = \left(2,\,2\right),\) \(\Sigma = \mathbb{I}_2.\) Consider also matrices \(\mathbb{A} = \left( 1,\,1\right),\) \(\mathbb{B} = \left( 1,\,-1\right).\) Show that the random varaibles \(\mathbb{A}X,\) \(\mathbb{B}X\) are independent.

The covariance of the random variables \(\mathbb{A}X\) and \(\mathbb{B}X\) is \[\begin{gather} cov\lbrace \mathbb{A}X,\mathbb{B}X \rbrace = \mathbb{A} \, var(X) \, \mathbb{B}^T = \mathbb{A} \, \mathbb{I}_2 \, \mathbb{B}^T = \begin{pmatrix} 1 & 1 \\ \end{pmatrix} \begin{pmatrix} 1 \\ -1 \\ \end{pmatrix} =1-1=0. \end{gather}\] The random variables \(\mathbb{A}X,\) \(\mathbb{B}X\) are uncorrelated and normally distributed (since the distribution of \(X\) is normal), hence they are independent.

The uncorrelatedness can be seen from a regression line.

Since the slope of the regression line is very close to zero, we see that the random variables are indeed uncorrelated (and consequently independent).

2.

The joint normality of a random vector implies normality of all marginals. On the other hand, the normal distribution of all marginals does not imply the joint normality of the whole random vector. An example of that is: \(X \sim \mathcal{N} \left( 0,\,1\right)\) and \(Y = \pm X\) each with probability \(\frac{1}{2}.\) Then \(Y \sim \mathcal{N} \left( 0,\,1\right)\) (since \(\pm X \sim \mathcal{N}\left( 0,\,1\right)\)). We see that both marginal distributions of the vector \(\left( X,\,Y\right)^T\) are normal. The joint distribution is however not normal. To see that, we can realize that any linear transformation of a normally distributed vector is also normally distributed. Because \(X + Y\) is not normally disributed (it cannot be as it is equal to zero with a probability of 0.5), the random the vector \(\left( X,\,Y\right)^T\) cannot be normally distributed.

To verify that \(X + Y\) is indeed not normally distributed and \(X\) and \(Y\) are, we can use graphical tools, for example histograms.