Random Matrices
Introduction to Random Matrices
Recommended Prerequesites
- Probability
- Probability II
- Sampling
- Introduction to Multivariate Distributions
Building on Prior Sections
In the last couple of chapters, we covered multivariate distributions. Now, we shall extend this idea to random matrices, where some or all of the elements are random variables.
To understand a random matrix, consider a matrix where each entry is determined by a random experiment. For example, imagine a matrix representing the returns of multiple stocks over several days. Each row corresponds to a stock, and each column to a day. The entries are random because stock returns are inherently unpredictable.
Visually, a random matrix can be thought of as a grid or table filled with random numbers. Each element varies according to a specified probability distribution, and the entire matrix captures the collective randomness of a system with multiple interrelated components.
Wishart Distribution
Introduction
The Wishart distribution extends the concept of the chi-squared distribution to multidimensional settings, dealing with matrices instead of scalar values.
Definition
Let \(X\) be a \(p\times n\) matrix whose columns \(x_i\) are iid p-dimensional normal vectors with mean zero and covariance matrix \(\Sigma\).
$$x_i\sim N_p(0,\Sigma),\quad i=1,2,\dots,n$$
Then the matrix
$$S=\sum_{i=1}^{n}x_ix_{i}^T=XX^T$$
follows a Wishart distribution with n degrees of freedom and scale matrix \(\Sigma\), denoted \(S\sim W_p(n,\Sigma)\).
Chi-Square Distribution
Since it was mentioned earlier, let's revisit the Chi-Square Distribution. If \(z_1,z_2,\dots,z_n\) are independent standard normal random variables, then the sum of their squares follows a chi-square distribution with n degrees of freedom:
$$Q=\sum_{i=1}^nz_{i}^2\sim \chi_{n}^2$$
Back to Wishart
Let's look at the Wishart distribution when \(p=1\). Well, since the columns are p-dimensional normals and \(p=1\), it is a scalar normal random variable \(x_i\sim N(0,\sigma^2)\). The Wishart matrix \(S\) reduces to:
$$S=\sum_{i=1}^nx_{i}^2$$
Dividing both sides by \(\sigma^2\),
$$=\frac{S}{\sigma^2}=\sum_{i=1}^n\left(\frac{x_i}{\sigma}\right)^2$$
Since a 0-mean normal divided by its standard deviation is a standard normal along with our prior description of the Chi-Square distribution:
$$\frac{S}{\sigma^2}\sim \chi_{n}^2$$
Density Function
The PDF of the Wishart distribution is given by:
$$f(S)=\frac{|S|^{(n-p-1)/2}e^{-\frac{1}{2}\text{tr}(\Sigma^{-1}S)}}{2^{np/2}|\Sigma|^{n/2}\Gamma_p(\frac{n}{2})}$$
where
- \(|S|\) is the determinant of S
- \(\text{tr}(\Sigma^{-1}S)\) is the trace of the product \(\Sigma^{-1}S\)
- \(\Gamma_p(\frac{n}{2})\) is the multivariate gamma function.