Biostat 216 Homework 6

Due Nov 11 Friday @ 11:59pm

Submit a PDF (scanned/photographed from handwritten solutions, or converted from RMarkdown or Jupyter Notebook) to Gracescope on BruinLearn.

Eigenvalues and eigenvectors

• Q1. Diagonalize (show the steps to find eigenvalues and eigenvectors) $$ \mathbf{A} = \begin{pmatrix} 2 & -1 \\ -1 & 2 \end{pmatrix} $$ and compute $\mathbf{X} \boldsymbol{\Lambda}^k \mathbf{X}^{-1}$ to prove the formula $$ \mathbf{A}^k = \frac 12 \begin{pmatrix} 1 + 3^k & 1 - 3^k \\ 1 - 3^k & 1 + 3^k \end{pmatrix}. $$

• Q2. Suppose the same $\mathbf{X}$ diagonalize both $\mathbf{A}$ and $\mathbf{B}$. That is they have the same eigenvectors in $\mathbf{A} = \mathbf{X} \boldsymbol{\Lambda}_1 \mathbf{X}^{-1}$ and $\mathbf{B} = \mathbf{X} \boldsymbol{\Lambda}_2 \mathbf{X}^{-1}$. Prove that $\mathbf{A} \mathbf{B} = \mathbf{B} \mathbf{A}$.

• Q3. Suppose the eigenvalues of a square matrix $\mathbf{A} \in \mathbb{R}^{n \times n}$ are $\lambda_1, \ldots, \lambda_n$. Show that $\det (\mathbf{A}) = \prod_{i=1}^n \lambda_i$. Hint: $\lambda_i$s are roots of the characteristic polynomial.

• Q4. Ture of false. For each statement, indicate it is true or false and gives a brief explanation.

  1. If the columns of $\mathbf{X}$ (eigenvectors of a square matrix $\mathbf{A}$) are linearly independent, then (a) $\mathbf{A}$ is invertible; (b) $\mathbf{A}$ is diagonalizable; (c) $\mathbf{X}$ is invertible; (d) $\mathbf{X}$ is diagonalizable.

  2. If the eigenvalues of $\mathbf{A}$ are 2, 2, 5 then the matrix is certainly (a) invertible; (b) diagonalizable.

  3. If the only eigenvectors of $\mathbf{A}$ are multiples of $\begin{pmatrix} 1 \\ 4 \end{pmatrix}$, then the matrix has (a) no inverse; (b) a repeated eigenvalue; (c) no diagonalization $\mathbf{X} \boldsymbol{\Lambda} \mathbf{X}^{-1}$.

• Q5. Let $\mathbf{A} \in \mathbb{R}^{m \times n}$ and $\mathbf{B} \in \mathbb{R}^{n \times m}$. Show that $\mathbf{A} \mathbf{B}$ and $\mathbf{B} \mathbf{A}$ share the same non-zero eigenvalues. Hint: examine the eigen-equations.

• Q6. Find the eigenvalues of $\mathbf{A}$, $\mathbf{B}$ and $\mathbf{A} + \mathbf{B}$: \begin{eqnarray*} \mathbf{A} = \begin{pmatrix} 3 & 0 \\ 1 & 1 \end{pmatrix}, \quad \mathbf{B} = \begin{pmatrix} 1 & 1 \\ 0 & 3 \end{pmatrix}. \end{eqnarray*} Are eigenvalues of $\mathbf{A} + \mathbf{B}$ equal to the sum of eigenvalues of $\mathbf{A}$ and $\mathbf{B}$?

• Q7. Suppose $\mathbf{A}$ has eigenvalues 0, 3, 5 with independent eigenvectors $\mathbf{u}$, $\mathbf{v}$, $\mathbf{w}$ respectively.

  1. Give a basis for $\mathcal{C}(\mathbf{A})$ and a basis for $\mathcal{N}(\mathbf{A})$.
  2. Find a particular solution to $\mathbf{A} \mathbf{x} = \mathbf{v} + \mathbf{w}$. Find all solutions.
  3. Show that the linear equation $\mathbf{A} \mathbf{x} = \mathbf{u}$ has no solution.

Positive definite matrices

• Q8. Suppose $\mathbf{C}$ is positive definite and $\mathbf{A}$ has independent columns. Apply the energy test to show that $\mathbf{A}' \mathbf{C} \mathbf{A}$ is positive definite.

• Q9. Show that the diagonal entries of a positive definite matrix are positive.

• Q10. Suppose $\mathbf{S}$ is positive definite with eigenvalues $\lambda_1 \ge \lambda_2 \ge \cdots \ge \lambda_n > 0$ with corresponding orthonormal eigenvectors $\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_n$.

  1. What are the eigenvalues of the matrix $\lambda_1 \mathbf{I} - \mathbf{S}$? Is it positive semidefinite?
  2. How does it follow that $\lambda_1 \mathbf{x}'\mathbf{x} \ge \mathbf{x}' \mathbf{S} \mathbf{x}$ for every $\mathbf{x}$?
  3. Draw the conclusion: The maximum value of the Rayleigh quotient $$ R(\mathbf{x}) = \frac{\mathbf{x}'\mathbf{S}\mathbf{x}}{\mathbf{x}'\mathbf{x}} $$ is $\lambda_1$.
  4. Show that the maximum value of the Rayleigh quotient subject to the condition $\mathbf{x}\perp \mathbf{u}_1$ is $\lambda_2$. Hint: expand $\mathbf{x}$ in eigenvectors $\mathbf{u}_1,\ldots,\mathbf{u}_n$.
  5. Show that the maximum value of the Rayleigh quotient subject to the conditions $\mathbf{x}\perp \mathbf{u}_1$ and $\mathbf{x}\perp \mathbf{u}_2$ is $\lambda_3$.
  6. What is the maximum value of $\frac 12 \mathbf{x}' \mathbf{S} \mathbf{x}$ subject to the constraint $\mathbf{x}'\mathbf{x}=1$. Hint: write down the Lagrangian and set its derivative to zero.