Today

  • Concluding remarks. Three pillars of machine learning: analysis/probability/statistics, linear algebra/optimization, and computing/algorithms.

  • Some applications.

  • Multivariate calculus and optimization (cont’d).

  • Course evaluation: https://my.ucla.edu/

Announcement

  • Send me you questions before the last lecture. I will try to answer them in the last lecture.

Some issues I saw in HW6

  • HW6 Q10. Recognize SVD.

  • HW6 Q12. Explanations?

  • HW6 Q13. Invariance of the spectral norm (L2), nuclear norm, and Frobenius norm under orthogonal rotation.

  • HW6 Q17. Alternative proof of global optimality of $\hat{\Omega}$ using Cholesky decomposition (optional).

Q&A

  • HW6 Q11.6 (MP inverse produces the smallest least squares solution). $\beta^\star = (\beta^\star - \beta^+) + \beta^+$, where $\beta^\star - \beta^+ \in \mathcal{N}(X’X) = \mathcal{N}(X)$ and $\beta^+ \in \mathcal{C}(X’)$. Thus $\beta^\star - \beta^+ \perp \beta^+$ and $|\beta^\star|^2 = |\beta^\star - \beta^+|^2 + |\beta^+|^2 \ge |\beta^+|^2$.

  • HW4, Q3.5 (Householder transformation for QR decomposition).

  • HW4 BV 12.2.

  • Direct sum of vector spaces.