Linear Algebra Review

Matrices and Vectors

Matrix: rectangular array of numbers, for example:

\(A = \left[\begin{matrix} 1 & x \\ 2 & y \\ 4 & z \\ \end{matrix} \right]\)
Dimension of matrix: rows x columns. Above example is a 3 x 2 matrix
Mnemonic: flying RC toy inside the matrix

Matrix Elements: \(A_{ij}\) = "\(i, j\) entry" in the \(i^{th}\) row, \(j^{th}\) column
So, \(A_{11} = 1, A_{32} = z\) in above example

Vector: is an n x 1 matrix, for example:

\(v = \left[\begin{matrix} 1 \\ x \\ 2 \\ y \\ \end{matrix} \right]\)
Dimension of vector: number of rows. Above example is a 4-dimensional vector
Vector Elements: \(v_i = i^{th}\) element
Can be either 1-indexed or 0-indexed, by default for this course, it is 1-indexed

Convention: uppercase to refer to matrices, lowercase to refer to scalars, vectors


Addition and Scalar Multiplication:



Matrix Vector Multiplication:

Trick to solve all in one go:

Matrix Matrix Multiplication:

Trick to solve all in one go:

Matrix Multiplication Properties:


  • Not commutative A x B \(\ne\) B x A
  • Is associative: A x B x C = (A x B) x C = A x (B x C)
  • Identity matrix (unit matrix), I: square matrix 1s on the main diagonal and zeros elsewhere
  • A x I = I x A

Inverse and Tranpose:


  • If A is an m x m matrix, and if it has an inverse, then \(AA^{-1} = A^{-1} A = I\)
  • Matrices that don't have an inverse are "singular" or "degenerate"
  • B is transpose of A, \(B = A^T\) then \(B_{ij} = A_{ji}\)


Resources:
https://www.coursera.org/learn/machine-learning/lecture/38jIT/matrices-and-vectors
https://en.wikipedia.org/wiki/Identity_matrix

Comments