Linear Algebra Review

Matrices and Vectors

Matrix: rectangular array of numbers, for example:

$A = \left[\begin{matrix} 1 & x \\ 2 & y \\ 4 & z \\ \end{matrix} \right]$

Dimension of matrix: rows x columns. Above example is a 3 x 2 matrix
Mnemonic: flying RC toy inside the matrix

Matrix Elements: $A_{ij}$ = "$i, j$ entry" in the $i^{th}$ row, $j^{th}$ column
So, $A_{11} = 1, A_{32} = z$ in above example

Vector: is an n x 1 matrix, for example:

$v = \left[\begin{matrix} 1 \\ x \\ 2 \\ y \\ \end{matrix} \right]$

Dimension of vector: number of rows. Above example is a 4-dimensional vector
Vector Elements: $v_i = i^{th}$ element
Can be either 1-indexed or 0-indexed, by default for this course, it is 1-indexed

Convention: uppercase to refer to matrices, lowercase to refer to scalars, vectors

Addition and Scalar Multiplication:

Matrix Vector Multiplication:

Trick to solve all in one go:

Matrix Matrix Multiplication:

Trick to solve all in one go:

Matrix Multiplication Properties:

• Not commutative A x B $\ne$ B x A
• Is associative: A x B x C = (A x B) x C = A x (B x C)
• Identity matrix (unit matrix), I: square matrix 1s on the main diagonal and zeros elsewhere
• A x I = I x A

Inverse and Tranpose:

• If A is an m x m matrix, and if it has an inverse, then $AA^{-1} = A^{-1} A = I$
• Matrices that don't have an inverse are "singular" or "degenerate"
• B is transpose of A, $B = A^T$ then $B_{ij} = A_{ji}$

Resources:
- https://www.coursera.org/learn/machine-learning/lecture/38jIT/matrices-and-vectors
- https://en.wikipedia.org/wiki/Identity_matrix