Linear Algebra

What can vectors describe, and what can we build from them?

0. 1.0Opening: What is a vector?
1. 1.1Arrows, movement, and coordinates
2. 1.2Adding vectors
3. 1.3Scaling vectors
4. 1.4Linear combinations
5. 1.5What can we reach?
6. 1.6First encounter with span
7. 1.7Coordinates are descriptions
8. 1.8Python Lab: vector arithmetic
9. 1.9Proof Lab: tiny proofs
10. 1.10Common traps
11. 1.11Practice set
12. 1.12Chapter checkpoint
13. 1.13Where we go next

Which directions actually contribute new information?

1. 2.1Span as a generated world
2. 2.2Redundant directions
3. 2.3Linear dependence
4. 2.4Linear independence
5. 2.5Basis, intuitive version
6. 2.6Coordinates relative to a basis
7. 2.7Different bases, same space
8. 2.8Python Lab: independence and coordinates
9. 2.9Exercises

What does “linear” mean geometrically and algebraically?

1. 3.1Moving an entire space
2. 3.2Scaling, rotation, reflection, shear, projection
3. 3.3Grid-line intuition
4. 3.4Additivity
5. 3.5Homogeneity
6. 3.6Formal definition of a linear map
7. 3.7A map is determined by a basis
8. 3.8Composition
9. 3.9Injective and surjective intuition
10. 3.10Interactive transformation lab
11. 3.11Exercises

Where do matrices come from?

1. 4.1Images of basis vectors become columns
2. 4.2Matrix-vector multiplication as column combination
3. 4.3Rectangular matrices
4. 4.4Dimension-changing maps
5. 4.5Matrix entries as coordinates
6. 4.6Matrix addition and scalar multiplication
7. 4.7Identity, zero, diagonal, triangular, and permutation matrices
8. 4.8Row and column viewpoints
9. 4.9Python Lab: matrix-vector multiplication
10. 4.10Exercises

1. 5.1Composition becomes multiplication
2. 5.2Why order matters
3. 5.3Matrix multiplication by columns
4. 5.4The row-column rule
5. 5.5Associativity from composition
6. 5.6Identity matrices
7. 5.7Inverse transformations
8. 5.8Inverse matrices
9. 5.9When inverses fail
10. 5.10Python Lab: matrix multiplication
11. 5.11Exercises

1. 6.1Three views of Ax = b
2. 6.2Equation view
3. 6.3Column-combination view
4. 6.4Transformation and preimage view
5. 6.5Homogeneous systems
6. 6.6Nonhomogeneous systems
7. 6.7Consistency
8. 6.8No, one, or infinitely many solutions
9. 6.9Particular plus homogeneous solution
10. 6.10First encounter with null space and column space
11. 6.11Exercises

1. 7.1Gaussian elimination
2. 7.2Elementary row operations
3. 7.3Echelon form
4. 7.4Pivots and free variables
5. 7.5Rank as pivot count
6. 7.6Elementary matrices
7. 7.7Elimination as multiplication
8. 7.8LU factorization
9. 7.9Pivoting and PA = LU
10. 7.10Column-row factorization A = CR
11. 7.11Floating-point caution
12. 7.12Python Project: Gaussian elimination
13. 7.13Exercises

What remains when we remove arrows and coordinates?

1. 8.1Extracting the rules from earlier chapters
2. 8.2Definition of a real vector space
3. 8.3Polynomials as vectors
4. 8.4Functions as vectors
5. 8.5Matrices as vectors
6. 8.6Sequences as vectors
7. 8.7Fields, lightly
8. 8.8Subspaces
9. 8.9Sums of subspaces
10. 8.10Direct sums
11. 8.11Affine sets are not subspaces
12. 8.12Exercises

1. 9.1Linear maps revisited abstractly
2. 9.2Kernel as destroyed information
3. 9.3Image as reachable output
4. 9.4Kernel and injectivity
5. 9.5Image and surjectivity
6. 9.6Null space and column space of a matrix
7. 9.7Row space
8. 9.8Left null space
9. 9.9Matrix of a linear map, revisited
10. 9.10Exercises

1. 10.1Basis, formal version
2. 10.2Finite-dimensional spaces
3. 10.3Dimension
4. 10.4Extending independent lists
5. 10.5Reducing spanning lists
6. 10.6Rank as dimension of the image
7. 10.7Nullity
8. 10.8Rank–nullity theorem
9. 10.9Dimensions of matrix subspaces
10. 10.10Maximal linearly independent sets
11. 10.11Exercises

What changes when coordinates change, and what stays the same?

1. 11.1A vector is not its coordinate tuple
2. 11.2Coordinate maps
3. 11.3Same vector, different basis
4. 11.4Change-of-coordinate matrices
5. 11.5Matrix of a map under arbitrary bases
6. 11.6Similar matrices
7. 11.7Why a good basis matters
8. 11.8Python Lab: coordinates and similarity
9. 11.9Exercises

1. 12.1Oriented area
2. 12.2Oriented volume
3. 12.3Determinant as scaling factor
4. 12.4Determinant and invertibility
5. 12.5Multiplicativity
6. 12.6Row operations
7. 12.7Cofactor expansion
8. 12.8Volume applications
9. 12.9Cramer’s rule, briefly
10. 12.10Optional: alternating multilinear viewpoint
11. 12.11Python Lab: determinant algorithms
12. 12.12Exercises

1. 13.1Invariant directions
2. 13.2Eigenvectors and eigenvalues
3. 13.3Eigenspaces
4. 13.4Matrix formulation
5. 13.5Characteristic polynomial
6. 13.6Computing eigenvectors
7. 13.7Distinct eigenvalues imply independent eigenvectors
8. 13.8Rotation breaks the real story
9. 13.9Complex numbers enter naturally
10. 13.10Complex eigenvalues
11. 13.11Repeated transformations
12. 13.12Markov-chain preview
13. 13.13Python Lab: power iteration
14. 13.14Exercises

1. 14.1Invariant subspaces
2. 14.2Diagonal matrices
3. 14.3Diagonalization as basis selection
4. 14.4Conditions for diagonalizability
5. 14.5Algebraic and geometric multiplicity
6. 14.6Powers of matrices
7. 14.7Differential-equation application
8. 14.8When diagonalization fails
9. 14.9Jordan form preview
10. 14.10Exercises

1. 15.1Dot product geometrically
2. 15.2Length
3. 15.3Angle
4. 15.4Orthogonality
5. 15.5Abstract inner products
6. 15.6Norms
7. 15.7Cauchy–Schwarz
8. 15.8Orthogonal complements
9. 15.9Linear functionals and the shadow of duality
10. 15.10Complex inner products
11. 15.11Exercises

1. 16.1Why orthonormal coordinates are special
2. 16.2Orthonormal lists
3. 16.3Orthonormal bases
4. 16.4Coordinates become inner products
5. 16.5Gram–Schmidt
6. 16.6QR factorization
7. 16.7Orthogonal matrices
8. 16.8Unitary matrices
9. 16.9Numerical stability preview
10. 16.10Python Lab: Gram–Schmidt and QR
11. 16.11Exercises

1. 17.1Projection onto a line
2. 17.2Projection onto a subspace
3. 17.3Projection matrices
4. 17.4Orthogonal decomposition
5. 17.5When Ax = b has no solution
6. 17.6Least squares
7. 17.7Normal equations
8. 17.8Data fitting
9. 17.9Pseudoinverse
10. 17.10Numerical warning: do not worship normal equations
11. 17.11Python Project: fit noisy data
12. 17.12Exercises

1. 18.1Transpose revisited
2. 18.2Adjoint operators
3. 18.3Self-adjoint operators
4. 18.4Symmetric matrices
5. 18.5Normal operators
6. 18.6Positive operators
7. 18.7Positive definite matrices
8. 18.8Cholesky factorization
9. 18.9Energy and covariance interpretations
10. 18.10Exercises

1. 19.1Why symmetric matrices are special
2. 19.2Orthogonality of eigenspaces
3. 19.3Real spectral theorem
4. 19.4Orthogonal diagonalization
5. 19.5Complex spectral theorem
6. 19.6Normal operators
7. 19.7Positive operators through spectra
8. 19.8Geometry of eigenspaces
9. 19.9Exercises

1. 20.1From xᵀAx to geometry
2. 20.2Bilinear forms
3. 20.3Symmetric bilinear forms
4. 20.4Quadratic forms
5. 20.5Principal axes
6. 20.6Orthogonal reduction to standard form
7. 20.7Completing the square
8. 20.8Positive definite quadratic forms
9. 20.9Inertia
10. 20.10Optimization and Hessian preview
11. 20.11Python Lab: classify quadratic forms
12. 20.12Exercises

1. 21.1Every matrix has a geometric story
2. 21.2Unit sphere to ellipsoid
3. 21.3Singular values
4. 21.4Right singular vectors
5. 21.5Left singular vectors
6. 21.6Derivation from A*A
7. 21.7SVD of a linear map
8. 21.8SVD of a matrix
9. 21.9Rank and nullity through SVD
10. 21.10Pseudoinverse revisited
11. 21.11Best low-rank approximation
12. 21.12PCA
13. 21.13Image compression
14. 21.14Python Capstone: truncated SVD
15. 21.15Exercises

Where does the structure go when it leaves the classroom?

1. 22.1Differential equations
2. 22.2Markov chains
3. 22.3Perron–Frobenius intuition
4. 22.4Computer graphics
5. 22.5Image compression
6. 22.6PCA and covariance
7. 22.7Optimization
8. 22.8Linear programming and duality preview
9. 22.9Neural-network linear layers
10. 22.10Conditioning and numerical stability
11. 22.11Iterative methods preview
12. 22.12Capstone project gallery