What Is Linear Algebra?
Linear algebra is the branch of mathematics concerned with vectors, vector spaces, and linear transformations — functions that preserve addition and scalar multiplication. It is the mathematical backbone of an enormous range of applications: computer graphics, machine learning, signal processing, quantum mechanics, economics, and engineering all rely on linear algebra in fundamental ways.
Despite its broad applicability, linear algebra has a compact core. The central objects are matrices and vectors, and the central operation is solving systems of linear equations. Everything else — determinants, eigenvalues, decompositions — builds on this foundation.
The standard reference is Gilbert Strang's Introduction to Linear Algebra, which accompanies MIT's 18.06 course (freely available on MIT OpenCourseWare). David Lay's Linear Algebra and Its Applications is another widely used text with a more applied focus.
Scalars, Vectors, and Matrices
A scalar is a single real number: 3, −7, 1/2, √2. In linear algebra, scalars are used to scale vectors.
A vector in ℝⁿ is an ordered list of n real numbers, written as a column:
v = [3 ]
[-1]
[2 ]This is a vector in ℝ³. Vectors represent points in space, directions, or abstract quantities with n components. The zero vector 0 has all components equal to 0.
A matrix is a rectangular array of numbers arranged in rows and columns. An m×n matrix has m rows and n columns:
A = [ 1 2 3 ] (2×3 matrix: 2 rows, 3 columns)
[ 4 5 6 ]Matrices represent linear transformations, store data, encode systems of equations, and much more. The entry in row i and column j is written A[i][j] or aᵢⱼ.
Vector Operations
Addition: Add component-by-component. Vectors must have the same dimension.
[1] [4] [5] [2] + [5] = [7] [3] [6] [9]
Scalar multiplication: Multiply every component by the scalar.
3 · [1] [3]
[2] = [6]
[3] [9]Dot product: Sum of component-wise products. The result is a scalar.
[1, 2, 3] · [4, 5, 6] = 1·4 + 2·5 + 3·6 = 4 + 10 + 18 = 32
The dot product u · v = |u||v|cos(θ), where θ is the angle between u and v. When u · v = 0, the vectors are orthogonal (perpendicular).
Linear Systems of Equations
A linear system is a collection of linear equations in the same set of variables:
2x + y - z = 8 -3x - y + 2z = -11 -2x + y + 2z = -3
"Linear" means each variable appears only to the first power — no x², no xy, no sin(x). This restriction is what makes the subject tractable and the solutions well-structured.
Linear systems have exactly one of three outcomes: (1) exactly one solution, (2) infinitely many solutions, or (3) no solution. The system above has the unique solution x = 2, y = 3, z = −1.
The Matrix Equation Ax = b
Every linear system can be written as a matrix equation Ax = b, where A is the coefficient matrix, x is the column vector of unknowns, and b is the column vector of constants.
[ 2 1 -1 ] [x] [ 8 ] [-3 -1 2 ] [y] = [-11] [-2 1 2 ] [z] [-3 ]
This compact notation makes it clear that solving a linear system is equivalent to finding a vector x that maps to b under the linear transformation A.
Row Reduction and RREF
The standard method for solving linear systems is Gaussian elimination (or its extension, Gauss-Jordan elimination), which systematically simplifies the system using three elementary row operations:
- Swap two rows
- Multiply a row by a non-zero constant
- Add a multiple of one row to another
The goal is to reach Reduced Row Echelon Form (RREF), where the solution can be read off directly. The RREF of a matrix is unique — every matrix has exactly one RREF. Try the RREF calculator to see this process step by step.
Augmented matrix: RREF: [ 2 1 -1 | 8 ] → [ 1 0 0 | 2 ] [-3 -1 2 | -11] → [ 0 1 0 | 3 ] [-2 1 2 | -3] → [ 0 0 1 | -1 ]
Reading off: x = 2, y = 3, z = −1.
Rank, Null Space, and Linear Independence
The rank of a matrix A is the number of pivots in its RREF. It equals the number of linearly independent rows and the number of linearly independent columns — these are always equal (row rank = column rank). Rank measures "how much information" the matrix contains.
The null space (kernel) of A is the set of all vectors x such that Ax = 0. Its dimension is called the nullity of A. The rank-nullity theorem states: rank(A) + nullity(A) = n (number of columns). Free variables in RREF correspond to null space dimensions.
Vectors v₁, v₂, ..., vₖ are linearly independent if no one of them can be written as a linear combination of the others. Equivalently, the only solution to c₁v₁ + c₂v₂ + ... + cₖvₖ = 0 is c₁ = c₂ = ... = cₖ = 0. A set of vectors that is linearly dependent contains redundancy.
Span and Basis
The span of a set of vectors {v₁, ..., vₖ} is the set of all linear combinations: {c₁v₁ + ... + cₖvₖ : cᵢ ∈ ℝ}. Geometrically, the span is the subspace "reachable" from those vectors.
A basis of a vector space V is a set of vectors that (1) spans V and (2) is linearly independent. Every basis of V has the same number of elements — this number is the dimension of V.
The standard basis of ℝ³ is {e₁ = [1,0,0], e₂ = [0,1,0], e₃ = [0,0,1]}. Every vector in ℝ³ is a unique linear combination of these three vectors.
Geometric Meaning of Span and Linear Independence
These abstract definitions become concrete once you picture them geometrically.
In ℝ²: A single non-zero vector v spans a line through the origin — all scalar multiples cv. Two vectors u and v span ℝ² (the entire plane) if and only if they are not parallel (not scalar multiples of each other). If they are parallel, their span is still just a line — adding a parallel vector adds no new directions.
In ℝ³: A single non-zero vector spans a line. Two non-parallel vectors span a plane (a 2D subspace through the origin). Three vectors span all of ℝ³ if and only if they are not all in the same plane — i.e., no one of them is a linear combination of the other two. Three vectors that all lie in one plane are linearly dependent; their span is a plane, not all of ℝ³.
Linear independence, geometrically: Vectors v₁, ..., vₖ are linearly independent if no one of them lies in the span of the others. In ℝ², two vectors are independent if they point in genuinely different directions (not parallel). In ℝ³, three vectors are independent if none lies in the plane spanned by the other two.
How to test: Form a matrix with the vectors as columns and compute its RREF using the RREF calculator. If every column is a pivot column (rank = k), the vectors are linearly independent. If any column is a non-pivot column (rank < k), the corresponding vector can be written as a linear combination of the pivot-column vectors — the set is linearly dependent.
Finding a Basis Using RREF
Given a set of vectors (or a matrix), RREF identifies a basis for the column space (also called the range) directly.
Procedure: Form a matrix A whose columns are the given vectors. Compute RREF(A). The pivot columns of A — the original columns (before row reduction) corresponding to pivot positions in RREF — form a basis for the column space of A.
Example: Find a basis for the column space of A = [[1,2,3],[2,4,5],[3,6,8]].
RREF(A) = [ 1 2 0 ]
[ 0 0 1 ]
[ 0 0 0 ]Pivot columns are columns 1 and 3 (columns with leading 1s in RREF). So a basis for the column space of A consists of the original columns 1 and 3: [[1,2,3]] and [[3,5,8]]. Column 2 [2,4,6] is not in the basis because it is 2 times column 1 — a dependent vector. Rank = 2.
Key rule: Use the pivot columns of the original matrix A, not the columns of RREF(A). The RREF columns have the right positions, but the original columns are the actual basis vectors for the original column space.
Rank-Nullity Theorem: Worked Example
The rank-nullity theorem states: for any m×n matrix A, rank(A) + nullity(A) = n. Here n is the number of columns. This single equation connects the "output" dimension (rank) with the "lost" dimension (nullity).
Example: Let A be the 3×5 matrix:
A = [ 1 2 0 1 0 ]
[ 0 0 1 2 0 ]
[ 0 0 0 0 1 ]This matrix is already in RREF. Pivot positions are in columns 1, 3, and 5. Therefore:
- rank(A) = 3 (three pivot columns)
- nullity(A) = 5 − 3 = 2 (two non-pivot columns, 2 and 4)
- Free variables: x₂ and x₄ (non-pivot columns)
A basis for the null space (all solutions to Ax = 0): set x₂ = 1, x₄ = 0 → [−2, 1, 0, 0, 0]. Set x₂ = 0, x₄ = 1 → [−1, 0, −2, 1, 0]. The null space is a 2-dimensional subspace of ℝ⁵. rank + nullity = 3 + 2 = 5 = n ✓.
Matrix Operations
Addition and subtraction: Add/subtract entry-by-entry. Matrices must have the same dimensions.
Scalar multiplication: Multiply every entry by the scalar.
Matrix multiplication (AB): Entry C[i][j] = dot product of row i of A with column j of B. A must be m×n and B must be n×p; C is m×p. Not commutative in general. Try the matrix multiplication calculator.
Transpose (Aᵀ): Flip rows and columns. Entry Aᵀ[i][j] = A[j][i]. Try the transpose calculator.
Inverse (A⁻¹): Exists only for square non-singular matrices. A · A⁻¹ = I. Try the matrix inverse calculator.
Determinant (det A): A scalar encoding volume scaling and invertibility. det(A) ≠ 0 iff A is invertible. Try the determinant calculator.
Eigenvalues and Eigenvectors
An eigenvector of a square matrix A is a non-zero vector v such that Av = λv for some scalar λ. The scalar λ is the corresponding eigenvalue. Geometrically, eigenvectors are directions that A stretches or compresses without rotating — the transformation merely scales them by λ.
Finding eigenvalues requires solving the characteristic equation det(A − λI) = 0. For an n×n matrix, this gives a degree-n polynomial in λ, whose roots are the eigenvalues. Each eigenvalue λ has an associated eigenspace — the null space of (A − λI) — containing all corresponding eigenvectors.
Eigenvalues and eigenvectors are used in principal component analysis (PCA), differential equations, Google's PageRank algorithm, quantum mechanics, and stability analysis of dynamical systems.
Key Theorems to Know
- Rank-Nullity Theorem: rank(A) + nullity(A) = n (# columns of A)
- RREF Uniqueness: Every matrix has exactly one RREF
- Invertible Matrix Theorem: For an n×n matrix A, the following are equivalent: A is invertible, det(A) ≠ 0, rank(A) = n, Ax = 0 has only the trivial solution, and more
- Cauchy-Schwarz Inequality: |u · v| ≤ |u||v|
- Spectral Theorem: Every real symmetric matrix is diagonalizable with real eigenvalues and orthogonal eigenvectors
Next Steps
After mastering these basics, the natural progression in a first course includes: solving systems with RREF, matrix transformations in ℝ², eigenvalue problems, and orthogonality (dot products, projections, Gram-Schmidt). A second course typically covers inner product spaces, SVD, Jordan normal form, and abstract vector spaces.
Use the calculators on this site to check your work as you practice. The RREF calculator is particularly useful for verifying row reduction, and the determinant calculator confirms eigenvalue computations.