Vector Spaces and Subspaces

Vector spaces and subspaces are central concepts in linear algebra, forming the foundational structure within which vectors operate. Understanding these concepts is crucial for grasping more advanced topics like linear transformations, eigenvalues, and eigenvectors. This article explores what vector spaces and subspaces are, their key properties, and their significance in the broader context of linear algebra.

---

1. What is a Vector Space?

A vector space (also called a linear space) is a collection of vectors that can be added together and multiplied by scalars, satisfying certain axioms (rules). Vector spaces provide the framework for analyzing and solving linear equations and transformations.

1.1 Definition of a Vector Space

A vector space $V$ over a field $F$ (such as the real numbers $\mathbb{R}$ or complex numbers $\mathbb{C}$) is a set equipped with two operations:

1. Vector addition: An operation $+$ that takes any two vectors $\mathbf{u}, \mathbf{v} \in V$ and produces another vector $\mathbf{u} + \mathbf{v} \in V$.
2. Scalar multiplication: An operation $\cdot$ that takes a scalar $c \in F$ and a vector $\mathbf{v} \in V$ and produces another vector $c \mathbf{v} \in V$.

1.2 Axioms of a Vector Space

To qualify as a vector space, the set $V$ and the operations must satisfy the following axioms:

1. Associativity of addition: $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$
2. Commutativity of addition: $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$
3. Identity element of addition: There exists an element $\mathbf{0} \in V$ such that $\mathbf{v} + \mathbf{0} = \mathbf{v}$ for all $\mathbf{v} \in V$.
4. Inverse elements of addition: For each $\mathbf{v} \in V$, there exists an element $-\mathbf{v} \in V$ such that $\mathbf{v} + (-\mathbf{v}) = \mathbf{0}$.
5. Compatibility of scalar multiplication with field multiplication: $a(b \mathbf{v}) = (ab) \mathbf{v}$ for all $a, b \in F$ and $\mathbf{v} \in V$.
6. Identity element of scalar multiplication: $1 \mathbf{v} = \mathbf{v}$ for all $\mathbf{v} \in V$, where $1$ is the multiplicative identity in $F$.
7. Distributivity of scalar multiplication with respect to vector addition: $a(\mathbf{u} + \mathbf{v}) = a\mathbf{u} + a\mathbf{v}$ for all $a \in F$ and $\mathbf{u}, \mathbf{v} \in V$.
8. Distributivity of scalar multiplication with respect to field addition: $(a + b) \mathbf{v} = a\mathbf{v} + b\mathbf{v}$ for all $a, b \in F$ and $\mathbf{v} \in V$.

1.3 Examples of Vector Spaces

1. Euclidean Space: $\mathbb{R}^n$ is a vector space where each element (vector) is an $n$-tuple of real numbers.
2. Polynomial Space: The set of all polynomials of degree $n$ or less is a vector space over $\mathbb{R}$.
3. Function Space: The set of all continuous functions from $\mathbb{R}$ to $\mathbb{R}$ is a vector space.

---

2. What is a Subspace?

A subspace is a subset of a vector space that is itself a vector space under the same operations of addition and scalar multiplication.

2.1 Definition of a Subspace

A subset $W \subseteq V$ is a subspace of the vector space $V$ if:

1. The zero vector: $\mathbf{0} \in W$.
2. Closed under addition: If $\mathbf{u}, \mathbf{v} \in W$, then $\mathbf{u} + \mathbf{v} \in W$.
3. Closed under scalar multiplication: If $\mathbf{v} \in W$ and $c \in F$, then $c \mathbf{v} \in W$.

2.2 Examples of Subspaces

1. The set of all vectors on a line through the origin in $\mathbb{R}^2$: This is a subspace of $\mathbb{R}^2$.
2. The set of all polynomials of degree at most $n-1$: For $n\geq1$, this is a subspace of the vector space of all polynomials of degree at most $n$; writing “at most” also includes the zero polynomial without assigning it a conventional degree.

2.3 Properties of Subspaces

1. Intersection: The intersection of two subspaces is also a subspace.
2. Sum of Subspaces: The sum of two subspaces is the set of all possible sums of vectors from each subspace, and it is also a subspace.

---

3. Linear Independence

Vectors $\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k$ in a vector space $V$ are said to be linearly independent if the only solution to the equation:

$$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \dots + c_k \mathbf{v}_k = \mathbf{0}$$

is $c_1 = c_2 = \dots = c_k = 0$. If there exists a non-trivial solution (i.e., not all $c_i$ are zero), the vectors are linearly dependent.

3.1 Importance of Linear Independence

Linear independence is crucial for determining the dimension of a vector space, understanding the structure of vector spaces, and for applications such as solving systems of linear equations.

---

4. Basis and Dimension

A basis of a vector space $V$ is a set of linearly independent vectors that span the entire space. The dimension of $V$ is the number of vectors in any basis for $V$.

4.1 Basis of a Vector Space

A set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_n\}$ is a basis for a vector space $V$ if:

1. The vectors are linearly independent.
2. The vectors span $V$, meaning any vector in $V$ can be written as a linear combination of these basis vectors.

4.2 Dimension of a Vector Space

The dimension of a vector space $V$, denoted $\text{dim}(V)$, is the number of vectors in a basis of $V$. For example:

• The dimension of $\mathbb{R}^n$ is $n$.
• The dimension of the space of polynomials of degree at most $n$ is $n+1$.

---

5. Applications of Vector Spaces and Subspaces

5.1 Solution Spaces of Linear Systems

The solution set of a homogeneous system of linear equations forms a subspace of $\mathbb{R}^n$. Understanding the subspaces associated with a matrix helps in understanding the structure of the solutions.

5.2 Principal Component Analysis (PCA)

In PCA for centered real-valued data, principal directions can be chosen as orthonormal eigenvectors of the sample covariance matrix. A complete set forms a basis for the ambient feature space; retaining only the leading directions instead gives a basis for a lower-dimensional principal subspace.

5.3 Computer Graphics

In computer graphics, linear transformations such as rotations and scalings are represented by matrices acting on vector spaces. Translations are affine transformations and can be included in matrix calculations by using homogeneous coordinates. The subspaces associated with these transformations help describe how objects are manipulated in space.

---

Conclusion

Vector spaces and subspaces are foundational concepts in linear algebra, providing the structure within which vectors exist and operate. Understanding these concepts is essential for deeper explorations into linear transformations, eigenvalues, and eigenvectors, as well as for practical applications in data science, engineering, and computer graphics.