Linear combination

Linear combinations are a concept central to linear algebra and related fields of mathematics. Most of this article deals with linear combinations in the context of a vector space over a field, with some generalisations given at the end of the article.

Table of contents 1 Definition 2 Examples and counterexamples 2.1 1: Examples related to analytic geometry 2.2 2: Examples related to functional analysis 2.3 3: Examples related to algebraic geometry 3 The linear span 4 Other related concepts 5 Generalisations

Definition

Suppose that K is a field and V is a vector space over K. As usual, we call elements of V vectors and call elements of K scalars. If v₁,...,v_n are vectors and a₁,...,a_n are scalars, then the linear combination of those vectors with those scalars as coefficients is

a₁v₁ + a₂v₂ + a₃v₃ + ... + a_nv_n.

In a given situation, K and V may be specified explicitly, or they may be obvious from context. In that case, we often speak of a linear combination of the vectors v₁,...,v_n, with the coefficients unspecified (except that they must belong to K). Or, if S is a subset of V, we may speak of a linear combination of vectors in S, where both the coefficients and the vectors are unspecified, except that the vectors must belong to the set S (and the coefficients must belong to K). Finally, we may speak simply of a linear combination, where nothing is specified (except that the vectors must belong to V and the coefficients must belong to K).

Note that you can only take a linear combination of finitely many vectors (except as described in Generalisations below); that is, the number n may be finite. However, the set S that the vectors are taken from (if one is mentioned) can still be infinite; each individual linear combination will only involve finitely many vectors. Also, there's no reason that the finite number n can't be zero; in that case, we declare by convention that the result of the linear combination is the zero vector in V.

Examples and counterexamples

1: Examples related to analytic geometry

Let the field K be the set R of real numbers, and let the vector space V be the Euclidean space R³. Consider the vectors e₁ := (1,0,0), e₂ := (0,1,0) and e₃ = (0,0,1). Then any vector in R³ is a linear combination of e₁, e₂ and e₃.

To see that this is so, take an arbitrary vector (a₁,a₂,a₃) in R³, and write:

(a₁,a₂,a₃) = (a₁,0,0) + (0,a₂,0) + (0,0,a₃) = a₁(1,0,0) + a₂(0,1,0) + a₃(0,0,1) = a₁e₁ + a₂e₂ + a₃e₃.

2: Examples related to functional analysis

Let K be the set C of all complex numbers, and let V be the set C_C(R) of all continuous functions from the real line R to the complex plane C. Consider the vectors (functions) f and g defined by f(t) := e^it and g(t) := e^−it. (Here, e is the base of the natural logarithm, about 2.71828..., and i is the imaginary unit, a square root of −1.) Some linear combinations of f and g are:

• cos t = (1/2)e^it + (1/2)e^−it;
• 2 sin t = (−i)e^it + (i)e^−it.

On the other hand, the constant function 3 = (3)(e^it)(e^−it) is not a linear combination of f and g, because you're not allowed to multiply vectors with vectors when forming linear combinations.

3: Examples related to algebraic geometry

Let K be any field (R, C, or whatever you like best), and let V be the set P of all polynomials with coefficients taken from the field K. Consider the vectors (polynomials) p₁ := 1, p₂ := x + 1, and p₃ := x² + x + 1.

Is the polynomial x² − 1 a linear combination of p₁, p₂, and p₃? To find out, consider an arbitrary linear combination of these vectors and try to see when it equals the desired vector x² − 1. Picking arbitrary coefficients a₁, a₂, and a₃, we want

a₁(1) + a₂(x + 1) + a₃(x² + x + 1) = x² − 1.

Multiplying the polynomials out, this means

(a₁) + (a₂x + a₂) + (a₃x² + a₃x + a₃) = x² − 1,

and collecting like powers of x, we get

a₃x² + (a₂ + a₃)x + (a₁ + a₂ + a₃) = 1x² + 0x + (−1).

Two polynomials are equal if and only if their corresponding coefficients are equal, so we can conclude

a₃ = 1,    a₂ + a₃ = 0,    a₁ + a₂ + a₃ = −1.

This system of linear equations can easily be solved. First, the first equation simply says that a₃ is 1. Knowing that, we can solve the second equation for a₂, which comes out to −1. Finally, the last equation tells us that a₁ is also −1. Therefore, the only possibile way to get a linear combination is with these coefficients. Indeed,

x² − 1 = −1 − (x + 1) + (x² + x + 1) = −p₁ − p₂ + p₃,

so x² − 1 is a linear combination of p₁, p₂, and p₃.

On the other hand, what about the polynomial x³ − 1? If we try to make this vector a linear combination of p₁, p₂, and p₃, then following the same process as before, we'll get the equation

0x³ + a₃x² + (a₂ + a₃)x + (a₁ + a₂ + a₃) = 1x³ + 0x² + 0x + (−1).

However, when we set corresponding coefficients equal in this case, the equation for x³ is

0 = 1,

which is always false. Therefore, there is no way for this to work, and x³ − 1 is not a linear combination of p₁, p₂, and p₃.

The linear span

Take an arbitrary field K, an arbitrary vector space V, and let v₁,...,v_n be vectors (in V). It's interesting to consider the set of all linear combinations of these vectors. This set is called the linear span (or just span) of the vectors v₁,...,v_n. In symbols,

Sp(v₁,...,v_n) := {a₁v₁ + ... + a_nv_n : a₁,...,a_n ∈ K}.

(This expression is an example of set builder notation.)

Theorem 1: Sp(v₁,...,v_n) is a subspace of V. Furthermore, this span is the smallest subspace of V that the vectors v₁,...,v_n all belong to.

This fact (which is proved later in this section) is one reason why the span is important.

Now let S be a subset of the vector space V. The linear span of S consists of all linear combinations of elements of S. In symbols,

Sp(S) = {a₁v₁ + ... + a_kv_k : k ∈ N, a₁,...,a_k ∈ K, v₁,...,v_k ∈ S},

where N is the set of natural numbers (including zero). Notice that this time the number of vectors involved in the linear combination can vary, from zero on up, but it must still be finite each time.

Theorem 2: Sp(S) is also a subspace of V. Furthermore, this span is the smallest subspace of V that is a superset of S.

The rest of this section is a proof of Theorem 1. Theorem 2 is very similar, but it's a bit messier to write down, since the vectors involved in any given linear combination can vary.

Proof of Theorem 1:

Property 1:
The most general possible two elements of the span are x := a₁v₁ + ... + a_nv_n and y := b₁v₁ + ... + b_nv_n. We have to show that x + y is also a linear combination. By using associativity and commutativity of addition and the distributive law, we can write

x + y = (a₁ + b₁)v₁ + ... + (a_n + b_n)v_n,

and since a_i + b_i is a scalar for every i, we see that x + y is indeed a linear combination of the given vectors.

Property 2:
Let c be a scalar and again take x := a₁v₁ + ... + a_nv_n. We have to show that cx is also a linear combination. Now,

cx = (ca₁)v₁ + ... + (ca_n)v_n,

and since ca_i is a scalar for every i, we are done.

Property 3:
The zero element 0_V of V is a linear combination because we can write

0_V = 0_Kv₁ + 0_Kv₂ + ... + 0_Kv_n.

(Here, 0_K is the zero element of the field K.) This equation is true because in every vector space we have 0_Kv = 0_V.

Minimality:
Suppose W is another subspace of V which contains the vectors v₁,...,v_n. Then W is closed under scalar multiplication and addition of vectors, so we can prove by mathematical induction that a₁v₁ + ... + a_nv_n is an element of W for any scalars a₁,...,a_n. Thus, sp(v₁,...,v_n), the set of all such linear combinations, is a subset of W.

Other related concepts

Sometimes, some single vector can be written in two different ways as a linear combination of v₁,...,v_n. If that is possible, then v₁,...,v_n are called linearly dependent; otherwise, they are linearly independent. Similarly, we can speak of linear dependence or independence of an arbitrary set S of vectors.

If S is linearly independent and the span of S equals V, then S is a basis for V.

We can think of linear combinations as the most general sort of operation on a vector space. The basic operations of addition and scalar multiplication, together with the existence of an additive identity and additive inverses, can't be combined in any more complicated way than the generic linear combination. Ultimately, this fact lies at the heart of the usefulness of linear combinations in the study of vector spaces.

Generalisations

If V is a topological vector space, then there may be a way to make sense of certain infinite linear combination, using the topology of V. For example, we might be able to speak of a₁v₁ + a₂v₂ + a₃v₃ + ..., going on forever. Such infinite linear combinations don't always make sense; we call them convergent when they do. Allowing more linear combinations in this case can also lead to a different concept of span, linear independence, and basis. The articles on the various flavours of topological vector spaces go into more detail about these.

If K is a commutative ring instead of a field, then everything that has been said above about linear combinations generalises to this case without change. The only difference is that we call spaces like V modules instead of vector spaces. If K is a noncommutative ring, then the concept still generalises, with one caveat: Since modules over noncommutative rings come in left and right versions, our linear combinations may also come in either of these versions, whatever is appropriate for the given module. This is simply a matter of doing scalar multiplication on the correct side.

A more complicated twist comes when V is a bimodule[?] over two rings, K_L and K_R. In that case, the most general linear combination looks like

a₁v₁b₁ + ... + a_nv_nb_n,

where a₁,...,a_n belong to K_L, b₁,...,b_n belong to K_R, and v₁,...,v_n belong to V.