ELEMENTARY LINEAR ALGEBRA AND VECTOR CALCULUS MATHEMATICS FOR UNDERGRADUATES Antony L. Foster Department of Mathematics (NAC 6/273) The City College of New York New York, New York 10031. Chapter 1. LINEAR EQUATIONS AND MATRICES 1.1. SYSTEMS OF LINEAR EQUATIONS One of the most frequently recurring practicalproblems in almost all fields of study—such as mathematics, physics, biology,chemistry,economics,allphasesofengineering,operationsresearch,thesocialsciences,andsoforth—isthat of solving a system of linear equations. The equation b = a x + a x + + a x , (1) 1 1 2 2 n n ··· which expresses b in terms of the variables x , x , ..., x and the constants a , a , ..., a , is called a linear 1 2 n 1 2 n equation. In many applications we are given b and must find numbers x , x , ..., x satisfying (1). 1 2 n A solution to a linear equation (1) is a sequence of n numbers s , s , ..., s such that (1) is satisfied when 1 2 n x = s , x = s , ..., x = s are substituted in (1). Thus x = 2, x = 3, and x = 4 is a solution to the 1 1 2 2 n n 1 2 3 − linear equation 6x 3x + 4x = 13, 1 2 3 − − because 6(2) 3(3) + 4( 4) = 13. − − − More generally, a system of m linear equations in n unknowns, or a linear system, is a set of m linear equations each in n unknowns called x , x , ..., x . 1 2 n A linear system can be conveniently written as a x + a x + + a x = b 11 1 12 2 1n n 1 ··· a x + a x + + a x = b 21 1 22 2 2n n 2 . . ··· . . (2) . . . . . . . . a x + a x + + a x = b . m1 1 m2 2 mn n m ··· Thus the ith equation in the system is a x + a x + + a x = b . i1 1 i2 2 in n i ··· In (2) the a are known constants. Given the values b , b , ..., b , we want to find values of x , x , ..., x that ij 1 2 n 1 2 n will satisfy each equation in (2). A solution to a linear system (2) is a sequence of n numbers s , s , ..., s , which have the property that each 1 2 n equation in (2) is satisfied when x = s , x = s , ..., x = s are substituted. 1 1 2 2 n n If the linear system (2) has no solution, it is said to be inconsistent; if it has a solution, it is called consistent. If b = b = = b = 0, then (2) is called a homogeneous system. The solution x = x = = x = 0 1 2 n 1 2 n ··· ··· to a homogeneous system is called the trivial solution. A solution to a homogeneous system in which not all of x , x , ..., x are zero is called a nontrivial solution. 1 2 n Consider another system of r linear equations in n unknowns: c x + c x + + c x = d 11 1 12 2 1n n 1 ··· c x + c x + + c x = d 21 1 22 2 2n n 2 . . ··· . . (3) . . . . . . . . c x + c x + + c x = d . r1 1 r2 2 rn n r ··· We say that (2) and (3) are equivalent if they both have exactly the same solutions. Example 1.1.1. The linear system x 3x = 7 1 − 2 − (4) 2x + x = 7 1 2 has only the solution x = 2 and x = 3. The linear system 1 2 8x 3x = 7 1 2 − 3x 2x = 0 (5) 1 2 − 10x 2x = 14 1 2 − also has only the solution x = 2 and x = 3. Thus (4) and (5) are equivalent. 1 2 To find solutions to a linear system we shall use a technique called the method of elimination; that is, we eliminate some variables by adding a multiple of one equation to another equation. Elimination merely amounts to the development of a new linear system which is equivalent to the original system but is much simpler to solve. Readers have probably confined their earlier work in this area to linear systems in which m = n, that is, linear systemshavingasmanyequationsasunknowns. Inthiscourseweshallbroadenouroutlookbydealingwithsystems in which we have m = n, m < n, and m > n. Indeed, there are numerous applications in which m = n. 6 Example 1.1.2. Consider the linear system x 3x = 3 1 − 2 − (6) 2x + x = 8. 1 2 To eliminate x , we subtract twice the first equation from the second, obtaining 1 7x = 14, 2 and equation having no x term. Thus we have eliminated the unknown x . Then solving for x , we have 1 1 2 x = 2, 2 and substituting into the first equation of (6), we obtain x = 3. 1 Then x = 3, x = 2 is the only solution to the given linear system. 1 2 Example 1.1.3. Consider the linear system x 3x = 7 1 − 2 − (7) 2x + x = 7. 1 2 Again, we decide to eliminate x . We subtract twice the first equation from the second one, obtaining 1 0 = 21, which makes no sense. This means that (7) has no solution; it is inconsistent. We could have come to the same conclusion from observing that in (7) the left side of the second equation is twice the left side of the first equation, but the right side of the second equation is not twice the right side of the first equation. Example 1.1.4. Consider the linear system x + 2x + 3x = 6 1 2 3 2x 3x + 2x = 14 (8) 1 2 3 − 3x + x x = 2. 1 2 3 − − To eliminate x , we subtract twice the first equation from the second and three times the first equation from the 1 third, obtaining 7x 4x = 2 − 2 − 3 (9) 5x 10x = 20. 2 3 − − − This is a system of two equations in the unknowns x and x . We divide the secondequation of (9) by -5, obtaining 2 3 7x 4x = 2 2 3 − − x + 2x = 4, 2 3 which we write, by interchanging equations as x + 2x = 4 2 3 (10) 7x 4x = 2. 2 3 − − We now eliminate x in (10) by adding 7 times the first equation to the second one, to obtain 2 10x = 30, 3 or x = 3. (11) 3 Substituting this value of x into the first equation of (10), we find that x = 2. Substituting these values of x 3 2 2 − and x into the first equation of (8), we find that x = 1. We might observe further that our elimination procedure 3 1 has actually produced the linear system x + 2x + 3x = 6 1 2 3 x + 2x = 4 (12) 2 3 x = 3. 3 obtainedbyusingthefirstequationsof(8)and(10)aswellas(11). Theimportanceoftheprocedureisthatalthough the linear systems (8) and (12) are equivalent, (12) has the advantage that it is easier to solve. Example 1.1.5. Consider the linear system x + 2x 3x = 4 1 2 3 − − (13) 2x + x 3x = 4. 1 2 3 − Eliminating x , we subtract twice the first equation from the second equation, to obtain 1 3x + 3x = 12. (14) 2 3 − We must now solve (14). A solution is x = x 4, 2 3 − where x can be any real number. Then from the first equation of (13), 3 x = 4 2x + 3x 1 2 3 − − = 4 2(x 4) + 3x 3 3 − − − = x + 4. 3 Thus a solution to the linear system (13) is x = x + 4 1 3 x = x 4 2 3 − x = any real number. 3 This means that the linear system (13) has infinitely many solutions. Every time we assign a value to x we obtain 3 another solution to (13). Thus, if x = 1, then 3 x = 5, x = 3, and x = 1 1 2 3 − is a solution, while if x = 2, then 3 − x = 2, x = 6, and x = 2 1 2 3 − − is another solution. These examples suggest that linear system may have a unique solution, no solution, or infinitely many solutions. Consider next a linear system of two equations in the unknowns x and x . 1 2 a x + a x = c 1 1 2 2 1 (15) b x + b x = c 1 1 2 2 2 Thegraphofeachoftheseequationsisastraightline,whichwedenotebyl andl respectively. Ifx = s , x = s 1 2 1 1 2 2 is a solution to the linear system (15), then the point (s , s ) lies on both lines l and l . Conversely, if the point 1 2 1 2 (s , s ) lies on both lines l and l , then x = s , x = s is a solution to the linear system (15). Thus we are led 1 2 1 2 1 1 2 2 geometrically to the same three possibilities mentioned above in (Figure 1.1). xxx222 x2 x2 l 2 l 2 x x x 1 1 1 l 1 l 2 l 1 l 1 (a) A unique solution. (b) No solution. (c) Infinitely many solutions. If we examine the method of elimination more closely, we find that it involves three manipulations that can be performed on a linear system to convert it into an equivalent system. These manipulations are as follows: 1. We may interchange the ith and jth equations of the system. 2. We may multiply an equation in the system by a nonzero constant. 3. We may replace the ith equation by c times the jth equation plus the ith equation, i = j. 6 That is, replace a x + a x + + a x = b i1 1 i2 2 in n i ··· by (a + ca )x + (a + ca )x + + (a + ca )x = b + cb . i1 j1 2 i2 j2 2 in jn n i j ··· It is not difficult to prove that performing these manipulations on a linear system leads to an equivalent system. Example 1.1.6. Suppose that the ith equation of a linear system such as (2) is multiplied by the nonzero constant c, obtaining the linear system a x + a x + + a x = b 11 1 12 2 1n n 1 ··· a x + a x + + a x = b 21 1 22 2 2n n 2 . . ··· . . . . . . . . . . (16) ca x + ca x + + ca x = cb i1 1 i2 2 in n i . . ··· . . . . . . . . . . a x + a x + + a x = b . m1 1 m2 2 mn n m ··· If x = s , x = s , x = s is a solution to (2), then it is a solution to all the equations in (16) except 1 1 2 2 n n ··· possibly for the ith equation. For the ith equation we have c(a s + a s + + a s ) = cb i1 1 i2 2 in n i ··· or ca s + ca s + + ca s = cb . i1 1 i2 2 in n i ··· Thus the ith equation of (16) is also satisfied. Hence every solution to (2) is also a solution to (16). Conversely, every solution to (16) also satisfies (2). Hence (2) and (16) are equivalent systems. In the next Section we develop methods that will enable us to further discuss and solve linear systems of equations. SUGGESTED EXERCISES FOR SECTION 1 In Exercises 1 through 14, solve the given linear system by the method of elimination. (I suggest that you hold off on these exercises until later when we discuss the method of Gaussian Elimination.) x + 2x = 8 Exercise 1.1.1. 1 2 . 3x 4x = 4 1 2 − 2x 3x + 4x = 12 1 2 3 − − Exercise 1.1.2. x 2x + x = 5 . 1 2 3 − − 3x + x + 2x = 1 1 2 3 3x + 2x + x = 2 1 2 3 Exercise 1.1.3. 4x + 2x + 2x = 8. 1 2 3 x x + x = 4 1 2 3 − x + x = 5 Exercise 1.1.4. 1 2 3x + 3x = 10. 1 2 2x + 4x + 6x = 12 1 2 3 − Exercise 1.1.5. 2x 3x 4x = 15 1 2 3 − − 3x + 4x + 5x = 8. 1 2 3 − x + x 2x = 5 Exercise 1.1.6. 1 2 − 3 2x + 3x + 4x = 2. 1 2 3 x + 4x + 6x = 12 Exercise 1.1.7. 1 2 3 3x + 8x 2x = 4. 1 2 3 − 3x + 4x x = 8 Exercise 1.1.8. 1 2 − 3 6x + 8x 2x = 3. 1 2 3 − x + x + 3x = 12 Exercise 1.1.9. 1 2 3 2x + 2x + 6x = 6. 1 2 3 x + x = 1 1 2 Exercise 1.1.10. 2x x = 5 1 2 − 3x + 4x = 2. 1 2 2x + 3x = 13 1 2 Exercise 1.1.11. x 2x = 3 1 2 − 5x + 2x = 27. 1 2 x 5x = 6 1 2 − Exercise 1.1.12. 3x + 2x = 1. 1 2 5x + 2x = 1 1 2 x + 3x = 4 1 2 − Exercise 1.1.13. 2x + 5x = 8 1 2 − x + 3x = 5. 1 2 − 2x + 3x x = 6 1 2 3 − Exercise 1.1.14. 2x x + 2x = 8 1 2 3 − − 3x x + x = 7. 1 2 3 − − Exercise 1.1.15. Show that the linear system obtained by interchanging two equations in (2) is equivalent to (2). Exercise 1.1.16. Show that the linear system obtained by adding a multiple of an equation in (2) to another equation is equivalent to (2). 1.2. MATRICES; MATRIX OPERATIONS Before continuing the study of solving linear systems, we now introduce the notion of a matrix, which will greatly simplify our notational problems, and develop tools to solve many important applied problems. If we examine the method of elimination described in Section 1.1, we make the following observation: Only the numbers in front of the unknowns x , x , ..., x are being changed as we perform the steps in the method of 1 2 n elimination. Thus we might think of looking for a way of writing a linear system without having to carry along the unknowns. Matrices enables us to do this—that is, to write linear systems in a compact form that makes it easier to automate the elimination method on an electronic computer in order to obtain a fast and efficient procedure for finding solutions. Their use is not, however, merely that of a convenient notation. We now develop operations on matrices and will work with matrices according to the rules they obey; this will enable us to solve systems of linear equations and to do other computational problems in a fast and efficient manner. Of course, as any good definition should do, the notion of a matrix provides not only a new way of looking at old problems but also gives rise to a great many new questions, some of which we study in these notes. WHAT ARE MATRICES? Definition 1.2.1. A matrix (plural matrices) is a rectangular array of objects (usually numbers) denoted by a a a a 11 12 1j 1n ··· ··· a a a a 21 22 2j 2n  ··· ···  . . . . . . . . . . . . A =  ··· .  a a a a   i1 i2 ··· ij ··· in   .. .. .. ..   . . . .   ···  a a a a   m1 m2 ··· mj ··· mn   Unless stated otherwise, we assume that all our matrices are composed entirely of real numbers. The ith row of A is (a a a ) (1 6 i 6 m) i1 i2 in ··· while the jth column of A is a 1j a 2j  .  (1 6 j 6 n) . . a   mj   If a matrix A has m rows and n columns, we saythat A is an m by n (m n) matrix. If m = n, we say that A is × a square matrix of order n and that the elements a , a , ..., a are on the main diagonal of A. We refer 11 22 nn to a as the (i, j) entry (entry in ith row and jth column) or (i, j)th element and we often write ij A = [a ] or A = [a ] ij ij m n × We shall also write A to indicate that A has m rows and n columns. If A is n n, we merely write A . m n n × × Example 1.2.1. The following are matrices: 1 2 3 A = 4 5 6 , B = (1 2 9)   − 7 8 9   2 1 0 3 C = −3 , and D = 1 2 . (cid:18)− − (cid:19) 4     In matrix A, the entry a = 8; in matrix C, the entry c = 4. Here A is a 3 3 square matrix, B is a 1 3, 32 41 × × matrix C is a 4 1 matrix, and matrix D is a 2 2 square matrix. In A, the entries a = 1, a = 5 and a = 9 11 22 33 × × are on the main diagonal. Whenever a new object is introduced in mathematics, one must determine when two such objects are equal. For example,in the setofallrationalnumbers, the numbers 2 and 4 arecalledequal, althoughthey arenotrepresented 3 6 in the same manner. What we have in mind is the definition that a/b equals c/d when ad = bc. Accordingly, we now have the following definition. EQUALITY OF MATRICES Definition 1.2.2. Two m n matrices A = [a ] and B = [b ] are equal if they agree entry by entry, that is, if ij ij × a = b for i = 1,2,...,m and j = 1,2,...,n. ij ij Example 1.2.2. The matrices 1 2 1 1 2 w − 2 3 4 and B = 2 x 4  −    0 4 5 y 4 z − −     are equal if and only if w = 1, x = 3, y = 0, and z = 5. − − We next define a number of operations that will produce new matrices out of given matrices; this will enable us to compute with the matrices and not deal with the equations from which they arise. These operations are also useful in the applications of matrices. ADDITION OF MATRICES Definition 1.2.3. If A = [a ] and B = [b ] are both m n matrices, then their matrix sum A + B is an ij ij × m n matrix C = [c ] defined by c = a + b , i = 1,2,..., m; j = 1,2,..., n. Thus, to obtain the sum of A ij ij ij ij × and B, we merely add corresponding (i, j) entries. Example 1.2.3. Let 1 2 3 0 2 1 A = − and B = . 2 1 4 1 3 4 (cid:18) − (cid:19) (cid:18) − (cid:19) Then 1 2 3 0 2 1 1 + 0 2 + 2 3 + 1 1 0 4 A + B = − + = − = . 2 1 4 1 3 4 2 + 1 1 + 3 4 4 3 2 0 (cid:18) − (cid:19) (cid:18) − (cid:19) (cid:18) − − (cid:19) (cid:18) (cid:19) It should be noted that the sum of the matrices A and B is defined only when A and B have the same number of rows and the same number of columns, that is, only when A and B are of the same size. We now establish the conventionthat when A + B is formed, both A and B are of the same size. The basic properties of matrix addition are considered in the following section and are similar to those satisfied by the real numbers.