Systems of Linear Equations and Matrices

1
Linear Algebra

• Lecturers
• Heru Suhartanto, PhD, heru_at_cs.ui.ac.id
• Yova Ruldeviyani, MKom, yovayg_at_gmail.com
• Schedule (at C3111)
• Tuesday, 1.00 2.40 PM
• Thursday, 1.00 1.50 PM
• Marking scheme
• 2 exams (mid 15, final 20) reqs attendance
  75
• 4 quizes, 40
• 5 assignments, 25
• Reference
• Horward Anton, Elementary Linear Algebra, 8-th
  Ed, John Wiley Sons, Inc, 2000
• More information (will be updated soon) at
• http//telaga.cs.ui.ac.id/WebKuliah/LinearAlgebra0
  5/

2
Systems of Linear Equation and Matrices

• CHAPTER 1
• FASILKOM UI 05

3
Introduction Matrices

• Information in science and mathematics is often
  organized into rows and columns to form
  rectangular arrays.
• Tables of numerical data that arise from physical
  observations
• Example (to solve linear equations)
• Solution is obtained by performing appropriate
  operations on this matrix

4

• 1.1 Introduction to
• Systems of Linear Equations

5
Linear Equations

• In x y variables (straight line in the xy-plane)
• where a1, a2, b are real constants,
• In n variables
• where a1, , an b are real constants
• x1, , xn unknowns.
• Example 1 Linear Equations
• The equations are linear (does not involve any
  products or roots of variables).

6
Linear Equations

• The equations are not linear.
• A solution of
  is a sequence of n numbers s1, s2, ..., sn ?
  they satisfy the equation when x1s1, x2s2, ...,
  xnsn (solution set).
• Example 2 Finding a Solution Set
• 1 equation and 2 unknown, set one var as the
  parameter (assign any value)
• or
• 1 equation and 3 unknown, set 2 vars as parameter

7
Linear Systems / System of Linear Equations

• Is A finite set of linear equations in the vars
  x1, ..., xn
• s1, ..., sn is called a solution if x1s1, ...,
  xnsn is a solution of every equation in the
  system.
• Ex.
• x11, x22, x3-1 the solution
• x11, x28, x31 is not, satisfy only the first
  eq.
• System that has no solution inconsistent
• System that has at least one solution consistent
• Consider

8
Linear Systems

• (x,y) lies on a line if and only if the numbers x
  and y satisfy the equation of the line. Solution
  points of intersection l1 l2
• l1 and l2 may be parallel
• no intersection, no solution
• l1 and l2 may intersect
• at only one point one solution
• l1 and l2 may coincide
• infinite many points of intersection,
• infinitely many solutions

9
Linear Systems

• In general Every system of linear equations has
  either no solutions, exactly one solution, or
  infinitely many solutions.
• An arbitrary system of m linear equations in n
  unknowns
• a11x1 a12x2 ... a1nxn b1
• a21x1 a22x2 ... a2nxn b2
• am1x1 am2x2 ... amnxn bm
• x1, ..., xn unknowns, as and bs are constants
• aij, i indicates the equation in which the
  coefficient occurs and j indicates which unknown
  it multiplies

10
Augmented Matrices

• Example
• Remark when constructing, the unknowns must be
  written in the same order in each equation and
  the constants must be on the right.

11
Augmented Matrices

• Basic method of solving system linear equations
• Step 1 multiply an equation through by a nonzero
  constant.
• Step 2 interchange two equations.
• Step 3 add a multiple of one equation to
  another.
• On the augmented matrix (elementary row
  operations)
• Step 1 multiply a row through by a nonzero
  constant.
• Step 2 interchange two rows.
• Step 3 add a multiple of one equation to another.

12
Elementary Row Operations (Example)

• r2 -2r1 r2
• r3 -3r1 r3

13
Elementary Row Operations (Example)

• r2 ½ r2
• r3 -3r2 r3
• r3 -2r3

14
Elementary Row Operations (Example)

• r1 r1 r2
• r1 -11/2 r3 r1
• r2 7/2 r3 r2
• Solution

15

• 1.2 Gaussian Elimination

16
Echelon Forms

• Reduced row-echelon form, a matrix must have the
  following properties
• If a row does not consist entirely of zeros the
  the first nonzero number in the row is a 1
  leading 1
• If there are any rows that consist entirely of
  zeros, then they are grouped together at the
  bottom of the matrix.
• In any two successive rows that do not consist
  entirely of zeros, the leading 1 in the lower row
  occurs farther to the right than the leading 1 in
  the higher row.
• Each column that contains a leading 1 has zeros
  everywhere else.

17
Echelon Forms

• A matrix that has the first three properties is
  said to be in row-echelon form.
• Example
• Reduced row-echelon form
• Row-echelon form

18
Elimination Methods

• Step 1 Locate the leftmost non zero column
• Step 2 Interchange
• r2 ? r1.
• Step 3 r1 ½ r1.
• Step 4 r3 r3 2r1.

19
Elimination Methods

• Step 5 continue do all steps above until the
  entire matrix is in row-echelon form.
• r2 -½ r2
• r3 r3 5r2
• r3 2r3

20
Elimination Methods

• Step 6 add suitable multiplies of each row to
  the rows above to introduce zeros above the
  leading 1s.
• r2 7/2 r3 r2
• r1 -6r3 r1
• r1 5r2 r1

21
Elimination Methods

• 1-5 steps produce a row-echelon form (Gaussian
  Elimination). Step 6 is producing a reduced
  row-echelon (Gauss-Jordan Elimination).
• Remark Every matrix has a unique reduced
  row-echelon form, no matter how the row
  operations are varied. Row-echelon form of matrix
  is not unique different sequences of row
  operations can produce different row- echelon
  forms.

22
Back-substitution

• Bring the augmented matrix into row-echelon form
  only and then solve the corresponding system of
  equations by back-substitution.
• Example Solved by back substitution

23
Back-Substitution

• Step 3. Assign arbitrary values to the free
  variables parameters, if any

24
Homogeneous Linear Systems

• A system of linear equations is said to be
  homogeneous if the constant terms are all zero.
• Every homogeneous sytem of linear equations is
  consistent, since all such systems have
  x10,x20,...,xn0 as a solution trivial
  solution. Other solutions are called nontrivial
  solutions.

25
Homogeneous Linear Systems

• Example Gauss-Jordan Elimination

26
Homogeneous Linear Systems

• The corresponding system of equations is
• Solving for the leading variables yields
• The general solution is
• The trivial solution is obtained when st0

27
Homogeneous Linear Systems

• Theorem
• A homogeneous system of linear equations with
  more unknowns than equations has infinitely many
  solutions.

28

• 1.3 Matrices and Matrix Operations

29
Matrix Notation and Terminology

• A matrix is a rectangular array of numbers with
  rows and columns.
• The numbers in the array are called the entries
  in the matrix.
• Examples
• The size of a matrix is described in terms of the
  number of rows and columns its contains.
• A matrix with only one column is called a column
  matrix or a column vector.
• A matrix with only one row is called a row matrix
  or a row vector.

30
Matrix Notation and Terminology

• aij (A)ij the entry in row i and column j of
  a matrix A.
• 1 x n row matrix a a1 a2 ... an
• m x 1 column matrix
• A matrix A with n rows and n columns is called a
  square matrix of order n. Main diagonal of A
  a11, a22, ..., ann

31
Operations on Matrices

• Definition
• Two matrices are defined to be equal if they have
  the same size and their corresponding entries are
  equal.
• If A aij and B bij have the same size,
  then AB if and only if (A)ij(B)ij, or
  equivalently aijbij for all i and j.
• Definition
• If A and B are matrices of the same size, then
  the sum AB is the matrix obtained by adding the
  entries of B to the corresponding entries of A,
  and the difference AB is the matrix obtained by
  subtracting the entries of B from the
  corresponding entries of A. Matrices of different
  sizes cannot be added or subtracted.

32
Operations on Matrices

• If A aij and B bij have the same size,
  then
• (AB)ij (A)ij (B)ij aij bij and
• (A-B)ij (A)ij (B)ij aij - bij
• Definition
• If A is any matrix and c is any scalar, then the
  product cA is the matrix obtained by multiplying
  each entry of the matrix A by c. The matrix cA is
  said to be a scalar multiple of A.
• If A aij, then (cA)ij c(A)ij caij.

33
Operations on Matrices

• Definition
• If A is an mxr matrix and B is an rxn matrix,
  then the product AB is the mxn matrix whose
  entries are determined as follows. To find the
  entry in row i and column j of AB, single out row
  i from the matrix A and column j from the matrix
  B. Multiply the corresponding entries from the
  row and column together and then add up the
  resulting products.

34
Partitioned Matrices
35
Matrix Multiplication by Columns and by Rows
36
Matrix Products as Linear Combinations
37
Matrix Form of a Linear System
38
Transpose of a Matrix
39

• 1.4 Inverses Rules of Matrix Arithmetic

40
Properties of Matrix Operations

• ab ba for real numbers a b, but AB ? BA even
  if both AB BA are defined and have the same
  size.
• Example

41
Properties of Matrix Operations

• Theorem Properties of
• AB BA
• A(BC) (AB)C
• A(BC) (AB)C
• A(BC) ABAC
• (BC)A BACA
• A(B-C) AB-AC
• (B-C)A BA-CA
• a(BC) aBaC
• a(B-C) aB-aC

• Math Arithmetic
• (Commutative law for addition)
• (Associative law for addition)
• (Associative for multiplication)
• (Left distributive law)
• (Right distributive law)
• (ab)C aCbC
• (a-b)C aC-bC
• a(bC) (ab)C
• a(BC) (aB)C

42
Properties of Matrix Operations

• Proof (d)
• Proof for both have the same size
• Let size A be r x m matrix, B C be m x n (same
  size).
• This makes A(BC) an r x n matrix, follows that
  ABAC is also an r x n matrix.
• Proof that corresponding entries are equal
• Let Aaij, Bbij, Ccij
• Need to show that A(BC)ij ABACij for all
  values of i and j.
• Use the definitions of matrix addition and matrix
  multiplication.

43
Properties of Matrix Operations

• Remark In general, given any sum or any product
  of matrices, pairs of parentheses can be inserted
  or deleted anywhere within the expression without
  affecting the end result.

44
Zero Matrices

• A matrix, all of whose entries are zero, such as
• A zero matrix will be denoted by 0 or 0mxn for
  the mxn zero matrix. 0 for zero matrix with one
  column.
• Properties of zero matrices
• A 0 0 A A
• A A 0
• 0 A -A
• A0 0 0A 0

45
Identity Matrices

• Square matrices with 1s on the main diagonal and
  0s off the main diagonal, such as
• Notation In n x n identity matrix.
• If A m x n matrix, then
• AIn A and InA A

46
Identity Matrices

• Example
• Theorem If R is the reduced row-echelon form of
  an n x n matrix A, then either R has a row of
  zeros or R is the identity matrix In.

47
Identity Matrices

• Definition If A B is a square matrix and same
  size ? AB BA I, then A is said to be
  invertible and B is called an inverse of A. If no
  such matrix B can be found, then A is said to be
  singular.
• Example

48
Properties of Inverses

• Theorem
• If B and C are both inverses of the matrix A,
  then B C.
• If A is invertible, then its inverse will be
  denoted by the symbol A-1.
• The matrix
• is invertible if ad-bc ? 0, in which case the
  inverse is given by the formula

49
Properties of Inverses

• Theorem If A and B are invertible matrices of
  the same size, then AB is invertible and (AB)-1
  B-1A-1.
• A product of any number of invertible matrices is
  invertible, and the inverse of the product is the
  product of the inverses in the reverse order.
• Example

50
Powers of a Matrix

• If A is a square matrix, then we define the
  nonnegative integer powers of A to be
• A0I An AA...A (ngt0)
• n factors
• Moreover, if A is invertible, then we define the
  negative integer prowers to be A-n (A-1)n
  A-1A-1...A-1
• n factors
• Theorem Laws of Exponents
• If A is a square matrix, and r and s are
  integers, then ArAs Ars Ars
• If A is an invertible matrix, then
• A-1 is invertible and (A-1)-1 A
• An is invertible and (An)-1 (A-1)n for n 0,
  1, 2, ...
• For any nonzero scalar k, the matrix kA is
  invertible and (kA)-1 1/k A-1.

51
Powers of a Matrix

• Example

52
Polynomial Expressions Involving Matrices

• If A is a square matrix, m x m, and if
• is any polynomial, then we define
• Example

53
Properties of the Transpose

• Theorem If the sizes of the matrices are such
  that the stated operations can be performed, then
• ((A)T)T A
• (AB)T AT BT and (A-B)T AT BT
• (kA)T kAT, where k is any scalar
• (AB)T BTAT
• The transpose of a product of any number of
  matrices is equal to the product of their
  transpose in the reverse order.

54
Invertibility of a Transpose

• Theorem If A is an invertible matrix, then AT is
  also invertible and (AT)-1 (A-1)T
• Example

55
Exercise

• Show that if a square matrix A satisfies
  A2-3AI0, then A-13I-A
• Let A be the matrix
• Determine whether A is invertible, and if so,
  find its inverse. Hint. Solve AX I by equating
  corresponding entries on the two sides.

56

• 1.5 Elementary Matrices and
• a Method for Finding A-1

57
Elementary Matrices

• Definition
• An n x n matrix is called an elementary matrix if
  it can be obtained from the n x n identity matrix
  In by performing a single elementary row
  operation.
• Example
• Multiply the second row of I2 by -3.
• Interchange the second and fourth rows of I4.
• Add 3 times the third row of I3 to the first row.

58
Elementary Matrices

• Theorem (Row Operations by Matrix
  Multiplication)
• If the elementary matrix E results from
  performing a certain row operation on Im and if A
  is an m x n matrix, then the product of EA is the
  matrix that results when this same row operation
  is performed on A.
• Example
• EA is precisely the same matrix that results when
  we add 3 times the first row of A to the third
  row.

59
Elementary Matrices

• If an elementary row operation is applied to an
  identity matrix I to produce an elementary matrix
  E, then there is a second row operation that,
  when applied to E, produces I back again.
• Inverse operation

60
Elementary Matrices

• Theorem Every elementary matrix is invertible,
  and the inverse is also an elementary matrix.
• Theorem (Equivalent Statements)
• If A is an n x n matrix, then the following
  statements are equivalent, that is, all true or
  all false.
• A is invertible
• Ax 0 has only the trivial solution.
• The reduced row-echelon form of A is In.
• A is expressible as a product of elementary
  matrices.

61
Elementary Matrices

• Proof
• Assume A is invertible and let x0 be any
  solution of Ax0.
• Let Ax0 be the matrix form of the system

62
Elementary Matrices

• Assumed that the reduced row-echelon form of A
  is In by a finite sequence of elementary row
  operations, such that
• By theorem, E1,,En are invertible. Multiplying
  both sides of equation on the left we obtain
• This equation expresses A as a product of
  elementary matrices.
• If A is a product of elementary matrices, then
  the matrix A is a product of invertible matrices,
  and hence is invertible.
• Matrices that can be obtained from one another by
  a finite sequence of elementary row operations
  are said to be row equivalent.
• An n x n matrix A is invertible if and only if it
  is row equivalent to the n x n identity matrix.

63
A Method for Inverting Matrices

• To find the inverse of an invertible matrix, we
  must find a sequence of elementary row operations
  that reduces A to the identity and then perform
  this same sequence of operations on In to obtain
  A-1.
• Example
• Adjoin the identity matrix to the right side of
  A, thereby producing a matrix of the form AI
• Apply row operations to this matrix until the
  left side is reduced to I, so the final matrix
  will have the form IA-1.

64
A Method for Inverting Matrices

• Added 2 times the first row to the second and
  1 times the first row to the third.
• Added 2 times the second row to the third.
• Multiplied the third row by 1.
• Added 3 times the third row to the second and 3
  times the third row to the first.
• We added 2 times the second row to the first.

65
A Method for Inverting Matrices

• Often it will not be known in advance whether a
  given matrix is invertible.
• If elementary row operations are attempted on a
  matrix that is not invertible, then at some point
  in the computations a row of zeros will occur on
  the left side.
• Example
• Added -2 times the first row to the second
  and
• added the first row to the third.
• Added the second row to the third.

66
Exercises

• Consider the matrices
• Find elementary matrices, E1, E2, E3, and E4,
  such that
• E1AB
• E2BA
• E3AC
• E4CA

67
Exercises

• Express the matrix
• in the form A E F G R, where E, F, G are
  elementary matrices, and R is in row-echelon form.

68

• 1.6 Further Results on
• Systems of Equations and Invertibility

69
Linear Systems

• Theorem
• Solving Linear Systems by Matrix Inversion
• If A is an invertible n x n matrix, then for
  each n x 1 matrix b, the system of equations Ax
  b has exactly one solution, namely, x A-1b.
• Linear systems with a common coefficient matrix.
• Axb1, Axb2, Axb3, ..., Axbk
• If A is invertible, then the solutions
• x1A-1b1, x2A-1b2, x3A-1b3, ..., xkA-1bk
• This can be efficiently done using Gauss-Jordan
  Elimination on Ab1b2...bk

70
Linear Systems

• Example (a) (b)
• The solution
• (a) x11, x20, x31
• (b) x12, x21, x3-1

71
Properties of Invertible Matrices

• Theorem Let A be a square matrix.
• If B is a square matrix satisfying BAI, then
  BA-1.
• If B is a square matrix satisfying ABI, then
  BA-1.
• Theorem Equivalent Statements
• A is invertible
• Ax0 has only the trivial solutions
• The reduced row-echelon form of A is In
• A is expresssible as a product of elementary
  matrices
• Axb is consistent for every n x 1 matrix b
• Axb has exactly one solution for every n x 1
  matrix b

72
Properties of Invertible Matrices

• Theorem Let A and B be square matrices of the
  same size. If AB is invertible, then A and B must
  also be invertible.
• A fundamental problem.
• Let A be a fixed m x n matrix. Find all m x 1
  matrices b such that the system of equations Axb
  is consistent.

73
Exercises

• Solve the system by inverting the coefficient
  matrix.
• Find condition that bs must satisfy for the
  system to be consistent.

74

• 1.7 Diagonal, Triangular,
• and Symmetric Matrices

75
Diagonal Matrices

• A square matrix in which all the entries off the
  main diagonal are zero. Example
• A diagonal matrix is invertible if and only if
  all of its diagonal entries are nonzero.

76
Diagonal Matrices

• Example

77
Triangular Matrices

• Lower triangular a square matrix in which all
  the entries above the main diagonal are zero.
• Upper triangular a square matrix in which all
  the entries under the main diagonal are zero.
• Triangular a matrix that is either upper
  triangular or lower triangular.

78
Triangular Matrices

• Theorem (basic properties of triangular
  matrices)
• The transpose of a lower triangular matrix is
  upper triangular, and the transpose of an upper
  triangular matrix is lower triangular.
• The product of lower triangular matrices is lower
  triangular, and the product of upper triangular
  matrices is upper triangular.
• A triangular matrix is invertible if and only its
  diagonal entries are all nonzero.
• The inverse of an invertible lower triangular
  matrix is lower triangular, and the inverse of an
  invertible upper triangular matrix is upper
  triangular.

79
Triangular Matrices

• Example
• The matrix A is invertible, since its diagonal
  entries are nonzero, but the matrix B is not.
• This inverse is upper triangular.
• This product is upper triangular.

80
Symmetric Matrices

• A square matrix A is called symmetric if A AT.
• A matrix A aij is symmetric if and only if
  aijaji for all values of i and j.

81
Symmetric Matrices

• Theorem If A and B are symmetric matrices with
  the same size, and if k is any scalar, then
• AT is symmetric
• AB and A-B are symmetric
• kA is symmetric
• Theorem
• If A is an invertible matrix, then A-1 is
  symmetric.
• If A is an invertible matrix, then AAT and ATA
  are also invertible.

82
Exercise

• Find all values of a, b, and c for which A is
  symmetric.
• Find all values of a and b for which A and B are
  both not invertible.