1 / 82

Linear Algebra

- Lecturers
- Heru Suhartanto, PhD, heru_at_cs.ui.ac.id
- Yova Ruldeviyani, MKom, yovayg_at_gmail.com
- Schedule (at C3111)
- Tuesday, 1.00 2.40 PM
- Thursday, 1.00 1.50 PM
- Marking scheme
- 2 exams (mid 15, final 20) reqs attendance

75 - 4 quizes, 40
- 5 assignments, 25
- Reference
- Horward Anton, Elementary Linear Algebra, 8-th

Ed, John Wiley Sons, Inc, 2000 - More information (will be updated soon) at
- http//telaga.cs.ui.ac.id/WebKuliah/LinearAlgebra0

5/

Systems of Linear Equation and Matrices

- CHAPTER 1
- FASILKOM UI 05

Introduction Matrices

- Information in science and mathematics is often

organized into rows and columns to form

rectangular arrays. - Tables of numerical data that arise from physical

observations - Example (to solve linear equations)
- Solution is obtained by performing appropriate

operations on this matrix

- 1.1 Introduction to
- Systems of Linear Equations

Linear Equations

- In x y variables (straight line in the xy-plane)
- where a1, a2, b are real constants,
- In n variables
- where a1, , an b are real constants
- x1, , xn unknowns.
- Example 1 Linear Equations
- The equations are linear (does not involve any

products or roots of variables).

Linear Equations

- The equations are not linear.
- A solution of

is a sequence of n numbers s1, s2, ..., sn ?

they satisfy the equation when x1s1, x2s2, ...,

xnsn (solution set). - Example 2 Finding a Solution Set
- 1 equation and 2 unknown, set one var as the

parameter (assign any value) - or
- 1 equation and 3 unknown, set 2 vars as parameter

Linear Systems / System of Linear Equations

- Is A finite set of linear equations in the vars

x1, ..., xn - s1, ..., sn is called a solution if x1s1, ...,

xnsn is a solution of every equation in the

system. - Ex.
- x11, x22, x3-1 the solution
- x11, x28, x31 is not, satisfy only the first

eq. - System that has no solution inconsistent
- System that has at least one solution consistent
- Consider

Linear Systems

- (x,y) lies on a line if and only if the numbers x

and y satisfy the equation of the line. Solution

points of intersection l1 l2 - l1 and l2 may be parallel
- no intersection, no solution
- l1 and l2 may intersect
- at only one point one solution
- l1 and l2 may coincide
- infinite many points of intersection,
- infinitely many solutions

Linear Systems

- In general Every system of linear equations has

either no solutions, exactly one solution, or

infinitely many solutions. - An arbitrary system of m linear equations in n

unknowns - a11x1 a12x2 ... a1nxn b1
- a21x1 a22x2 ... a2nxn b2
- am1x1 am2x2 ... amnxn bm
- x1, ..., xn unknowns, as and bs are constants
- aij, i indicates the equation in which the

coefficient occurs and j indicates which unknown

it multiplies

Augmented Matrices

- Example
- Remark when constructing, the unknowns must be

written in the same order in each equation and

the constants must be on the right.

Augmented Matrices

- Basic method of solving system linear equations
- Step 1 multiply an equation through by a nonzero

constant. - Step 2 interchange two equations.
- Step 3 add a multiple of one equation to

another. - On the augmented matrix (elementary row

operations) - Step 1 multiply a row through by a nonzero

constant. - Step 2 interchange two rows.
- Step 3 add a multiple of one equation to another.

Elementary Row Operations (Example)

- r2 -2r1 r2
- r3 -3r1 r3

Elementary Row Operations (Example)

- r2 ½ r2
- r3 -3r2 r3
- r3 -2r3

Elementary Row Operations (Example)

- r1 r1 r2
- r1 -11/2 r3 r1
- r2 7/2 r3 r2
- Solution

- 1.2 Gaussian Elimination

Echelon Forms

- Reduced row-echelon form, a matrix must have the

following properties - If a row does not consist entirely of zeros the

the first nonzero number in the row is a 1

leading 1 - If there are any rows that consist entirely of

zeros, then they are grouped together at the

bottom of the matrix. - In any two successive rows that do not consist

entirely of zeros, the leading 1 in the lower row

occurs farther to the right than the leading 1 in

the higher row. - Each column that contains a leading 1 has zeros

everywhere else.

Echelon Forms

- A matrix that has the first three properties is

said to be in row-echelon form. - Example
- Reduced row-echelon form
- Row-echelon form

Elimination Methods

- Step 1 Locate the leftmost non zero column
- Step 2 Interchange
- r2 ? r1.
- Step 3 r1 ½ r1.
- Step 4 r3 r3 2r1.

Elimination Methods

- Step 5 continue do all steps above until the

entire matrix is in row-echelon form. - r2 -½ r2
- r3 r3 5r2
- r3 2r3

Elimination Methods

- Step 6 add suitable multiplies of each row to

the rows above to introduce zeros above the

leading 1s. - r2 7/2 r3 r2
- r1 -6r3 r1
- r1 5r2 r1

Elimination Methods

- 1-5 steps produce a row-echelon form (Gaussian

Elimination). Step 6 is producing a reduced

row-echelon (Gauss-Jordan Elimination). - Remark Every matrix has a unique reduced

row-echelon form, no matter how the row

operations are varied. Row-echelon form of matrix

is not unique different sequences of row

operations can produce different row- echelon

forms.

Back-substitution

- Bring the augmented matrix into row-echelon form

only and then solve the corresponding system of

equations by back-substitution. - Example Solved by back substitution

Back-Substitution

- Step 3. Assign arbitrary values to the free

variables parameters, if any

Homogeneous Linear Systems

- A system of linear equations is said to be

homogeneous if the constant terms are all zero. - Every homogeneous sytem of linear equations is

consistent, since all such systems have

x10,x20,...,xn0 as a solution trivial

solution. Other solutions are called nontrivial

solutions.

Homogeneous Linear Systems

- Example Gauss-Jordan Elimination

Homogeneous Linear Systems

- The corresponding system of equations is
- Solving for the leading variables yields
- The general solution is
- The trivial solution is obtained when st0

Homogeneous Linear Systems

- Theorem
- A homogeneous system of linear equations with

more unknowns than equations has infinitely many

solutions.

- 1.3 Matrices and Matrix Operations

Matrix Notation and Terminology

- A matrix is a rectangular array of numbers with

rows and columns. - The numbers in the array are called the entries

in the matrix. - Examples
- The size of a matrix is described in terms of the

number of rows and columns its contains. - A matrix with only one column is called a column

matrix or a column vector. - A matrix with only one row is called a row matrix

or a row vector.

Matrix Notation and Terminology

- aij (A)ij the entry in row i and column j of

a matrix A. - 1 x n row matrix a a1 a2 ... an
- m x 1 column matrix
- A matrix A with n rows and n columns is called a

square matrix of order n. Main diagonal of A

a11, a22, ..., ann

Operations on Matrices

- Definition
- Two matrices are defined to be equal if they have

the same size and their corresponding entries are

equal. - If A aij and B bij have the same size,

then AB if and only if (A)ij(B)ij, or

equivalently aijbij for all i and j. - Definition
- If A and B are matrices of the same size, then

the sum AB is the matrix obtained by adding the

entries of B to the corresponding entries of A,

and the difference AB is the matrix obtained by

subtracting the entries of B from the

corresponding entries of A. Matrices of different

sizes cannot be added or subtracted.

Operations on Matrices

- If A aij and B bij have the same size,

then - (AB)ij (A)ij (B)ij aij bij and
- (A-B)ij (A)ij (B)ij aij - bij
- Definition
- If A is any matrix and c is any scalar, then the

product cA is the matrix obtained by multiplying

each entry of the matrix A by c. The matrix cA is

said to be a scalar multiple of A. - If A aij, then (cA)ij c(A)ij caij.

Operations on Matrices

- Definition
- If A is an mxr matrix and B is an rxn matrix,

then the product AB is the mxn matrix whose

entries are determined as follows. To find the

entry in row i and column j of AB, single out row

i from the matrix A and column j from the matrix

B. Multiply the corresponding entries from the

row and column together and then add up the

resulting products.

Partitioned Matrices

Matrix Multiplication by Columns and by Rows

Matrix Products as Linear Combinations

Matrix Form of a Linear System

Transpose of a Matrix

- 1.4 Inverses Rules of Matrix Arithmetic

Properties of Matrix Operations

- ab ba for real numbers a b, but AB ? BA even

if both AB BA are defined and have the same

size. - Example

Properties of Matrix Operations

- Theorem Properties of
- AB BA
- A(BC) (AB)C
- A(BC) (AB)C
- A(BC) ABAC
- (BC)A BACA
- A(B-C) AB-AC
- (B-C)A BA-CA
- a(BC) aBaC
- a(B-C) aB-aC

- Math Arithmetic
- (Commutative law for addition)
- (Associative law for addition)
- (Associative for multiplication)
- (Left distributive law)
- (Right distributive law)
- (ab)C aCbC
- (a-b)C aC-bC
- a(bC) (ab)C
- a(BC) (aB)C

Properties of Matrix Operations

- Proof (d)
- Proof for both have the same size
- Let size A be r x m matrix, B C be m x n (same

size). - This makes A(BC) an r x n matrix, follows that

ABAC is also an r x n matrix. - Proof that corresponding entries are equal
- Let Aaij, Bbij, Ccij
- Need to show that A(BC)ij ABACij for all

values of i and j. - Use the definitions of matrix addition and matrix

multiplication.

Properties of Matrix Operations

- Remark In general, given any sum or any product

of matrices, pairs of parentheses can be inserted

or deleted anywhere within the expression without

affecting the end result.

Zero Matrices

- A matrix, all of whose entries are zero, such as
- A zero matrix will be denoted by 0 or 0mxn for

the mxn zero matrix. 0 for zero matrix with one

column. - Properties of zero matrices
- A 0 0 A A
- A A 0
- 0 A -A
- A0 0 0A 0

Identity Matrices

- Square matrices with 1s on the main diagonal and

0s off the main diagonal, such as - Notation In n x n identity matrix.
- If A m x n matrix, then
- AIn A and InA A

Identity Matrices

- Example
- Theorem If R is the reduced row-echelon form of

an n x n matrix A, then either R has a row of

zeros or R is the identity matrix In.

Identity Matrices

- Definition If A B is a square matrix and same

size ? AB BA I, then A is said to be

invertible and B is called an inverse of A. If no

such matrix B can be found, then A is said to be

singular. - Example

Properties of Inverses

- Theorem
- If B and C are both inverses of the matrix A,

then B C. - If A is invertible, then its inverse will be

denoted by the symbol A-1. - The matrix
- is invertible if ad-bc ? 0, in which case the

inverse is given by the formula

Properties of Inverses

- Theorem If A and B are invertible matrices of

the same size, then AB is invertible and (AB)-1

B-1A-1. - A product of any number of invertible matrices is

invertible, and the inverse of the product is the

product of the inverses in the reverse order. - Example

Powers of a Matrix

- If A is a square matrix, then we define the

nonnegative integer powers of A to be - A0I An AA...A (ngt0)
- n factors
- Moreover, if A is invertible, then we define the

negative integer prowers to be A-n (A-1)n

A-1A-1...A-1 - n factors
- Theorem Laws of Exponents
- If A is a square matrix, and r and s are

integers, then ArAs Ars Ars - If A is an invertible matrix, then
- A-1 is invertible and (A-1)-1 A
- An is invertible and (An)-1 (A-1)n for n 0,

1, 2, ... - For any nonzero scalar k, the matrix kA is

invertible and (kA)-1 1/k A-1.

Powers of a Matrix

- Example

Polynomial Expressions Involving Matrices

- If A is a square matrix, m x m, and if
- is any polynomial, then we define
- Example

Properties of the Transpose

- Theorem If the sizes of the matrices are such

that the stated operations can be performed, then - ((A)T)T A
- (AB)T AT BT and (A-B)T AT BT
- (kA)T kAT, where k is any scalar
- (AB)T BTAT
- The transpose of a product of any number of

matrices is equal to the product of their

transpose in the reverse order.

Invertibility of a Transpose

- Theorem If A is an invertible matrix, then AT is

also invertible and (AT)-1 (A-1)T - Example

Exercise

- Show that if a square matrix A satisfies

A2-3AI0, then A-13I-A - Let A be the matrix
- Determine whether A is invertible, and if so,

find its inverse. Hint. Solve AX I by equating

corresponding entries on the two sides.

- 1.5 Elementary Matrices and
- a Method for Finding A-1

Elementary Matrices

- Definition
- An n x n matrix is called an elementary matrix if

it can be obtained from the n x n identity matrix

In by performing a single elementary row

operation. - Example
- Multiply the second row of I2 by -3.
- Interchange the second and fourth rows of I4.
- Add 3 times the third row of I3 to the first row.

Elementary Matrices

- Theorem (Row Operations by Matrix

Multiplication) - If the elementary matrix E results from

performing a certain row operation on Im and if A

is an m x n matrix, then the product of EA is the

matrix that results when this same row operation

is performed on A. - Example
- EA is precisely the same matrix that results when

we add 3 times the first row of A to the third

row.

Elementary Matrices

- If an elementary row operation is applied to an

identity matrix I to produce an elementary matrix

E, then there is a second row operation that,

when applied to E, produces I back again. - Inverse operation

Elementary Matrices

- Theorem Every elementary matrix is invertible,

and the inverse is also an elementary matrix. - Theorem (Equivalent Statements)
- If A is an n x n matrix, then the following

statements are equivalent, that is, all true or

all false. - A is invertible
- Ax 0 has only the trivial solution.
- The reduced row-echelon form of A is In.
- A is expressible as a product of elementary

matrices.

Elementary Matrices

- Proof
- Assume A is invertible and let x0 be any

solution of Ax0. - Let Ax0 be the matrix form of the system

Elementary Matrices

- Assumed that the reduced row-echelon form of A

is In by a finite sequence of elementary row

operations, such that - By theorem, E1,,En are invertible. Multiplying

both sides of equation on the left we obtain - This equation expresses A as a product of

elementary matrices. - If A is a product of elementary matrices, then

the matrix A is a product of invertible matrices,

and hence is invertible. - Matrices that can be obtained from one another by

a finite sequence of elementary row operations

are said to be row equivalent. - An n x n matrix A is invertible if and only if it

is row equivalent to the n x n identity matrix.

A Method for Inverting Matrices

- To find the inverse of an invertible matrix, we

must find a sequence of elementary row operations

that reduces A to the identity and then perform

this same sequence of operations on In to obtain

A-1. - Example
- Adjoin the identity matrix to the right side of

A, thereby producing a matrix of the form AI - Apply row operations to this matrix until the

left side is reduced to I, so the final matrix

will have the form IA-1.

A Method for Inverting Matrices

- Added 2 times the first row to the second and

1 times the first row to the third. - Added 2 times the second row to the third.
- Multiplied the third row by 1.
- Added 3 times the third row to the second and 3

times the third row to the first. - We added 2 times the second row to the first.

A Method for Inverting Matrices

- Often it will not be known in advance whether a

given matrix is invertible. - If elementary row operations are attempted on a

matrix that is not invertible, then at some point

in the computations a row of zeros will occur on

the left side. - Example
- Added -2 times the first row to the second

and - added the first row to the third.
- Added the second row to the third.

Exercises

- Consider the matrices
- Find elementary matrices, E1, E2, E3, and E4,

such that - E1AB
- E2BA
- E3AC
- E4CA

Exercises

- Express the matrix
- in the form A E F G R, where E, F, G are

elementary matrices, and R is in row-echelon form.

- 1.6 Further Results on
- Systems of Equations and Invertibility

Linear Systems

- Theorem
- Solving Linear Systems by Matrix Inversion
- If A is an invertible n x n matrix, then for

each n x 1 matrix b, the system of equations Ax

b has exactly one solution, namely, x A-1b. - Linear systems with a common coefficient matrix.
- Axb1, Axb2, Axb3, ..., Axbk
- If A is invertible, then the solutions
- x1A-1b1, x2A-1b2, x3A-1b3, ..., xkA-1bk
- This can be efficiently done using Gauss-Jordan

Elimination on Ab1b2...bk

Linear Systems

- Example (a) (b)
- The solution
- (a) x11, x20, x31
- (b) x12, x21, x3-1

Properties of Invertible Matrices

- Theorem Let A be a square matrix.
- If B is a square matrix satisfying BAI, then

BA-1. - If B is a square matrix satisfying ABI, then

BA-1. - Theorem Equivalent Statements
- A is invertible
- Ax0 has only the trivial solutions
- The reduced row-echelon form of A is In
- A is expresssible as a product of elementary

matrices - Axb is consistent for every n x 1 matrix b
- Axb has exactly one solution for every n x 1

matrix b

Properties of Invertible Matrices

- Theorem Let A and B be square matrices of the

same size. If AB is invertible, then A and B must

also be invertible. - A fundamental problem.
- Let A be a fixed m x n matrix. Find all m x 1

matrices b such that the system of equations Axb

is consistent.

Exercises

- Solve the system by inverting the coefficient

matrix. - Find condition that bs must satisfy for the

system to be consistent.

- 1.7 Diagonal, Triangular,
- and Symmetric Matrices

Diagonal Matrices

- A square matrix in which all the entries off the

main diagonal are zero. Example - A diagonal matrix is invertible if and only if

all of its diagonal entries are nonzero.

Diagonal Matrices

- Example

Triangular Matrices

- Lower triangular a square matrix in which all

the entries above the main diagonal are zero. - Upper triangular a square matrix in which all

the entries under the main diagonal are zero. - Triangular a matrix that is either upper

triangular or lower triangular.

Triangular Matrices

- Theorem (basic properties of triangular

matrices) - The transpose of a lower triangular matrix is

upper triangular, and the transpose of an upper

triangular matrix is lower triangular. - The product of lower triangular matrices is lower

triangular, and the product of upper triangular

matrices is upper triangular. - A triangular matrix is invertible if and only its

diagonal entries are all nonzero. - The inverse of an invertible lower triangular

matrix is lower triangular, and the inverse of an

invertible upper triangular matrix is upper

triangular.

Triangular Matrices

- Example
- The matrix A is invertible, since its diagonal

entries are nonzero, but the matrix B is not. - This inverse is upper triangular.
- This product is upper triangular.

Symmetric Matrices

- A square matrix A is called symmetric if A AT.
- A matrix A aij is symmetric if and only if

aijaji for all values of i and j.

Symmetric Matrices

- Theorem If A and B are symmetric matrices with

the same size, and if k is any scalar, then - AT is symmetric
- AB and A-B are symmetric
- kA is symmetric
- Theorem
- If A is an invertible matrix, then A-1 is

symmetric. - If A is an invertible matrix, then AAT and ATA

are also invertible.

Exercise

- Find all values of a, b, and c for which A is

symmetric. - Find all values of a and b for which A and B are

both not invertible.