Saturday, June 09, 2007

Cofactor Expansion

In today's blog, I provide a general method for computing the determinant using cofactor expansion. This is taken from Matrices and Linear Transformations by Charles G. Cullen.

Lemma 1R:

If the rows of an n x n matrix A are linearly dependent, det(A)=0

Proof:

(1) Let A be an n x n matrix with rows that are linearly dependent.

[See Definition 1, here for review of linearly independent]

(2) Then, there exists αi such that:

α1R1 + ... + αnRn = 0.

(3) Then, it is possible to use elementary operations to set R1 = 0 since:

(a) Multiply R1 with α1

(b) For each row i (where i = 2 .. n), add αi*Ri to R1.

(c) By step #2, it is clear after (a) and after each addition (b), R1 = 0.

(4) So we have:

A(0R1) = EnEn-1...E2E1A where Ei is an elementary operation. [See Theorem 5, here for review of how elementary operations map to multiplication]

(5) Using the definition of determinants [See Definition 4, here], we have:

det(A(0R1)) = 0 [See Lemma Corollary 4.2, here]

(6) Using our previous results, we know that all det(Ei) ≠ 0. (See Lemma 4, Lemma 7, and Lemma 9 here)

(7) It therefore follows that det(A) = 0 since det(A(0R1)) = det(En)det(En-1)...det(E2)det(E1)det(A)

QED

Corollary 1.1R: If A is a matrix with two identical rows, then det(A)=0

Proof:

(1) Let A be an n x m matrix where row i = row j and i ≠ j.

(2) We can define the set αk such that:

if k = i, then αk=1
if k = j, then αk=-1
otherwise, αk=0

(3) Clearly, α1R1 + ... + αnRn=0 where not all αk = 0.

(4) Therefore, the rows of A of linearly dependent and using Lemma 1 above, det(A)=0.

QED

Lemma 1C:

If the columns of an n x n matrix A are linearly dependent, det(A)=0

Proof:

(1) Let A be an n x n matrix with columns that are linearly dependent.

[See Definition 1, here for review of linearly independent]

(2) Then, there exists αi such that:

α1C1 + ... + αnCn = 0.

(3) Then, it is possible to use elementary operations to set C1 = 0 since:

(a) Multiply C1 with α1

(b) For each column i (where i = 2 .. n), add αi*Ci to C1.

(c) By step #2, it is clear after (a) and after each addition (b), C1 = 0.

(4) So we have:

A(0C1) = AEnEn-1...E2E1 where Ei is an elementary operation. [See Theorem 5, here for review of how elementary operations map to multiplication]

(5) Using the definition of determinants [See Definition 4, here], we have:

det(A(0C1)) = 0 [See Lemma Corollary 4.2, here]

(6) Using our previous results, we know that all det(Ei) ≠ 0. (See Lemma 4, Lemma 7, and Lemma 9 here)

(7) It therefore follows that det(A) = 0 since det(A(0C1)) = det(En)det(En-1)...det(E2)det(E1)det(A)

QED

Corollary 1.1C: If A is a matrix with two identical columns, then det(A)=0

Proof:

(1) Let A be an n x m matrix where row i = row j and i ≠ j.

(2) We can define the set αk such that:

if k = i, then αk=1
if k = j, then αk=-1
otherwise, αk=0

(3) Clearly, α1R1 + ... + αnRn=0 where not all αk = 0.

(4) Therefore, the rows of A of linearly dependent and using Lemma 1 above, det(A)=0.

QED

Lemma 2: det(A) is a linear function of the rows of A

That is,

if three matrices A,B,C are exactly the same except for one row where this row is α for A, β for B, and (α + β) for C, then:

det(C) = det(A) + det(B)

Proof:

(1) Assume that the rows which are the same in A,B, and C are linearly dependent. [See Definition 1, here]

(2) Then det(A)=0, det(B)=0, and det(C)=0. [See Lemma 1 above]

(3) So that we have 0 = det(C) = det(A) + det(B) = 0 + 0

(4) Assume that all the rows but the first row are the same and that none of these rows are linearly dependent.

(5) Let F1 x n be the set of all possible rows of length n.

(6) F1 x n is a vector space. [See Definition 2, here for definition of vector space]

(7) Since [ R2, R3, ..., Rn ] are linearly independent, we can use this list to build a basis: [ ε, R2, R3, ..., Rn]. [See Theorem 1, here]

(8) Since [ ε, R2, R3, ..., Rn] is a basis (see Definition 2, here for definition of basis), we have:

Row1(A) = α = a1ε + a2R2 + ... + anRn

Row1(B) = β = b1ε + b2R2 + ... + bnRn

Row1(C) = (α + β) = (a1 + b1)ε + (a2 + b2)R2 + ... + (an + bn)Rn

(9) For A,B,C we can use I(kRj + Ri) to simplify the first row [See Definition 3, here for definition of A(kRj + Ri)].

For A, we set each k=ai and then use Ri. In this way, we can reduce α to a1. Likewise, we can reduce β to b1 and finally, we can reduce (α + β) to (a1 + b1.

(10) Since det(AB) = det(A)det(B) [See Definition 4, here] and since det(I(kRj + Ri) = 1 [See Lemma 7, here] , it follows that:

det(A) =



det(B) =



det(C) =



(11) It is also true that (see Definition 4, here):

det(A) =



det(B) =



det(C) =



(12) And it is true that:

det(C) = det(A) + det(B)

(13) Now, we have one last case to deal with. In this case, A,B,C are all the same except for a row r that is not the first row and all other rows are linearly independent.

(14) Now, from the elementary operations (see Lemma 9, here), we know that:

det(A*I(Ri ↔ R1)) = det(A)*(-1)

det(B*I(Ri ↔ R1)) = det(B)*(-1)

det(C*I(Ri ↔ R1)) = det(C)*(-1)

(15) Now from step #4 thru step #12, we know that:

det(C*I(Ri ↔ R1)) = det(A*I(Ri ↔ R1)) + det(B*I(Ri ↔ R1)) =

= -det(C) = -det(A) + -det(B)

(16) Multiplying both sides by -1 gives us:

det(C) = det(A) + det(B)

QED

Corollary 2.1: det(A) is a linear function of the rows of A

That is,

if three matrices A,B,C are exactly the same except for one column where this row is α for A, β for B, and (α + β) for C, then:

det(C) = det(A) + det(B)

Proof:

(1) Using Lemma 2 above, we have:

det(CT) = det(AT) + det(BT)

(2) Using Theorem 9 here, we have:

det(CT) = det(C) det(AT) = det(A)' det(BT) = det (B)
(3) Putting this all together gives us:
det(C) = det(A) + det(B)

QED

Definition 1: Minor Mi,j(A)

For an n x n matrix A, the minor of ai,j, Mi,j(A), is the n-1 x n-1 submatrix of A obtained by deleting row i and column j of A.

Definition 2: Cofactor

For an n x n matrix A, the cofactor of ai,j, cofi,j(A), is defined as (-1)i+jdet(Mi,j(A)).

Lemma 3:

Let B be an Fn x n matrix of the form:



then:

det(B) = det(M1,1(B)) = cof1,1(B).

Proof:

(1) Using the elementary row operation I(kRj + Ri) [See Definition 3, here] we can reduce B to the following form:



(2) Let E1*....*Et be a representation of M1,1(B) as a product of n-1 x n-1 elementary matrices. [See Theorem 6, here]

(3) Now, we have:









det(E1)*...*det(Et) = det(E1*...*Et) = det(M11(B))

QED

Corollary 3.1:

Let B be an Fn x n matrix of the form:



then:

det(B) = det(M1,1(B)) = cof1,1(B).

Proof:

(1) Using Lemma 3 above, we have:

det(BT) = det(M1,1(B)) = cof1,1(B).

(2) Using Theorem 9 here, we have:

det(B) = det(BT)

QED

Theorem 4: Cofactor Expansion of rows

The determinant of an n x n matrix can be computed as the sum of products of the elements of any row and their cofactors.

That is:

det(A) = ∑(j=1,n) ai,jcofi,j(A)

Proof:

(1) Let A be an n x n matrix.

(2) Let i be the row that we wish to use to compute det(A).

(3) Using Row operation II (I(Ri ↔ Rj)), we move row i to row 1 by switching row i ↔ row (i-1) and row (i-1) ↔ row (i-2) ↔ ... ↔ row(2) ↔ row(1).

(4) We can see that there are i-1 such operations and at the end of it, we are left with the following order:



(5) Since det(I(Ri ↔ Rj)) = -1, the (i-1) row operations gives us:





(6) Using Lemma 2 above, we have:
det(A) =



(7) Using Column Operation II, we can move the nonzero column at column j (where j represents the current column) to column 1. This consists of the same type of operation as before:

column i ↔ column (j-1) and column (j-1) ↔ column (j-2) ↔ ... ↔ column(2) ↔ column(1).

(8) This gives us (-1)0 for j=1, (-1)1 for j=2, and so on so that we get:

det(A) = (-1)(i-1)[(-1)(1-1)det([ai,1*]) + (-1)(2-1)det([ai,2*]) + ... + (-1)(n-1)det([ai,n*]) ]

where [ai,j*] is the determinant of the [ai,j] after it has been moved to column 1 using column operations.

(9) Since (-1) to an even number = 1 and (-1) to an odd number = -1, we can simplify step #8 to the following:

det(A) = (-1)i-1[



]

(10) Using row operation I (I(kRi)), we further get:

det(A) = (-1)i-1[



]

(11) We can now plug in Lemma 3 above to complete the proof:

det(A) =
(-1)i-1[ai,1det(Mi,1(A)) - ai,2det(Mi,2(A)) + ai,3det(Mi,3(A)) - ai,4det(Mi,4(A)) + ...] =

(-1)i-1[ai,1cofi,1(A) - ai,2cofi,2(A) + ai,3cofi,3(A) - ai,4cofi,4(A) + ... ]

QED

The implication of Theorem 4 is that we can now establish a recursive method for obtaining the determinant of an n x n matrix.

Algorithm 1: Find the determininant of any n x n matrix

(1) If n x n is a 2 x 2 matrix, then the determinant is:



(2) If n x n is not a 2 x 2 matrix, then the determinant can be derived from the determinant of the Minor of n x n so that:

det(n x n) = a1,1det(M1,1) - a1,2det(M1,2) + ... + (-1)(n-2)a1,(n-1)det(M1,(n-1) + (-1)(n-1)a1,ndet(M1,n).

Corollary 4.1: Cofactor Expansion of columns

The determinant of an n x n matrix can be computed as the sum of products of the elements of any column and their cofactors.

That is:

det(A) = ∑(i=1,n) ai,jcofactori,j(A)

Proof:

(1) Let A be an n x n matrix.

(2) Let j be the column that we wish to use to compute det(A).

(3) Using Column operation II (I(Ci ↔ Cj)), we move column j to column 1 by switching column j ↔ column (j-1) and column (j-1) ↔ column (j-2) ↔ ... ↔ column(2) ↔ column(1).

(4) We can see that there are j-1 such operations and at the end of it, we are left with the following order:



(5) Since det(I(Ci ↔ Cj)) = -1,

det(A) = (-1)(j-1)det([Colj Col1 ... Coln])

(6) Using Corollary 2.1 above, we have:
det(A) = (-1)(j-1)[det([a1,j]) + det([a2,j]) + ... + det([an,j])]

where [ai,j] is the matrix derived from A where all values in column j are empty except for ai,j.

(7) Using Row Operation II (I(Ri ↔ Rj)), we can move this nonzero row at row i to row 1. This consists of the same type of operation as before:

row i ↔ row (i-1) and row (i-1) ↔ row (i-2) ↔ ... ↔ row(2) ↔ row(1).

(8) This gives us (-1)0 for i=1, (-1)1 for i=2, and so on so that we get:

det(A) = (-1)(j-1)[(-1)(1-1)det([a1,j*]) + (-1)(2-1)det([a2,j*]) + ... + (-1)(n-1)det([an,j*]) ]

where [ai,j*] is the determinant of the [ai,j] after it has been moved to row 1 using row operations.

(9) Since (-1) to an even number = 1 and (-1) to an odd number = -1, we can simplify step #8 to the following:

det(A) = (-1)(j-1)[det([a1,j*]) - det([a2,j*]) + ... {+/-} det([an,j*]) ]

where [ai,j*] is the determinant of the [ai,j] after it has been moved to row 1 using row operations.

(10) We can use column operation I (I(kCi))to convert each matrix into this form:

det(A) = (-1)(j-1)[a1,jdet([ai,j**]) - a2,jdet([a2,1**] + ... ]

where ai,jdet([ai,j**]) = det([ai,j*])

(11) We can now plug in Corollary 3.1 above to complete the proof:

det(A) = (-1)i-1[ai,1det(Mi,1(A)) - ai,2det(Mi,2(A)) + ai,3det(Mi,3(A)) - ai,4det(Mi,4(A)) + ...] =

(-1)i-1[ai,1cofi,1(A) - ai,2cofi,2(A) + ai,3cofi,3(A) - ai,4cofi,4(A) + ... ]

QED

References

Thursday, June 07, 2007

Determinant of an n x n matrix

In a previous blog, I defined the determinant of a 2 x 2 matrix as:



In today's blog, I will offer a more general definition that is taken from Matrices and Linear Transformations by Charles G. Cullen.

Before offering this definition, I need to establish some notation for elementary operations of matrices. See here for review of these elementary operations.

Definition 1: A(kRi)

This is the matrix that results from multiplying a scalar k to the ith row of the matrix A.

Definition 2: A(Ri ↔ Rj)

This is the matrix that results from interchanging row i with row j in the matrix A.

Definition 3: A(kRj + Ri)

This is the matrix that results from adding (a scalar k * row j) to row i in the matrix A.

Now, I can use these definitions to offer a more general definition of the determinant of an n x n matrix:

Definition 4: Determinant of an n x n matrix A det(A)

A function is said to be the determinant of an n x n matrix if:
(I) det(AB) = det(A)*det(B)
(II) det(I(kR1)) = k

NOTE: I(kR1) is the identity matrix In with k multiplied to row 1 in I. [See here for details on how modifying the identity matrix and how this relates to elementary operations.]

For example, I(5R1) =



since for identity I2:




Now, from our previous result, we have shown that the above rule applies to 2 x 2 matrices.

Lemma 1: 2 x 2 matrices are determinants by Definition 4 above

Proof:

(1) Let A, B be 2 x 2 matrices.

(2) det(AB) = det(A)det(B) [See Lemma 1 here]

(3) I(kR1) =



(4) Using the definition of determinant for 2 x 2 matrices gives us:

k*1 - 0*0 = k

(5) Thus, we have shown that our proposed definition is consistent with the previous definition for 2 x 2 matrices.

QED

Lemma 2: I(kRj) = I(Rj ↔ R1)I(kR1)I(Rj ↔ R1)

Proof:

(1) I =




(2) I(Rj ↔ R1) =



(3) I(kR1)I(Rj ↔ R1)



(4) I(Rj ↔ R1)I(kR1)I(Rj ↔ R1)



QED

Lemma 3: I(Rj ↔ R1)I(Rj ↔ R1) = I.

Proof:

(1) I =



(2) I(Rj ↔ R1) =



(3) I(Rj ↔ R1)I(Rj ↔ R1) =



QED

Lemma 4: det(I(kRj)) = k

Proof:

(1) det(I(kRj)) = det( I(Rj ↔ R1)I(kR1)I(Rj ↔ R1) ) [See Lemma 2 above]

(2) det( I(Rj ↔ R1)I(kR1)I(Rj ↔ R1) ) = det( I(Rj ↔ R1) )det( I(kR1) )det( I(Rj ↔ R1) ) [See Definition 4 above]

(3) det( I(Rj ↔ R1) )det( I(kR1) )det( I(Rj ↔ R1) ) = det( I(kR1) )det( I(Rj ↔ R1) )det( I(Rj ↔ R1) ) [from the Commutativity of scalars]

(4) det( I(kR1) )det( I(Rj ↔ R1) )det( I(Rj ↔ R1) ) = det( I(kR1) )det( I(Rj ↔ R1)I(Rj ↔ R1) ) [ See Definition 4 above]

(5) det( I(kR1) )det( I(Rj ↔ R1)I(Rj ↔ R1) ) = det( I(kR1) )det(I) [See Lemma 3 above]

(6) det( I(kR1) )det(I) = det(I(kR1))det(I(1R1))

(7) det( I(kR1))det(I(1R1)) = k*1 = k [See Definition 4 above]

QED

Corollary 4.1: A = I(kRj)B → det(A) = k*det(B)

Proof:

(1) A = I(kRj)B

(2) det(A) = det(I(kRj)B) = det(I(kRj))det(B) = k*det(B) [See Lemma 4 above]

QED

Corollary 4.2: det(I(0Ri)) = 0

Proof:

(1) det(I(0Ri)) = 0 [This follows directly from Lemma 4 above for k=0]

QED

Corollary 4.3: det(I) = 1

Proof:

(1) det(I) = det(I(1Ri)) = 1. [This follows directly from Lemma 4 above for k = 1]

QED

Lemma 5: I(kRj + Ri) = I(k-1Rj)I(1Rj + Ri)I(kRj)

Proof:

(1) I(kRj + Ri) =



(2) I(kRj) =




(3) I(1Rj + Ri)I(kRj) =



(4) I(k-1Rj)I(1Rj + Ri)I(kRj) =




QED

Lemma 6: I(k-1Rj)I(kRj) = I

Proof:

(1) I(kRj) =



(2) I(k-1Rj)I(kRj) =



QED

Lemma 7: det(I(kRj + Ri)) = 1

Proof:

(1) Assume k = 0

(2) det(I(kRj + Ri)) = det(I(0Rj + Ri)) = det(I) = det(I(kR1)) = 1 [See Definition 4 above]

(3) Assume k ≠ 0

(4) det(I(kRj + Ri)) =det( I(k-1Rj)I(1Rj + Ri)I(kRj) ) [See Lemma 5 above]

(5) det( I(k-1Rj)I(1Rj + Ri)I(kRj) ) = det( I(k-1Rj) )det( I(1Rj + Ri) )det( I(kRj) ) [See Definition 4 above]

(6) det( I(k-1Rj) )det( I(1Rj + Ri) )det( I(kRj) ) = det( I(1Rj + Ri) )det( I(k-1Rj) )det( I(kRj) ) [from the Commutativity of scalars]

(7) det( I(1Rj + Ri) )det( I(k-1Rj) )det( I(kRj) ) = det( I(1Rj + Ri) )det( I(k-1Rj) I(kRj) ) [see Definition 4 above]

(8) det( I(1Rj + Ri) )det( I(k-1Rj) I(kRj) ) = det( I(1Rj + Ri) )det( I ) [See Lemma 6 above]

(9) det( I(1Rj + Ri) )det( I ) = det( I(1Rj + Ri) I) = det(I(1Rj + Ri)) [See Definition 4 above]

(10) So, we have shown that: det(I(kRj + Ri)) = det(I(1Rj + Ri))

(11) Now, (I(1Rj + Ri))2 = (I(1Rj + Ri))(I(1Rj + Ri)) = I(2Rj + Ri)

(12) But for k=2, we have:

det( (I(1Rj + Ri))(I(1Rj + Ri))) = det(I(2Rj + Ri) ) = det(I(1Rj + Ri))

(13) Now, this can only be true if det(I(1Rj + Ri)) = 0 or det(I(1Rj + Ri)) = 1.

(14) We note that I(1Rj + Ri)I(-1Rj + Ri) = I.

(15) Since det(I) = 1, it follows that det(I(1Rj + Ri)I(-1Rj + Ri)) = 1 so it is impossible for det(I(1Rj + Ri)) = 0.

(16) Therefore, det(I(1Rj + Ri)) = 1.

QED

Corollary 7.1: A = I(kRj + Ri)B → det(A) = det(B)

Proof:

(1) A = I(kRj + Ri)B

(2) det(A) = det(I(kRj + Ri)B) = det(I(kRj + Ri))det(B) = 1*det(B) = det(B) [See Lemma 7 above]

QED

Lemma 8: I(Ri ↔ Rj) = I(-Ri)I(1Ri + Rj)I(-Rj + Ri)I(1Ri + Rj)

Proof:

(1) I(Ri ↔ Rj) =



(2) I(1Ri + Rj) =



(3) I(-Rj + Ri)I(1Ri + Rj) =



(4) I(1Ri + Rj)I(-Rj + Ri)I(1Ri + Rj) =



(5) I(-Ri)I(1Ri + Rj)I(-Rj + Ri)I(1Ri + Rj) =



QED

Lemma 9: det(I(Ri ↔ Rj)) = -1.

Proof:

(1) det( I(Ri ↔ Rj) ) = det( I(-Ri)I(1Ri + Rj)I(-Rj + Ri)I(1Ri + Rj) ) [See Lemma 8 above]

(2) det( I(-Ri)I(1Ri + Rj)I(-Rj + Ri)I(1Ri + Rj) ) = det( I(-Ri) )det( I(1Ri + Rj) )det( I(-Rj + Ri) )det( I(1Ri + Rj) ) [See Definition 4 above]

(3) det( I(-Ri) ) = -1 [See Lemma 4 above]

(4) det( I(1Ri + Rj) ) = det( I(-Rj + Ri) ) = 1 [See Lemma 7 above]

(5) So, det( I(-Ri) )det( I(1Ri + Rj) )det( I(-Rj + Ri) )det( I(1Ri + Rj) ) = (-1)*(1)*(1)*(1) = -1.

QED

Corollary 9.1: A =I(Ri ↔ Rj)B → det(A) = -det(B).

Proof:

(1) A =I(Ri ↔ Rj)B

(2) det(A) = det(I(Ri ↔ Rj)B) = det(I(Ri ↔ Rj))det(B) = (-1)*det(B) = -det(B)

QED

Lemma 10:

Let Ei be an elementary matrix

Let A =



then:

det(A) = det(Ei)

Proof:

(1) There are only three types of elementary matrices so this proof is complete if I show it is true for each type.

(2) Type I: Ei = I(kRi)

(a) From Lemma 4 above, det(Ei) = k

(b) A, in this case, is equal to I(kRi+1)

(c) Using Lemma 4 above, det(A) = k

(3) Type II: Ei = I(kRj + Ri)

(a) From Lemma 7 above, det(Ei) = 1

(b) A, in this case, is equal to I(kRj+1 + Ri+1)

(c) Using Lemma 7 above, det(A) = 1.

(4) Type III: Ei = I(Ri ↔ Rj)

(a) From Lemma 9 above, det(Ei) = -1.

(b) A, in this case, is equal to I(Ri+1 ↔ Rj+1)

(c) Using Lemma 9 above, det(A) = -1

QED

References