Suppose and are fields with , and the field operations on being the same as those on .  In such a situation we say that is a subfield of , and that is an extension field of .  We will denote this relationship between and as and simply refer to as an extension.  For example, we may say that is an extension, as is a subfield of .  Notice that if , we may regard as an -vector space.  To see this, simply note that as is a field containing , all of the axioms for a vector space are immediately satisfied (clearly addition of elements in stays inside , multiplying an element of by an element of remains in , etc.).  The dimension of as an vector space is referred to as the degree of the extension and is denoted .  If the degree of is finite, we say that is a finite extension of .

As an example, notice that is an extension field of .  In particular, we may regard as an -vector space with basis , and so .  We could also regard as an extension of (and hence a vector space over) .  In doing so we have an infinite dimensional vector space, so .

Given a subset of , we denote by the smallest extension field of which lies in , but contains .  This is given by intersecting all fields satisfying both and .  We call the extension field of generated by .  In the event that is finite, we write in place of .  If is a singleton, we call a simple extension.  Notice that we may calculate a simple extension by simply taking all of the rational functions with coefficients in and evaluating them at .

(This is essentially saying the elements of are given by taking all sums, powers, multiples, and inverses of with the other elements of .)

Notice that even though is generated by one element, the degree of over may not be two.  That is, it’s tempting to assume that is a basis for as an -vector space, but this may not be the case.  For instance, consider as a subfield of .  In order for this to be a field, it must contain , which we can not express as a linear combination of , so our basis is in fact .

Last time we defined the eigenvalues and eigenvectors of a matrix, but didn’t really discuss how to actually calculate the eigenvalues or eigenvectors; we said that if it so happened that your matrix was similar to a diagonal matrix, the non-zero entries of the diagonal matrix were the eigenvalues, and the columns of the change-of-basis matrix were the eigenvectors.  Now we’re going to discuss how to find the eigenvalues using the matrix’s characteristic polynomial.

Notice that if is an eigenvalue of with associated eigenvector we have the following.

Of course, satisfies this equation, but that’s a trivial solution.  For any other, non-trivial, solution we’d require that is non-singular, and so .  Thus if is an eigenvalue of , we must have .

Now suppose that is such that .  Then there is a non-trivial solution to , so , and is an eigenvalue.  We’ve shown that is an eigenvalue of if and only if .  Furthermore, is a polynomial in (this is obvious if is , and inductively we can show that this is true for matrices).  This means that with the characteristic polynomial, the problem of finding eigenvalues is reduced to finding the roots of a polynomial.

As an example, suppose

Then the characterstic polynomial is

So we see that the eigenvalues are , , and (notice the last two are complex conjugates of one another).

Now, once we’ve found the eigenvalues, the next step is to find the eigenvectors.  Since

what we want is to find the nullspace of , since these are all the vectors that will take to zero.  In our particular example, for ,

Now we take the row-reduced echelon form of this matrix, since it shares the same null space:

This tells us that

So the eigenvectors associated with the eigenvalue are the multiples of .  We’d repeat the above process with to find the other eigenvectors.

Let’s suppose that is an matrix which is similar to a diagonal matrix, .  This means there is an invertible (change-of-basis) matrix such that

Now since is a change of basis matrix, each of its columns gives the coordinates to a basis vector of some basis.  Let’s call that basis and let through be the elements of that basis.  Now, if we take the above equation and multiply by on the right, notice that

That is, the -th column of is equal to the -th column of , which is just times the -th column of .  Since each column of is just a linear combination of the columns of , though, we have

This means that when we plug in the -th column of to the linear transformation represented by , we get back a multiple of that column.  Calling the linear transformation , we have that

.

Vectors such as whose image under is just a multiple of the vector are called eigenvectors of .  That multiple, the above, is called an eigenvalue of .  These eigenvectors and eigenvalues are associated with a particular linear transformation, so when we talk about the eigenvectors and eigenvalues of a matrix, we really mean the eigenvectors and eigenvalues of the transformation represented by that matrix.  Notice that this means that eigenvalues are independent of the chosen basis; since similar matrices represent the same transformation just with respect to different bases, similar matrices have the same eigenvalues.

We assumed that was similar to a diagonal matrix above, but this isn’t always true.  If is similar to a diagonal matrix, say , then as we’ve just shown, the columns of are eigenvectors of .  Since these form the columns of a non-singular matrix, the eigenvectors of form a basis for the vector space.  Also, if the eigenvectors of form a basis, let’s take those basis vectors as columns of .

So a matrix is diagonalizable (similar to a diagonal matrix) if and only if its eigenvectors form a basis for the vector space.

Let’s suppose we have a linear transformation on which performs the following:

Now, the matrix representation of this transformation with respect to the standard basis is clearly

But suppose we were to use a different basis for , like say .  We see that our transformation maps these basis vectors as follows:

Notice that with the vectors we have on both the left and the right above are the coordinates with respect to the standard basis.  We’d like to see what the matrix representing looks like with respect to the basis, so let’s convert the vectors on the right to -coordinates.  Recalling that a change of basis is simply a system of equations where the columns of the coefficient matrix are the coordinates of the basis vectors (and the inverse of this matrix if we want to go the other way), we have that

So with respect to our basis, the representation of is

We will denote this matrix as where the right-most subscript means that inputs are in -coordinates, and the left-most subscript means the outputs are in -coordinates as well.  Note that we could calculate by going from coordinates to standard coordinates, using the earlier matrix, then going back to -coordinates.  That is,

where refers to the -to-standard basis change of basis matrix.  For notational convenience, define the following

Then the above becomes

Any matrices and for which there exists a with satisfying this equation are called similar matrices.  Note that two matrices are similar if and only if they represent the same linear transformation, but with respect to different bases.  Also note that every non-singular matrix represents a change of basis matrix.  Similarity forms an equivalence relation on the set of square matrices / the set of linear transformations from a finite dimensional vector space to itself.

A common theme in mathematics is (or seems to be) looking at sets with a particular structure, and then looking at functions between those sets which preserve that structure.  In groups we have homomorphisms; in topological spaces we have continuous maps; in general categories we have morphisms.  In the particular case of vector spaces, though, there are two particular “structures” we want to preserve: vector addition and scalar multiplication.  The maps which preserve these are what we refer to as linear transformations.

Specifically, suppose and are vector spaces over the field .  A function is called a linear transformation if for all scalars and for all vectors we have

Note that because of this linearity, a linear transformation is completely determined by how it maps the basis vectors of the domain.  Suppose that is a basis for V.  Let be any vector in with .  We then have

.

So if we know each , we can figure out where any other vector will be sent by .  This does not mean that is necessarily a basis for the range, .  It could be that , in which case these vectors are linearly dependent and can’t both be in the basis.  We do have that span , however, so as long as they’re linearly independent they’ll form a basis.

The main thing we want to notice about linear transformations for right now is that if both and are finite dimensional, then a linear transformation can be represented as a matrix.  Suppose that is -dimensional with the basis mentioned above, and that is -dimensional with basis .  Note that the properties of matrix multiplication tell us that any matrix defines a linear transformation from to :

Now suppose is any other linear transformation.  Suppose that the coordinate vector of with respect to the basis is

Now let .  We then have

Thus a linear transformation between finite dimensional vector spaces can be represented as a matrix.  Notice that the entries of our matrix depend on our particular chosen bases: if one basis were altered, the matrix would change, even though the transformation is the same.  We will denote the matrix representing with respect to the and bases as

We’ve yet to describe a way to calculate determinants in any easy way; we’ve seen some nice properties, but still have to resort to writing a non-elementary matrix as a product of elementary matrices in order to calculate its determinant.  What we want to do now is describe a recursive procedure for calculating a determinant by looking at determinants of submatrices.  Let’s first agree to call the submatrix of with the -th row and -th column deleted the -minor of , which we’ll denote .  So, supposing

if we delete the first row and second column we have

Let’s also note that if the first row of is all zeros except for the first entry, the determinant of is simply the determinant of multiplied by that first entry.  That is,

To see this, first note that we can zero out the entries in the first column below the by performing a sequence of elementary row operations that don’t change the determinant.  Now we clearly have

Supposing we can write as a product of elementary matrices, , to calculate its determinant, we can then obtain the matrix above by looking at the product

Each of these is then an elementary matrix whose determinant is the same as the determinant of the associated matrix, so we have our result.

Now, applying the linearity we discussed last time,

Notice that we can can zero out the elements in the first column below giving us

In general, for the -th column, we want to do a series of column swaps bringing the -th column to the front of the matrix, but keeping the other columns in order.  (For this reason a single column swap won’t work, since that permutes the remaining columns.)  Each time we swap columns, the determinant is multiplied by -1.  If we move the -th column to the left by swapping with column , then with column , and so on, we perform swaps.  Thus

So in total we have

We could also perform this expansion along another row, but we’d have to perform some row swaps to move that row to the top first.  Supposing we decide to expand along the -th row, we’ll perform swaps, each time multiplying the determinant by -1.  We’d multiply our result above, then by .  Distributing that across our sum though, we note that

So, if we expand along the -th row, our formula becomes

Each term of the sum with the factor removed is called a cofactor; The -th cofactor, which we’ll denote is

The procedure we’ve just described is known as the Laplace expansion (or cofactor expansion) for the determinant, and so now we have a more efficient way of calculating determinants.  (Notice that we could expand along a column instead of a row: just repeat the above procedure on the transpose of the matrix.)

One easy consequence of our definition of determinant from last time is that any singular matrix must have determinant zero. Suppose is a singular matrix and that is the matrix which puts into row reduced form. Then we have

If is singular, once we put it in row reduced form it must have a row of zeros. We can now break up into a product of elementary matrices, one of which will have be of the form . We know that this matrix will have determinant zero, so the product will be zero, and thus . Likewise, if , we can write as a product of elementary row matrices and one will be , so is singular. Now we know that a matrix is singular if and only if its determinant is zero.

Suppose now that the first row of can be written as for some vectors and . We wish to show that

First suppose that the rows through form a linearly dependent set. Then our matrix is singular so has determinant zero. The determinants on the right in the above equation are zero too, so our we have our result.

Suppose now that through are linearly independent. We can then extend these to a basis for by adding a vector, call it . Then there exist scalars for such that

Some simple manipulations from last time give us the following.

And likewise,

Now we combine these results,

Since we can swap rows without altering the determinant, this result holds for any rows.  A similar argument shows the result also holds for columns.

I remember that when I took linear algebra, I had learned determinants in a very “algorithmic” sort of way; a determinant to me was a function defined on square matrices by a particular recursive procedure.  In Charles Cullen’s Matrices and Linear Transformations, however, he defines a determinant not by a rule, but by two properties which completely characterize the determinant.  A determinant, according to Cullen, is an function which satisfies the following.

1. For any two matrices and ,
2. The determinant of is .

That’s it.  That’s all you need to define the determinant.  Of course, now we have to worry about whether a function with these properties even exists or not, and if so, is that function is unique?  Before answering either of those questions, though, we need to establish that every matrix can be written as a product of elementary matrices (where by an elementary matrix we mean one which results in an elementary row (or column) operation when multiplied on the left (or right), including zeroing out a row or column).  To see this, recall that for every matrix there exist non-singular matrices and such that

where .  In the above, represents a sequence of elementary row operations, and gives a sequence of elementary column operations; and are the products of elementary matrices.  Since these are non-singular we have

The middle matrix is clearly a product of matrices of the form .  If we agree to also call such matrices elementary, then every square matrix is a product of elementary matrices.

Supposing we have a function satisfying the properties given above, we can calculate the determinant of an elementary matrix pretty easily.  In the following, means we multiply the -th row by in the identity matrix; means we swap rows and ; means we add times row to row .

First we’ll look at scalar multiples of a particular row:

(these are scalars in a field, so they commute)

Using similar identities we can show and .  Now we know the determinants for all elementary matrices.  Since every square matrix is a product of elementary matrices, and we can split the determinant up across a product of matrices, we can even calculate the determinant of any square matrix based solely on the two properties given earlier.  (Note this isn’t necessarily an efficient way to calculate the determinant, just a possible way.)

At this point it shouldn’t seem too surprising that the two properties above are all we need to define a determinant: every square matrix is a product of elementary matrices, and we can “massage” elementary matrices into a form whose determinant we can calculate.  In coming posts we’ll see how to expand this to define the Laplace / cofactor expansion for the determinant; the relationship between determinants and inverses; and Cramer’s rule, which tells us how to compute a single coordinate in the solution vector to a system of equations.

In any non-trivial vector space there will be several possible bases we could pick.  In , for instance, we could use or .  This first basis is known as the standard basis, and in general for an -dimensional vector space over , we’ll refer to as the standard basis, and will let denote the vector with a 1 in the -th position, and zeros elsewhere.

When we write down a vector like we implicitly mean that these are the coordinates to use with the vectors in the standard basis; these are the coefficients we multiply the basis vectors by to get describe our vector.

But how would we find the appropriate coordinates if we were to use as our basis?  Let’s suppose our coefficients are .  Then what we want to do is find the values of the such that

So what we have is a system of equations:

Notice that since the columns of this matrix form a basis for , the matrix is invertible, and so we can easily solve for the .

In general, if we have a basis , we will write

to mean that the are the coefficients of the vectors in our basis, and will let be the matrix converts the coordinates for the standard basis to coordinates in our basis.  Likewise, will be the matrix which converts coordinates from the basis to coordinates with the standard basis.

Generalizing on the argument above we see that we can calculate by simply using our basis vectors as our columns.  For we take the inverse of this matrix.

Now suppose we have two bases, and .  The matrix which will take coordinates with respect to the basis and convert them into coordinates for the basis is denoted .  One way to calculate is to convert our -coordinates into standard coordinates, and then convert those into -coordinates:

Alternatively, we could note that

(with the 1 in the -th spot)

(recall is the notation I use for the -th column of )

So the -th column of is simply -th vector from our basis, but in -coordinates.

In the last post we defined the column and row space of a matrix as the span of the columns (in the case of the column space) or rows (for the row space) of the matrix.  There’s a third important subspace of a matrix, called the null space, which is the set of all vectors which maps to zero.  That is, if we think of an matrix as a function from to , then the null space of , , is simply the kernel of the map.

The dimension of the null space is sometimes called the nullity of the matrix.  There’s an important relationship between the column space, row space, and null space which we’ll now state and prove: if is an matrix, then .

We begin by assuming is a basis for .  Since is a subspace of , we may extend to a basis for all of by adding properly chosen vectors.  Call these vectors so that is a basis for .  Since this is a basis, let and suppose

Now we multiply on the left by .  Since we have

where the last step follows from the fact each of are mapped to zero by .

Since our choice of was arbitrary, spans .  If we can now show that are linearly independent we’ll have that and have proven our theorem.

So now we want to find the such that

Now since is a basis for , we have that , so is a linearly independent set which spans , so where .  Since , we have our result.

Notice that if , clearly , so that .  If is non-singular, however, it has a trivial null space and so .  Combining this with the above theorem we have that if is non-singular,

Likewise, if is non-singular.  (Apply the previous argument with .)

Picking up from last time, we have if and are the non-singular matrices such that , then

So the dimension of the column space equals the dimension of the row space.  This common value is called the rank of , and is denoted .  For this reason the theorem we proved above is known as the rank-nullity theorem.