Throughout our narrative, we have tried, as much as possible, to avoid the language of matrices. This was, in part, to show that the structures that present themselves in the course of a tensorial analysis can be handled without the use of matrices. Scalar variants of any order can be thought of simply as indexed lists of numbers that need not be organized into tables. For example, it has been sufficient to summarize the $27$ elements of the Christoffel symbol $\Gamma_{jk}^{i}$ in cylindrical coordinates by listing its three nonzero elements This economic way of capturing $\Gamma_{jk}^{i}$ proved quite effective in helping us interpret the formula for the Laplacian in cylindrical coordinates, where That calculation was a microcosm of the general realization that Tensor Calculus does not need Matrix Algebra.

Yet, a few of the distinct advantages of Matrix Algebra are impossible to ignore. First, the Matrix Algebra treatment of systems as whole indivisible units helps stimulate our algebraic intuition which is deeply ingrained in our mathematical culture. Consider, for example, the concept of the matrix inverse and the identity for the inverse of the product of two matrices: This identity is a concise expression of the fundamental idea that to reverse a combination of two actions, we must combine the individual reverse actions in the opposite order. For example, if our trip to work consists of traveling by train followed by traveling by bus then the trip home will require traveling back by bus followed by traveling back by train. The indicial notation does not have an effective way of capturing this elementary idea.

To prove the formula simply multiply $B^{-1}A^{-1}$ by $AB$. We have as we set out to show. In this concise derivation, the great utility of treating matrices as indivisible whole units is on full display. Furthermore, the formula can be used to derive the analogous equation for the inverse $\left( ABC\right) ^{-1}$ by the following chain of identities where the justification of each step is left as an exercise. In summary, From this derivation, it is clear that the inverse of the product of any number of matrices equals the product of the individual inverses in the opposite order. This derivation is not only straightforward, but also highly insightful from the algebraic point of view. Meanwhile, the same set of ideas cannot be effectively expressed in the indicial notation.

We should also call attention to another purely algebraic fact which we have relied upon in our narrative to a significant degree. Namely, it is the fact that It is because of this fact that we have been able to contract the product of the covariant metric tensor $Z_{ij}$ and the contravariant metric tensor $Z^{ij}$ on any valid combination of indices to produce the Kronecker delta.

Let us give the classical matrix-based proof of this fact which will once again demonstrate the utility of the algebraic way of thinking afforded to us by matrices. It follows from Linear Algebra considerations that if $A$ is a square matrix with linearly independent columns then there exists a unique matrix $R$ such that We will refer to $R$ as the "right inverse" of $A$. A square matrix with linearly independent columns has linearly independent rows and therefore there exists a unique matrix $L$, referred to as the left inverse of $A$, such that Our goal is to show that the left inverse and the right inverse are the same matrix, i.e. Subsequently, if we denote the common value of $L$ and $R$ by $B$, then the equivalence of $AR$ and $LA$ (both being the identity matrix) becomes the equivalence of $AB$ and $BA$.

To show that multiply both sides of the identity by $R$ on the right, i.e. Since $AR=I$, $LI=L$, and $IR=R$, we find as we set out to show. Note that we glossed over an application of the associative property of matrix multiplication. Indeed, multiplying $LA$ by $R$ yields $\left( LA\right) R$. By the associative property, we find that $\left( LA\right) R=L\left( AR\right)$ and the rest of the argument can proceed as described above. Thus, the equivalence of the left and right inverses is a direct consequence of the associative property of multiplication.

Much like the preceding proofs, this proof, too, cannot be effectively carried out in indicial notation. In fact, one could argue that this fundamental property of matrices cannot be discovered in indicial notation. This proof completes our discussion of the significant logical advantages of the matrix notation.

Another advantage of the matrix notation is that there are numerous software packages that implement various matrix operations. Consequently, any calculation formulated in terms of matrices, matrix products, and even more advanced matrix operations, can then be very effectively carried out by computers. Most readers can probably hardly remember the last time they multiplied two matrices by hand.

Finally, we must acknowledge the ubiquity of the language of matrices. Matrices represent our go-to way of visualizing data, economizing notation, and injecting algebraic structure into otherwise unstructured frameworks in a way that enables the leveraging of ideas from Linear Algebra.

Tensor Calculus which, as we stated, does not need matrices, is a case in point. After all, it is practically impossible not to think of first- and second-order systems as matrices. We have consistently associated metric tensors with matrices, even though we preferred the phrase corresponds to to the word equal in order to preserve the logical separation between systems and matrices. For example, we would say that, in spherical coordinates, Consequently, in spherical coordinates, the dot product $\mathbf{U} \cdot\mathbf{V}$ of vectors $\mathbf{U}$ and $\mathbf{V}$ with components $U^{i}$ and $V^{i}$, given by the tensor identity can be expressed in the language of matrices as follows: This form was frequently mentioned in Chapter 10. To many readers it likely helped clarify the interpretation of the then-novel expression $Z_{ij}U^{i}V^{j}$.

Having just described the indisputable benefits of the language of matrices, we ought to reiterate some of the advantages of the tensor notation.

First, by virtue of treating systems not as whole indivisible units but as collections of individual elements, the tensor notation allows direct access to those elements. We have taken great advantage of this feature on numerous occasions, including the analysis of quadratic form minimization in Section 8.7 and in establishing the derivative of the volume element $\sqrt{Z}$ with respect to the coordinate $Z^{i}$ in Section 16.12.

Second, the tensor notation continues to work for systems of order greater than two. This is the case for a number of objects that we have already encountered, including the Christoffel symbol $\Gamma_{jk}^{i}$, the Levi-Civita symbols $\varepsilon_{ijk}$ and $\varepsilon^{ijk}$, and the Riemann-Christoffel tensor $R_{klij}$. In order for an identity to be beyond the scope of the matrix notation, it need not be as complicated as the celebrated tensor equation Even an identity involving second-order systems may require the tensor notation. For example, the identity requires the tensor notation since it has four live indices.

Last but not least, the tensor notation has its own unique way of inspiring algebraic intuition. In particular, the natural placement of indices always accurately predicts the way an object transforms under a change of coordinates and suggests the ways in which it can be meaningfully combined with other objects. This feature of the tensor notation will be showcased throughout this Chapter.

In Section 2.7, we contrasted our approach to vectors to that often taken by Linear Algebra. According to our approach, vectors are directed segments, subject to addition according to the tip-to-tail rule and multiplication by numbers. It can be demonstrated that these operations satisfy a number of desirable properties, such as associativity and distributivity. In Linear Algebra, vectors are defined as generic objects subject to abstract operations of addition and multiplication by numbers that, by definition, satisfy the same set of desirable properties. That makes geometric vectors, i.e. directed segments, a special case of the abstract vector as defined by Linear Algebra. To distinguish between the two categories of vectors, we have used bold capital letters, such as $\mathbf{U}$, $\mathbf{V}$, and $\mathbf{W}$, for geometric vectors, and bold lowercase letters, such as $\mathbf{x}$, $\mathbf{y}$, and $\mathbf{z}$, for vectors in the sense of Linear Algebra.

Tensor Calculus and Linear Algebra take similarly different approaches to the related concepts of the dot product and inner product. The dot product of two geometric vectors $\mathbf{U}$ and $\mathbf{V}$ is defined by the equation It can be demonstrated that the dot product satisfies commutativity, and distributivity The distributive property can also be referred to as linearity and is often described by stating that the dot product is linear in each argument. It is also obvious that the dot product of a nonzero vector $\mathbf{U}$ with itself is positive, i.e. This property, known as positive definiteness, is hardly worthy of mention for geometric vectors but becomes part of the definition for generic vectors.

The Linear Algebra concept of an inner product is an axiomatic adaptation of the classical dot product. An inner product is an operation that takes two vectors and produces a number. The inner product of vectors $\mathbf{x}$ and $\mathbf{y}$ is denoted by $\left( \mathbf{x} ,\mathbf{y}\right)$ and is defined by commutativity distributivity, also known as linearity, and positive definiteness A vector space endowed with an inner product is called a Euclidean space -- so close is the analogy with Euclidean spaces as we introduced them in Chapter 2.

The final difference between our approach to vectors and that of Linear Algebra is the way in which a basis emerges. In Tensor Calculus, a coordinate system is chosen arbitrarily and the covariant basis $\mathbf{Z}_{i}$ is constructed by differentiating the position vector function $\mathbf{R}\left( Z\right)$ with respect to the coordinate $Z^{i}$, i.e. Thus, the covariant basis (at a given point) is specific to the chosen coordinate system. The dependence of $\mathbf{Z}_{i}$ on the choice of coordinates is underscored by the term variant. In Linear Algebra, a basis $\mathbf{b}_{i}$ is arbitrarily selected. Any complete linearly independent set of vectors represents a legitimate basis. Thus, in both approaches we can talk about a change of basis, although in Tensor Calculus, a change of basis is induced by a change of coordinates.

Having contrasted the origins of the key concepts, we must now point out that their uses in the two subjects are almost identical. Tensor Calculus and Linear Algebra share the concept of the component space in which analysis is performed in terms of the components of vectors rather than vectors themselves. For example, with the help of the covariant metric tensor $Z_{ij}$ defined by the component space expression for the dot product $\mathbf{U}\cdot\mathbf{V}$ of vectors $\mathbf{U}$ and $\mathbf{V}$ with components $U^{i}$ and $V^{i}$ reads Similarly, the inner product matrix $M$, whose entries $M_{ij}$ are can be used to express the inner product $\left( \mathbf{x,y}\right)$ of vectors $\mathbf{x}$ and $\mathbf{y}$ with components $x^{i}$ and $y^{i}$ as follows: In the language of matrices, which is one of the topics discussed in this Chapter, the same operation is captured by the equation where $x$ and $y$ are the column matrices consisting of the elements $x^{i}$ and $y^{i}$.

The covariant metric tensor $Z_{ij}$ and matrix $M$ are essentially the same object, differing only in notation and the terminology used to describe them. We will therefore use $M$ in many of the same ways that we have used $Z_{ij}$, including utilizing the symbol $M^{ij}$ to denote the entries of $M^{-1}$, and performing index juggling by contracting with $M_{ij}$ and $M^{ij}$.

As we have already mentioned, only first- and second-order systems can be effectively represented by matrices. As a matter of convention, let us agree to represent first-order systems by $n\times1$ matrices. Thus, a system $x^{i}$ in a three-dimensional space will be represented by the column matrix We could have also represented $x^{i}$ by a row matrix $\left[ x^{1}\ \ x^{2}\ \ x^{3}\right]$. However, limiting ourselves to column matrices only will help reduce the number of possible matrix representations of contractions.

The flavor of the index, whether it indicates a tensor property of the corresponding variant or is used for convenience, has no bearing on the matrix representation of a system. Thus, a system $x_{i}$ is also represented by a column matrix, i.e.

Second-order systems correspond to square $n\times n$ matrices. Note that in Chapter 8, we encountered systems that correspond to rectangular matrices. As a matter of fact, systems of that sort will begin to arise naturally when we study embedded surfaces in the next volume. Nevertheless, in this Chapter, we will limit our focus to square matrices. However, all points made in this Chapter will remain valid for systems corresponding to rectangular matrices.

As we have done throughout the book, and as the most commonly accepted convention dictates, the first index of the system corresponds to the row the element is in while the second index corresponds to the column. Thus, it is essential to have complete clarity as to the order of indices. For systems with two superscripts or two subscripts, the order is obvious. For mixed systems, i.e. systems with one superscript and one subscript, the order of the indices is usually indicated by the dot placeholder technique introduced in Section 7.2. For example, in the symbols and the dot makes it clear that $i$ is the first index and $j$ is second.

Interestingly, we have not seen the dot placeholder that much in our narrative so far. The reason for this is that the only mixed second-order system that we have consistently encountered is the Kronecker delta $\delta_{j}^{i}$ which corresponds to the symmetric matrix and, therefore, the order of the indices does not matter. However, as we will describe below in Section 19.5, this exception can be made only for symmetric systems, in the sense of the term symmetric that differs from that for matrices.

Another category of mixed second-order systems for which the use of the placeholder is not required is the Jacobians $J_{i^{\prime}}^{i}$ and $J_{i}^{i^{\prime}}$, for which we can simply agree that the superscript is first and the subscript is second. What makes this convention reliable in this case is the fact that the indices of a Jacobian are never juggled. In other words, the symbol $J_{i^{\prime}}^{i}$ is never used to represent the combination $J_{j}^{j^{\prime}}Z_{i^{\prime}j^{\prime}}Z^{ij}$. Had index juggling been allowed, the symbol $J_{i^{\prime}}^{i}$ would be ambiguous and the use of the dot placeholder would be in order. More generally, the possibility of index juggling is the very reason for not being able to use the flavors of indices as a mechanism for determining their order.

Any combination of contractions involving first- and second-order systems can be represented by matrix multiplication. In this Section, we will review the basic mechanics of matrix multiplication and subsequently show how some of the most common contractions can be expressed by matrix products.

Consider three matrices $A$, $B$, and $C$ with entries $A_{ij}$, $B_{ij}$ and $C_{ij}$. Suppose that $A$ is an $l\times m$ matrix, $B$ is $m\times n$, and $C$ is $l\times n$. Then, by definition, $C$ is the product of $A$ and $B$, i.e. if Note that we are following the Linear Algebra tradition of using only subscripts to reference the individual entries of a matrix.

An essential element of matrix multiplication is that the summation takes place over the second index of $A$ and the first index of $B$. Since the first index indicates the row of the entry and the second indicates the column, matrix multiplication combines the $i$-th row of $A$ with the $j$-th column of $B$ to produce $C_{ij}$, i.e. the entry in the $i$-th row and $j$-th column of $C$. These mechanics are illustrated in the following figure: Thus, the mechanics of matrix multiplication are very rigid and it is up to us to use the two available "levers" -- the order of the operands and the transpose -- to make sure the products reflect the contractions in a given tensor expression. We will now go through a series of examples converting contractions to matrix products in increasing order of complexity.

Let us start with the contraction that represents the inner product of vectors $\mathbf{x}$ and $\mathbf{y}$. Since $x_{i}$ corresponds to and $y^{i}$ corresponds to the combination $x_{i}y^{i}$ corresponds either to In either case, the matrix product involves the transpose of one of the matrices. We will find this to be the case in the more complicated situations as well.

Note that the tensor product $x_{i}y^{j}$ corresponds to the matrix product Thus, $x_{i}y^{i}$ can also be captured evaluating the trace of the resulting matrix, i.e.

Let us now turn our attention to the contraction involving a second-order system $A_{\cdot j}^{i}$ and a first-order system $x^{j}$. If $A$ is the matrix corresponding to $A_{\cdot j}^{i}$ and $x$ is the matrix corresponding to $x^{i}$, then Note that the convention that the first index indicates the row while the second indicates the column is essential for reaching this conclusion. Also note that the alternative product $x^{T}A^{T}$ results in the same values, although arranged into a $1\times n$ row matrix. Therefore, by our convention, where first-order systems are represented by column matrices, $x^{T}A^{T}$ does not properly represent the resulting system.

As we already mentioned, the flavor of the indices has no bearing on the matrix representation. Therefore, the combinations which represent variations in the flavors of indices but not their order, are also represented by the product $Ax$.

The contraction on the other hand, is principally different in that it is the first index of $A_{\cdot j}^{i}$ that is engaged in the contraction. As a result, the only way to represent $A_{\cdot j}^{i}x_{i}$ by a matrix product is As before, the product $x^{T}A$ produces the same values as a row matrix and therefore does not properly represent the result.

Let us now express the transformation rules for first-order tensors in matrix form. Recall from Chapter 17, that we let Now suppose that $T^{i}$ is a contravariant tensor, i.e. Since first-order tensors correspond to column matrices, we will use lowercase letters to denote those matrices. If the matrix $x$ represents $T^{i}$ and $x^{\prime}$ represents $T^{i^{\prime}}$, then Similarly, if $T_{i}$ is a covariant tensor, i.e. then its matrix representations $x$ and $x^{\prime}$ are related by the equation

A contraction of two second-order systems produces another second-order system. Thus we expect that a tensor identity featuring such a contraction corresponds to one of the many variations of the matrix equation that differ by the order of $A$ and $B$ and their transposition. In fact, we can construct $16$ different variations, such as and so on. The fact that makes half of the $16$ variations equivalent. For example, are equivalent as can be seen by taking the transpose of both sides of the first equation. This reduces the number of truly distinct equations to $8$ and also means that every indicial relationship can be captured in two different, albeit equivalent, ways.

Recall, once again, that the flavors of indices have no bearing on the corresponding matrix representations. Thus, we will choose the placements of indices completely at will.

Consider three second-order systems represented by the matrices $A$, $B$, and $C$. For our first example, consider the relationship This relationship very clearly corresponds to the matrix identity However, if the order of the indices on $C_{\cdot j}^{i}$ is switched,i.e. then the corresponding matrix identity is or, equivalently,

As the above two examples illustrate, the question of representing the contraction $A_{\cdot k}^{i}B_{\cdot j}^{k}$ by a matrix product has two legitimate answers: $AB$ and $B^{T}A^{T}$, even though these expressions result in two different matrices. The two different interpretations are possible because the combination $A_{\cdot k}^{i}B_{\cdot j}^{k}$, on its own, does not give us an indication of the order of the indices in the resulting system. If, in the result, $i$ is the first index and $j$ is the second, then $A_{\cdot k}^{i}B_{\cdot j}^{k}$ corresponds to $AB$. Otherwise, $A_{\cdot k}^{i}B_{\cdot j}^{k}$ corresponds to $B^{T}A^{T}$.

For another example, consider the equation This contraction takes place on the second index of $A_{\cdot k}^{i}$ and the second index of $B_{j}^{\cdot k}$. The entries in the matrices $A$ and $B$ involved in the contraction for $i=2$ and $j=3$ are illustrated in the following figure: Clearly, this arrangement does not correspond to a valid matrix product. In order to invoke matrix multiplication, the matrix $B$ needs to be transposed. Thus, corresponds to

Finally, consider the identity The summation on the right takes place on the first index of $A_{k}^{\cdot i}$ and the first index $B_{\cdot j}^{k}$, which suggests the combination $A^{T}B$. Additionally, as indicated by the symbol $C_{j}^{\cdot i}$, the result of the contraction must be arranged in such a way that $j$ is the first index and $i$ is the second. Thus, the matrix form of the identity must use $C^{T}$ instead of $C$. In summary, the above relationship corresponds to the matrix identity or, equivalently,

Let us now express the transformation rules for second-order tensors in matrix form. Suppose that $T^{ij}$, $T_{\cdot j}^{i}$, and $T_{ij}$ are tensors of the type indicated by their indicial signatures, i.e. If the symbols $T$ and $T^{\prime}$ represent the matrices corresponding to the alternative manifestations of each of the three tensors, then the transformation rules in the matrix notation read

Two systems $A_{ij}$ and $B_{ij}$, related by the identity correspond to matrices that are the transposes of each other. For example, if then The best way to convince yourself of this relationship is to recall the technique of unpacking described in Section 7.4. For example, $B_{32}$ equals $A_{23}$ which equals $8$. In other words, the entry in the $3^{\text{rd}}$ row and $2^{\text{nd}}$ column of the matrix corresponding to $B_{ij}$ is $8$ -- the same as the entry in the $2^{\text{nd}}$ row and $3^{\text{rd}}$ column of the matrix corresponding to $A_{ij}$. Thus, the two matrices are, indeed, the transposes of each other.

The same statement holds for systems $A^{ij}$ and $B^{ij}$ with superscripts. If then the two systems correspond to matrices that are the transposes of each other.

Similarly, two mixed systems $A_{\cdot j}^{i}$ and $B_{i}^{\cdot j}$ related by the identity correspond to matrices that are the transposes of each other. It is noteworthy, however, that in order for this relationship to hold, the order in which the superscript and the subscript appear in the two systems must be reversed.

A system $A_{ij}$ is called symmetric if it satisfies the identity Note that a symmetric system $A_{ij}$ corresponds to a symmetric matrix. Recall that a matrix $A$ is symmetric if it equals its transpose, i.e. The term symmetric makes sense since the values of the entries exhibits a mirror symmetry with respect to the main diagonal, e.g.

Similarly, a system $A^{ij}$ with two superscripts is called symmetric if Such a system, too, corresponds to a symmetric matrix.

Interestingly, the concept of symmetry appears to be problematic for a mixed system $A_{\cdot j}^{i}$. The would-be definition of symmetry is invalid on the notational level. This should give us serious pause. Over the course of our narrative, we have learned to trust the intimation given to us by the tensor notation. We must, therefore, accept the fact that the concept of symmetry cannot be applied to a mixed system in the above form. If we ignore what the notation is telling us, and call a mixed system symmetric if it corresponds to a symmetric matrix, we will run into contradictions later on. Specifically, suppose that a "symmetric" $A_{\cdot j}^{i}$ is the manifestation of a tensor in some coordinate system or, to use the language of Linear Algebra, the manifestation of a tensor with respect to a particular basis. Then $A$ transforms under a change of coordinates (or a change of basis) according to the rule Thus, if $A_{\cdot j}^{i}$ corresponds to a symmetric matrix, then $A_{\cdot j^{\prime}}^{i^{\prime}}$ will most likely not. For a simple illustration, note that for a symmetric matrix the combination equals which is no longer symmetric. Thus, when a mixed system $A_{\cdot j}^{i}$ corresponds to a symmetric matrix, it is as much a characterization of the particular coordinate system (or basis) as it is of $A_{\cdot j}^{i}$ itself. As we mentioned earlier, the Kronecker delta $\delta_{j}^{i}$ is the sole exception to this rule because it corresponds to the identity matrix in all coordinate systems.

Fortunately, the tensor notation not only warns us of a potential problem, but also presents us with a clear path forward. Namely, no rules of the tensor notation would be violated if we called a system $A_{\cdot j}^{i}$ symmetric when This works on the notational level, but what is the system $A_{j}^{\cdot i}$ and how is it related to $A_{\cdot j}^{i}$? The tensor framework provides the answer to this question, as well: $A_{j}^{\cdot i}$ is $A_{\cdot j}^{i}$ with a lowered first index and a raised second index. In other words, where we are using the inner product matrix (or, in the language of tensors, the metric tensor) $M$ for juggling indices. This enables us to define what it means for a mixed system $A_{\cdot j}^{i}$ to be symmetric, albeit not in isolation but in the context of an inner product. In other words, in the context of the associated Euclidean space framework.

Specifically, a mixed system $A_{\cdot j}^{i}$ is said to be symmetric if where $A_{j}^{\cdot i}$ is the result of index juggling on $A_{\cdot j}^{i}$.

It is worth reiterating that this new definition of symmetry does not correspond to the concept of symmetry as applied to matrices. In particular, the matrix corresponding to a system $A_{\cdot j}^{i}$ that satisfies the identity above, will not be symmetric, except in very special circumstances. Also note that the identity indicates that the matrices corresponding to $A_{\cdot j}^{i}$ and $A_{j}^{\cdot i}$ are not equal, but rather the transposes of each other. Note, however, that for a system $A_{\cdot j}^{i}$ that satisfies this identity, the placeholder may be safely omitted, i.e. the symbol $A_{\cdot j}^{i}$ can be replaced with $A_{j}^{i}$. After all, the above identity tells us that it does not matter whether $A_{j}^{i}$ is interpreted as $A_{\cdot j}^{i}$ or $A_{j}^{\cdot i}$.

Also be reminded that, unlike systems with two subscripts or two superscripts, the concept of symmetry, as applied to mixed systems, requires the availability of an inner product, i.e. the context of a Euclidean space. To make the above definition more explicit, we could have written it in the form although this form seems to obfuscate, rather than clarify, the matter. However, this form does represent an explicit recipe for testing whether a system $A_{\cdot j}^{i}$ is symmetric. For example, suppose that We will now show that if the system then it is symmetric. In order to apply the definition note that the combination $A_{\cdot s}^{r}M_{rj}M^{si}$ corresponds to $MAM^{-1}$, i.e. The resulting matrix is the transpose of the matrix corresponding to $A_{\cdot j}^{i}$ proving that $A_{\cdot j}^{i}$ is symmetric.

The symmetry criterion can be simplified by applying a single index juggling step to the equation Lowering the index $i$ on both sides, we find Thus, $A_{\cdot j}^{i}$ is symmetric if the related system $A_{ij}$ is symmetric. This is a simpler criterion since a system with two superscripts is symmetric if it corresponds to a symmetric matrix. To illustrate this criterion with the example above, note that $A_{ij}=M_{ik}A_{\cdot j}^{k}$ which corresponds to the matrix product $MA$, i.e. Since the resulting matrix is symmetric, $A_{ij}$ is symmetric and therefore $A_{\cdot j}^{i}$ is symmetric.

The concept of a linear transformation is one of the most fundamental elements in Linear Algebra. Recall that a linear transformation $T$ can be represented by a matrix $A$ in the component space. In this regard, linear transformations and inner products are similar in that both are represented by matrices in the component space: the matrix $A$ in the case of linear transformations and the matrix $M$ in the case of inner products. However, this similitude appears stronger than it really is and the tensor notation alerts us to a fundamental difference between the two matrices.

A linear transformation $T$ maps one vector to another. The original vector is known as the preimage and the result of the transformation is known as the image. Denote the preimage by $\mathbf{x}$ and its image by $\mathbf{y}$, i.e. The role of the matrix $A$ is to convert the components $x^{i}$ of the preimage into the components $y^{j}$ of the image. Thus, in the tensor notation, the matrix $A$ naturally corresponds to a mixed system $A_{\cdot j}^{i}$ which enables us to write

An inner product, on the other hand, takes two vectors $\mathbf{x}$ and $\mathbf{y}$ as inputs and produces a scalar. This is why the matrix $M$ naturally appears with two subscripts, so as to enable the combination that results in $\left( \mathbf{x,y}\right)$.

Thus, under a change of basis, the matrices $A$ and $M$ transform by different rules. By the quotient theorem, discussed in Chapter 14, the variant $A_{\cdot j}^{i}$ is a tensor of the type indicated by its indicial signature and therefore transforms according to By the same theorem, the variant $M_{ij}$ is a tensor of the type indicated by its indicial signature and therefore transforms according to

If $A^{\prime}$ is the matrix representing the linear transformation $T$ in the alternative basis $\mathbf{b}_{i^{\prime}}$, then the equation tells us that where $J$ denotes the matrix corresponding to the Jacobian $J_{j^{\prime}} ^{j}$. This formula can be found in equation $\left( 11\right)$ of Chapter $2$ of I.M. Gelfand's Lectures on Linear Algebra.

Similarly, if $M^{\prime}$ represents the same inner product in the alternative basis $\mathbf{b}_{i^{\prime}}$, then the equation tells us that This formula can be found in equation $\left( 7\right)$ of Chapter $1$ of the same book.

Thus, it may be said that the matrix notation blurs the distinction between the component space representations of inner products -- or, more generally, bilinear forms -- and linear transformations since both concepts are represented by matrices. In fact, this similitude creates the temptation to consider a one-to-one correspondence between bilinear forms and linear transformation. Such a correspondence indeed exists but, in order to avoid internal contradictions, it needs to be carefully framed. The tensor notation can be instrumental in helping us establish the right correspondence -- the task to which we now turn.

An inner product is a special case of a more general operation known as a bilinear form $B\left( \mathbf{x},\mathbf{y}\right)$ defined as an operation that takes two vectors and produces a number subject only to linearity in each argument, i.e. and Thus, an inner product is a symmetric positive definite bilinear form, where the term symmetric refers to the commutative property. Rather than introduce a special letter to distinguish inner products from general bilinear forms, Linear Algebra simply drops the letter altogether for inner products resulting in the symbol $\left( \mathbf{x},\mathbf{y}\right)$.

Much like inner products, bilinear forms are represented by matrices in the component space. If $\mathbf{x}=x^{i}\mathbf{b}_{i}$ and $\mathbf{y} =y^{i}\mathbf{b}_{i}$, then, by linearity, Thus, if the matrix $B$ with entries $B_{ij}$ is defined by then or, in the language of matrices, Naturally, $B$ transforms from under a change of basis by the rule