Let us begin with linear combinations, the fundamental algebraic building block of Linear Algebra. As a sum of products, a linear combination has the potential to be interpreted as a contraction but, first, an important adjustment needs to be made.

Consider the decomposition of a vector $\mathbf{U}$ as a linear combination of the basis vectors $\mathbf{b}_{1}$, $\mathbf{b}_{2}$, and $\mathbf{b}_{3}$ with coefficients $\alpha_{1}$, $\alpha_{2}$, and $\alpha_{3}$, i.e. Since both indices are subscripts, the above summation does not represent a valid contraction. In order to make it one, either the coefficients $\alpha_{i}$ or the basis vectors $\mathbf{b}_{i}$ must be enumerated by a superscript. We arbitrarily choose the coefficients. With the coefficients denoted by $\alpha^{1}$, $\alpha^{2}$, and $\alpha^{3}$, the linear combination becomes As a result, it is now a valid contraction. Invoking the summation convention, we can now express $\mathbf{U}$ by the remarkably compact equation

This example perfectly illustrates the guiding aspect of the tensor framework. Our decision to enumerate the coefficients $\alpha^{i}$ by a superscript was governed by nothing other than our desire to invoke the summation convention. However, in doing so, we have made a prediction of how the coefficients $\alpha^{i}$ transform under a change of basis. Crucially, our prediction is correct, as will be confirmed in Chapter 14. We have highlighted this surprising aspect of the tensor notation before but it is worth reiterating: it not only captures the properties of objects, but also predicts them.

We will frequently encounter the situation where the decomposition of one and the same vector with respect to one and the same basis is arrived at in two different ways. Since, as it is well known from elementary Linear Algebra, the decomposition is unique, the decomposition coefficients -- i.e. the components -- in the two alternatively-derived expansions must coincide. In other words, from the equality of two linear combinations we can conclude the equality of the coefficients. While this is a straightforward matter, the tensor notation captures linear combinations in such a compact manner that this logic may sometimes prove elusive. It is therefore worth illustrating with a few examples.

Suppose that the basis consists of the vectors $\mathbf{b}_{i}$. Then from we can conclude that To see how this conclusion is reached, unpack both sides of the former equation, i.e. With the equation in this form, it becomes clear that the Linear Algebra principle of equating expansion coefficients applies. Also, the unpacked form prevents you from slipping into the incorrect argument of "dividing" both sides of the identity $\alpha^{i}\mathbf{b}_{i}=\gamma^{i}\mathbf{b}_{i}$ by $\mathbf{b}_{i}$. Note that even as your experience with the tensor notation grows, you will not cease to practice unpacking. You will simply learn to imagine and process the unpacked form more quickly.

The equality of linear combinations may appear in more algebraically complicated forms. For example, from the identity we can conclude that Note that we need not concern ourselves with the interpretation of the combination $A_{j}^{i}\alpha^{j}$ in order to reach this conclusion.

In situations where the repeated indices enumerating the elements of the basis are different on the two sides of the equation, e.g. you will need to rename one of the pairs of the repeated indices so that the two pairs match. In the above example, switch the roles of $i$ and $j$ on the right, i.e. From this form of the identity, we are able to conclude that

I hope that the foregoing examples illustrate that while the tensor notation may require some practice, its compactness does not obfuscate the ideas that you may be used to seeing in the unpacked form.

In the near future, we will study transformation of systems under a change of coordinates. The corresponding topic in Linear Algebra is referred to as change of basis. In this Section, we will use the tensor notation to demonstrate that the matrices relating the two bases are the inverses of each other.

Suppose that $\mathbf{b}_{i}$ and $\mathbf{c}_{i}$ are two alternative bases. Let the system $B_{j}^{i}$ express the elements of the basis $\mathbf{b}_{j}$ with respect to $\mathbf{c}_{i}$, i.e. and $C_{j}^{i}$ express the elements of the basis $\mathbf{c}_{j}$ with respect to $\mathbf{b}_{i}$, i.e. In other words, $B_{j}^{i}$ is the component of $\mathbf{b}_{j}$ with respect to $\mathbf{c}_{i}$ and $C_{j}^{i}$ is the component of $\mathbf{c}_{j}$ with respect to $\mathbf{b}_{i}$. Because the systems $B_{j}^{i}$ and $C_{j}^{i}$ perform opposite conversions, we may anticipate that they are the inverses of each other in the matrix sense. This is, indeed, the case. Furthermore, this fact is naturally demonstrated in the tensor notation and will offer us an opportunity to practice combining indicial expressions.

The idea is to substitute the identity $\mathbf{c}_{j}=C_{j}^{i}\mathbf{b} _{i}$ into $\mathbf{b}_{j}=B_{j}^{i}\mathbf{c}_{i}$ and thus express the basis $\mathbf{b}_{i}$ in terms of itself. However, since the equation features $\mathbf{c}_{j}$ (with a $j$) while the equation features $\mathbf{c}_{i}$ (with an $i$), substitution is not possible until we properly coordinate the index names. In the identity rename $i$ into $k$ (in order to free up $i$ for the next step), i.e. and then $j$ into $i$, i.e. Next, substitute the above identity into which yields From here, one can proceed according to one of two approaches: the tensor-novice or the tensor-expert. We will describe both approaches. The tensor-novice approach analyzes the elementary operations that take place under the hood. The tensor-expert approach showcases the elegance and the efficiency of the tensor notation.

For the tensor-novice approach, begin by unpacking the identity on the dummy index $k$, i.e. and subsequently on the live index $j$ which, as we discussed in the previous Chapter, expands the equation into a set of three, i.e. where the identities are still not fully unpacked: each equation contains three un-unpacked contractions on the index $i$.

The above identities express the elements of the basis $\mathbf{b}_{i}$ with respect to itself. From elementary Linear Algebra, we know that there is a unique way of doing so and that is Since equality of linear combinations implies equality of coefficients, we arrive at the following nine identities: Finally, note that with the help of the indicial notation and the Kronecker delta $\delta_{j}^{k}$, these nine identities can be captured by the single tensor equation

We are now able to formulate our conclusion. Since the Kronecker delta corresponds to the identity matrix, this identity states precisely that the matrices corresponding to $B_{j}^{i}$ and $C_{j}^{i}$ are the inverses of the other, as we set out to show. The advantage of the approach that we have just described is that it exposed almost every elementary low-level detail of the calculation. On the other hand, its exhaustive details may have obscured the simple algebraic structure of the equations. This approach therefore cannot be considered an effective use of the tensor notation.

The tensor-expert approach brings out the elegance of the argument by keeping the equations compact. In the equation replace $\mathbf{b}_{j}$ with the equivalent expression $\delta_{j} ^{k}\mathbf{b}_{k}$. The resulting identity shows the equivalence of the two linear combinations with respect to the same basis, which is precisely the situation we described earlier. Equating the coefficients, we find Switching the sides, i.e. we immediately recognize it as precisely the equation we set out to prove.

Recall from Chapter 2, that for two vectors $\mathbf{U}$ and $\mathbf{V}$, the matrix form of the component space expression for $\mathbf{U}\cdot\mathbf{V}$ reads where $U$ and $V$ are the $n\times1$ matrices representing the components of $\mathbf{U}$ and $\mathbf{V}$, and $M$ is the $n\times n$ matrix of pairwise dot products of the basis vectors $\mathbf{b}_{i}$, i.e. As elegant as the equation may be, its tensor analogue is every bit as aesthetically pleasing. In fact, with the introduction of index juggling in Chapter 11, it will become even more so.

The entries of the matrix $M$ are naturally enumerated by a pair of subscripts, i.e. Once again, the choice of subscripts is dictated strictly by the rules of the tensor notation rather than some a priori insight into the nature of the system $M_{ij}$ or knowledge of how it transforms under a change of basis. The tensor notation stipulates that all terms in an identity must have matching indicial signatures. Thus, the fact that the expression $\mathbf{b}_{i}\cdot\mathbf{b}_{j}$ has two subscripts means that we have no choice but to enumerate the entries of $M$ by subscripts. The fact that this choice accurately predicts the manner in which $M_{ij}$ transforms under a change of basis, which will be confirmed in Chapter 14, is simply yet another illustration of the great predictive ability of the tensor notation.

Next, following the convention adopted earlier, which was also dictated by the rules of the tensor notation, enumerate the components $U^{i}$ and $V^{i}$ of $\mathbf{U}$ and $\mathbf{V}$ by superscripts, i.e. We will now show that in terms of $M_{ij}$, $U^{i}$, and $V^{i}$, the expression for $\mathbf{U}\cdot\mathbf{V}$ reads First, let us convince ourselves that this formula is correct by fully unpacking the contractions on the right that represent a sum of nine terms, i.e. This equations is identical to the equation derived in Chapter 2, except for the fact that the components of the vectors are now enumerated by superscripts. Thus, the combination $M_{ij}U^{i}V^{j}$ indeed represents the dot product $\mathbf{U} \cdot\mathbf{V}$.

Thanks to the summation convention, the expression $M_{ij}U^{i}V^{j}$ is every bit as compact as its matrix analogue $U^{T}MV$. Furthermore, the indicial form offers a few notational advantages over the matrix form. First, it does not require the operation of the transpose. As we discussed in the previous Chapter, this speaks to the extreme economy of operations in Tensor Calculus. Second, as we also discussed in the previous Chapter, the order of the multiplicative terms in the expression $M_{ij}U^{i}V^{j}$ is immaterial. Third, the expression $M_{ij}U^{i}V^{j}$ gives access to the individual entries of the matrix $M$ as well as the individual components of $\mathbf{U}$ and $\mathbf{V}$. This will prove to be of crucial advantage in numerous applications, including quadratic form minimization discussed below. Finally, note that with the help of index juggling introduced in Chapter 11, the expression $M_{ij}U^{i}V^{j}$ will be supplanted by the remarkably compact equivalent $U_{i}V^{i}$ which, while valid for all bases, exhibits the utmost simplicity of the dot product expressed with respect to an orthonormal basis.

Next, let us re-derive the identity strictly in the tensor notation, as opposed to by matching up $M_{ij} U^{i}V^{j}$ with a previously established result. On the one hand, the upcoming calculation is almost too simple to be called a derivation. On the other hand, it does require a careful manipulation of indices and will therefore serve as a worthwhile exercise of the tensor notation.

Start with the decompositions of $\mathbf{U}$ and $\mathbf{V}$ in terms of the basis $\mathbf{b}_{i}$, i.e. Since we are about to combine these expressions in a single product, they cannot both use the index $i$ -- otherwise, we would end up with an invalid combination $U^{i}\mathbf{b}_{i}\cdot V^{i}\mathbf{b}_{i}$. Thus, we will keep $i$ in the expression for $\mathbf{U}$ and switch to $j$ in the expression for $\mathbf{V}$, i.e. Dotting the two identities, we find (Recall from Section 7.8.2 the discussion concerning the subtleties inherent in expressions that feature two or more simultaneous contractions.) Rearrange the terms on the right to bring the two vectors together, i.e. Since we arrive at the desired result

The foregoing discussion is important for two reasons. First, the dot product is a central operation in Geometry, and therefore in Tensor Calculus, and its component space representation -- later to be referred to as the coordinate space representation -- is of utmost value. Second, the discussion illustrated that the tensor notation is an effective tool for deriving algebraic relationships. Note that in Chapter 2, we essentially guessed the equation and subsequently observed its correctness. In this Section, with the help of the tensor notation, we were able to arrive the equivalent equation by straightforward algebraic manipulation.

Many of the fundamental identities in Tensor Calculus are obtained by differentiating identities involving composite functions. As a result, such analyses rely heavily on the use of the chain rule. Fortunately, as we are about to demonstrate, the multivariate chain rule lends itself perfectly to the tensor notation and illustrates another one of its natural applications.

We will begin with a function $H\left( t\right)$ of a single variable given by the composition where $F\left( x,y,z\right)$ is a function of three variables, and each of $a\left( t\right)$, $b\left( t\right)$, and $c\left( t\right)$ are functions of $t$. According to the chain rule, the derivative $H^{\prime}\left( t\right)$ is given in terms of the partial derivatives of $F\left( x,y,z\right)$ and the ordinary derivatives of $a\left( t\right)$, $b\left( t\right)$, and $c\left( t\right)$ by the identity Note the convention that we will use throughout our narrative of suppressing the arguments of functions when they are clear from the context. With the full detail of the arguments included, the above equation would read which makes it evident why we prefer

Our present goal is to express the right side in the tensor notation. Fortunately, being a sum of products, it is ready to be interpreted as a contraction. To this end, denote the arguments of $F$ by $z^{1}$, $z^{2}$, and $z^{3}$, turning $F\left( a,b,c\right)$ into $F\left( z^{1} ,z^{2},z^{3}\right)$. Also, denote the functions of $a\left( t\right)$, $b\left( t\right)$, and $c\left( t\right)$ by $c^{1}\left( t\right)$, $c^{2}\left( t\right)$, and $c^{3}\left( t\right)$. In terms of the new symbols, $H\left( t\right)$ is given by while the expression for its derivative reads Now the expression on the right can be easily captured with the help of the summation convention, i.e. As we discussed at the end of Section 7.3, the index $i$ in the symbol $\partial F/\partial z^{i}$ can be thought of as a subscript because it is a superscript in the "denominator" of a "fraction". Therefore, the repeated index appears once as a subscript and once as a superscript, thus properly triggering Einstein's summation convention.

Finally, if you still find it helpful to visualize the matrix form of indicial expressions, note that the above identity can be captured by the equation We will continue to provide the corresponding matrix forms for each of the chain rule identities in this Section.

Let us now consider a function $H\left( u,v\right)$ of two variables formed by composing the function $F\left( x,y,z\right)$ with functions of two variables $a\left( u,v\right)$, $b\left( u,v\right)$, and $c\left( u,v\right)$, i.e. Even though in this example there are only two independent variables, our analysis will apply to functions with an arbitrary number of arguments.

Applying the chain rule for each independent variable, we find Let us convert these identities into a single indicial equation. In addition to denoting the independent variables of $F$ by $z^{i}$ and the functions $a$, $b$, and $c$ by $c^{i}$, denote the independent variables as $u^{1}$ and $u^{2}$, i.e. Collectively, we will refer to $u^{1}$ and $u^{2}$ as $u^{\alpha}$. The superscript $\alpha$ is taken from a different alphabet to highlight the fact that the number of independent variables $u^{\alpha}$ is different from the number of the arguments of $F$. In terms of the new symbols, the above differential identities read

We can now "pack" these identities into the single indicial equation As usual, the repeated index represents a summation while the free index enumerates independent equations. Note that the object $\partial c^{i}/\partial u^{\alpha}$ represents six partial derivatives: the derivative of each of the three functions $c^{i}\left( u^{1},u^{2}\right)$ with respect to each of the two variables $u^{\alpha}$. If the Latin index is treated as first and the Greek as second, the above identity can be captured in matrix form by the equation

A note on terminology is in order. The phrase differentiation with respect to $u^{\alpha}$ refers to the evaluation of the partial derivatives of a function with respect to each of the independent variables $u^{\alpha}$. However, from the point of view of the mechanics of differentiation, the derivatives are evaluated as if with respect to a single variable, such as $u^{1}$ or $u^{2}$. In other words, thanks to the tensor notation, the simultaneous nature of the operation does not increase the complexity of the analysis compared to the evaluation of a single derivative. As a matter of basic tensor proficiency, you should be able to go fluently from the equation defining $H\left( u^{1},u^{2}\right)$ in terms of $F$ and $c^{i}$ to the equation that gives its derivatives in terms of the derivatives of $F$ and $c^{i}$.

We will now turn our attention to the most general case of several functions of several variables.

Finally, let us consider the general case of several functions $F^{p}$ of several variables $z^{i}$ composed with a matching number of functions $c^{i}$ of several variables $u^{\alpha}$. We have run out of alphabets, so we going to use a letter, $p$, from a different part of the Latin alphabet to indicate that the number of functions $F^{p}$ may be different from the number of arguments $z^{i}$ in each function $F^{p}$ and from the number of independent variables $u^{\alpha}$. For the sake of concreteness, suppose that there are $4$ functions $F^{p}$, i.e. $F^{1}$, $F^{2}$, $F^{3}$, and $F^{4}$, and thus there are $4$ composite functions $H^{p}$, i.e. Pack these equations into a single one with the help of a live index $p$, i.e. Differentiating the combined equation with respect to $u^{1}$ and $u^{2}$ yields These equations represent a total of $8=4\times2$ identities, as each equation represents $4$ identities corresponding to $p=1$, $2$, $3$, and $4$. Express the contractions on the right by using a dummy index $i$, i.e. and, subsequently, combine the two equations into a single one by using a free index $\alpha$ , i.e. This single equation captures the $8$ partial derivatives $\partial H^{p}/\partial u^{\alpha}$ of the functions $H^{p}\left( u^{1},\upsilon ^{2}\right)$ in terms of the partial derivatives of $F^{p}$ and $c^{i}$. The application of the chain rule to multivariate composite functions will be one of the most common operations going forward.

Finally, let us give the matrix form of this equation. If the superscript enumerating the functions is considered first and the subscript enumerating the variables second, then the corresponding equation in matrix form reads

The natural ability of the tensor notation to handle the chain rule pays immediate dividends in the analysis of inverse functions, to which we now turn.

As we have already mentioned on a number of occasions, the tensor property describes how a system transforms under a change of coordinates. A change of coordinates is, in turn, specified by two sets of inverse functions. In this Section, we will derive the relationship between the partial derivatives of those sets of functions. This exercise will serve the dual purpose of preparing us for future analyses of coordinate changes as well as increasing our fluency with the tensor notation.

To describe a change of coordinates in an $n$-dimensional space requires a set of $n$ functions of $n$ variables. Indeed, we must specify how each of the $n$ new coordinates is obtained from the $n$ old coordinates. For example, the functions that describe the transformation from Cartesian coordinates to polar coordinates are Naturally, the inverse coordinate transformation -- from the new coordinates back to the old coordinates -- is also described by a set of $n$ functions of $n$ variables. For the same coordinate transformation, the functions that describe the inverse transformation are Thus, we have two sets of functions, i.e. those that translate from old coordinates to new and those that translate from new to old. By definition, the two sets are function inverses of each other. In other words, if one set maps the values $a^{1},\cdots,a^{n}$ to $b^{1},\cdots,b^{n}$, then the other sends $b^{1},\cdots,b^{n}$ back to $a^{1},\cdots,a^{n}$.

Remaining in two dimensions for now, denote the functions that translate the old coordinates to the new coordinates by $F^{1}\left( u,v\right)$ and $F^{2}\left( u,v\right)$ or, collectively, $F^{i}\left( u,v\right)$. Denote the functions that translate in the opposite direction by $G^{1}\left( u,v\right)$ and $G^{2}\left( u,v\right)$ or, collectively, $G^{i}\left( u,v\right)$. For the transformation between Cartesian and polar coordinates, we have Let us confirm that these sets of functions are indeed the inverses of each other by evaluating the composite functions We expect $G^{1}\left( F^{1}\left( u,v\right) ,F^{2}\left( u,v\right) \right)$ to recover $u$ and $G^{2}\left( F^{1}\left( u,v\right) ,F^{2}\left( u,v\right) \right)$ to recover $v$. Substituting $F^{1}\left( u,v\right)$ and $F^{2}\left( u,v\right)$ for the arguments of $G^{1}\left( u,v\right)$ and $G^{2}\left( u,v\right)$, we find It is left as an exercise to show that and therefore as we set us to show. It is left as an exercise to show that composing the functions in the opposite way yields the same result, i.e.

In $n$ dimensions, the collection of the partial derivatives of the functions $F^{i}$ with respect each of their independent variables is a second-order system with $n^{2}$ elements. The same is true of the partial derivatives of $G^{i}$ with respect to each of their independent variables. The two systems are connected by an extraordinarily elegant relationship which we are about to derive. We will discover that the tensor notation, and its particular effectiveness in expressing the chain rule, truly shines in this application.

Let us start our discussion with functions of one variable. Consider two ordinary functions $F\left( x\right)$ and $G\left( x\right)$ that are the inverses of each other, i.e. if $F\left( x\right)$ maps the number $a$ to the number $b$, then $G\left( x\right)$ maps $b$ back to $a$. In other words, if then Here are a few examples of inverse functions that we may use to confirm the derived relationships: Let us also document their derivatives: While, on the face of it, the derivatives $F^{\prime}\left( x\right)$ and $G^{\prime}\left( x\right)$ do not appear to be related in an obvious way, you may remember from ordinary Calculus that they are, in a certain sense, the algebraic reciprocals of each other. Of course, they are not reciprocals in the sense that which can be immediately seen from the examples above. Instead, the proper relationship can be described like this: if $b$ is the image of $a$ under $F$, i.e. or, equivalently, then In other words, the derivatives are the reciprocals of each other at the appropriate values of their arguments -- namely $G^{\prime}$ must be evaluated at $F\left( a\right)$, i.e. the image of $a$, rather than $a$ itself. It is left as an exercise for the reader to remind themselves why this relationship makes perfect sense when one considers the relationship between the graphs of the inverse functions $F\left( x\right)$ and $G\left( x\right)$.