As we discovered at the end of the last Chapter, if $T^{i}$ is a contravariant tensor, i.e. then the variant is not a tensor in its own right since it transforms according to the identity As a result, we do not easily see how the variant $\partial T^{i}/\partial Z^{j}$ can be used in an expression that would produce the same value in all coordinate systems and would therefore be considered an invariant. In other words, our analysis has lost its grasp on the geometric meaning.

In order to restore the lost geometric meaning, we must reconnect our analysis to tangible geometric objects. Let us imagine that $T^{i}$ is not some abstract tensor but is, in fact, the components of an invariant vector $\mathbf{T}$, and that the whole purpose of differentiating $T^{i}$ with respect to the coordinates $Z^{j}$ is to capture the rate of change of the vector $\mathbf{T}$. In fact, let us switch from the letter $T$ to the letter $U$ which we have commonly used to denote geometric vectors in a Euclidean space. Once our present exploration suggests to us a way of retaining the geometric meaning in the course of our analysis, we will switch back from $U$ to $T$ and extend our analytical insight to abstract tensors.

Consider an invariant vector field $\mathbf{U}$ with components $U^{i}$, i.e. Recall, that both $U^{i}$ and $\mathbf{Z}_{i}$ are tensors. Furthermore, the collection of partial derivatives being the result of differentiating an invariant, is a tensor in its own right (with vector elements). Since we should expect that analyzing the expression should present us with the insights we need for preserving the geometric meaning. (Note that we are differentiating with respect to $Z^{k}$ rather than $Z^{j}$ because we would like to save the index $j$ for our upcoming analysis of second-order tensors.)

By the product rule, we have Recall that, by the very definition of the Christoffel symbol, Therefore, Importantly, neither term on the right is a tensor -- but their sum is! Thus, we know to keep the two terms together.

Both terms represent linear combinations with respect to one and the same basis. We should, therefore, be able to combine them in a component-by-component manner. To do so while staying in the tensor notation, re-index the second term so that the basis appears with the index $i$. This can be accomplished by switching the roles of indices $i$ and $m$, which is valid since both indices are dummy. As a result, we are able to factor out $\mathbf{Z}_{i}$, i.e. Since the Christoffel symbol is symmetric in its subscripts, i.e. we arrive at the identity This identity brings us right to the main point: the combination being the components of a tensor, is a tensor in its own right. This alone should signal to us that this combination ought to take the place of the partial derivative as the differential operator for measuring spatial variability. Besides being a tensor, the combination is decidedly preferable over $\partial U^{i}/\partial Z^{k}$ alone since it actually represents the components of the derivatives of the vector field $\mathbf{U}$. And it is able to do so by referencing only those objects that are available in the coordinate space. This is possible thanks to the term containing the Christoffel symbol which can be said to account for the variability of the accompanying basis.

Let us now explore the same analytical argument for the covariant component $U_{j}$ of $\mathbf{U}$. Differentiate the identity with respect to $Z^{k}$, i.e. By the product rule, we have Since we find that Renaming the indices and rearranging the terms as before yields Thus, we conclude that the combination is a tensor in its own right and therefore ought to replace the partial derivative as the primary differential operator. In the next Section, we will use the combinations and as the inspiration for the definition of the covariant derivative for first-order variants.

Since we are turning our attention to general tensors which may not be the components of a vector field, let us switch from the letter $U$ back to $T$. For a first-order contravariant tensor $T^{i}$ and for a first-order covariant tensor $T_{j}$, the definitions of the covariant derivative read The cornerstone property of the covariant derivative, which we will discuss last, is that it produces tensor outputs for tensor inputs.

It is a signature characteristic of the covariant derivative that the different flavors of tensors receive different treatments. At first sight, the fact that there are two different definitions for the two different types of tensors may create a certain sense of inelegance and perhaps of excessive complexity. However, we should point out that the strong indicial structure of these definitions makes it easy to correctly arrange the indices in the Christoffel term. Indeed, note that the indicial signature of while that of Thus, where in the last Christoffel symbol, the order of the subscripts does not matter due to symmetry. Since two of the three indices have "placed themselves", it leaves only one possibility for the remaining index $m$, i.e. Thus, the only aspect of the definition of the covariant derivative that needs to be memorized is the sign: plus for a contravariant tensor and minus for a covariant tensor.

Note another interesting characteristic of the covariant derivative: it cannot be applied to the elements of the input vector individually. For example, $\nabla_{k}T^{1}$ cannot be evaluated without referring to all of the remaining elements of $T^{i}$ since those appear in the term with the Christoffel symbol. This is unlike the partial derivative where $\partial T^{1}/\partial Z^{k}$ can be evaluated individually, without a reference to the other elements of $T^{i}$.

The definitions presented above are valid only for first-order tensors. Before we give the general definition of the covariant derivative applicable to tensors of arbitrary order, we will explore one crucial property, known as the metrinilic property, of the first-order definitions presented so far.

The definitions of the covariant derivative -- as applied to first-order tensors -- that we have just formulated in the previous Section can be applied to the covariant basis $\mathbf{Z}_{i}$ as well as the contravariant basis $\mathbf{Z}^{i}$. In both cases, the result is as surprising as it is important.

Let us begin with the covariant basis $\mathbf{Z}_{i}$. By the definition of the covariant derivative for a covariant tensor, $\nabla_{k}\mathbf{Z}_{i}$ is given by However, by the very definition of the Christoffel symbol, and therefore the two terms on the right of the preceding equation cancel each other. Thus, we conclude that i.e. the covariant derivative of the covariant basis vanishes.

The very same conclusion awaits us for the contravariant basis $\mathbf{Z} ^{i}$. By the definition of the covariant derivative for a contravariant tensor, $\nabla_{k} \mathbf{Z}^{j}$ is given by and, since we can conclude that i.e. the covariant derivative of the contravariant basis vanishes.

This characteristic of the covariant derivative is referred to as the metrinilic property -- metrinilic being the combination of the words metric and nil. We will soon discover that the metrinilic property extends to the metric tensors, as well as to the soon-to-be-introduced Levi-Civita symbols $\varepsilon^{ijk}$ and $\varepsilon_{ijk}$ also considered parts of the metrics family.

The metrinilic property of the covariant derivative has far reaching implications. Recall that the partial derivative $\partial/\partial Z^{k}$ produces zero when applied to the coordinate basis vectors $\mathbf{i}$, $\mathbf{j}$, and $\mathbf{k}$ in affine coordinates, i.e. This property is the key to our ability to differentiate vector fields by differentiating their affine components. Indeed, for a vector field $\mathbf{F}\left( Z\right)$ with components $F^{1}\left( Z\right)$, $F^{2}\left( Z\right)$, and $F^{3}\left( Z\right)$, i.e. we have, by the product rule, Since the partial derivatives of the basis vectors vanish, we are left with Thus, the components of the derivative $\partial \mathbf{F}/\partial Z^{k}$ are i.e. the derivatives of the corresponding components. In informal words, $\mathbf{F}$ can be differentiated by differentiating its components.

Thanks to the metrinilic property, the same principle applies to the covariant derivative. If or, omitting the independent variables, then, by an application of the product rule (to be demonstrated later), we have By the metrinilic property, we have So, analogously to the partial derivative, the covariant derivative can be applied in component-wise fashion.

Finally, let us express the same calculation in indicial form. If then, by taking the covariant derivatives of both sides of this equation, i.e. we find, by the product rule, that By the metrinilic property, the second term vanishes and we are left with

On an intuitive level, the effect of the metrinilic property may be described by saying that the covariant derivative sees the basis as a constant and thus lets it pass through, similar to the way the ordinary derivative allows constants to pass through, e.g. Similarly,

Rather than outright stating the definition of the covariant derivative for variants of arbitrary order, we will show how it inevitably arises from the combination of the metrinilic property and the product rule. To this end, consider a second-order tensor $T_{j}^{i}$. The definitions of the covariant derivative $\nabla_{k}$ available to us so far apply only to variants of order one and therefore do not apply to $T_{j}^{i}$. However, if we contract $T_{j}^{i}$ with $\mathbf{Z}^{j}$, the resulting tensor is of order one and is therefore subject to the definition According to this definition, we have Substituting $T_{j}^{i}\mathbf{Z}^{j}$ for $\mathbf{T}^{i}$ and $T_{j} ^{m}\mathbf{Z}^{j}$ for $\mathbf{T}^{m}$ and applying the ordinary product rule to the partial derivative, we find Since we have Finally, switching the roles of $j$ and $m$ in the second term, we find This completes our analysis of the first-order tensor $\mathbf{T}^{i}$ under the already-established covariant derivative for first-order variants.

Now let us turn our attention to the yet-to-be-defined covariant derivative $\nabla_{k}$ applicable to second- and higher-order tensors that could therefore be applied to the equivalent combination $T_{j}^{i}\mathbf{Z}^{j}$. If we desire that it satisfies the product rule and the metrinilic property, then its application to the identity will yield Comparing the two expressions for $\nabla_{k}\mathbf{T}^{i}$ we conclude that the only viable definition for $\nabla_{k}T_{j}^{i}$ is This identity will indeed serve as a blueprint for the general definition of the covariant derivative. The structure of the expression on the right can be summarized in words as follows: there is a Christoffel term for each index in the indicial signature of the variant. In each term, the relevant index participates in the indicial pattern that was established for first-order variants, while the remaining indices remain where they are.

The number of terms in the definition of the covariant derivative depends on the order of the variant. Therefore, we will, as we typically do, capture the general definition by presenting an expression for a variant $T_{j}^{i}$ with a representative collection of indices. The definition reads It is to be understood in the sense that to each index in the indicial signature of the variant under the derivative, there corresponds an appropriate additive term involving the Christoffel symbol, where the relevant index participates in the kind of indicial pattern that was established for first-order variants, while the remaining indices remain where they are.

For example, when applied to the contravariant tensor $T^{rst}$, the formula reads For a covariant tensor $T_{rst}$, we have Finally, for a tensor $T_{uv}^{rs}$, we have

We are, once again, obliged to state the most crucial property, known as the tensor property, of the covariant derivative -- namely, that it produces tensor outputs for tensor inputs. More specifically, the resulting tensor is of one covariant order greater than the input tensor.

The newly created index can, of course, be raised by contraction with the contravariant metric tensor. The resulting operator, represented by the combination is aptly denoted by the symbol $\nabla^{l}$, i.e. and can be referred to as the contravariant derivative, although this term is not frequently used.

Finally, note that the covariant derivative can be applied to variants that are not tensors. For example, In this case, it is likely that the output variant is also not a tensor.

A tensor of order zero is an invariant. As we pointed out in the previous Chapter, the fact that the general definition of a tensor applies to variants of order zero is not a minor edge case but is, in fact, the heart of the matter.

Similarly, the general definition of the covariant derivative applies to variants of order zero. Since a variant $T$ of order zero has no indices, the covariant derivative reduces to the ordinary partial derivative, i.e. Note that this fact justifies our earlier use of the symbol $\nabla_{k}$ to denote partial derivatives of variants of order zero.

Recall that the Christoffel symbol vanishes in affine coordinates, i.e. As a result, the covariant derivative coincides with the partial derivative, i.e. in affine coordinates.

This seemingly simple fact actually finds frequent and important applications. For instance, assume for a moment that the tensor property of the covariant derivative has already been demonstrated. Then, from our discussion in Section 14.2 it follows if the partial derivatives of a tensor vanish in an affine coordinate system, then its derivative vanishes in all coordinates. In particular, the metrinilic property follows from the fact that the partial derivatives of the coordinate basis vectors $\mathbf{i}$, $\mathbf{j}$, and $\mathbf{k}$ vanish in affine coordinates.

Furthermore, from the fact that partial derivatives commute it will follow that covariant derivatives commute as well. This important insight will be discussed later in this Chapter.

Crucially, the covariant derivative satisfies the familiar product rule. For example, Let us demonstrate this particular identity and, as always, it will be apparent that the rule holds generally for variants with arbitrary indicial signatures.

By definition, the covariant derivative $\nabla_{k}\left( T^{i}U_{j}\right)$ of $T^{i}U_{j}$ is given by An application of the ordinary product rule to the partial derivative on the right yields Now, group the terms on the right in the following way: Since we arrive at the desired result We encourage the reader to repeat the calculation for variants with more complicated indicial signatures, such as $T_{j}^{i}U_{lm}$ and make sure that the expected product rule is valid.

Admittedly, the above demonstration of the product rule proved to be rather straightforward and not entirely unexpected. On the other hand, at the time when the definition of the covariant derivative first occurred to its inventors, the question of whether the product rule held was not at all obvious. We can only imagine their sense of satisfaction at discovering that it does indeed hold, allowing the idea to move forward.

Earlier, we established that the covariant and contravariant bases vanish under the covariant derivative, i.e. The product rule, which is valid for dot products, allows us to easily extend this result to the metric tensors. Since the covariant metric tensor $Z_{ij}$ is given by the dot product an application of the product rule yields and since each term on the right vanishes, we can conclude that Similarly, the identity yields the analogous result for the contravariant metric tensor Finally, since the Kronecker delta $\delta_{j}^{i}$ is given by we are able to conclude that it, too, vanishes under the covariant derivative, i.e. In Applications of Tensor Analysis, J.J. McConnell refers to the identities $\nabla_{k}Z_{ij}=0$ and $\nabla_{k}Z^{ij}=0$ as Ricci's lemma.

Thanks to the metrinilic property, the metrics freely pass through the covariant derivative. We have already observed this for the covariant and contravariant bases in Section 15.3. For example, as can be seen by applying the product rule on the right and subsequently appealing to the metrinilic property. Similarly, the metric tensor also seamlessly passes through the covariant derivative, e.g. This important observation finds numerous applications. In particular, it makes index juggling safe, as we will demonstrate in the next Section.

Finally, we should point out that the above proof of the metrinilic property, by virtue of its reliance on the covariant basis and the geometric dot product, is essentially Euclidean. However, in a Euclidean space, we could give another justification for the metrinilic property. Note that the identities are obviously true in Cartesian coordinates -- or any affine coordinates, for that matter. Indeed, in such coordinates, the bases, the metric tensors, and, of course, the Kronecker delta have constant elements while the covariant derivative coincides with the partial derivative. Thus, the result is zero. Meanwhile, by the flagship tensor property of the covariant derivative, which will be demonstrated below, the variants $\nabla_{k}\mathbf{Z}_{i}$, $\nabla_{k}\mathbf{Z}^{i}$, $\nabla_{k}Z_{ij}$, $\nabla_{k}Z^{ij}$, and $\nabla_{k}\delta_{j}^{i}$ are tensors. As such, if they vanish in one coordinate system, they vanish in all coordinate systems, which completes the argument.

This argument is also restricted Euclidean since it relies on the availability of an affine coordinate system. However, it is important to know that the metrinilic property with respect to the metric tensors and the Kronecker delta continues to hold in the more general Riemannian spaces. A more general way to demonstrate the metrinilic property that holds up in a Riemannian space involves a direct application of the definition of the covariant derivative. This approach can be found in one of the exercises at the end of the Chapter.

There are two operations related to index juggling to which we have become accustomed. The first is juggling a free index on both sides of an identity. For example, if then lowering $i$ on both sides of the identity yields The second is allowing two dummy indices in a contraction to exchange flavors. For example, if then exchanging the flavors of the index $j$ between the two variants on the right yields

Do these operations remain valid in the presence of a covariant derivative? For example, consider the identity First, let us ask whether the index $i$ can be lowered on both sides to produce In order to answer this question, we must recall the underlying mechanics of index juggling. The lowering of the index $i$ in the identity is achieved by contracting both sides with the covariant metric tensor $Z_{ir}$, i.e. Since, as we pointed out in the previous Section, the metric tensor $Z_{ir}$ moves freely across the covariant derivative $\nabla_{k}$, we have Therefore, and, after renaming $r$ into $i$, we have In summary, the initial identity implies In other words, thanks to the metrinilic property, a free index can indeed be raised or lowered on both sides of an identity even if one or more variants are found under the covariant derivative.

Similarly, it can be shown that a dummy index can be juggled across the covariant derivative, i.e. The demonstration of this fact is left as an exercise.

Consider the expression and note that it can be interpreted in two ways with respect to the order in which the covariant derivative and contraction are applied. On the one hand, it can be interpreted as the covariant derivative applied to the first-order variant $S_{j}=T_{ij}^{i}$. In this interpretation, the expanded expression for the covariant derivative will have a single Christoffel term. On the other hand, it can be seen as the covariant derivative applied to the third-order variant $T_{lj}^{i}$ with the result subsequently contracted on $i$ and $l$. In this interpretation, the expression for the covariant derivative will have three Christoffel terms. Fortunately, as we are about to show, both interpretations lead to the same result.

Let us first interpret $\nabla_{k}T_{ij}^{i}$ as the covariant derivative applied to the variant $S_{j}=T_{ij}^{i}$ of order one. We have In the alternative interpretation, let us apply the covariant derivative to the third-order variant $T_{lj}^{i}$ and subsequently perform the contraction. By the definition of the covariant derivative, we have Now, contract $i$ and $l$: Note that the first two Christoffel terms, i.e. those that correspond to the indices $i$ and $l$, cancel each other out. This can be seen by exchanging the names of the indices $i$ and $m$ in, say, the first term. Thus, which is consistent with the first interpretation.

We now turn to the most crucial property of the covariant derivative, i.e. its tensor property. It states that the result of applying the covariant derivative to a tensor is a tensor of one additional covariant order.

Our proof of this property will be based on an elegant inductive argument. For tensors of order zero, i.e. invariants, the tensor property of $\nabla_{k}T$ follows from the fact that the covariant derivative $\nabla_{k}T$ coincides with the partial derivative $\partial T/\partial Z^{k}$, for which the tensor property was demonstrated in the previous Chapter. We will now turn our attention to first-order tensors and then show how to extend the proof to second- and higher-order tensors.

Consider a covariant tensor $T_{i}$. Our goal is to demonstrate the tensor property of $\nabla_{k}T_{i}$. Of course, we could do this by considering the invariant vector field $\mathbf{T}=T_{i}\mathbf{Z}^{i}$ and then repeating the argument given at the beginning of this Chapter based on the tensor property of $\nabla_{k}\mathbf{T}$. However, that argument has a few shortcomings. For example, it is not applicable to a tensor $\mathbf{T}_{i}$ with vector elements, since there is no such thing as $\mathbf{T}_{i}\mathbf{Z}^{i}$. However, even for tensors with scalar components, it is important to have a proof that utilizes only those objects that are available in the component space. Such a proof would remain valid in the context of Riemannian spaces which we will soon describe. Thus, we will choose to give a direct proof based on demonstrating that $\nabla_{k}T_{i}$ and $\nabla_{k^{\prime}}T_{i^{\prime} }$ are related by the proper transformation rule. Importantly, the analytical logistics of this approach also work for tensors $\mathbf{T}_{i}$ with vector elements. Furthermore, this proof will show the precise manner in which the non-tensor contributions to the transformation rules cancel each other out to produce a tensor.

In the primed coordinates, the variant $\nabla_{k^{\prime}}T_{i^{\prime}}$ is given by We will now relate each of the elements on the right to their unprimed counterparts.

Let us start with $\partial T_{i^{\prime}}/\partial Z^{k^{\prime}}$, for which the transformation rule was given in the previous Chapter, but never derived. Since $T_{i}$ is a tensor, we have As we have done previously on a number of occasions, treat this equation as an identity in the primed coordinates, i.e. and differentiate both sides with respect to $Z^{k^{\prime}}$. By the product rule, we find Since we have Thus, we have the transformation rule for the first term on the right in equation

Moving on to the second term, recall that or Thus, since $J_{m}^{m^{\prime}}T_{m^{\prime}}=T_{m}$, we have

Putting the two terms together, we find The two terms containing the second-order Jacobians -- i.e. the two non-tensor terms! -- cancel each other and we are left with or, factoring out the Jacobians, Finally, since the quantity in parentheses corresponds to $\nabla_{k}T_{i}$, we have precisely as we set out to show.

Naturally, the tensor property of $\nabla_{k}T^{i}$ for a contravariant tensor $T^{i}$ can be demonstrated in the exact same manner as we just used for a covariant tensor $T_{i}$. Pursuing this approach is left as an exercise. Instead, we will now use a different approach based on the use of the quotient theorem. The beauty of this approach is that it can also be used in the inductive step of the overall proof.

Consider a contravariant tensor $T^{i}$ and, for any other tensor $V_{i}$, form the invariant Apply the contravariant derivative to both sides of the above identity: By the product rule, we have The variant $\nabla_{k}T^{i}$, whose tensor property we are trying to demonstrate, is found in the term $\nabla_{k}T^{i}~V_{i}$ on the right. Solving for that term, we find: The combination $\nabla_{k}T^{i}~V_{i}$ is clearly a tensor since each term on the right is a tensor according to the properties proven earlier. Thanks to the arbitrariness of $V_{i}$, the quotient theorem tells us that $\nabla _{k}T^{i}$ is a tensor in its own right, as we set out to prove.

For higher-order tensors, the tensor property of the covariant derivative can be shown by increasing the order by one unit at a time. For a tensor order two, say $T_{j}^{i}$, construct a first-order tensor $T^{i}$ by contracting $T_{j}^{i}$ with an arbitrary tensor $V^{j}$: As before, take the covariant derivative of both sides By the product rule, As before, solve for the term containing $\nabla_{k}T_{j}^{i}$: Once again, all terms on the right are tensors which leads to the conclusion that $\nabla_{i}T_{k}^{j}$ is a tensor by the quotient theorem.

It is clear that continuing in this fashion, we can demonstrate the tensor property of the covariant derivative with respect to tensors of any order and any indicial signature. This completes the proof of this linchpin property.

The tensor property guarantees the geometric meaningfulness of the covariant derivative and opens the floodgates for producing differential invariants. For example, for a tensor $U^{i}$, the variant $\nabla_{i}U^{j}$ is a tensor in its own right. Therefore, the combination is an invariant. In fact, it is the celebrated divergence operator. We may not immediately know its precise geometric meaning but we can be confident that this quantity is worthy of analysis. Similarly, for an invariant $U$, the combination $Z^{ij}\nabla_{i}\nabla_{j}U$, which can also be expressed in the more compact form is an invariant. Of course, you may recognize it as the Laplace operator or the Laplacian of $U$. Note that both, the divergence and the Laplacian, have elegant geometric interpretations that will discussed in Chapter 18.