Einstein summation, index notation and numpys np.einsum

As a Linear algebra addict and fan of vectors and matrices, it was unclear for me for a long time, why I should use Einstein notation at all. But When I got interested in backpropagation calculus, I got to a point, where tensors got involved and I then realised that thinking in terms of matrices limits my thinking to 2 dimensions. In this article, I will nevertheless use many matrix and vector analogies, so that the topic becomes easier to grasp.

Free indices are indices, which occur on both sides of an equation. For example:

𝑣 could now represent a row or a column vector.

That’t exactly the point of index notation. You free yourself from any concrete representation of vectors.

We can also have two free indices:

We can imagine this equation describing the rows and columns of a matrix 𝐴.

However if we continue to increase the number of free indices, it becomes increasingly difficult to image a concrete representation.

With 3 free indices, it would be a tensor and we could try to imagine it as a vector of matrices.

Dummy can occur on one side of an equation and as indices, they occur an even amount of times in every product.

An example would be like:

This equation could also be written as an inner product of a row and a column vector.

When we use apply this convention, we sum over the dummy indices even if there is no sum symbol. This convention is useful, because the summation over dummy indices happens very often in linear algebra.

Applying this convention, the last equation can be rewritten as follows:

Some people apply the following convention and some people don’t. I myself apply it, if I have to convert between index notation and a vectorized form quickly.

Using this convention, we write both lower and upper indices.Please do not confuse the upper indices with “to the power of”.

Then only same indices diagonal to each other are summed over.

Example of a repeated index, over which we sum:

Another example:

Example of a repeated index, over which we don’t sum:

We can also see, that we don’t have to sum over the index j, because it occurs on both sides of the equation and therefore is a free index.

Most of the times we find free and dummy indices in the same equation. For some…

Continue reading: https://towardsdatascience.com/einstein-index-notation-d62d48795378?source=rss—-7f60cf5620c9—4

Source: towardsdatascience.com