In general, the metric MDS calculates distances between each pair of points in the original high-dimensional space and then maps it to lower-dimensional space while preserving those distances between points as well as possible.

Note, the number of dimensions for the lower-dimensional space can be chosen by you. Typically, one would choose either 2D or 3D as it allows for the data to be visualized.

So let’s take a look at high-level steps performed by metric MDS. I have tried to keep maths to a minimum in this explanation, but it was impossible to avoid it altogether.

## Steps used by metric MDS algorithm

Step 1 — The algorithm calculates distances between each pair of points, as illustrated below.

Step 2 — With the original distances known, the algorithm attempts to solve the optimization problem by finding a set of coordinates in a lower-dimensional space that minimizes the value of Stress.

Multiple approaches can be used to optimize the above cost function, such as Kruskal’s steepest descent method or De Leeuw’s iterative majorization method. However, I will not delve into maths this time to keep this article focused on high-level explanation.

One important thing to note is that both aforementioned methods are iterative approaches, sometimes giving different results since they are sensitive to the initial starting position.

However, Sklearn’s implementation of MDS allows us to specify how many times we want to initialize the process. In the end, the configuration with the lowest stress is picked as the final result.

Leaving theory behind, let’s get into the fun bits and use MDS for dimensionality reduction in Python.

Typically, you would want to use MDS for high-dimensionality data, for example, images of hand-written text or numbers. However, I am keen to demonstrate what the data looks like before and after. Hence, I will use MDS on simpler data that can be visualized in both 3D and 2D.

## Setup

We will use the following data and libraries:

Let’s start by importing libraries.