In general, the metric MDS calculates distances between each pair of points in the original high-dimensional space and then maps it to lower-dimensional space while preserving those distances between points as well as possible.

Note, the number of dimensions for the lower-dimensional space can be chosen by you. Typically, one would choose either 2D or 3D as it allows for the data to be visualized.

So let’s take a look at high-level steps performed by metric MDS. I have tried to keep maths to a minimum in this explanation, but it was impossible to avoid it altogether.

Steps used by metric MDS algorithm

Step 1 — The algorithm calculates distances between each pair of points, as illustrated below.

Points and their distances in 3D space. Image by author.

Step 2 — With the original distances known, the algorithm attempts to solve the optimization problem by finding a set of coordinates in a lower-dimensional space that minimizes the value of Stress.

Stress cost function. Image by author.

Multiple approaches can be used to optimize the above cost function, such as Kruskal’s steepest descent method or De Leeuw’s iterative majorization method. However, I will not delve into maths this time to keep this article focused on high-level explanation.

One important thing to note is that both aforementioned methods are iterative approaches, sometimes giving different results since they are sensitive to the initial starting position.

However, Sklearn’s implementation of MDS allows us to specify how many times we want to initialize the process. In the end, the configuration with the lowest stress is picked as the final result.

Leaving theory behind, let’s get into the fun bits and use MDS for dimensionality reduction in Python.

Typically, you would want to use MDS for high-dimensionality data, for example, images of hand-written text or numbers. However, I am keen to demonstrate what the data looks like before and after. Hence, I will use MDS on simpler data that can be visualized in both 3D and 2D.

Setup

We will use the following data and libraries:

Let’s start by importing libraries.

Next, we create some data using Sklearn’s make_swiss_roll and display it on a 3D plot.

Interactive 3D swiss roll. Graph by author.

Make sure to explore the above interactive graph from every angle by rotating it.

Performing MDS

We will now use MDS to map this 3D structure to 2 dimensions while preserving distances between points as best as possible. Note that the depth of the swiss roll is…

Continue reading: https://towardsdatascience.com/mds-multidimensional-scaling-smart-way-to-reduce-dimensionality-in-python-7c126984e60b?source=rss—-7f60cf5620c9—4

Source: towardsdatascience.com