François CholletVerified account

A common beginner mistake is to misunderstand the meaning of the term "interpolation" in machine learning. Let's take a look

Some people think it always refers to linear interpolation -- for instance, interpolating images in pixel space. In reality, it means interpolating *on a learned approximation of the latent manifold of the data*. That's a very different beast!

However, if you interpolate between two digits *on the latent manifold of the MNIST data*, the mid-point between two digits still lies on the manifold of the data, i.e. it's still a plausible digit.

But wait, what does that really mean? What's a "manifold"? What does "latent" mean? How do you learn to interpolate on a latent manifold?