Most of the applications I’ve seen have been only a few dimensions, usually for time and space. But they can also be used for general regression. Radford Neal showed back in the early 90s that the limit as the hidden layers grow of a neural network is a GP. So it’s a very general non-linear curve fitting model.
The right way to think about data dimensionality is in topological terms. For example, a sphere is a 2-dimensional surface embedded in 3 dimensions. While on a sphere, you only have two degrees of freedom (latitude and longitude). The natural distance might be something like great circle distance (more generally, a geodesic).
In general, the trick’s finding the right embeddings for GPs. To tackle the birthday problem (cover of BDA 3), Aki modified distances so that all Mondays are near each other, all days in a given season are near each other, etc. It doesn’t just have to be Euclidean distance.