For a few centuries, scientists have described natural phenomena by means of relatively simple mathematical models such as Newton's law of gravitation or Snell's law of refraction. Sometimes, they found these models deductively, starting from fundamental considerations; more frequently, however, they derived the models inductively from data. With increasing amounts of data available for all sorts of (natural and social) systems, one may argue that we are now in a position to inductively uncover new interpretable models for these systems. But can this process be automatized? That is, can we design algorithms that automatically learn, from data, the closed-form mathematical models that generated them? And if so, are the true generating models always learnable? Here we will discuss how network inference approaches can help us to answer these questions. Moreover, we will show that there is a transition occurring between: (i) a learnable phase at low observation noise, in which the true model can in principle be learned from the data; and (ii) an unlearnable phase, in which the observation noise is too large for the true model to be learned from the data by any method.