One class of examples would be problems where probing the environment is expensive. In many robotics applications, this can be tricky because the robot might break or not return to a suitable initial condition without human oversight. Moreover, the physical experiment can relatively slow so little data is generated, and what is generated may not generalize well. That being said, how to overcome these challenges is a very active research topic in the community.
