“It is vitally necessary to be able to check the simulation against real data.”
Actually, that’s not enough either. In spam training, you have to have a completely set of data that you use for the final test but never, ever for training. Why? Because otherwise your spam filter will learn how to properly classify every message you train it with and nothing else.
For simulated science, this means it’s not safe to say, “Well, we trained the simulation until it could reproduce 2000-2010 when we gave it 1990-2000 as input data.” If you do that, your predictions for 2010+ are probably going to be worthless, since your simulation is like a cheating student who only knows the right answers on old tests he stole, not how to pass any test in general.
Actually, that’s not enough either. In spam training, you have to have a completely set of data that you use for the final test but never, ever for training. Why? Because otherwise your spam filter will learn how to properly classify every message you train it with and nothing else.
For simulated science, this means it’s not safe to say, “Well, we trained the simulation until it could reproduce 2000-2010 when we gave it 1990-2000 as input data.” If you do that, your predictions for 2010+ are probably going to be worthless, since your simulation is like a cheating student who only knows the right answers on old tests he stole, not how to pass any test in general.