This is stretching it, but if you accept automata theory as a part of CS (not very controversial) and accept cellular automata like the game of life as a branch of automata theory (not very controversial) and accept some research methods involving experimentation and observation of the behavior of random seeds as part of cellular automata theory (not very controversial) then the very extended argument stretches from hard core CS to 'stick a random seed in the CA, run it, and ta da, we observe the un predicted existence of gliders!'. Its stretching it but I think its acceptable.
I was going to list some formal percolation theory studies in as an applied version of imaginary network routing protocols or could be seen as a very weird emulation of a CA, kinda, but that's probably going way too far. Practical simulation of a specific algo is obviously cheating, but simulation of an abstract concept is maybe not cheaty. Possibly there is, or will be, an analytic topological pure math theory that could be applied to mathematically prove constants currently only discovered by experiment in perc theory. If you're ever bored and want to run some simulations, perc theory is very project euler like in that a short(ish) question quickly results in "I guess I gotta run it and see".
Another "stretch the limits" is there is no unclassified explanation of some peculiar corners of some crypto algos. Hard to know if they're truly random, determined by experimental runs against possibly classified cracking techniques, or intentional back doors, or intentional classified design techniques. But relying on military intel classification "we can't rule out experimental methods being used in secret" is kinda cheaty in the spirit of the discussion.
Seriously, just about everything we do in computer science is experimentation, at least if it involves...physical computers.
Every time we run a program, we run an experiment. At least if we have some expectation of what we want the program to do. Every time we write a test, or run a test suite, we are running an experiment.
Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments.
At least if we're doing it right.
If you're not doing experiments, I have a hard time categorizing what you are doing as computer science.
In physics, we do experiments because we don't know the rules of the natural world. That's the only way we can learn them.
In CS/programming, if we're doing any experiments, they're done because we're too stupid to infer the right answer beforehand. Insofar as you're dealing with the theoretical part of the problem, you shouldn't need any experiments, because the results are purely mathematic and should be perfectly predictable. And if you're doing hardware experiments, it's not computer science either - that's electrical engineering or applied physics. Whatever other tests you may be doing to save some time are still the engineering type.
The point is, computer science doesn't have any area in which it could use experiments for discovery. It's fair to say then, that it's not really a science. (It's also fair to point out that the line between science and engineering is getting blurry nowadays.)
> In CS/programming, if we're doing any experiments, they're
> done because we're too stupid to infer the right answer
That's the same as for physics. In theory, there is no difference between theory and practice. In practice, there is.
More specifically, for CS: what you're describing is an alternate universe where the halting problem[1] doesn't exist, or rather has been solved. In short, there is no way to predict, in the general case, whether a program will terminate or not, other than running it (= doing the experiment).
More generally, if you have a mechanism that predicts with perfect accuracy what the output of my program is, more quickly than running the program, then I will just call that new mechanism "the program", and use that instead of the original program. And then you're back to running the program to find out what it does.
UPDATE: To me, this is a (if not THE) fundamental results of theoretical computer science: a precise description of its limits. And since this is an article by Knuth: "Beware of bugs in the above code; I have only proved it correct, not tried it"[2]
You're describing software engineering, not computer science.
CS is big-O, Chomsky hierarchy, finite automata, type theory, and computability classes. It's the study of algorithms & data structures, not their implementations. In other words, CS is a branch of math.
Theoretical computer science is a division or subset of general computer science and mathematics that focuses on more abstract or mathematical aspects of computing and includes the theory of computation.
Yeah you "explore stuff" in any field. Science, art, math, every day life, when learning a craft (welding etc). Just because you explore doesn't mean that is what the core of what computer science is.
> Every time we implement some sort of abstraction, every time we vary a parameter, every time we optimize a program, we are running experiments.
Ok, what in life then isn't science because well everything is running experiments? I am going to buy milk -- it's an experiment to see what happens. Maybe I won't find parking. I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake.
By that token we've diluted the word science to not really mean anything anymore.
"An experiment is a procedure carried out to verify, refute, or establish the validity of a hypothesis. Experiments provide insight into cause-and-effect by demonstrating what outcome occurs when a particular factor is manipulated. Experiments vary greatly in goal and scale, but always rely on repeatable procedure and logical analysis of the results."
I have done a lot of performance work in my career, and there you run experiments almost continuously. At least you do if you actually want to have an effect. Admittedly, many people try to optimize without running experiments. That tends to not be very effective.
And yes, computer systems are easily sufficiently complex that reasoning it out ahead of time without running the experiment tends to be completely futile except for the most trivial arrangements. In fact, even in some situations that seem completely trivial.
> I am going to buy milk -- it's an experiment to see what happens.
If you have a hypothesis and you're trying to find out if it is true or not, sure, that can be an experiment. Usually it probably won't be.
> I am drinking coffee in the morning, it's an experiment to see if it will still keep me awake.
That certainly can be an experiment, for example if you vary the amount of coffee to see how much of will be keep you awake. Though it seems unlikely that you will be isolating the variables sufficiently. With computers, isolating the variables is often a lot simpler.
> By that token we've diluted the word science to not really mean anything anymore.
See you are proving my own point. Just because you do exploratory stuff and run "experiments" that doesn't make what you science. What isn't science then?
> I have done a lot of performance work in my career, and there you run experiments almost continuously
Yes me too. But I am not a scientist like Einstein.
There is a stretch in going from I am debugging some program and trying various things to saying I am a scientist.
You can sure call yourself a scientist but you have to explore why you want to do that. Is to make yourself feel better because the word "programmer" is boring or implies a lower status. What about craftsman, you are" crafting software" is that imply something undesirable, if so why? What about business problem solver?
Licensed engineers even scoff at software "engineers" even calling themselves "engineers" (even though what most computer scientist do is kind of like engineering), because they don't have a license etc.
Wow. Just wow. Please don't project your insecurities on others.
Look, I tend to say that "Computer Science" is a bit pretentious, "playing with computers" is overall more appropriate for the level our field is at. Not sure where you got Einstein from. And, in the words of Wolfgang Pauli: "Ach, that isn't even wrong".
Bringing Einstein, who was a theorist, into a discussion about experiments is downright silly, and makes me question whether you have any idea as to what you're talking about.
Second, assuming for the sake of argument that Einstein was an experimenter, it would be just about the biggest straw-man imaginable. If experiments are only experiments if performed by one of the greatest scientists who ever lived, then science effectively doesn't exist and there are no scientists, to several significant figures.
However, the question was not who is an awesome scientist, but whether there are experiments in CS, whether empiricism is relevant especially vs. the entire field being a trivial application of mathematical theory.
And that is, for me, easy to answer: it is a highly empirical field. If you think it is a trivial application of mathematical theory, you don't know what you're talking about, neither in theory nor in practice.
And yes: there is a lot of science being done that isn't at the level of Einstein or the people at CERN or Fermilab.
