The study may or may not be flawed, but what's really interesting to me is the reaction. We need more science in our computer science, which means more experiments and more results like this. We should also be open to the truth that we use the tools we like because we like them rather than because they're technically superior, even though we pimp them ad nauseum as though they are.
I once read an article about a technique Intel had developed for improving cooling of processors by changing the shape of the fan. I related this to some of my co-workers. One of them proceeded to tell me that this can't possibly work, backing up his argument with "reasoning" based on off-the-cuff remarks about the way air and physics "must" work. The fact that Intel had actually done this seemed to have no effect on his eagerness to continue the "debate" about this scientific fact.
I have a hard time believing that I get no benefit from using Haskell over Smalltalk, but if a body of science were to appear that cast doubt on that belief, the appropriate thing to do is change the belief, not stand around debating from imagined first principles why all the science is wrong and can't be so. Shut up, design an experiment and go prove it!
Perhaps there's little of this kind of actual science in our computer science because it would mean asking hard questions and accepting difficult truths. "The prisoner falls in love with his chains."
I've only scanned the comments, but the chief reaction seems to be exactly what you want for a scientific approach--people are arguing about the scope and methodology of the study. A paper like this is not some broad-reaching conclusion--it's very specific and based on some potentially flawed methodology. You want people to qualify exactly how specific it is and talk about potential flaws in its approach. That's how you improve the general knowledge.
Also, I suspect there are a couple of reasons such studies are uncommon in computer science. For one, CS isn't really a science; programmers and computer scientists are not trained in the scientific method or experimentation (beyond their general education); almost no CS papers I've read have contained empirical studies. If anything, they are closer to math papers than science papers!
Additionally, this sort of study is basically sociology. (Or something similar.) These sorts of fields are considered a little shady by hard scientists, and CS people tend to empathize more with the latter. I think this explains the immediate attack on methodology.
All that said, having more studies done about these questions would be great. I'm just not sure who's the best to do them. Maybe HCI researchers? I can't help thinking that the really intense PL people I know wouldn't be very interested in doing this.
"Almost no CS papers I've read have contained empirical studies."
You aren't reading the right papers then. At least in Software Engineering, you can't get into the main conferences (ICSE and FSE) without a pretty significant empirical study.
That's a broad brush, and though it's probably a good categorization, I don't find this distinction to be all that useful as a boundary. For instance, the field of artificial intelligence and cognitive psychology branched at one point, so much of my work in cognitive architecture and algorithmic modeling necessitates user studies. One would be hard-pressed to bucket AI into software engineering though. Likewise, in machine learning, I've seen a push from classical data-driven to modern "data-informed" approaches to analyzing these results. Computational linguistics (NLP) and computational narrative are yet additional examples of fields that often requires user studies or other empirical data.
More to the point, the distinction of what is and isn't computer science has become even more blurry in the research community because research in itself has become more inter-disciplinary. There seems to be little to gain from attempting to "bucket" research into distinct taxonomies.
The only problem is, people have been doing these experiments for 30 years, and do you know what the net effect it's had on the world of programmers: none at all. Saying "no no, this time really listen to this study" seems to be having no effect.
There are a lot of causes for this, not least of all the things you mention (nobody cares about science, people like or dislike based on non-scientific evidence).
But it's also because people know that all these studies are flawed. As much as I hate to be the "nitpicker" who takes apart studies, EVERY SINGLE STUDY on programming doesn't even come close to real-life scenarios. In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them.
But take a look at the Dynamic vs. Static argument. For years, the Dynamic-fans have been saying "Quicker to program, so it's better", while the Static-fans have been saying "Quicker to program, but harder to maintain, hard to use with large teams". So now we have a study that doesn't even come close to addressing most of the issues that have been argued for years! Of course this isn't going to convince anyone.
"In fact, one of the few studies that people actually believe is the "some programmers are 10x better" study, and that was actually fairly well conducted - many students were given identical tasks, and a fairly large amount of time to do them."
Can you by chance point me to this paper? I'd like to add it to my paper collection, since most of the studies I've seen concerning programmer variability use members of the workforce. I'm not aware of the one involving students, but such replication studies are easy to miss.
I'm not sure how that link helps me. I've already seen many of those studies. Which one satisfies, "students were given identical tasks, and a fairly large amount of time to do them"?
That is the specific study I am searching for to add to my list of papers. Did you give me this link because you were referring to Humphrey (A Discipline for Software Engineering), or something else? I can track down Humphrey, but it will take me a few days, since it's a physical book.
My apologies. I misread what you were looking for. Mostly by not actually reading what you wrote. :(
Perhaps the previous author was thinking of the Prechelt "An Empirical Comparison of .." paper? http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprtTR.pdf . Section 5.7 has "work times" for Java and C/C++ programmers using well-observed times. However, that is not for a "fairly large amount of time."
There is no such study. It's a well-regarded and popular myth.
Laurent Bossavit does a masterful job of researching the origins of this myth (and others) in his new book The Leprechauns of Software Engineering, available on LeanPub [1].
Not at all, that study was not well conducted, it compared people just learning to program with people that already had experience, in a time when programming was much different from what it is today... Just go read the damn paper...
I don't know how science can help you here. In the case of the improved fan design, well that's easily testable.
However, I don't want to go down the rabbit hole and argue social vs natural sciences.
I find it dumbfounding that in this day and age, people can still create rather arbitrary social experiments with only 49 people and then think they can draw grandiose conclusions from their "data".
Unfortunately, user studies are difficult to conduct in computer science mainly because it is difficult to get users that you can study. Like it or not, 49 people is actually a pretty big study relative to what is out there. For better or for worse, I have seen much smaller studies accepted/published by top tier conferences.
Software development is so complicated that there is always an endless supply of objections to fire at any study at odds with one's beliefs. And that is exactly how all these discussions go. All we're doing is repeating shibboleths.
The most interesting studies would be ones that changed somebody's beliefs. That doesn't happen very often in our field. Does it ever?
No study can control all the variables enough to convince a fanboy that his favorite language isn't the greatest ever. However, the current state of PL research is as close to astrology as you can get. Why are we working on type systems and modules and concurrency primitives et. al. without a scrap of evidence that any of it contributes to programmer productivity? There's no science there.
In fact, everyone here should flip this around and ask: can you design a practical experiment to compare productivity of dynamic vs. static languages? Will others find it convincing? Probably not. PL advocates are no different than religious missionaries. They have no objective proof of any of their claims. And both will murder the natives if they don't convert.
PL research is not necessarily for "contributing to programmer productivity." Formalization of various programming language concepts into a type system[0] allows researchers to apply analysis and verification techniques. The simplest example I can think of is the Maybe type. Instead of having null pointers and, if you forget to check for NULL, getting a runtime error (segfault, NullPointerException, whatever), instead you would fail to compile since the "null-like" pointer would be a Maybe type, not the concrete value. You can't use it without unwrapping it. Concretely, the code:
int foo(int *p) {
return *p + 17; //well, probably something more complicated.
}
if p is NULL, this code doesn't work. In a language with a more expressive type system, we would write:
foo :: Maybe Int -> Maybe Int
foo p = case p of
Just x -> Just (x + 17)
Nothing -> Nothing
(Note: there are more succinct ways to write this example in Haskell, but I'm trying to illustrate the code.)
In this case, we have demonstrably covered every case. That's what an expressive type system gets you: you can completely preclude certain classes of bugs like null pointer dereferences by having a sufficiently expressive type system. It's the same in more relevant PL research: people are attempting to formalize systems so that whole classes of bugs can be removed at compile time. It's not about user case studies or anything like that: those can come later, when features look like they'd be useful to integrate into languages.
It looks like the featured article gives a case study about a pretty bad type system. A sufficiently expressive type system doesn't get in the way--it aids the programmer, not hinders her. Heck, in Haskell and ML, you don't even have write down types--the compiler will infer them for you. (It is Haskell practice to type-annotate toplevel functions anyway).
[0] When I say type system, I mean a static type system. For the purposes of this discussion, dynamically typed programs are statically typed, just with not-very-useful types.
You say: "when features look like they'd be useful to integrate into languages". How do you know a feature would be "useful" without compelling evidence that this is really a problem for programmers? Now you're back to the original problem of figuring out what the most significant problems are for programmers. Wouldn't it be better to figure this out BEFORE PL people plunge into a particular topic?
Remember all the research done on typestates? The motivation section of those papers was usually a few paragraphs of total BS. AFAIK, there was no real data nor experiment that demonstrated this was a real problem for professional programmers.
FYI: I like static type systems and used Haskell et. al. But no one has demonstrated that it is better than even Visual Basic!
Not really. The research, again, isn't in programmer usability. It's in formal logic and analysis of the semantics of computer programs. Benefits for programmers are just a side-benefit. The goal is advancing humanity's understanding of computer science, not in helping programmers, although sometimes the two goals are somewhat linked and deeper understanding occasionally yields industry benefits. In my example, and in many instances of expressive type systems, research has yielded tangible benefits for industry.
Well, one thought that comes to mind is that the topic is essentially a subset of ergonomics, so perhaps experimental protocols should take a few more cues from ergonomics research.
For example, controlling variables has to be done by constructing artificial systems from the ground up. You can't just pull two commercial products off the shelf and then pretend you're examining the impact of only one of the hundered different things that differs between the two.
Similarly, if we wanted to compare the impact of dynamic vs. static typing, we'd have to make sure that that is the only variable. Which means you basically have to construct a new programming language from the ground up, so that you can easily create new dialects of it that differ in only one very specific characteristic.
The problem is that there are so many soft factors affecting the outcome that it's very hard to tell if even a great scientific study will apply to your case. The number of people on your team, their experience with the technology being used, their relationships between each other, the politics of the working space and thousands other factors may have much bigger effect on the productivity and quality then the choice of a typing system, and it may be pretty impossible to control for them in the study. I agree that it's good to approach things scientifically, but at the same time it's also good to question how much science applies to the problems being solved.
Agreed with your overall premise, but this particular study doesn't meet the criteria to have any conclusions drawn from its results.
The number of programmers is too small and skills not representative (49 undergrad programmers). The problem (writing a simple scanner / parser) isn't one that will really benefit from a decent type system. All you need need are ints, strings, and arrays and you're good to go.
Reasoning from basic principles is a valuable tool to evaluate conclusions. Sure, it has strong weaknesses (hidden assumptions which are wrong, insufficient imagination about what could happen, rationalizing one's biases). But it is useful, when you dont have complete trust in the quality or the scope of the experiment. For instance, claims about quantum computing solving NP complete problems are legitimately held in doubt because of theoretical reasons. Also, whenever there are short term positives hiding a long term negative, like say unsustainable financial or ecological behavior, the negatives might be only seen by a chain of reasoning and not by direct experiments.
I agree in the sense that we see so many cases in the other direction - reasoning full of holes being trusted over empirics. The interesting thing is in any given situation, how much trust to give to the different tools that we have to evaluate a claim.
Counterexample: JavaScript. It is the best demonstration I know of that our choice of languages and tools is mostly based on historical accident rather than any technical criteria.
I second that.It's one of the reasons i quit my day job to do a Master's degree...though i chose a naive enough college and course that am now back to a programming job where i am surrounded by the same type of crowd again.
Oh. Sorry, for that snarky reply(tired i guess). I have a master's in cognitive science(from india). And currently a python programmer here. Mainly because there aren't that many jobs(in cog. sci.) , here and i was not focused enough to get published in a journal during my master's. As for types, well, one ex: is a situation where i was trying to defend my choice of Dvorak kb saying usability and got a dismissive, snorty laugh as a response.
Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously. It's easy to hold a mental model of the code you write in your head and minimize mistakes, it's harder to hold a mental model of a larger program consisting of code written by many programmers.
I'd like to see them hand a pre-written codebase of say, 10,000 lines to a bunch of students, and ask them to make large changes/new features to the codebase.
And good on them. But that doesn't mean it's easy to write large websites in dynamically-typed programming languages, it just means that they are very good at what they do.
- Multi-site development at least in 3 countries;
- Sometimes up to 100+ developers
- CI systems
- Source code of several MB of source code
- Enterprise like infrastructures for Fortune 500 companies
- Different skill sets from the "just out of the university" to the "top developer"
Maybe YouTube beats this, but Google only has Phd guys able to crack out crazy algorithms/data structures in minutes. Not typical in most software houses.
I agree dynamic languages result in smaller codebases, but that's only because they tend to be so much more concise than most statically typed languages.
Also, as for large and complex projects done by distributed teams, I don't have to point much further than Django or Plone to prove dynamic typing works well in that context.
The plural of anecdote is not proof. Heck, it's not even data.
Heck, the examples you give aren't even anecdotes. They're just name-dropping. One would have to be pretty familiar with the codebases in question and the history of their development in order to be able to give a clear assessment of what, if any, impact dynamic typing might have had on them.
Fair enough. They prove large projects built by distributed teams are possible in at least one dynamic language.
There was a time I was familiar with both codebases (I have some catching up to do) and that's why I mentioned them. Both projects carry heavy heritage and are experiencing huge pressures to evolve and both are doing very well (from what I hear on the dev lists).
Sure, but even then one merely has an observation that there are these two large projects that are written in dynamic languages and are being successfully maintained.
That doesn't imply that they are being successfully maintained because they are written in dynamic languages, or despite being written in dynamic languages. And of course there's also the possibility that static vs. dynamic is a wash and doesn't really have an impact at all. Or that the potential impact of going with static or dynamic is heavily influenced by other factors - does the static language have type inference, does the dynamic language support duck typing, stuff like that.
Long story short, correlation does not, in and of itself, imply causation.
In any case, it's mostly an apples to oranges comparison. Observing from my heavily biased toward web applications experience, there aren't as many Python codebases as there are Java ones at least in part because you can do a lot more with a small Python codebase than you can do with a similarly sized Java codebase.
By limiting the damage a single crappy programmer can do.
Everywhere people talk about 'large' teams maintaining a large codebase in this thread, substitute 'mediocre' teams stuck with a poor bloated codebase that is the vehicle for their ambitions. It's just not worth anybody's time to understand its unique needs in detail, especially since any improvement you make today risks being messed up tomorrow.
I can understand using static typing if you already have a large codebase that's statically typed, but is there any reason you'd start a project with a statically typed language?
Yeah the benefits of static typing are front-loaded at the start of a project. I might rewrite in a statically-typed language for performance if it ever needs it, but I wouldn't start statically-typed.
My lisp interpreter above allows me to tear out a lisp function and replace it with a C function, while leaving the unit tests untouched.
>Studying individuals is the wrong experiment. Static typing benefits tooling most when you have larger groups of programmers who much collaborate, often asynchronously.
Absolutely.
And also: static typing benefits is not about "development time" but about maintenance, adding stuff, refactoring etc.
If anything, dynamic typing would be expected to lead to faster development time, which is also why it's used in most prototyping.
Lastly: small sample. Did enough of them use an editor/IDE that could take advantage of a static type system?
Did enough of them use an editor/IDE that could
take advantage of a static type system?
Just because the language is static, that doesn't mean it benefits from an IDE.
For instance, Scala has fairly good integration on top of Eclipse and IntelliJ IDEA, however every time I try doing some work in Scala, I end up cursing and screaming, because these Scala IDE plugins are slow, incomplete and unstable and get in my way. And I can't blame their authors, because Scala is a difficult language to deal with.
Another example would be C++, which has been around for a very long time and is one of the most popular programming languages ever. And yet even Visual Studio has problems with its refactoring/IntelliSense support.
And then there are the Smalltalk environments, still around, still kicking ass.
Here's the thing ... there are static languages, and then there are languages designed for usage within an IDE ;-) Java, C# and Smalltalk are like that, while Scala, Haskell and C++ aren't.
i agree totally with both parents -- i wanted to comment the same...
static typing would more benefit:
* large groups of programmers
* large codebases
* maintenance (which is usually taking more resources then the initial development of a large code base)
aditionally i've come to believe that not all static typing is created equal. most allow `null` to be returned instead of adhering to the type, then some have exceptions and not many languages type side effects (like IO). i'm basically saying that C/C++/Java/C#'s kind of typing is not Haskell's kind of typing, and that the potential gains from Haskell's type system are much bigger on the long run (while it also comes with a steeper learning curve).
If anything, I think this study is actually in favor of static typing. Why? Well, most of the proponents of dynamic typing claim it makes development faster. Most proponents of static typing concede this point, but claim that while it may slow down initial development, it makes maintenance much easier.
Of course, this study is fairly limited and drawing conclusions from it is probably premature. I would not base any policy decisions on it alone.
But, if static typing does not slow you down, it seems worth using even if benefits to maintainability are slight. And if they're nonexistent, using a statically typed language over a dynamically typed one still doesn't hurt you.
Besides, I think static typing is awesome. And I'm a college student--you can trust me! That should be all the validation you need :).
"In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment)."
Maybe the abstract is lying, but that seems to agree with my original premise. That is, the type system had no effect on development time.
Now, the paper makes no comment either way--all it says is that the type system did not have an effect on development speed. My argument is that not having an effect is actually a point in static typing's favor: the main advantage of dynamic typing is supposed to be faster development, after all.
Read further than the abstract. It's just an abstract.
To quote:
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Statistically speaking, 'a significant negative impact' doesn't mean the same as it does in regular English. It doesn't say much about how large the difference is, only that they have enough data to show that there _is_ a difference. When an abstract summarizes that the difference wasn't much to talk about, that does mean a lot more than you give credit.
Now, judging from their test setup, and also from the very low number of testers, I find it very hard to agree with what it seems to be you're taking away from this article. Static typing has more benefit in large code-bases, with multiple programmers, and for avoiding hard to find bugs related to dynamic typing. This doesn't seem to be well reflected by the setup they had.
Also, don't think you should repeat the same comments throughout the HN articles, you are not replying to individuals, but to general readers.
In the first case, the use of a statically typed programming language had a significant negative impact
Considering the size of the sample, I'd guess the difference is rather large to considered significant. By looking at the numbers quickly, it seems to be around 25%. I'm a bit shocked, in fact, because in my own experience, the difference is much larger, but this experiment controls for language and my experience doesn't.
Yes, I was able to read that in both your comments, as well as in the paper.
I suspect you are not understanding my point, or what I said about the meaning of the use of the word "significance" when used in statistics.
edit:
Say you flip a loaded coin that is 50.1% likely to be heads. Now you want to test whether this is loaded, and flip the coin a certain number of times and count the outcomes. If the number of times you flip the coin is too low, you won't be able to say the coin is 'significantly loaded'. It might be either way, you don't have enough data. If you flip it enough times, you will be able to say something about it -- i.e. that either it significantly is, or significantly isn't loaded.
However, in vernacular English, you would still say that the difference isn't very significant. Who cares if it is 50% or 50.1%.
There is no reason to quarrel about the meaning of 'significance' in statistical testing vs. in natural language. Just look at figs. 4 and 5 in the paper, and see that mean times spent for the scanner task were:
And that the difference in statistically significant (p=0.04, Mann-Whitney U-test). Whether 5 vs. 8 hours is significant in the natural language sense, everyone can decide for themselves.
Yes, but I'd risk that a 25% deviation is significant when the sample is 49 students. The smaller the sample, the larger the deviation must be to be significant, but 25% is quite a difference.
Also, it's worth to notice they controlled for language - they used the same language in two flavors - to isolate the typing system difference. It's not a Lisp vs. C thing.
This is an interesting paper, but I'd dismiss the findings as a strong argument in the dynamic vs static typing debate. I think the setup is a little bit skewed as it's plays into the strengths of dynamic languages:
* The whole api consisted of 14 classes, which makes the conclusion that static typing doesn't help with API discoverability somewhat moot.
* We are not talking about Haskell, Scala or even Java/C++ flavor of static typing. The statically typed language offers no generics, no support for encapsulation, no type inference etc.
* IDE support seems to be absolutely minimal.
A methodical problem is that they seem to skip over the abysmal success rate: Only one(!) guy or gal was able to get 100%, more than halve of them was unable to implement a meaningful parser at all. To be honest, it looks a little bit like the subjects were overwhelmed by the unfamiliar syntax and/or most of them where really inexperienced (the paper says that none of them ever implemented a scanner or parser).
The scope of the study also seems suspect to me. From the paper:
"The introduced study is a relatively large study with 49 subjects and required for each subject approximately 27 hours pure development time. Including the teaching time for each subject, between 43 and 45 hours working time was required per subject."
If I were working on a software project where I expected 30 hours of development work, I'm not sure static vs dynamic typing would matter at all. I'd pick the language that would get the job done in 30 hours.
I'm surprised this was accepted in a peer-reviewed conference. I'd give this study more credence if this involved the same number of subjects working on or inheriting a much larger, longer-lived code base. Though, based on the subtitle of the paper, it sounds like the author could be convinced otherwise.
Final paragraph of section 3.2, page 4: "...we have to make sure that the programming task given to the subjects cannot be solved by the subjects completely..."
The conclusion contains a paragraph that matches my initial reaction to the headline and opening. "It seems that the experiment at least does not contradict with a common believe that is often articulated in the area of type systems: that dynamically typed systems have a benefit in small projects while this benefit is reduced (or even negated) in larger projects." (It goes on, but I can't seem to copy actual text out of it, I typed that).
My feeling about this has always been that dynamically typed languages will save time on initial implementation and small projects, and statically typed languages will win on large projects and things that require more maintenance. They acknowledge this, and didn't find anything to contradict it.
Their own graphs seem to show that effect too. In some areas, dynamically and statically typed languages were equal, while in others dynamically typed languages had lower times. I don't actually see where the conclusion that a "static type system had no effect on development time" came from, it appears that it had an overall (if not very large) negative effect. Just like I would expect for a small (27 hours) project.
I didn't crunch the numbers though, maybe I'm just eyeballing it wrong. (again, I can't seem to copy-paste out of it, and I don't feel like reentering all of them)
I wonder whether they are actually measuring the right thing here. I think the real value to static typing is not so much the initial development of small (tiny, in this case) programs, but rather large systems that have long lifetimes and maintenance periods involving multiple developers.
I just started diving into Ruby. I'm about a month into it, after having spent over a decade doing Java and other static languages. The productivity boost for new development in Ruby is very real; I converted a 10K LoC java program to < 1K LoC Ruby in about a week, despite not knowing the language well at all. It almost feels like a "runner's high" to be so incredibly productive. But there are several things that I find frustrating about dynamic typing.
In dynamic languages it's very hard to navigate existing code and libraries. In Java, you can click on a variable and instantly navigate to its declaration and class definition. In a well-configured environment (say Intellij + Maven) you can even click on imported libraries and drill right down into their source, even the JRE. It's totally natural to know exactly what a method takes as arguments, and what it returns.
In Ruby it always feels like a mystery. When you ask the IDE to jump into a library, it pops up a dialog of 20 choices and makes you guess which one to open. It feels like the old days of running find/grep over the source tree. And if you somehow manage to guess correctly and find the code you're looking for, you often have to read the comments (if they exist) to figure out what it expects/returns. I'm sure a lot of this is due to my inexperience, but code navigation is really painful in a dynamic environment. I can only imagine how hard it would be in a non-trivial project after a couple of years with a few different devs.
Of course refactoring is way better in a static environment. Even if you throw out the IDE, you can more or less instantly find where a piece of code is used by simply renaming it and recompiling to see what breaks. In Ruby you have to rely on tests for that kind of thing, instead of getting it for free from the compiler.
Don't get me wrong- I'm excited about Ruby, and I think dynamic languages will play a major role in the rest of my career. I just wish that code navigation could be better.
> The productivity boost for new development in Ruby is very real; I converted a 10K LoC java program to < 1K LoC Ruby in about a week, despite not knowing the language well at all
It's important not to conflate the benefits of Ruby here with benefits of dynamic typing.
Haskell is statically typed but can be as concise as Ruby (sometimes more concise, sometimes less).
I disagree. There's simply no way that the IDE can know everything about a variable without running the code and I don't believe that eclipse does this.
It's ok at refactoring but it breaks down all the time for me, renaming unrelated variables or missing cases. I don't believe it can handle kwargs for example (no computer in front of me, feel free to prove me wrong)
In theory, you're right. In practical, day-to-day programming, an IDE can get it right the vast majority of times and do exactly the right thing. See a previous post of mine: http://news.ycombinator.com/item?id=4037602
I believe that "works almost every time" is great, but that last inch is also meaningful. Knowing you can trust your IDE's refactorings is reassuring and allows you to dare do things you might otherwise not dare.
My big problem with this study is that the project sizes and teams are too small. Static typing really starts to get it's power as the team and project get bigger. This is especially true as the skill levels from larger teams start to spread from amazing all the way down to terrible. In smaller teams you can often work with higher quality developers, but with larger corporate systems, you don't have that luxury. You have to deal with all kinds of experience levels, etc. You need to be able to modulize the project, etc.
From my personal experience, dynamic typing usually works well with 1-2 developers and continues to be useful until about 4-5 developers (static typing too for that matter). After that, static typing usually wins pretty consistently.
My personal preference is static typing. I find it makes the code more readable and catches a lot of oopses early, at compile time. Otherwise you just end up writing a ton of test cases and hope to catch the same issues at runtime. So you have less protection (you have to catch it with a test), and you just end up spending more time writing tests anyways.
I do understand the benefits of both, I just prefer static typing because of the way I work. Both work and have their places. Like everything else, you need to use the right tool for the right job AND the right people.
I don't think the size (or the revenue) of the company matters as much as the size of the projects that you're working on within those companies.
Someone mentioned maintenance, so I will also ask how old the codebases that you're working on are? For dynamic typed languages, as the code grows older (with more bug fixes and add-ons and one-offs) then it gets harder and harder to maintain as 100% error free (let alone debug).
I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass. Some code branches have system level calls changed, and unless you have very good unit test coverages (which may or may not need to be re-written on a library version upgrade) then you may or may not catch the problem during the upgrade. With a statically typed language, a static analysis tool can tell you exactly where and how the library or interface upgrade affects your codebase.
"I will also throw in the fact that doing software library version upgrades on dynamically typed systems is a pain in the ass."
This is very true, but you gain a lot with dynamic languages in other areas. The last company I worked for had about 300 programmers in the office I was in. Roughly half did Java and the other half did Ruby. I was one of the few that moved between both ruby and java projects. I did not see these great benefits of static typing that people always insist exist for large teams at large companies.
Have you also used dynamic languages extensively as part of large teams at your employer? Dealing with lots of people over long periods of time is messy, no matter what language/tool you use. I think the claim is just that static typing gives you better results than dynamic typing. All relative..
The main disadvantage of dynamic typing for me is the lack of verifiable documentation about what data a function needs, and which data comes out of it. This becomes a problem when
a) the data is complex (e.g. dictionary of lists of items with certain properties)
b) I haven't looked at the function for a while.
In those cases, I find statically typed code easier to reason about.
On the other hand, it absolutely maddens me that static type systems force me to spell everything out, even where it's trivial. It slows me down, it bloats the code (especially when you have to create a new class for everything) - which again makes it harder to see the purpose of the code instantly.
That's why I decided to make my own little pragmatic experiment:
In cases where complexity is expected (or experienced) - and only there- I use the Clojure pre- and post-conditions to check the shape of selected parameters and/or its return value.
It looks like this:
(defn german-holidays
"Returns map of german holidays, in the year of d."
[d]
{:post [(like {(date 1 1 2012) :easter} %)]}
...)
You can instantly see how the data is supposed to look like, it will be checked automatically, and it is close to the actual code where you need it (unlike e.g. Unit Tests).
I'm not yet sure how this experiment will fare in the future, so any suggestions or warnings are appreciated.
I use tests for documentation. They seem like more work at the start, but they're more flexible. You can self-document things with tests that you cannot with just types.
You're right that tests aren't close to code. But that can be good or bad. Since it's not next to the code it can be more verbose and thorough without adding a constant reading burden. And when I wonder, "why is this line here?" I can comment it out and run the tests to find out.
I think of tests as records of the input space of my program. Most of the time we go to great lengths to make explicit what we do but not quite what aspects of its behavior we truly care about. Where are the dont-cares in this program? If I refactor it am I restricted to a bit-for-bit identical behavior? In practice that's impossible, and without tests we fudge random parts of the state space without rigorously knowing what is important and what isn't.
Static type systems don't excuse you from writing tests. But they do effectively write a whole class of tests for you, and run them much faster and give you more precise errors than manual testing ever will.
People complain about C++ compile times but I've heard of big Ruby projects with test suites that take 20+ hours to run.
Ah, but a lot of bugfixes are developed by making small changes to understand better what is going on internally during the bug symptom, and there the compile time is more important than the compile/test time.
> You can self-document things with tests that you cannot with just types
Absolutely.
But I would say that is a different debate. In my case, I don't want to specify the behavior, but only the form, which is the counterpart of a static type, but more flexible (e.g. {} is likened to any structure that behaves like a map - the class doesn't have to be exactly the same).
With my like-function, I don't have to spend too much time specifying what is expected, and I get a lot of bang for the buck.
I want to do just enough so the code stays comprehensible and thus manageable.
(fn parse-int [str] ...) needs no type or unit-test to be understood.
(fn parse-appointment [str] ...) is better understood when it has {:post [(like {:id (UUID.) :name "me" :date (java.util.Date .)} %)]}
And as soon as the function gets really smart, and it's smartness isn't revealed directly by the code, it would be time for either a good comment or a Unit Test (or make the code better so that it does, which is an often forgotten option).
You might want to read up on Design By Contract, if you haven't already. It's an approach to writing robust software by incorporating explicit pre- and post-conditions into code. IIRC they were used to generate documentation for functions as well as to do the run-time checks. It originally came from the Eiffel community I believe, but has spread to other languages as well.
I recently talked to a friend of mine who expressed the frustration of not knowing what a function returns because it's a language without static typing. It was something I hadn't thought of before personally. I thought about it a bit and realized that as a Ruby programmer, I depended on two things to figure out what a function returns, and neither is the source or documentation (most of the time at least):
1) a REPL. Just give it a whirl and see what you get.
2) intuitive method names and good abstractions. If I
call @user.posts in a Rails app, I can assume I will get an Array-like object that I can treat as an array without any problems. The actual type does not matter 90% of the time.
> a REPL. Just give it a whirl and see what you get.
A REPL is in no-way tied to having a dynamically typed or interpreted programming language. E.g. Haskell's GHCi has an excellent REPL environment where you can, in addition to evaluating code, ask for the type of an expression and do other static analyses.
I agree that a REPL makes a huge (mostly positive) difference on how you write code.
that's one aspect of dynamic typing that i strongly dislike. you can't figure out what a function does by looking at its signature. if documentation is absent (which is almost always the case), a coding task turns into a reverse engineering task.
A good repl that automatically prints out the function's heredocs is usually enough for me. I like bpython particularly because it shows the argument list and the heredoc as soon as you type the opening parenthesis for a function call. In other REPLs you can usually call a docs(fn) or a source(fn) to handle the rest.
I see where you're going but thats not really what we're doing with that ... @user.posts actually returns an array of posts and @user.posts.first returns an actual Post object.
You could also change @user.posts to be @user.contributions, for example, and that would hide the fact that contributions was just an array of Post objects.
I think we are talking about different things. Duck-typing doesn't help in depicting what the return type of a function is when looking at the function signature only. You need to look at the function implementation (the actual duck) to tell what the return type is.
My point I guess was that with proper abstractions and duck typing you really don't have to care about the type. I only notice something isn't a duck when I try to use it like a duck and it fails - which is surprisingly infrequent.
> a REPL. Just give it a whirl and see what you get.
In a large system you might have to jump through a lot of hoops to get an accurate result from the REPL. What if the thing you're trying to puzzle out is 20 call levels deep? As a new Ruby dev I struggle with this.
In python if I was trying to figure out something complex 20 levels deep, I would stick in an "import pdb; pdb.set_trace()" to get a REPL at that spot. There's probably an equivalent for Ruby?
(Off-topic, the werkzeug debugger even lets you debug browser apps by jumping into a REPL anywhere in the stack trace, in the browser, after an exception is thrown. I think it also has a command that bakes you a pie.)
If it is a pure function it makes the call depth irrelevant. While not always the case, it seems to be pretty common in Ruby development to keep things as pure as possible.
No, it's not irrelevant. If you're trying to figure out what gets passed up the call stack, you need to trace it through. Being able to plug in your own values into a single method call via the REPL isn't helpful if you're trying to analyze what really happens when it runs. Either you need to run it under a debugger, or manually trace it through. In a static language you just have to read the type signatures.
As mentioned by JackC, if you need to understand the context in which a method is called, use a REPL. But to understand what a pure method does should be context free, by definition.
The study followed 49 undergraduates who had taken Java classes. I would argue that, because of this, the students were probably more experienced with static type systems.
A good point. Also, 49 programmers is very little. But even with thousands of developers using all sorts of languages, it's still moot to draw any conclusion from. It's very subjective and dependant on people's experience/taste/State-of-mind..
For me, switching from static (java) to dynamic typing (common lisp) has saved me a lot of development time. Or did it? Maybe it was Emacs? Maybe it was functional programming? Maybe it was the REPL? Maybe experience?
I think in a lot of cases programmers are happiest with their new hotness simply because they became better and more experienced programmers in the process of learning the new hotness, not because the new hotness is 'good' necessarily.
Using students (especially undergrads) is a typical problem of programmer productivity studies. N=49 and 27h of experimentation time per subject isn't a lot either. Doing a study that can draw actually useful conclusions may just be prohibitievely expensive.
It seems to be very basic static typing along the lines what Java or C has stashed on top an Smalltalkesque object-oriented language, so I am not surprised, the many years I used dynamically typed languages I maybe once or twice had a type error of the kind "confusing int with string" (e.g. when selecting a number from database and it being returned as string).
For the benefits of static typing I would rather look into something like Haskell which has a much stronger notion of type and builds many language features around this notion. It's probably harder to design a reasonable study comparing that with for example Smalltalk, though.
I'd be more interested in comparing Haskell to one of those even stronger typed programming languages. I'm forgetting the name of them, but it would be interesting to see how much is too much.
Are you thinking of Agda/Coq/Idris, which have dependent types? If you understand a bit of type theory, this is an interesting paper: http://www.cs.cmu.edu/~fp/papers/popl99.pdf
Not to mention being able to rule out large classes of errors at compile time, and ease of refactoring large code bases. You mean I get all that for free? Where do I sign up?
Obligatory language argument: If by ease of refactoring you mean having to change everything whenever anything changes just to avoid a little messiness. :)
only 20x? What happens when you reevaluate it with the python code not having any pieces that are C under the covers? (my understanding that a lot of python idioms for fast code basically turn into ah "use the approach that pushes as much work into C as possible")
Not in the same way at all.
Bytestring or hmatrix may have their primops in C, but the control flow stuff code for libs like Repa or Vector are pure Haskell (and in fact current state of the art work on Repa has it matching if not beating C code).
The crucial difference is that in python, these faster python idioms are for using C as the control code
The paper's title is casting doubts on the positive impact of statically typed languages as they had a negative impact on development time. (though not quality)
In the experiments the existence of the static type system has neither a positive nor a negative impact on an application's development time (under the conditions of the experiment).
What's the point of dynamic languages if they don't even make the initial implementation faster? They're certainly harder to maintain afterwards.
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
Here's my pet unprovable theory. Development time is dominated by a programmer's understanding of a problem. Static languages force programmers to say a lot about their understanding of a problem.
Dynamic code can be very fast, but in lisp, after algorithms, that means actually going and finding all of the calls to elt and replacing them by calls to nth. With a static language the compiler has a much better chance of specializing the code for you.
I don't think you can show much of anything in 27 hours of development time. Neither the advantages nor the drawbacks of static typing will have any impact on productivity for such a small one off, single developer, throw away project.
It seems the only question this experiment could possibly claim to answer is whether or not the additional finger typing required by static languages affects the coding speed of these particular 49 undergrads.
49 undergraduate students working independently on a small project over the course of a single week using a previously unseen language with no consideration of the performance or maintainability of the resulting code. I can't see how there are any meaningful conclusions to be drawn from this study other than "inexperienced programmers find learning new dynamically typed languages easier".
Not at all. When learning any new programming language there's going to be a period where you're working by trial and error. Removing compile-time typing generally means less syntax to learn which makes the feedback loop tighter which I would imagine gets you the point of familiarity quicker. The study seems to back that idea, but not much more.
In my own experience: if I write dynamic code I'm still thinking of types. Especially in Python.. What's mutable, what isn't... This is what my professors at school say too. Less typing (pressing keys) doesn't make you think faster!
Seconded. Even though i write python code, i am still thinking of types. Infact, it's one of my gripes with the language. There are some tired mental states, when i don't think/forget to think about types, but just write and call functions. It's at those times that i find having to go back and read the original function for a type or having to go to REPL for testing the type painful. I would be happy to have a compiler tell me what type is expected instead. I guess more experience will lead me to infer this from the actual error message itself.(ex: Nonetype has no function iter)
You should strive to not rely on type-checking, and trust and rely more on duck-typing. If at some point you need to ensure something is not None, test it and act accordingly in the alternate case. If something absolutely must not be None (or must have a specific method), ever, use assert() so that things blow up upfront where you can infer (or even read) the reason, and not deeper in the code where the actual error message, as you mention, has to be parsed according to the whole call stack to make sense of it.
Sometimes you may need to type-dispatch when you want to do smart functions. A typical example is a function that takes a sequence/iterable/generator of strings or a string. Since a string is iterable in python, you have no choice but to check for BaseString.
Duck typing requires more mental effort to track the variable types. And in python some type errors are detected only in runtime, like when you call a function with a wrong parameter count.
I'm a bit puzzled which role programmer ability has in this experiment. What strikes me is the enormous spread in programmer quality within each group. Varying from 0.7 to 15 hours for the scanner assignment and 50% to 100% of succeeded test cases. The spread occurs over both groups.
That programmer quality shows way stronger effects than the measured effect of a typing system would suggest to me that
- the measurement was flawed?
- programmer ability beats language choice by orders of magnitude?
That programmer ability beats language choice by orders of magnitude, is already a familiar fact, so that would seem a satisfactory explanation for the observed variance.
What about maintenance? I've never heard a static-typing proponent claim they can develop faster. It's always that they can maintain the software with much less work and have less overhead at run time.
I've wondered about languages that are "hybrid" in the sense that the type system can be turned on and off, because from an intuitive sense, whether the language is meant to be used as a dynamic language has an impact on its design, similarly with a static language.
You take Smalltalk for example, as it is the original archetype of a language that benefit from blurring the lines of the type system. A language like Haskell on the other hand was design with the intention that the programmer would use the type system to do a lot of heavy lifting.
Of course, no real-world language is "pure" in that sense. We've graft type systems onto Smalltalk's "way of doing things", and the GHC Haskell compiler now allows you to treat type errors as warnings.
The change in GHC is just to make development easier. Basically, just imagine that instead of stopping at a type error, the compiler replaces the incorrect function with code to throw an error at runtime. So here the poorly typed function always errors, no matter what you pass it; in a dynamically typed language, it would work as long as the arguments passed at runtime have the correct types.
So it isn't anything like dynamic typing, or even quite like making type errors warnings--it's just a way to compile partially incorrect code to make changing the code incrementally easier. It's also not designed to be used in production.
It just automates the existing practice of temporarily commenting out offending functions and replacing them with `undefined' to get the whole file to compile.
Let's summarise what the study actually found. The researchers go in with the expectation that static languages will have advantages in their study. They designed a brand new language - 'Purity' - in both dynamically and statically typed variants, so there was no difference in documentation or other language features. The only difference is that one forces you to declare variable types, and checks them at compile time rather than at run time.
They take 49 students and train them in the language - 16 hours for the dynamic version, 18 for the static, because of "the additional effort for teaching the static type system". The students have 27 hours to complete two related tasks. All activity was logged, so they could look at the time from a test run failing with type errors to a successful test run.
- Scanner task: overall, students using the dynamically typed language were significantly quicker to complete a scanner that passed the tests - on average, about a third faster. Within that, the time spent debugging type errors doesn't differ between the groups.
- Parser task (builds on the scanner): The implementations are tested after coding has finished, showing no significant difference in success between the languages. In both groups, about half the students didn't manage a meaningful parser at all. In this part, students with dynamic typing spent significantly less time debugging type errors.
I think there are several flaws in this study, but alas, its subject is quite inflammatory and it is easy for people to gloss over the details and start drawing conclusions.
Here are a few things to keep in mind, IMO:
- Good/capable/flexible static languages are much much harder to design than dynamic ones. Case in point is Haskell, where hundreds (thousands?) of researches have been working for years to widen the area of expression vs. area of safety. The devised language for this study seems very minimalistic to be useful. Most dynamic languages, on the other hand, pretty much let you do anything even when they have a simplistic design.
- I find that people take longer to become fast/useful in static languages. While you can just throw things together and tweak until it works with a dynamic language, a static one will force you more to know what you're doing. I had a semi-technical colleague who had a much much easier time getting up to speed with Python than with any other static language. Therefore, unless the study involves a long prep time, it will naturally be skewed towards lower devel time in dynamic.
- Static languages start shining as the project size/time-line grows. Key affected areas are refactoring, intra-team communication and incremental improvements on the code. To test for this, it would be interesting to stage a time lapse where developers are asked to modify their previous parsers after, say, a month of total separation.
I think a more interesting study would be to hire 20 highly decorated Python devs and 20 highly decorated Haskell devs and give them a few different computational tasks, overall adding up to a large # of hours. Naturally, the study needs to limit usage of 3-rd party libs and take other normalizing measures. Then we may get a sense for what the state-of-the-art in both fronts have to offer.
Well, definitely real-world tasks with focus on usefulness and other practical concerns. Ideas would be:
- Implement a parser for some protocol/data with testing, etc.
- Implement a communication protocol/framework/driver/API
- Implement a sophisticated-enough simulation for a real-world question you're trying to address certainly with some calculations inside.
- Implement a game AI as defined by some spec
- ...
No negative effect of static type systems for green field development sounds great to me.
I do like static type systems (like Scala) more than dynamic ones (like Ruby), and always thought it would perhaps people slow down. Good to hear it doesn't.
One would need to look into brown field development, if there is any positive/negative effects (I assume positive ones)
Looking over the comments, WOW, please please PLEASE read the paper! This title on HN is very misleading. A static type system had no POSITIVE effect on development time. The study is casting doubts on POSITIVE impact of static type systems.
To quote:
We measured two different points in the development: first, the development time until a minimal scanner has been implemented, and second the quality of the resulting software measured by the number of successful test cases fulfilled by the parser. In none of these measured points the use of a static type system turned out to have a successful impact. In the first case, the use of a statically typed programming language had a significant negative impact, in the latter one, no significant difference could be measured.
This study is way to small and limited to show much of anything... but many of the proponents and historical engineering studies that advocate static typing don't claim that it reduces "development time" so much as reduce the times/costs throughout the entire life cycle of the product. In fact, the "development time" or in this case "prototype time" hasn't been all that terribly interesting.
What would be interesting is to take these same projects, and same team members and add some features a year later and see what that looks like and then take the same project and give it to a new team after 6 months and have them add some minor feature or fix a bug and measure that. Or double the performance of the project and measure what that costs, if it's possible. etc.. It's fun to argue and advocate for what you like though.
Is it not so that development time is always maxed out automatically? Programmers improve the system as far as they can, and the study explicitly says that it did not address "design time, readability or maintenance time". Since software is almost never perfect (unless constructed with proofs, and you can make optimizations even there), I would think that programming time depends on the programmer's willingness to invest yet more time into its features - and this willingness is most likely not related to the language in use, but rather to financial or temporal constraints. Am I missing something?
If static typing wasn't useful, I don't think anyone would bother with types. At the same time some static type systems are restrictive while coding, so you're seeing more and more type inference in languages.
Go does it by allowing one to declare a variable with an expression " a := 1".
C++ 2011 has it with "auto"
Haskell is pretty darned good at it, and a lot of people don't write the types of their functions, though I think it's good practice to make sure you and the compiler agree on what you've written.
That said there's languages like Clojure which are dynamically typed in a sense, and I believe pay a slight performance penalty because they have to use reflection to behave that way. There's ways to annotate types in Clojure to get around that. This is an interesting case because Lisp is typically a dynamically typed language.
At the same time there's awesome languages and environments like Racket that have statically typed and dynamically typed versions of their scheme dialects.
My feeling is that a ruling on whether types are useful or not is sort of a pointless discussion. It's not even true that "the jury is out". You're just choosing tools with different properties and some make sense for some situations and others for others.
This study, while perhaps not very relevant to real world software engineering (development, maintenance, refactoring), seems to show that there when developers design code and applications, there's little difference for them between using dynamically or statically typed languages. And that's interesting, because it implies that the way developers think about code is somehow type independent.
The static versus dynamic typing "battle" is kind of from the kindergarten. You need both, similarly as to how both boys need their fathers even though both of them get declared the strongest in the world in turns.
With dynamic typing it's much more difficult to create large, reliable codebases. There's too much implicitness among the interactions of different actors, modules, and programmers.
With static typing it's much more difficult to create any codebase. The exploratory nature of programming means that you will have to constantly change and redefine the static types, signatures and interfaces.
What you want is a language where you can start prototyping dynamically and later on when the APIs stabilize gradually you can then stamp static type checking on top of them for cementation without actually having to rewrite anything, possibly in another language. When you can selectively force a freeze on a certain piece of code or API then you can gradually move from dynamic to "static enough" typing as the part of your development process.
If you want to reduce implicit interactions between components, using FP over OOP will probably give you larger gains that static typing over dynamic typing.
While interesting, I don't think we can consider the conclusion as the final word on it. There are many other variables to take in account.
The programmer's expertise is an important one and it is difficult to take in account. Static typing is to check code validity. If the programmers make no mistake, then of course specifying types is an overhead. When it comes to collaborative work, strong typing can save a lot of debugging time, meeting time and documentation writing and reading time.
Another aspect is the resulting code efficiency. Code is programmed once and executed many times. It is much easier for a compiler to generate efficient code when it is given more information to do so by the programmer. This benefit of the typing overhead is not taken in consideration.
As repeatedly said, use the appropriate tool for your application.
"According to the known guidelines for empirical research in software engineering (see for example [16, 17]), it is necessary to make explicit all elements that potentially threaten the validity of an experiment."
Or, y'know, according to any basic understanding of science whatsoever. Jesus.
They're essentially trying to test whether static typing optimises development time, and as with any optimisation, you only see a substantial increase if you're optimising the bottleneck. Static typing won't show any speed benefits if your coders are being slowed down by weird abstractions, poor tooling, or analysis of the problem.
The experiment didn't focus on use of the type system, but nor did it actually test anything close to a real-world scenario. All in all, it seems worthless to me.
The author appears to misunderstand statistical significance when he makes claims such as "the existence of a static type system has neither a positive nor a negative impact on ... development time."
The mean development time for dynamically typed languages is about 30% shorter in his first experiment, and it is about 20% shorter in the second experiment.
Statistical significance test determines how likely this difference would be if the difference development time in a large population was 0... It doesn't mean that the true effect actually is 0... it's just addressing whether we can completely rule that out.
A more accurate summary of his results would be "Dynamic type systems allowed 20% to 30% faster development in my experiments, but I had too few participants to reject equality."
Development time with static languages is not an issue.
Try comparing the time one needs to change the codebase written in ML and the one written in Lisp and come back. It is experimentation and change that is slow in static typing.
I do not believe so. In languages like ML, static typing is not just used for type safety. Proper use of static typing extends to be able to use it in establishing invariants all over your code. Now, your entire codebase becomes dependent on the type. No matter how expressive a language is, this causes a lot of friction.
If anything, it is my experience that static typing makes it much easier to refactor large amounts of code.
I remember numerous instances where I make a large change in one Haskell module resulting in a few changes in the module's external interface, the compiler systematically warns me of all the 20 other modules that needed changing, and lo-and-behold, if it compiles, it works.
An interesting study, but really hard to compare. It's like attempting a discussion of the effectiveness/efficiency of Engligh vs French (using conjugations or sex).
And in particular, trying to compare the effectiveness of English vs. French by using 49 undergraduate students who weren't proficient in either language.
Maintenance... Dynamic would be faster for me to write in from 'scratch' (I find it weird it has NO effect on dev time, but then again the study isn't really broad), however after your sprint(s) changing things is really much easier with static typing. That's what I feel at least, especially with Haskell (and Hoogle) (which unfortunately I don't get to use that often).
It could also be the case that it is faster to write text parsers in dynamic languages as opposed to static ones. There are so many different variables to come to a meaningful conclusion. What about working on very large programs with other people?
Type restrictions benefit larger teams, working on larger projects, with longer periods of time between looking at the code (working with legacy code, in other words.) A larger study should be tried.
Meh, dynamically typed systems are within an order of magnitude of statically typed systems, even on benchmarks that greatly benefit from the static type information.
This study starts with a bizarre premise. I've never heard anyone, ever suggest that statically typed languages make initially writing a program faster. Very much the opposite, java apologists claim that their slower development time is offset by easier maintenance and less bugs, explicitly acknowledging that initial development is slower in a java-esque static language.
Of course, the static typing proponents you find on HN are generally not advocating incredibly limited type systems like java or the one used in this "study". I find it hard to believe that a language with such a limited type system is representative of languages with useful type systems like ocaml or haskell.
Well I think the article hints at the real benefits from type systems. They took longer in the beginning understanding the type system, but made up for it later on by being able to finish in the same amount of time. Static systems be balling for refactoring.
Hopefully for the next experiment they have them tackle a longer project, and have the programmers tackle short problems in multiple languages to 'rank' them.
I don't know. I have an equal (equally small :() amount of experience with both. And, amusingly, I've disliked both.
What I found after using Java, Gosu (basically Java but less stupid), Python and JavaScript fairly significantly is that all of them have their own problems. In that particular set, I'd probably value a static type system a little more than a dynamic one, but it gets eclipsed by other language characteristics (e.g. support for functional programming).
In fact, thanks to being scarred by Java, for the longest time I really liked dynamically typed languages more. However, I later realized that it was the lambdas and the higher-order functions and the clever abstractions and the terseness that I liked more than the dynamic typing.
Once I learned Haskell (and later OCaml), I've been converted to liking static typing quite a bit. The beauty in these two languages is not just that the type system catches more bugs than Java's while being less of a burden, but also that they type system can actually make the code more expressive.
My favorite example of this is the read function from Haskell. It is the opposite of toString--it goes from a String to some value. However, the really awesome thing is that you do not need to explicitly specify what type to parse to--the type inference can figure it out for you! Imagine being able to use "5".fromString() the same way you use 5.toString(). In a dynamically typed language, you would have to somehow specify what to parse to explicitly. So your fromString code would something like Integer.fromString("5"), which is less nice and throws away the toString/fromString symmetry.
I have about the same experience with dynamic (Perl, Scheme, JavaScript, Python) and static (Java, Gosu, Haskell, OCaml) languages and I like good static typing most.
I lived almost entirely in Ruby from about 2003-2010. Going back to statically typed languages for iOS development has reminded me now nice it is to let the computer catch all my trivial mistakes.
On the rare occasion you really need the kind of anything-goes dynamism of Ruby/Python/JS you can just roll your own dispatch tables.
I once read an article about a technique Intel had developed for improving cooling of processors by changing the shape of the fan. I related this to some of my co-workers. One of them proceeded to tell me that this can't possibly work, backing up his argument with "reasoning" based on off-the-cuff remarks about the way air and physics "must" work. The fact that Intel had actually done this seemed to have no effect on his eagerness to continue the "debate" about this scientific fact.
I have a hard time believing that I get no benefit from using Haskell over Smalltalk, but if a body of science were to appear that cast doubt on that belief, the appropriate thing to do is change the belief, not stand around debating from imagined first principles why all the science is wrong and can't be so. Shut up, design an experiment and go prove it!
Perhaps there's little of this kind of actual science in our computer science because it would mean asking hard questions and accepting difficult truths. "The prisoner falls in love with his chains."