"Since every value in a dynamic language is classified in this manner, what we are doing is agglomerating all of the values of the language into a single, gigantic (perhaps even extensible) type."
Yes, that's right, but you're overlooking the upside of doing things this way. What this gives us is the ability to define new types -- to extend the universal type, if you want to put it that way -- at runtime. No longer do we need this strict separation between compilation time and runtime; no longer do we need the compiler to bless the entire program as being type-correct before we can run any of it. This is what gives us incremental compilation, which (as I just argued elsewhere, http://news.ycombinator.com/item?id=2345424) is a wonderful thing for productivity.
"[...] you are depriving yourself of the ability to state and enforce the invariant that the value at a particular program point must be an integer."
This is just false. Common Lisp implementations of the CMUCL family interpret type declarations as assertions, and under some circumstances will warn at compile time when they can't be shown to hold. Granted, not every CL implementation does this, and the ones that do don't necessarily do it as well as one would like; plus, the type system is very simple (no parametric polymorphism). Nonetheless, we have an existence proof that it's possible at least some of the time (of course, it's uncomputable in general).
"[...] you are imposing a serious bit of run-time overhead to represent the class itself (a tag of some sort) and to check and remove and apply the class tag on the value each time it is used."
For many kinds of programming, the price -- which is not as high as you suggest, anyway -- is well worth paying.
In particular, dynamicity is necessary whenever data live longer than the code manipulating them. If you want to be able to change the program arbitrarily while not losing the data you're working with, you need dynamicity. In dynamic languages, the data can remain live in the program's address space while you modify and recompile the code. With static languages, what you have to do is write the data into files, change your program, and read them back in. Ah, but when you read them in, you have to check that their contents are of the correct type: you've pushed the dynamicity to the edges of your program, but it's still there.
For this reason, database systems -- the prototypical case of long-lived data -- have to be dynamic environments, in which types (relational schemata, e.g.) can be modified without destroying the existing data.
So to argue -- rather arrogantly, I might add -- that dynamic languages are really static languages is to overlook an operational difference that is a commonplace to anyone who uses both.
Let me expand just a little on my point about files. When your data are in files, they're just "raw seething bits" (to quote a colorful phrase I once heard); they have no type structure. To turn them into structures in memory, you have to parse and validity-check the contents. (This can be an expensive operation!)
Dynamic languages give you a middle ground: the stuff in memory is more structured than "raw seething bits", but less structured than data in a statically typed program. This is often very handy, as it's much more convenient to operate on data in memory; the slight performance cost relative to fully statically typed data is often no big deal.
Isn't this a property of the implementation rather than the language, though? Couldn't you have a compiler option to keep the type tags attached to the bits, sacrificing some of the speed benefits of static typing but keeping the primary advantage of proofs/testing?
You would still need to verify the data is still typed correctly. You just can't trust bits in general.
I've come to thinking of a strongly-typed program as being like a cell. Once you cross the cell boundary and it lets you pass, you can trust it and do all the wonderful type-based magic and optimizations that we know and love, but you have to get past that cell boundary first. And as annoying as that may be, poking a gaping wound in the cell wall and just jamming stuff in is likely to have bad results. Even between two cells of the same type, you really ought to do the checking, lest it turn out they not be quite as same as you thought (versioning issues).
(And of course in both theory and practice you still can't truly fully "trust" it even after it gets past the cell wall, but at some point you hit the limits of what you can verify. Real-world cells have the same problem too.)
1) If CL can only sometimes enforce invariants about type sometimes, and in special cases, I'd argue that the original statement was true. You'll still be deprived in many (probably most) cases. (People have survived falls where their parachutes didn't open, but in an essay most people wouldn't say "You can of course jump out of a plane without a parachute" despite the existence proof. And saying "Although there have been cases where people have survived, you can't generally expected with any reasonable success rate to survive a fall from an airplane without a working parachute" just isn't good writing.)
2) I don't get the database analogy. When I add a column to a table, the size of the table on disk does indeed change. Not to mention the schema can be changed statically. If you add/remove/modify a Field in a FieldList, the type doesn't change. So that really isn't a static vs dynamic issue.
About 1): I think the point was that CL can only sometimes enforce invariants at compile time. If the compiler can't prove that something is always true or always false, it will simply emit code to do a runtime check.
But as a general principal, you can't reliably or consistently do this in a dynamic language in any sort of guaranteed fashion, which was implicit in the original statement being refuted.
Even with a runtime check, you still CAN pass in a bad value, and the program will just fail, either locally or globally.
(Not to mention that the paragraph immediately following the refuted statement starts with 'Now I am fully aware that “the compiler can optimize this away”, at least in some cases, but to achieve this requires one of two things (apart from unreachable levels of ingenuity that can easily, and more profitably, be expressed by the programmer in the first place)...')
1) I think the right question to ask here is, how often is the lack of compile-time checking a problem in practice, and how big a problem is it? I've worked extensively in both static and dynamic languages, and my experience has been that in dynamic languages, while I do miss static type checking occasionally, it happens quite a bit less often than your parachute analogy would lead one to imagine.
Look at it this way. Programs have lots of important properties. Some of these we have figured out how to encode as types so that we can verify them statically. But (although research in this area is ongoing) there are still a lot of important properties we require of our programs that can't be statically verified. The upshot is, we have to test them. Yes, testing is necessarily imperfect, but we have to do it anyway. In my experience, in the course of testing a program in a dynamic language, the kinds of errors that would have been caught by static typing are relatively easy to find by testing; usually, the errors that are hard to find by testing are also well beyond the scope of static typing. Maybe that will change eventually, though I'm skeptical, but it's certainly the state of type systems in common use at the moment.
So the parachute analogy is grossly exaggerated; it's more like leaving your shoelaces untied.
When Doel says "you're depriving yourself of the ability to state and enforce the invariant...", it's not clear whether he means "always" or "at least sometimes". I concede that he could have meant the latter, in which case you're right, my counterargument fails. But as I've just argued, there are lots of other invariants that we can't statically enforce anyway, and they tend to be the more important ones.
2) A table is a bag (multiset) of tuples of some specific type (schema). When you add a column, the table now has a different type, because it's now a bag of tuples of a different type.
Imagine that instead of using a database you keep all your data in a running process of a program written in a static language. (Let's ignore the possibility that the program or the machine might crash.) How are you going to structure the data? To make use of static typing, you need to use collections of records (or "structs" or whatever you like to call them). How then would you add a field to one of these record types? There's no way to do it without recompiling your program, and there's no way to do that without either losing your data or dumping them to files and reading them back in to the new version of the program.
Now, you could store your data differently, by explicitly modelling rows as maps from keys to values. But what's the range type of your map? Why, it's the union of the types of the possible data values you might want to store; you've lost static typing for the data values, and will have to check at runtime whether each value you read out of one of these row maps is of the correct type.
Programs written in Python are often augmented with programs written in C. Didn't Python become popular BECAUSE it worked seamlessly with C programs? Use both types of languages together.
But I'm interested in whether type-inferring languages will ever take off, bringing the best of both worlds, i.e. the code tersity of "dynamic" and the compile-time checking of "static". I'm not sure if types can be changed dynamically in these types of languages, though.
In particular, dynamicity is necessary whenever data live longer than the code manipulating them. If you want to be able to change the program arbitrarily while not losing the data you're working with, you need dynamicity. In dynamic languages, the data can remain live in the program's address space while you modify and recompile the code. With static languages, what you have to do is write the data into files, change your program, and read them back in.
Not completely true. While I don't know of many use cases for doing what you're talking about, there is at least one that I know of, which is debugging.
In the case of debugging, you can change code with static languages almost arbitrarily while the data stays live ine program's address space. Although one thing that is not possible to do, with current implementations, is change the type definition. But otherwise changes can be made to code w/ data never leaving main memory (I probably do this 10 times per day).
Now it is theoretically possible to also change type definitions, but that would require a notion of constructor that builds the new type from the old type. In some cases this can effectively just be an interface, then its really just a noop.
Yes, that's right, but you're overlooking the upside of doing things this way. What this gives us is the ability to define new types -- to extend the universal type, if you want to put it that way -- at runtime. No longer do we need this strict separation between compilation time and runtime; no longer do we need the compiler to bless the entire program as being type-correct before we can run any of it. This is what gives us incremental compilation, which (as I just argued elsewhere, http://news.ycombinator.com/item?id=2345424) is a wonderful thing for productivity.
"[...] you are depriving yourself of the ability to state and enforce the invariant that the value at a particular program point must be an integer."
This is just false. Common Lisp implementations of the CMUCL family interpret type declarations as assertions, and under some circumstances will warn at compile time when they can't be shown to hold. Granted, not every CL implementation does this, and the ones that do don't necessarily do it as well as one would like; plus, the type system is very simple (no parametric polymorphism). Nonetheless, we have an existence proof that it's possible at least some of the time (of course, it's uncomputable in general).
"[...] you are imposing a serious bit of run-time overhead to represent the class itself (a tag of some sort) and to check and remove and apply the class tag on the value each time it is used."
For many kinds of programming, the price -- which is not as high as you suggest, anyway -- is well worth paying.
In particular, dynamicity is necessary whenever data live longer than the code manipulating them. If you want to be able to change the program arbitrarily while not losing the data you're working with, you need dynamicity. In dynamic languages, the data can remain live in the program's address space while you modify and recompile the code. With static languages, what you have to do is write the data into files, change your program, and read them back in. Ah, but when you read them in, you have to check that their contents are of the correct type: you've pushed the dynamicity to the edges of your program, but it's still there.
For this reason, database systems -- the prototypical case of long-lived data -- have to be dynamic environments, in which types (relational schemata, e.g.) can be modified without destroying the existing data.
So to argue -- rather arrogantly, I might add -- that dynamic languages are really static languages is to overlook an operational difference that is a commonplace to anyone who uses both.