Actually, that's a pretty clever optimization for caching, IMO. The user tends to leave apps in memory, leading to much snappier performance on machines with sufficient RAM.
EDIT: And if you're savvy, you can always manage memory manually and quit from the top menu or from the Dock.
If the OS is intelligent, the memory doesn't get overwritten until it is needed. Even if an app is completely closed and its process is killed, doesn't it's image still hang around in memory for the next time it is run?
The binary and libraries and accessed (and perhaps saved) files should still be in whatever buffers/cache the operating system maintains in the unused portion of RAM, but the state of the application is gone (unless the application saved its state and will restore it upon starting up again, of course).
EDIT: And if you're savvy, you can always manage memory manually and quit from the top menu or from the Dock.