In order of decreasing importance, I recommend you: Constantly check that you're building the right thing. Get some kind of testing framework. Look into Test Driven Development as a methodology.
With a testing framework, you won't waste time rolling your own. For Java, you could start with JUnit.
TDD is a formalized approach to writing test functions as you work, which you already do, while simultaneously designing your code. There are many sources on it around the internet, but Kent Beck's book took the magic out of it (a good thing) for me:
http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0...
Of course, there are many other specific techniques you can use to test that your algorithms do the right thing, all dependent on what you've made. Regardless, I try to make sure that the a real user gets their hands on an up to date build as often as possible, and they're sure to show me all of the ways that my software is technically excellent, yet in no way solves their actual problem.
Also, please check out 'Behavior Driven Development's or BDD, which is basically "TDD done correctly." It alters the description of the process a bit, and in the end, you write better tests.
Since I'm a Rubyist, I use Cucumber and RSpec to do testing. http://cukes.infohttp://rspec.info (note that you can use cucumber to test any language, and I bet you could use JRuby to test java.
But really, conceptually, BDD isn't that complicated. The hardest bit is figuring out the tooling for whatever language you're using, getting comfortable with it, and practicing. I know the Ruby side of this well, but if you're using another language, I'm not sure I can be of much help.
Adding on to your comment, a friend let me read through a beta copy of The Rspec Book last year. At the time, only the first couple chapters were written and it was lacking a lot of content. It was a great resource though, and I would highly recommend it. The final release is due "Soon", and ordering it now gives you access to the current revision of the Beta, which I hear is complete sans-editing. I'm holding out until I can buy a paper copy.
These are all Ruby or Rails specific. Cucumber itself is a DSL to be used along with another language, and there are projects underway to implement Cucumber or Cucumber-like frameworks for other languages like Python, Java and .NET
Thanks. I almost linked to the RSpec Book, but since I haven't read it myself, I didn't feel comfortable endorsing it. Good to know it's shaping up! Maybe I'll have to pick up a copy.
With a testing framework, you won't waste time rolling your own. For Java, you could start with JUnit.
TDD is a formalized approach to writing test functions as you work, which you already do, while simultaneously designing your code. There are many sources on it around the internet, but Kent Beck's book took the magic out of it (a good thing) for me: http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0...
Of course, there are many other specific techniques you can use to test that your algorithms do the right thing, all dependent on what you've made. Regardless, I try to make sure that the a real user gets their hands on an up to date build as often as possible, and they're sure to show me all of the ways that my software is technically excellent, yet in no way solves their actual problem.