There is a legal problem. Unless the contract specifically states that the IP for the project belongs to the buyer, the software belongs to the programmer, even after it has been paid for. Threatening him with legal action is a waste of time.
It's unintuitive, but most software contracts do not explicitly give the buyer the IP. It doesn't sound like either of the parties involved know what they are doing, so it's unlikely the contract gives the buyer any power over the IP.
A contract is only valid if something is exchanged in both directions. This is a legal principle called consideration, and in the absence of consideration a contract has no legal legitimacy.
Without some transfer of IP rights either by copyright assignment or by bare license I don't see what consideration would be for a contract to produce custom software.
If the contract is determined to be invalid the buyer can then make claims to the IP under the doctrine of promissory or reliance-based estoppel since they have built a business on the expectation that the software would be delivered with appropriate rights to distribute and modify it.
What precisely is transferred depends on the contract and the legal jurisdiction.
I have some clients that were gypped for a lot more than USD2.5k - they thought they were getting the IP and instead they were getting a non-transferable license to use (and the software was buggy and had - appallingly - developer backdoors and remote off-switches).
Explicit terms in contracts is a very smart move; don't assume anything about what the local version of contract and tort law will deliver to you.
Of course, off-shored development makes it all a bit more complicated - which party actually holds source code and can be sued? It's a bit of a mine-field.
You don't have the source code? He just gives you a crippled binary to demonstrate his progress which only works after authenticating with his server?
I'm not sure why people here are telling you that there is either some ethical or legal problem with taking control of the work you already paid for.