Stop working, move on immediately to other work. If the amount merits legal action, go for it, if it's too small — just forget about it and move on.
Moving on is the most important thing: don't get fixated on this situation and its negativity. Every hour you spend thinking about this is an hour wasted.
In the future, monitor the amount of outstanding unpaid work closely. It seems you were doing that, which is great — you should always be aware of what that amount is. Don't let it grow, because once it grows too large, the client gains power over you. As long as it's small, you can always move on, and it seems it's what you should do in this case.
If you had hard to dispute conditions, why not sell the debt? You immediately receive anywhere between 40 and 80% of the amount due, and it's not your problem anymore...
Simply because they now have leverage over future payouts. If the amount outstanding is significant to you (say maybe 1/12th of your annual income) then the client can do things like:
"We're really close to being able to afford to pay your bill, but I won't be able to do that until you give us the new features we talked about."
So now you're at a crisis point where you have to decide if you're going to keep putting in "free" work with a client that owes you money in hope of future additional payout.
Whereas if the guy is just late on a day or two of pay and you can say "I can't do anymore work until you've paid your invoice", then you can walk away free and clear with little lost sleep.
It's like that old saying (rough paraphrase):
If you owe the bank $10,000, it's your problem.
If you owe the bank $10,000,000, it's theirs.
Sunk cost fallacy. You're exerting more effort with no reasonable expectation to see it repaid. The work you have already performed is a sunk cost, it's gone. The unpaid invoices are money you are owed but they apparently have no intention of paying you.
For all you know, they're not paying because they're bankrupt and dead in the water. That means not only will you not ever see the money they already owe you, but if you do any more work, it's effectively pro bono for a lost cause.
If you're perfectly rational, there is no psychological disadvantage to continuing to work for them but it's obvious to you why you shouldn't in the first place. Chances are, you don't act as rational as you think you do.
If a major client skips payments, it's perfectly understandable that you don't want to risk burning any bridges by insisting on being paid. But this is the fallacy. Whether the client is big or not doesn't matter if they're not going to pay you. Unless you want to build a business on not being paid.
Because they can use the amount of the debt as a bargaining tool. If you owe the bank £1000, you have a problem. If you owe the bank £1bn, the bank has a problem.
Because then you simply can't just walk away. You now have this massive amount you'd like to collect and the client has that over you. So you keep talking to him, he keeps giving you excuses, "maybe if we just get to this milestone I can start paying you a portion..." blah blah blah. You end up digging yourself further in the hole trying to recover the money you are already owed.
Because he can say: I will pay you once you've done this or that. If the outstanding amount is large, you are more likely to go along with this, because you're hoping it will get you your money without legal action. If the amount is small, you can tell him to fuck off and pay you now for the work you have done or find someone else.
If you don't have money in the bank and need payment to cover living expenses, then you'll be wasting your valuable time and resources pursuing the debt. That time and energy could be better spent on pursuing new clients.
By keeping client balances small, you are in a position of strength, and can walk away if the client doesn't pay.
It probably has to do with the debt owed to you and your missing funds. If you're counting on that cash for your livelihood and you need that $5K to pay rent and eat, you're going to be at their mercy. Legal actions or otherwise can sometimes take so long they aren't necessarily the best solution.
If it's $100 you walk away, and if it's $100,000, you're less likely to. The problem is, it's not quite a sunk cost -- you may still have a chance of recovering it.
But that hope is also what the person who owes you may use to con you into doing more work for free.
Moving on is the most important thing: don't get fixated on this situation and its negativity. Every hour you spend thinking about this is an hour wasted.
In the future, monitor the amount of outstanding unpaid work closely. It seems you were doing that, which is great — you should always be aware of what that amount is. Don't let it grow, because once it grows too large, the client gains power over you. As long as it's small, you can always move on, and it seems it's what you should do in this case.