There is a ton of wisdom in this comment. Others would do well to heed your advice. My experience as a consultant and product business owner has very nearly been exactly as you’ve described.
Same here, the comment summarized all the stages we've been through.
On top of the weekly invoices, I also created "subscription" invoices, which I charge automatically instead of waiting for invoices. It really was killing me last few years. I don't even want to check last year's lost invoices...
Freshbooks and Wave both offer repeatable invoices - I'm sure other invoicing/bookkeeping services do as well.
I "Flintstoned" my small SAAS with automated monthly Freshbooks invoices until I had the time and money to get Stripe properly integrated. It meant we could get paying customers from day one with one less programming/integration headache to worry about.
I am using Stripe with a self-hosted CRM software I bought (slightly modified for other reasons, but not the payment module.)
Stripe alone can take care of it. Subscription or any type of recurring invoice system would be sufficient enough to stop collecting the check. Although, it comes with nearly 3% fee, which is fine because I see it as "what if I couldn't collect one of these invoices". That 3% really doesn't hurt much at the end of the year as long as you collected 100% of your invoices. Missing 2-3 invoices can truly hurt though.
Not OP, but I use waveapps.com for subscription billing. It was among the lowest cost option I could find, it works well, and it produces nice invoices that you can white label. Once the customer has entered their CC, you can also process incidental charges as necessary above and beyond their subscription. Most clients don't want to deal with billing either, so this just runs the card on whatever frequency you want and allows you to run the card whenever else you need to. If someone has a better suggestion, please let me know.