I've worked at both large companies and small startups, and I've never seen taking on tech debt pay off. Features never get delivered as quickly as anyone wants, but sometimes tech debt completely prevents even thinking about features, and not being able to even imagine implementing something is a much worse problem than being 2 extra weeks late to the newest pivot. If I were starting a software company today, I'd want to keep my POC code about as clean as I'd keep production code at a large company. Well, cleaner. It's going to be alive as long as you are, and there will never ever be a good time to rewrite it.
Having said that, I am very skeptical about what some people identify as tech debt. Things like "we're using X logger instead of Y logger, this codebase is unmaintable" I tend to not prioritize, but things like "this system looks weird and we don't understand if the code is wrong or the documentation is wrong" is something that should be addressed with high urgency. It's very easy to spend a lot of time maintaining misbehavior that should just be fixed, and the first step is understanding whether or not something is wrong.
Having said that, I am very skeptical about what some people identify as tech debt. Things like "we're using X logger instead of Y logger, this codebase is unmaintable" I tend to not prioritize, but things like "this system looks weird and we don't understand if the code is wrong or the documentation is wrong" is something that should be addressed with high urgency. It's very easy to spend a lot of time maintaining misbehavior that should just be fixed, and the first step is understanding whether or not something is wrong.