Seems like you'd reach a point where it becomes worth it even with continuous improvement. If the speed of light really is a speed limit, then you'll get to a point where the maximum theoretical improvement is still some small amount. If there's a way to go faster than light, then you'll get to a point where it only takes two seconds to get to your destination.
Even without the speed limit, you can expect to reach this point. Let's say your technology gets 10% faster every 25-year generation, you should launch a 200-year trip but not a 250-year one. We'll probably keep improving at that pace or better for a few iterations at least, but eventually space travel will be a stable technology, speed improvements will be rare and incremental, and 1000-year journeys will be justified.
Right, barring time travel, once you can make the voyage at all, you'll always reach a point where it makes sense to depart on it. If you invent technology that takes a million years to arrive, then you have a maximum deadline of a million years. Past that point, even instantaneous travel won't be worth waiting for, if your goal is simply to get there as early as possible.
