If you let the drifters get out of control they will just keep destroying you before you can build your numbers up. If that happens, invest 1:1 in speed and self-replication. Their problem is that they can’t replicate, so at least the number of drifters won’t increase. You’ll be able to dodge them in combat until your numbers surpass theirs, then go back to the regular strategy of a balanced but replication-heavy profile.
With 30+ points you should be fine though.