The point about "not doing something in hardware if can be done in software", is although mostly true, is less so when one takes FPGA into consideration, where the difference between hard in soft is blurred.
Eh, not really. FPGA folks will tell you the exact same thing. Digital design and verification is much harder and more time consuming than SW. The suggestion has always been to start your implementation in SW and if you just can't make it work (ex. no reasonable uC can keep up) only then do you reach for an FPGA.
My point is that VHDL/Verilog is both hardware and software. OTOH I find FPGA "programming" with VHDL considerably easier than actual embedded programming. No need to fight for CPU clocks, no indeterminism is execution time etc.
Yes, and when doing it in software makes no sense. We have GPUs for a reason, for example. (Yes I'm aware graphics can be done on the CPU but there's a reason GPUs are faster)