The main reason they've been doing that is because it's been much easier to lower power consumption per core than increase performance per core. This way they could still claim "significant improvements" year over year.
It's also a way better strategy to make "better server chips", because it means you can add more cores.
That's assuming there's no trivial single-core advancements possible at this fab size, which there might not be. The run up to Ryzen had a lot of AMD types cheering about "lazy" intel getting beat but Ryzen's per core performance is on par with a 2600k from 2011 and is consistently edged out by a 7700k at the same, or below, pricing.
Also Intel's gains have been modest, but very real outside of power savings. CPU passmark benchmark:
2600k (2011) : 8484
7700k (2016): 12196
Both are 4 cores so its a one-to-one comparison. A ~30% increase per core is pretty impressive this late in the Moore's law game. If it wasn't then we'd see Ryzen doing much better per core, but its not. My workloads cannot make effective use of 8 cores so its academic that I can buy a 8c chip now. It would literally be downgrade from my intel for me.
It's also a way better strategy to make "better server chips", because it means you can add more cores.