I'd start with something like this[0]. The only search term I used was "xeon-d". You can easily find other form factors and combinations. This line of CPUs is lower power (the linked one is rated at 45W), so should be trivially cool-able with a very quiet system. They support ECC RAM. You're likely to find mostly Mini-ITX and uATX boards, so they will fit in just about any case you want.
I've done this a few times, and the killer has always been the RAM. By the time you're buying decommissioned enterprise hardware, it's hard (or just expensive) to source RAM from its QVL list - even harder so when you want ECC, which not all of the QVL'd SKUs will be... And, in my painful experience, if it's not QVL RAM there's a good chance it just won't post.
I've had no problem with getting the RAM (either new or just more eBay stuff) - and even got a replacement mainboard for relatively cheap when one burned up (it was easier to swap the mainboard than reconfigure everything for a new server).
The killer is the power consumption. It's better now compared to old enterprise gear of 20 years ago, but you can sometimes still pay for a brand new low-power system just in power savings alone.
Of course, if you have solar onsite that doesn't matter.
The power is part of why I prefer the embedded boards like the one I linked.
Especially putting one of these not in a rackmount chassis, you can opt for quiet cooling and fans that do not draw 10s of watts each. The idle loads can get fairly low, depending on your additional peripherals.
For very low power, I actually prefer small form factor machines, either used 1L SFF machines from the big OEMs (see STH's Project TinyMiniMicro[0]) or one of the many AliExpress mini PCs[1]. These sorts of machines tend to use low-power CPUs and a lot of the AliExpress ones are laptop parts.
As with all things, there are always tradeoffs. If you enjoy researching and bargain hunting for hardware, then this is a viable path. If you just want a machine that works, then this is onerous work.
The top spec 32GB RAM on the qualified list for the specific board I linked earlier runs $40-$41 on ebay. The same speed and capacity for generic DDR4 runs in the mid $30s. The board has 4 RAM slots. So the premium for QVL RAM for that linked board is $20 over generic if you want to max out at 128GB.
Supermicro did have RAM issues in the early teens.
This sort of research is the non-dollar price of finding deals on server-class hardware.
If you want to get something pre-validated, and usually with a warranty, it can be worth it to look for resellers for the major OEMs that also offer refurbished hardware. It's more expensive than the typical options available on eBay, but still much cheaper than new hardware.
[0] https://www.ebay.com/itm/166190039675?epid=17034031881&hash=...