Its a 7B model with GGUFs already on Hugging face, with down to a 2-bit quantization. It’ll probably run on an RPi; you definitely don’t need an RTX 4090.
They're amusing but they're definitely dangerous to the potential of AI in the medium term. People like to say that "these things will happen anyway" even if it's made illegal, but there's no way 250 million will be invested into a company to develop LLMs if they're made illegal or heavily regulated. ML is resource intensive as it is (in terms of number of PhDs required and compute) and if the decels and censors have their way they definitely can make a dent in progress (i.e. "it'll happen anyway" isn't true, or it'll be slowed down so much nothing interesting will happen within our lifetimes).
Training needs insane amounts of inter node bandwidth, to the extent that training clusters use specially built hardware for it. Decentralised training isn't physically possible anytime soon, maybe never.
I also found the part about effective accelerationists and "decels" to be amusing. Sometimes it feels like we're living in a badly written sci-fi.