If it makes money who cares if it counts as real intelligence? If we can automate ourselves into a post scarcity society then everyone and everything is free to be as intelligent and as stupid as it desires.
It'd be one thing if their goal was to build ML tools that they want to market as AI, but they explicitly aim to develop an artificial general intelligence. There's a mountain of risks and ethical questions that we should be answering before even attempting such a goal, and for them to waive it off under the banner of "intelligence is just economic value" is dangerous and horribly irresponsible. If we can automate ourselves into a post scarcity society then everyone and everything is free to be as intelligent and as stupid as it desires.
Even an AGI can't create natural resources out of thin, post scarcity is a pipe dream that at best means we kicked the can down the road.
Based on our actions, society doesn't actually even want to be automated away for a simple life of freedoms where all the basics of life are taken care of. We could have done that a long time ago, instead we just raise the bar and continue to make life more complicated and grow the list of "necessities". People seem to need a sense of purpose in life, taking that away with an AGI-powered system that gives them everything the AGI deems necessary wouldn't end well.
