More than that, it would look weird because restoring the color to a photo in a way that looks plausibly photorealistic is really hard. If you make something more stylized, the viewer doesn’t have as much reference to compare and realize that there’s something wrong.
To the grandparent: people have been trying to colorize black and white photos since the 1840s
True. But there was also tons of skepticism in the medium throughout the mid- to late 1800s. Oliver Wendell Holmes' writing on veracity of photography is a neat reminder that it took the public decades to come to terms that the photographic process was a somewhat-veritable facsimile of "real" life.
To the grandparent: people have been trying to colorize black and white photos since the 1840s; complaining isn’t going to stop them now, https://en.wikipedia.org/wiki/Hand-colouring_of_photographs