Arguably, the quality record only began with the launch of satellites. How much can we really trust some weather station in the middle of Montana from 1880? And of course pre-satellite the data for the southern half of the globe is relatively sparse.
When it comes to CO2 in particular, there are things like ice cores and plant fossil stomata that gives us some understanding over geological timeframes, but that's very different from a thirty year average.
Even if the measurements from the middle of Montana is correct, I could imagine there are different ways to extrapolate to average global temperature from just a few points of measurements. In the end, it will just be another estimate.
That reminds me of a thought experiment. Suppose you want to accurately measure the average temperature of a company cafeteria. How many thermometers do you need and where do you place them? Do you keep them in fixed locations or move them?
That thought experiment is readily adapted to other spaces, such as a high school gymnasium or the state of Iowa.
When it comes to CO2 in particular, there are things like ice cores and plant fossil stomata that gives us some understanding over geological timeframes, but that's very different from a thirty year average.