> For every set of "unique" insights, there is almost always many sets of unrelated data sources that can be analytically combined to deliver the same insights. In the slightly seedy underworld of data model brokers, I've seen some very impressive examples of this.
If you could share a couple of examples it would help a lot to get your point across.
FWIW, I agree with the thesis that data intensive computation could be one or two orders of magnitude more efficient than it currently is, with sufficient engineering. Probably Cassandra vs ScyllaDB is a good public example, and ScyllaDB is likely not close to the theoretical optimum at all. But I'm not sure about deriving data from alternative sources. How do you derive movement data for everyone with an Android phone if you're not Google?
If you could share a couple of examples it would help a lot to get your point across.
FWIW, I agree with the thesis that data intensive computation could be one or two orders of magnitude more efficient than it currently is, with sufficient engineering. Probably Cassandra vs ScyllaDB is a good public example, and ScyllaDB is likely not close to the theoretical optimum at all. But I'm not sure about deriving data from alternative sources. How do you derive movement data for everyone with an Android phone if you're not Google?