I'm not sure what you would want from a standard. The basic idea is very simple, crawl the entire web starting from a few known points and visiting every single url you see on every page. Now you have an index of the internet you can run searches on.
The only problem is meaningfully ordering those results and having the computing power to do it. An open standard doesn't solve those.
