>AI similarly has powerful network effects. OpenAI's current business model is genius. By allowing free usage of chatGPT, they are getting millions of "free" employees, all actively and eagerly providing more data and more training to the system.
I never buy this. This story is repeated so often in so many places. You hear it about Tesla all the time "Their dataset is so big because they have all those cars out there no one could compete!". But what do we see every time an actual technical article about building something with AI comes up? Throwing a shit tonne of data at a model doesn't work. The number 1 most important thing about training neural nets is carefully grooming the training data so that the NN learns what you're actually trying to teach it, and doesn't just cheat your tests. So no, I don't believe the 5 millionth user typing "Write a monologue in the style of Benoit Blanc about the Johnny Depp/Amber Heard trial" is helping.
I'm not saying there's no incumbency advantage - there clearly is, which is one of the reasons why Office is still dominant. But it's not about raw data, yes you need data, but at the scale that tech giants operate, multiple companies can have that scale of data, and certainly there's atleast a dozen companies that could scrape the entire web if they wanted to. There's also the simple fact that the team that built OpenAI is probably just... better at building AI products, so it's not really surprising that if they pull this off, they will continue to produce the best AI products, regardless of their data advantage.
I doubt the data OpenAI receives from the usage of ChatGPT is even remotely close to the amount of data Google or Facebook handles that could be used to build a better language model.
They have advantage over other AI startups, but I don't think they have much of an advantage over companies like Google and Facebook which both have massive amounts of data, money, and computing power. Which at the current state is the three most important things needed to build models that can operate at internet scale.
OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt. Hence why ChatGPT is extremely slow, often crashes, and is protected by a rate limiter.
Your email account must look verify different to mine, which consists almost entirely of marketing spam and GitHub notifications… I don’t recall the last time I sent an email.
> OpenAI can never run ChatGPT at the scale needed for massive usage. They would drown in debt.
They have access to Microsoft's billions and the compute powers of Azure. The other day there was a HN post [1] about Microsoft has build a top 5 super computer. That computer is probably the computer OpenAI used to train their model on.
This is an old narrative focused on the foibles of supervised learning. The data acquired by chatGPT and Tesla is extremely valuable even while unlabeled. It can be used for pretraining, both self and unsupervised. I guarantee you that having larger and more diverse datasets for pretraining provides a lot of advantages.
We know that OpenAI was able to classify the prompts it received, at first privately - now publicly, so I'm thinking they are sorting out by use type, not prompt targets. They have probably learned a lot about new RLHF targets to focus on. It'll blind you to see the actual process of OpenAI if you focus on the noise in people's usage of ChatGPT.
Another consideration here is whether or not OpenAI specifically is inherently special relative to other systems.
ChatGPT is popular, because most of the world has looked at it (myself included) and thought “wow”, but I don’t know what Google, Microsoft or anyone else has running in their labs.
Google’s LAMDA demo some time back was incredibly impressive, but it’s not led to anything publicly released yet, so comparisons can’t be made.
Microsoft may not be interested in OpenAI because they’ve looked at it and thought they’re further ahead with some new Cortana buried in a basement in Redmond.
OpenAI is already very dependent on Microsoft for access to data. And MS is one of their major investors. So there's no pressure on either party to act in a hurry.
A reason for MS to buy OpenAI would presumably be to deny access to third parties to the AI and make it some kind of exclusive MS only feature. The problem with that is of course that all the research has been published already and outside researchers and developers are already replicating their results. That cat is already out of the bag. E.g. Google has some AI models that it has chosen not to release yet that are similar to GPT. Also, companies like this are heavily dependent on their people and they tend to start jumping ship in case of an acquihire where they receive a lot of equity. So, there's a question of what it is they would be buying.
What's more likely, is that OpenAI will power a long tail of smaller AI companies and create a lot of value that way, while the bigger companies will use their data leverage, and access to vast amounts of infrastructure to use the same algorithms to produce better AI models.
MS and OpenAI are of course collaborating pretty deeply with OpenAI already. They are using Azure and have access to a lot of MS controlled training data. Historically, AI algorithms have been public research with the understanding that they are relatively low value without a lot of infrastructure and training data. The likes of wikipedia only get you so far. So, OpenAI is basically that plus MS provided infrastructure and data. You can machine learn all you want on your laptop but you won't get anywhere until you put a few billion on the table for access to infrastructure and data. MS has done that through OpenAI.
I imagine, MS has some pretty favorable licensing agreement with them as well in exchange. That kind of symbiotic relation ship is mutually beneficial. And with OpenAI set to generate some serious revenue in the next years, it's a pretty good investment too.
MS might be better off buying some of the more successful openai using startups in this space and then integrating their features rather than building those features in house. After they acquire those smaller startups at a relatively low price, they can level them up with better data and access to infrastructure. Leave the experimenting and risk taking to the startups. MS has had a relatively successful acquisition strategy in recent years with some real value added to their products via acquisitions.
It's a very different strategy than what Google is doing. Google is being secretive and probably not that effective in terms of generating revenue from what they have done so far (regardless of how good that is). Making search slightly better isn't going to move the needle for them. Google translate is no longer best in class. There are some gimmicky features on docs and gmail but it's not quite at the same level as OpenAI. Typical Google: lots of R&D but very little to show for their money so far.
I think I like the MS strategy better. There's a learning curve with this kind of stuff and doing it in the open speeds that up massively. Shorter feedback loops, more rapid iteration, etc. Google could do the same of course but so far they are choosing to do everything on their own. They are a bit like MS used to be. I don't think that's a good strategy.
This might sound naive, but wasn't the founding principle behind OpenAI that it should prevent powers around AI being concentrated in the hands of nations or big companies?
Sam Altman addresses that when he was asked that question in 'How I Built This' Podcast.
Essentially: OpenAI did not realize the massive scale they needed for them to be successful. When they realized this, they could not raise any funding as 'non-profit'. They asked govt, who did not want to fund it, and other sources, at the end they did not have any other recourse.
Sam Altaman says: 90% of funding was needed for compute power, but also was needed for things like buying dataset and then to pay employee so that they can compete with likes of Google to retain them. If they would not have done this, then very soon they would have become irreverent.
So to retain the earlier intent (for greater good) they put in bunch of 'safety features' around funding - Ex. 'Profit Cap' - investors would get only certain amount of profit and after that the profit would be distributed to the world (in some way). Similarly, there were few other 'safety feature he talked about.
The founding principle was to have a thin veil of "academic research" around their work so they could ignore intellectual property laws while they bootstrap.
Bingo. This does not bode well for their future which will be full of legal costs and result in content creators removing their content from publicly available methods that can easily be scraped by these data aggregators.
It was but isn't anymore. It's just a commercial company now as far as it appears from the outside. I too had the same hope, but it seems that it went out the window a while ago.
1. Publishing the Dall-E papers and pre-trained CLIP weights. This inspired Stable Diffusion/MidJourney, reducing the amount of time OpenAI had for their Dall-E service to get established by maybe a year. During that time, they could've gained long-term customers, generated some revenue, and established partnerships with Adobe and other graphics software makers. Now they're second string.
2. Publishing the GPT-3 paper and releasing ChatGPT for free online. Instead, they should've improved it by adding things like references and released it already integrated in Bing/Word/Cortana. This would've been more valuable to Microsoft. Now, Google has time to catch up and have their own model in their search engine not long after Bing if they do it right. And Anthropic is working on a chat model and will have a good one as well soon.
One possible counterargument could be that these haphazard releases allowed OpenAI to gain mindshare among researchers. But this is much more vague and speculative.
to quote the "Introducing OpenAI" article from 2015, dec 11:
"OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact."
The mentioned things are strategic blunders if their goal is to maximise profit (maybe). If they have some other goal, they might not be blunders. In fact they might be on track with what they want to achieve.
The legal form of the corporate structure is not the deciding factor here. What are their goals is.
Hard disagree, I think 2 is actually a genius move after major fail with dall-e 2 pricing model. In general, they are probably looking for stable diffusion like adoption curve, but unless they have an offering people can use for free that is flexible enough (giving users just a prompt box is not flexible enough), someone else is going to eat their lunch on LLM's too, when you won't need colab to run them.
True, they would. But there is a limit to how much you can get out of it with just a text box. Just look at dalle2 vs sd - even though dall-e is much more capable and clearly better one out of the two, sd gives much more consistent results if you know what you are doing. Because you have full access to model we now have novel ways of inlining some state to pieces of text to assign importance, whole algebra done on embeddings etc. When similar thing happens to llm's nobody will care about the text box in ms products.
It gets a confusing between DALL-E, DALL-E 2, Stable Diffusion 1.x and 2.x. But Imagen is using some ideas introduced by OpenAI and they also published the influential GLIDE paper. To greatly simplify, my impression is that OpenAI's big impact was proving that scale works and causing many other large scale projects to get going.
In order to remain competitive, OpenAI will need to make ChatGPT free. It's basically going to be a repeat of the browser war since all the major tech companies have their own version of AI that they can offer with their other products. The main issue is that they, OpenAI, don't have any other sources of revenue so they can't continue to give away the product without going broke. Yet, the big companies can do it forever since they have other sources of income. Also, it's unlikely that a big tech company like Microsoft or any FAANG would be able to get antitrust approval to buy OpenAI, so it's not going to happen.
Unless OpenAI becomes a non-profit and can get funding from people who don't want the big tech players to have control over AI, I don't see a bright future for it. It's more likely to find a place similar to firefox where the big search engines pay it for the traffic it sends to them.
MS buying it is not going to happen in 2023 or ever.
The comparison to the browser wars may not be a 100% match. Part of the browser wars was that not all of the web would be compatible with all the browsers, whereas all the AI companies (in a way) program against the same "universal interface" that is the English language (and other languages, potentially).
But they also get to work with proprietary data corpuses and training weights, and can be adapted to trigger proprietary APIs on particular devices (or browsers) in response to certain user queries
Indeed triggering those APIs has been the actual use case for the most widely used chatbots that were around before GPT
The underlying assumption here is that Sam would be willing to sell OpenAI.
I don't think that's the case. There's no amount of money that Microsoft could throw at Sam to convince him to sell. Not only is he already wealthy, but he's also a true believer in AI. And unlike Snapchat, it's going to be very hard for incumbents to clone.
DALL-E 2 was a LOT better than Stable Diffusion. Sure it was open source but it was clearly a generation behind. Great if you didn't mind your inputs looking like last year's AI output.
> There's no amount of money that Microsoft could throw at Sam to convince him to sell.
Every experience I ever had of people who achieve power and success tells me that you are wrong. His own wealth is irrelevant. His beliefs are also not relevant. It is very likely he will want to "take the next step". He can always use that next shit ton of money to invest in the next big thing in AI.
I usually assume OpenAI is actually Microsoft already, as they are one of the lead investors and from what I've heard quite involved there. It's kind of a trick to appear "open" and not tainted by the Microsoft brand but in the end it's Microsoft already
Does OpenAI have anything resembling "special sauce", or are they just a critical mass of ML engineers with good funding, loose constraints and the word "open" in their name?
If the former, a buyout seems like the obvious next step, antitrust notwithstanding. If the latter, a buyout would kill two of the company strengths - "open"-ness (as weak as it is with the Microsoft tie-ins, it is still notionally a selling point), and loose constraints on target use-cases. Might not seem like such a good deal if it just turns out to be an acqui-hire.
I was listening to a podcast[1] and they say the cost of inference (responding to a question) in ChatGPT is about 1-2 cent. Is that true? And how would that map to the overall cost microsoft can pay for integrating that into Bing, considering number of search queries in Bing.
No one has yet to mention that OpenAI is built on fair use of copyrighted information. If it's language and data is pulled from the web and their output is derivative, if they charge for the product, there will be significant intellectual property concerns. Microsoft would be wise to avoid opening their large balance sheet to that liability until it is litigated elsewhere.
I was about to write a similar post, all companies using open ai will need more than a prompt as their competitive advantage. Or be caught in a race to the bottom. Microsoft needs open ai or risk this commoditize their business like msft did to ibm or component manufacturers did to dell.
Imu they are still a nonprofit parent structure over a “capped profit” company. I haven’t been able to find details about their structure, and it’s weird this article doesn’t verify these details.
I think Microsoft is one of the few companies in the world that could replicate OpenAI's work. It's much easier to build something when you know it can be done and roughly how.
OpenAI is mostly likely using Azure for training as well, and Microsoft has the model weights directly (and officially! they are even allowed to use them), so that gives a lot of possibilities for Microsoft to replicate the tech, without purchasing OpenAI.
Because they invested $1B in OpenAI so presumably they have Most Favored Nation status. And they’re not dumb enough to let Google buy it after they’ve integrated it in Azure.
Microsoft lacks philosophy and imagination, so even if OpenAI were acquired by Microsoft, I don't think it would make the use of Microsoft products more valuable.
In fact, all Microsoft did with DALL-E was to display AI-generated images in Bing's image search. Who would want to use such a feature?
Also, history has proven that companies acquired by Microsoft, such as Skype and Nokia, are corrupt.
Therefore, I hope OpenAI will be an independent organization.
I never buy this. This story is repeated so often in so many places. You hear it about Tesla all the time "Their dataset is so big because they have all those cars out there no one could compete!". But what do we see every time an actual technical article about building something with AI comes up? Throwing a shit tonne of data at a model doesn't work. The number 1 most important thing about training neural nets is carefully grooming the training data so that the NN learns what you're actually trying to teach it, and doesn't just cheat your tests. So no, I don't believe the 5 millionth user typing "Write a monologue in the style of Benoit Blanc about the Johnny Depp/Amber Heard trial" is helping.
I'm not saying there's no incumbency advantage - there clearly is, which is one of the reasons why Office is still dominant. But it's not about raw data, yes you need data, but at the scale that tech giants operate, multiple companies can have that scale of data, and certainly there's atleast a dozen companies that could scrape the entire web if they wanted to. There's also the simple fact that the team that built OpenAI is probably just... better at building AI products, so it's not really surprising that if they pull this off, they will continue to produce the best AI products, regardless of their data advantage.