I honestly believe the best to make AI responsibly is to make it open source. That way no single entity has total control over it, and researchers can study them to better understand how they can be used nefariously as well as in a good way—doing that allows us to build defenses to minimize the risks, and reap the benefits. Meta is already doing that, but other companies and organizations should do that as well.
I'm not a doomer but I honestly don't understand this argument. If releasing model as open source helps researchers determine if it's safe, what about when it's not deemed safe? Then it's already out there, on the hard drives of half of 4chan. It's much easier and cheaper to fine-tune a model, distil and quantize it and put it on a killer drone, than it is to train it from scratch.
On the other hand I totally relate with the idea that it could be preferable that everyday has access to advanced AI and not just large companies and nation states.
Open source models in general. Meta has for instance released DINO which is a self supervised transformer model. LLMs are also going multi modal (see LLaVA for instance). The name "LLM" has stuck but they should really be called Large Transformer Models. LeCun is working on self supervised visual world models (I-JEPA) which if successful and released could form the basis for killer drones. It's still a lot of engineering work to fine tune and put a model like this on embedded hardware on a drone, but at some point it might be easy enough for small groups of determined people to pull it off.
For a drone, an LLM derived solution is far too slow, unreliable, heavy and not fit for purpose. Developments in areas like optical flow, better small CNNs for vision, adaptive control and sensor fusion are what's needed. When neural networks are used, they are small, fast, specialized and cheap to train.
A multimodal or segmentation algorithm is not the solution for bee-level path planning, obstacle avoidance or autonomous navigation. Getting LLMs to power a robot for household tasks with low latency to action and in an energy efficient manner is challenging enough, before talking about high-speed, highly maneuverable drones.
Tesla is running these models on 4 year old hardware to control a car in real time (30 fps). You don't need a full 100B model to control a drone, and it doesn't have to be as good as a car to cause a lot of damage. Reportedly both Ukraine and Russia are putting together on the order of a thousand drones a day at this point, Tesla includes the compute to run this in every car they make already today. Hardware is also moving fast, how come people forget about Moore's law and software improvements? To me there's no question that this tech will be in tens of thousands of drones within a few years.
a multimodal llm is a general purpose device to churn sensor inputs into a sequence of close to optimal decisions. the 'language' part is there to reduce the friction of the interface with humans, it's not an inherent limitation of the llm. not too farfetched to imagine a scenario where you point to a guy in a crowd and tell a drone to go get him, and the drone figures out a close to optimal sequence of decisions to make it so.
there are rumors that the latest gen drones in ukraine use crude embedded vision ai to increase terminal accuracy. launch and iterate, this will only get more lethal.
GNU/Linux is open source. Is it being used responsibly?
What is the "it" that no single entity has control over?
You have absolutely no control of what your next door neighbor is doing with open source.
Hey, if we want alcohol to be made responsibly, everyone should have their own still, made from freely redistributed blueprints. That way no single entity has control.
> Hey, if we want alcohol to be made responsibly, everyone should have their own still, made from freely redistributed blueprints.
Anyone who wants to can, in fact, find blueprints for making their own still. For example, https://moonshinestillplans.com/ contains plans for a variety of different types of stills and guidance on which type to build based on how you want to use it.
And in fact I think it's good that this site exists, because it's very easy to build a still that appears to work but actually leaves you with a high-methanol end product.
it's very easy to build a still that appears to work but actually leaves you with a high-methanol end product.
Is it? I've always seen concern about methanol in moonshine but I presume it came from intentional contamination from evil bootleggers. It's difficult to get a wash containing enough methanol to meaningfully concentrate in the first place if you're making whiskey or rum. Maybe with fruit wine and hard cider there's a bit more.
The physics of distillation kind of have your back here too. The lower temperature fractions with acetone and methanol always come out first during distillation (the "heads") and every resource and distiller will tell you to learn the taste and smell, then throw them out. The taste and smell of heads are really distinctive. A slow distillation to more effectively concentrate methanol also makes it easier to separate out. But even if you don't separate the heads from the hearts, the methanol in any traditional wash is dilute enough that it'll only give you a headache.
I think it's extremely hard to build a still that appears to work but creates a high methanol end product.
there’s no reason bootleggers would attempt to deliberately kill customers, at most you can argue about potential carelessness but in contrast there was indeed one party deliberately introducing methanol into the booze supply.
> GNU/Linux is open source. Is it being used responsibly?
Great example! Yes, linux being open source has been massively beneficial to society. And this is true despite the fact that some bad guys use computers as well.
I think the question mark is if AI is more akin to the nuclear bomb of the Internet.
If you don't put barriers, how quickly will AI bots take over people in online discourse, interaction and publication?
This isn't just for the sake of keeping the Internet an interesting place free of bots and fraud and all that.
But I've also heard that it's about improving AI itself. If AI starts to pollute the dataset we train AI on, the entire Internet, you get this weird feedback loop where the models could almost get worse over time, as they will start to unknowingly train on things their older versions produced.
It is very common for it not to be used responsibly. Alcohol causes health problems, and is implicated in accidents and violence.
Many countries have laws restricting either the sale of liquor in various ways, or its consumption in public places, or both, not to mention production and distribution.
It's entirely conceivable that even if AGI (or something comparably significant in terms of how impactful it would be to changing society or nation states) was achievable in our lifetime, it might be that:
1) Achieving it requires a critical mass of research talent in one place that perhaps currently exists at fewer than 5 companies - anecdotally only Google, Meta, and OpenAI. And a comparable number of world governments (At least in the US the best researchers in this field are at these companies, not in academia or government. China may be different.)
This makes it sound like a "security by obscurity" situation, and on a long enough timeline it may be. Without World War 2, without the Manhattan Project, and without the looming Cold War how long would it have taken for Humanity to construct a nuclear bomb? An extra 10 years? 20? 50? Hard to know. Regardless, there is a possibility that for things like AI, with extra time comes the ability to better understand and build those defenses before they're needed.
2) It might also require an amount of computing capacity that only a dozen companies/governments have.
If you open source all the work you remove the guard rails for the growth or what people focus investments on. It also means that hostile nations like Iran or North Korea who may not have the research talent but could acquire the raw compute could utilize it for unknown goals.
Not to mention that what nefarious parties on the internet would use it for. We only know about deep fake porn and generated vocal audio of family members for extortion. Things can get much much worse.
> there is a possibility that for things like AI, with extra time comes the ability to better understand and build those defenses before they're needed.
Or not, and damaging wrongheaded ideas will become a self-reinforcing (because safety! humanity is at stake!) orthodoxy, leaving us completely butt-naked before actual risks once somebody makes a sudden clandestine breakthrough.
> We don’t need to speculate about what would happen to AI alignment research during a pause—we can look at the historical record. Before the launch of GPT-3 in 2020, the alignment community had nothing even remotely like a general intelligence to empirically study, and spent its time doing theoretical research, engaging in philosophical arguments on LessWrong, and occasionally performing toy experiments in reinforcement learning.
> The Machine Intelligence Research Institute (MIRI), which was at the forefront of theoretical AI safety research during this period, has since admitted that its efforts have utterly failed. Other agendas, such as “assistance games”, are still being actively pursued but have not been significantly integrated into modern deep learning systems— see Rohin Shah’s review here, as well as Alex Turner’s comments here. Finally, Nick Bostrom’s argument in Superintelligence, that value specification is the fundamental challenge to safety, seems dubious in light of LLM's ability to perform commonsense reasoning.[2]
> At best, these theory-first efforts did very little to improve our understanding of how to align powerful AI. And they may have been net negative, insofar as they propagated a variety of actively misleading ways of thinking both among alignment researchers and the broader public. Some examples include the now-debunked analogy from evolution, the false distinction between “inner” and “outer” alignment, and the idea that AIs will be rigid utility maximizing consequentialists (here, here, and here).
> During an AI pause, I expect alignment research would enter another “winter” in which progress stalls, and plausible-sounding-but-false speculations become entrenched as orthodoxy without empirical evidence to falsify them. While some good work would of course get done, it’s not clear that the field would be better off as a whole. And even if a pause would be net positive for alignment research, it would likely be net negative for humanity’s future all things considered, due to the pause’s various unintended consequences. We’ll look at that in detail in the final section of the essay.
Getting the results is nice but that's "shareware" not "free software" (or, for a more modern example, that is like companies submitting firmware binary blobs into mainline Linux).
Free software means you have to be able to build the final binary from source. Having 10 TB of text is no problem, but having a data center of GPUs is. Until the training cost comes down there is no way to make it free software.
Free software means that you have the ability - both legal and practical - to customize the tool for your needs. For software, that means you have to be able to build the final binary from source (so you can adapt the source and rebuild), for ML models that means you need the code and the model weights, which does allow you to fine-tune that model and adapt it to different purposes even without spending the compute cost for a full re-train.
Exactly. The biggest question is why you would trust the single authority controlling the AI to be responsible. If there are enough random variables the good and the bad sort of cancel each other out to reach a happy neutral. But if an authority goes rogue what are you gonna do?
Making it open is the only way AI fulfills a power to the people goal. Without open source and locally trainable models AI is just more power to the big-tech industry's authorities.
No, the model is useful without the dataset, but its not functionally "open source", because while you can tune it if you have the training code, you can't replicate it or, more important, train it from scratch with a modified, but not completely new, dataset. (And, also, understanding the existing training data helps understand how to structure data to train that particular model, whether its with a new or modified data set from scratch, or for finetuning.)
For various industry-specific or specialized task models (e.g. recognizing dangerous events in self-driving car scenario) having appropriate data is often the big secret sauce, however, for the specific case of LLMs there are reasonable sets of sufficiently large data available to the public, and even the specific RLHF adaptations aren't a limiting secret sauce because there are techniques to extract them from the available commercial models.
One could argue that open source won’t change much with regard to China and Russia.
Both countries have access to LLMs already. And if they didn’t, they would have built their own or gotten access through corporate espionage.
What open source does is it helps us better understand & control the tech these countries use. And it helps level up our own homegrown tech. Both of these are good advantages to have.
That last paragraph is an opinion you seem to have just formed as you typed it stated as a fact that doesn’t seem to hold up to even the lightest scrutiny.
Afaik china is already pretty developed in this area, they already have a bunch of opensource llms that beat ours or at least are at the same lvl.
We can also argue that it'll have the same effect as banning chips but again, China succeeded to build dense nm chips even with sanctions, just a bit slower.
AI systems are the consequence of the pandora box that we've opened long time ago, about the time when humans got the curiosity to improve things. At this moment you can't stop the progress, the world is myltipolar, there'll always be players willing to go extra so the only solution is getting to the top faster or as fast as others
