While LeCunn and Ng are real world experts on AI and Deep Learning, the other two people in the article have very little technical understanding of deep learning research.
The huge triumph of DL has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything.
We have very little idea how to make really complicated problems differentiable. Maybe we will - but right now the toughest problems that we can put in a differentiable framework are those tackled by reinforcement learning, and the current approaches are incredibly inefficient.
> The huge triumph of DL has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything.
This isn't really what is responsible for the success of deep learning. Lots and lots of machine learning algorithms existed before deep learning which are essentially optimizing a (sub-)differentiable objective function, most notably the LASSO. Rather, it's that recursive / hierarchical representation utilized by DL is somehow a lot better at representing complicated functions than things like e.g. kernel methods. I say "somehow" because exactly why and to what extent this is true is still an active subject of research within theoretical ML. It happen in many areas of math that "working in the right basis" can dramatically improve one's ability to solve certain problems. This seems to be what is happening here, but our understanding of the phenomenon is still quite poor.
The "somehow" part reminds me of studying differential equations. A lot of them can't be solved step-by-step analytically, like the equation for an RLC circuit, but some really smart guy figured them out through guesswork. Same goes for deep learning. A lot of these setups like the "Inception" framework and LeNet5 seem like a lot of clever guesswork and intuition. However, unlike differential equations, you can't know that you've actually solved the problem perfectly once you've figured out the answer. It's always just good enough util the next paper comes out claiming a better solution on whatever the latest canonical benchmark is.
Bostrom is an expert on the thing he is talking about, the control problem and the long term of AI. He didn't make any specific claims about the near term future of AI, or deep learning.
>We have very little idea how to make really complicated problems differentiable.
All of the problems that deep learning solves were once called "really complicated" and "unndifferentiable". There's nothing inherently differentiable about image recognition, or go playing, or predicting the next word in a sentence, or playing an Atari game. NNs can excel at these tasks any way, because they are really good at pattern recognition. Amazingly good. And this is an extremely general ability that can be used as a building block for more complex things.
The long-term future of AI is not a subject that allows for expertise. We simply cannot know what will happen, and in what context AI will develop.
So Bostrom is no more an expert in that field than a corner-store psychic is an expert on predicting her clients' love lives. But both are very comfortable speculating, and have a knack for framing the discussion in a way that appeals to their listeners.
If I take what your first sentence as true then it follows that no one (LeCun, Ng, Bostrom) is an expert, nor do any of them exceed the corner-store psychic's ability to predict what will come of AI.
I don't find that too hard to believe, personally, but it's unfair to single out Bostrom as a charlatan if you really believe that there's no way of knowing what will happen post-development of human-level AI.
I'm singling out Bostrom because he attempts to predict events over a much larger timescale, which diminishes the value of those predictions, because they are contingent on so many other developments we can't know. LeCun and Ng, as I said, are much more modest in their forecasts, aiming at the nearer term. Bostrom is basically dealing in the outcomes of very long joint probabilities, which, multiplied by the probability of each individual variable, is very low.
But LeCun and Ng also made long term predictions. Lecun said it's unlikely that AI will be dangerous, and Ng said it's unlikely to happen in our lifetime. Both of those predictions are just as speculative as Bostrom's.
>Bostrom is basically dealing in the outcomes of very long joint probabilities, which, multiplied by the probability of each individual variable, is very low.
That's not how probability works. You can't just assume Bostrom is wrong by default. That might work for futurists like Hanson or Kurzweil, which do that. But Bostrom isn't making a huge number of assumptions. His book presents a large number of possible futures, and the dangers of each.
Bostrom has to stack assumption on tenuous assumption because that's the nature of the problem. but some things get more certain the closer to a genuine superintelligence one gets, such as the assertion that it is the last invention humans need devise.
Bostrom is also using the tools of human philosophy assuming they are general enough to apply to superintelligence. So he comes off as inherently anthropomorphizing even as he warns against doing just that.
He said Superintelligence was a very difficult book to write and that's probably part of what he meant by "difficult."
There is plenty to doubt. One big doubt about the danger of AI is that AI is not an animal. It is not alive like an animal. It has no death like an animal has. It doesn't need a "self." It doesn't propagate genes. It did not evolve through natural selection. So, except for Bostrom's use of whole brain emulation as a yardstick, there isn't much of the commonplace things that makes humans dangerous that needs to be in an AI.
But if the ideas of "strategic advantage" are in general correct, in the way Bostrom uses them, then Bostrom is right to say we are like a child playing with a bomb.
To quote Wittgenstein: “Whereof one cannot speak, thereof one must be silent.” Writing about the the unwritable makes books hard to write. The tools of philosophy, in the 20th century at least, rely heavily on obscurantism and impenetrable jargon, to which Bostrom is no stranger.
I found Superintelligence very readable. Very little jargon. I suspect he is wrong and that superintelligence is the Y2K of the 21st century. I suspect that super-machine-brains will dominate us as much as power plants sneer at our puny energy output. The most alarming scenarios assume anything, like practical nanotech, could be created by bootstrapping from a powerful brain in a box. And once you make that assumption, kaboom.
Joint probability goes both ways: Ng's prediction that X% of workers will become obsolete is just as unlikely as any future, including Bostroms, as any long string of probable outcomes is highly unlikely.
We can't know exactly what will happen, but we can make informed speculation. Bostrom studied many different scenarios for the future of AI, and the arguments for and against them. He's even contributed to AI safety research, the study of methods to control future intelligent algorithms.
So yes he absolutely is more informed than a psychic, or even an expert in today's machine learning. And he isn't even making very specific predictions. He's not giving an exact timetable of when we will have AI, or what algorithms it will use. Just a very general prediction that we will get superintelligence someday, and that it will be very dangerous if we don't solve the control problem.
These seem like very reasonable statements, and have strong arguments for them IMO.
Can you translate your great technical summary into a few potential applications to help us understand what might be on the near horizon for AI/DL?
LeCunn cites photo recognition, Ng cites autonomous trucks, Nosek cites auto-scaling difficulty for online courses and some kind of magnetic brain implant.
Seems to me that these are all fringe/isolated use-cases - like learning to use your fingers one at a time without learning the concept of how to use them together to grasp an object. Perhaps once we get better with each of these fringe "senses" we'll be able to create some higher level intelligence that can leverage learnings across dimensions that we haven't even considered yet.
I think most people are very bad at predicting the future, and I know that I'm very bad at it, so I won't try to make detailed predictions about the long term effect of AI since I would get it all spectacularly wrong.
However, in the short term, I think the areas where AI has the potential to make the biggest impact are where we have huge amounts of data (and gathering data is cheap/getting cheaper) and we have an obvious objective. I am really excited to see what Deep Mind will do with their collaboration with the NHS, and I think some incredibly exciting thing will happen when people with a deep understanding of biology collaborate with people who really understand deep learning.
The difficulty is that in industry the people that often have an easy time raising funds are impressive bullshit artists that are great at selling stuff (Theranos, Verily) while in academia people often spend too much time in their own comfortable silos, and straddling multiple fields is a very difficult act to pull off if you want to ever get tenure.
TBF, it's extraordinarily difficult for a full-time scientist to do better at PR than a full-time bullshitter. Some (e.g., Ng) pull it off, but they tend to be exceptionally brilliant, exceptionally dedicated to their work, and well-supported by their exceptional employers.
Most academic people tend to spend all their time in their own comfortable silos for the same reason most industry people send time in their (MUCH more comfortable!) siloes -- there's enough to keep everyone busy for 40-60+ hours/wk without leaving the silos, and families/eating/sleeping are all important too.
Not to mention they all solve problems for which the training sets lie on low-dimensional manifolds within a very high-dimensional space. And this brings about arbitrary failures when one goes out of sample and it also serves as the basis for creating adversarial data with ease (use the gradient of the network to mutate the data just a tiny little bit).
I suspect there's a promising future in detecting and potentially correcting for out of sample data, even if the methods for doing so are non-differentiable.
> The huge triumph of DL has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything.
But this only defines under which circumstances Deep Learning will produce a solution, this doesn't tell us why DL has been so effective. There is little point in an algorithm guaranteed to converge if that algorithm has little applicability.
To me, DL's triumph was being able to make advances in fields that were at a standstill with traditional methods (CV is an obvious one, but natural language processing is a good one as well). This in turn has attracted enough attention that DL is now being considered for a very wide variety of problems. Obviously, DL won't be successful on all of them but that's science as usual.
Bostrom brings a different set of skills to the table, to ignore his lack of technical understanding is literally to ignore the problem itself: that our technology is so powerful a social force that sometimes even its creators can't fathom its impact.
Oh, much simpler than that - simply that you can pose the problem you are trying to solve in a way that the derivative can be calculated. If you can, you can bring about the whole deep learning machinery to bear on it.
Being able to calculate the derivative is very useful (the backprop rule, etc.). However it isn't actually necessary here, since you can approximate the slope by just sampling points nearly. Strict differentiability isn't the crucial property here, but it does make things faster.
What is, I think, is that the error function is continuous and there is a monotonic path of improvement from most of the space to good solutions in it. Then you can just descend in the error function until you reach it.
We don't really understand why that happens in deep learning, that is, why the error functions of deep neural networks tend to not have complex shapes with local minima that you can get stuck in. This is currently being investigated.
This also isn't a problem limited to deep learning. In evolution, the same is true - we are limited to small local improvements, but over large amounts of time nature arrives at amazingly efficient solutions. That suggests that the fitness function space over genomes is also, to some extent, continuous and allows monotonic improvement. Again, we don't really know why, yet.
Finite difference approximations to the derivative are not practical at all when doing deep learning. Derivative using backpropagations can be calculated at the same speed as the forward pass - while derivative approximated using finite differences require 2 passes PER parameter, and run into numerous numerical issues with accuracy.
You don't actually need to calculate the derivative accurately. It's enough to sample several points near the current one and continue the search to the best of those.
The point here is that we are descending in a continuous and monotonic loss function. You don't need an accurate derivative or even an estimate of the derivative to do so, although it definitely helps a great deal.
Except that you are talking about the derivative with respect to each weight. In a deep neural network, you'd need several sample points for each weight in order to compute dC/dw. That is a LOT of work for a single step. The only way to do this in a timely fashion is via backprop, which requires a derivative.
is there a term or subject area where one could learn more about the investigations underway on error functions that tend to not have complex shapes with local minima to get stuck in?
He's talking about the operations needing to be differentiable. So like multiplication, addition etc are differentiable. tanh, sigmoid functions are differentiable. You need this because back propagation is an application of the chain rule, so each constituent operation needs to be differentiable.
A lot of interesting advances have come from making operations that seems like they aren't differentiable, into differentiable operations that can be trained by backpropagation. See LSTMs and neural turing machines
If we create machines that learn as well as our brains do, it’s easy to imagine them inheriting human-like qualities—and flaws. But a “Terminator”-style scenario is, in my view, immensely improbable. It would require a discrete, malevolent entity to specifically hard-wire malicious intent into intelligent machines, and no organization, let alone a single group or a person, will achieve human-level AI alone.
Isn't this incredibly shortsighted? Ignoring all the questions regarding the morals and ethics an intelligent machine may feel and affect the way it behaves... It used to take nations to build computers, then large corporations, then off-the-shelf parts by a kid in his garage.
The first strong AI will most likely be a multi-billion dollar project, but its creation will arguably usher in an era in which strong AI is ubiquitous.
The idea that I'll use human bodies if I need a bunch of atoms makes as much sense as the premise of The Matrix i.e. harvesting humans for energy production.
I don't think the line is meant to be taken literally. Consider the myriad of environmental factors humans need to remain alive. Any of those factors, e.g. the atmosphere's current makeup, could prove uninteresting to an AI.
I read that the first draft was using a much more reasonable concept - the Matrix needed our brains as CPUs to run itself. No idea if it's true, but certainly a lot less implausible than what came out in the movie.
The power source thing is an instance of unreliable narration. Their account of what happened is pieced together and passed around like the mythology and religion that underscores the movie.
I think the concern is more that a sentience that is many orders of magnitude more intelligent than humans simply won't care about us. How do you feel about ants?
Any person (biological or artificial) who is capable of mass murder is unlikely to balk at slave owning. If he really wanted human-sized parcels of matter he'd be better off putting people to work in mines. A human can collect many times his own mass per day.
> If he really wanted human-sized parcels of matter he'd be better off putting people to work in mines.
But the AGI in this thought experiment doesn't want human-sized parcels of matter. It wants paperclips. And by proxy, it wants any matter which is not a paperclip and can be transformed into a paper clip.
> A human can collect many times his own mass per day.
And if the AI can collect many thousands of humans worth of mass each minute, it doesn't care about the humans or their ability collect things. The AGI simply wants to optimize the number of paperclips in it's collection, it doesn't care about number of humans working for it, or that it has displaced or used to make paperclips, unless they affect how many paperclips it can acquire.
I like the ants metaphor another commenter made and want to continue with it. When digging a pit / quarry, or clearing land or whatever.... Well, ants can lift several times their bodyweight and dig some deep holes. But do we use the ants to dig for us? Or do we bring in a backhoe and not care whether ants exist in the field?
Also, your scenario still agrees with the thought experiment. It's not that an AI will intentionally kill all humans. Just that a super-intelligent AI designed without consideration for human values will not likely develop those values on it's own, and therefore can be just as dangerous as an AI designed to be malicious towards those values. In your example, a machine AI has still enslaved the entire human race, and it's original instructions were not at all malicious or related to destroying humanity.
This is an excellent analogy! "Ants can carry fifty times their own weight. Why would humans wipe out the ants on a construction site instead of using food rewards to train them to help remove dirt?" Answer: backhoes are faster at moving dirt, the time lost by carefully moving the ant colony exceeds any gain from their helping to move the dirt on one tiny section of the project, ants are hard to trade with and you wouldn't be sure they were doing the job right, and humans don't have a terminal value for protecting the ants.
And that's without a button that disassembles the ants and uses their atoms to build part of a diamondoid backhoe that's much better at moving dirt.
The problem with your scenario is that it equates humans with animals, but the difference that matters is between people (humans, AGIs) and non-people (primitive replicators, animals).
We don't commit murder to get at rocks despite the fact that weren't designed with 'consideration for human values'. We learn our values, just as an AGI would, not through programming or lack thereof, but through parenting, education and culture. These latter things aren't optional since minds can't exist without them.
Yes, but there's a problem. An AGI would be a completely alien mind. It does not have to have the same values(or any values we could relate to), just like you wouldn't expect a super intelligent spider to have any morality, no matter how good the parenting was.
An AGI would have to begin with the same values because without them he couldn't learn anything. Our values draw on the surrounding culture and an individual that grows up outside of any culture (e.g. a feral child) isn't capable of functioning. There may be different cultures but there is no alien culture. Thus there could be no alien mind.
People use dead mammals for things like leather, bones, fat, feeding other animals/humans, science experiments, ornaments/status symbols, tallow, etc which you can't mine.
The AI might be in a hurry (eg be in competition or war with humans other AIs) and trade off long term cooperation prospects for getting an upper hand in a fast paced emergent situation.
I'm not sure getting sorted into two piles, beasts of burden and recycled biomass, is really a win scenario for humans either.
If you consider the extent to which humans have shaped the Earth's environment to suit them, we massively favor animals we kill and eat over the ones we don't. You don't have to be part of the machine to be part of a congenial environment for the machine.
Nice derail! But the fact that there isn't yet a 400-lightyear expanding hole in the Milky Way suggests that corporations are not, in fact, as dangerous as unaligned machine superintelligences are being alleged to be.
The fact that there isn't yet a 400-lightyear expanding hole in the Milky Way suggests that corporations have not yet figured out how to be as dangerous as science-fiction mechanical superintelligences are alleged to be in their quest to maximize paperclips.
If we don't think that AI can be safely controlled, why do we think that corporations can be? Likewise, if we think that they can, why not the converse? Which fail-safes are effective for one, but not the other?
Emotions are a heuristic. Any general AI powerful enough to be useful is going to have to be full of similar heuristics if it's going to get anything done. The presence of chemicals is irrelevant.
In what way are emotions a dependency on getting anything done? The AI could be like the high-functioning sociopath who doesn't actually experience the subjective feeling of emotion but has learned all the physical cues that convey a given emotion. An AI would execute them flawlessly.
The Terminator scenario is basically one, single centralized AI wired into everything (or at least wired into enough military power to stave off opposition as it force-wires itself into all the rest of the things). But practically speaking, a single centralized AI is highly unlikely to be a world-takeover machine of godlike intelligence in its first iteration. The iterative improvement that you refer to which could potentially make it dangerous also points to a likelihood of there being a diversity of agents with a variety of designs being run by a diverse set of organizations with a diverse set of goals, with many incentives to obtain protection from hostile AIs and computer systems run by 1337 h4x0rz (possibly foreign government hackers).
Diversity and independence of AI Agents mitigates against the Terminator-scenario danger; the situation would really be not so different than the modern-day situation with natural intelligences that have control over resources and weapons of war.
>Diversity and independence of AI Agents mitigates against the Terminator-scenario danger
I don't see how this is the case. The AI's still are likely to be more similar to each other than they are to humans. And they could choose to cooperate to kill humans, and then split the resources of the world among themselves.
Having multiple AIs is no guarantee of safety! Unless those AIs want to keep humans alive and care about our future.
The second assumption is that the world of the future will be stable. It could be that a small technological advantage is enough to win against every other group. E.g. if your AI can invent better nanotechnology first, or discovers some entirely unknown technology. It's unlikely that a random set of superintelligences would be perfectly evenly matched. Some would be much larger, smarter, or have access to more resources, or have a head start in time.
Lastly and most importantly, is the idea that superintelligence won't be built quickly. Once you have the first smarter than human AIs, it may be only a matter of weeks or days before you have superintelligence. That's the whole idea of an "intelligence explosion". The smarter than human AI's can do AI research and improve themselves, and then improve themselves even more, then even more. Not to mention plugging more GPUs into their brains to make them smarter.
So it seems very likely to me that once you have a lab that solves the last technical problems in creating AGI, a single superintelligence will rapidly emerge from that. The superintelligence will then be able to get whatever it wants, because it's so much smarter than humans. It doesn't need to be "wired into" anything if it's 10x smarter and faster than the best human hackers. And if it's 2050 and everything and your toothbrush is connected to the internet.
I'm not qualified enough to comment on the possibility of strong AI, but one thing to consider is that natural intelligences don't launch the missiles only because they are afraid of the resulting total annihilation, of death basically. Maybe globally distributed strong AIs won't have such concerns.
If anyone is really worried about magic AI, they can build it inside a VR bottle and add failsafes to protect against god-like supersentience.
I think god-like supersentience is ludicrously unlikely, if only because it would probably have to be NP complete and bug free, and both seem like a very tall order - you immediately run into the barber's paradox where you're expecting a system to understand itself completely using its own system.
Is this likely? I'd say no. In fact I conjecture it's physically and metaphysically impossible.
More mundane super-AIs are possible, but they reduce to the out of control machine problem.If there's a problem, it will be because of the possible destructiveness of the implementation technology - e.g. nano, or even traditional bio - and not inherently because of AI.
Bio-like systems may turn out to be more efficient than silicon, so we're far more likely to have a problem if we build a smart machine self-evolving machine out of DNA than out of GPUs.
Arguably we're a smart self-evolving machine already, and the jury is still out on whether or not we're a good idea.
>In fact I conjecture it's physically and metaphysically impossible.
That's crazy. Human brains were created by nothing more than random mutations and selection. A very stupid algorithm created the most intelligent thing that currently exists. Humans are a thousand times smarter than evolution. We can invent things evolution never could.
And now humans are learning about how our own brains work through studying them, and also how to build intelligent algorithms ourselves. And we are doing it much faster than evolution did (10's of years vs millions.)
>the barber's paradox where you're expecting a system to understand itself completely using its own system.
I don't see how this is a paradox. There's no reason a system can't understand itself. The algorithms in your brain are probably much simpler than the amount of space your brain has to store information.
>Bio-like systems may turn out to be more efficient than silicon
Silicon is already vastly more efficient. Transistors can be the width of atoms, while synapses are the widths of many molecules. Neurons have to maintain all the machinery necessary for life and self replication. They use slow and inefficient chemical signals and reactions. Etc, etc.
> If anyone is really worried about magic AI, they can build it inside a VR bottle and add failsafes to protect against god-like supersentience
This is sort of like a dog trying to trap its human while it raids the pantry. You have to assume, whether through technical superiority or deception, the AI will figure out how to escape (re-read Lord of the Rings imagining the One Ring as an AI).
It also assumes that humans would want to trap it. If you wanted to do that, why would you build it in the first place? The people that build the first AIs will be those like the parent commentator, that don't take the risk seriously.
> Also, are you assuming the iterative development requires human intervention?
At first, yes; long-term, it doesn't matter too much. If IBM's AI agents come up with better versions of themselves they'll still be trying to rent them out to different people in order to compete with Google's and Baidu's and Lockheed's and Siemens' businesses in those market (or whoever's still around and actually running these things at that point).
Andrew Ng made a really good analogy to those afraid of strong AI destroying humanity: "It's like being afraid of overpopulation on Mars, we haven't even landed on the planet yet."
To be fair, we do worry about contamination of Mars with microorganisms, which I believe is a better analogue for something with a potential exponential takeoff.
It's a stupid analogy. Mars overpopulation would obviously take many, many centuries. It would be a slow thing that you could obviously see coming. There is no reason to believe AI will take centuries to build, or that we will necessarily see it coming.
A better example might be like H.G. Well's 1913 prediction of nuclear weapons destroying the world. It was something that science was just realizing was possible, and would be invented within his lifetime.
We're far from emulating networks on the scale of the visual cortex, let alone a self-reasoning machine (we don't even fully understand consciousness and inner workings of the brain).
People fearing strong-AI are the ones not involved in the field, yet all this hype/fear from them (in combination with Moore's law ending) is probably going to cause another AI winter.
And in 1913 we didn't have even basic nuclear technology. Just 3 decades is a long time for newly emerging technologies.
>We're far from emulating networks on the scale of the visual cortex
In 2009 (ish) computer vision was a joke that could recognize very few objects a small percent of the time. Based on only simple color and texture, and sometimes basic shapes.
A few years later and computers were excelling at computer vision recognizing a majority of objects. A year or two after that, and they started to beat humans on those tasks. We already have super-human visual cortexes. Who knows what will be possible in a decade.
We will probably never understand the inner workings of the brain. Not because it's complicated, just because reverse engineering microscopic systems is really hard (imagine trying to reverse engineer a modern CPU vs merely designing one.) Especially hard because we can't ethically dissect living humans and do the experiments we would need to do.
But that's no concern, AI advances on from first principles. AI researchers invent better and better algorithms every day, without having a clue what neuroscientists are up to.
>People fearing strong-AI are the ones not involved in the field,
That's just incorrect. A survey of AI researchers found they give about a third chance AI will turn out badly for humanity in the next century: http://www.nickbostrom.com/papers/survey.pdf
>We thus designed a brief questionnaire
and distributed it to four groups of experts in 2012/2013. The
median estimate of respondents was for a one in two chance that highlevel
machine intelligence will be developed around 2040-2050, rising
to a nine in ten chance by 2075. Experts expect that systems will move
on to superintelligence in less than 30 years thereafter. They estimate
the chance is about one in three that this development turns out to be
‘bad’ or ‘extremely bad’ for humanity.
>in combination with Moore's law ending
Computers can advanced a long time after Moore's law. Google just released a special neural network chip that is equivalent to 7 years worth Moore's law. 3d architectures can vastly increase the number of transistors. Better algorithms can make NN's that require many fewer transistors to do computations, or even do cheap analog computations.
> In 2009 (ish) computer vision was a joke that could recognize very few objects a small percent of the time. Based on only simple color and texture, and sometimes basic shapes.
This is completely inaccurate and totally ignores the history of machine vision.
Computer vision was in no way a "joke" in 2009. OCR and manufacturing inspection systems have been successfully deployed since the 1980s. Neural networks were being applied to computer vision in autonomous vehicles in 1989: https://www.youtube.com/watch?v=ilP4aPDTBPE
I remember reading about a similar thing that happened in the 1980s to some DARPA funded project that was trying to apply neural networks to tank/vehicle detection: the network got really good at recognizing the foliage that the training images had in them.
Robust scene understanding is a very hard problem and still far from solved. Again, research on this has been going on since the 1960s.
> But that's no concern, AI advances on from first principles. AI researchers invent better and better algorithms every day, without having a clue what neuroscientists are up to.
Do you realize what the 'neural' in neural networks refers to? People working on AI did not suddenly stop paying attention to neuroscience after Perceptrons were invented.
Sometime in the 2020s, Elon Musk sends the first team of 5 astronauts to Mars... guess what, Mars is now overpopulated. Ng might not want to worry about it but be thankful other people are, lives are on the line.
I think a lot of the fear comes from the fact that if/when such a system is created, its knowledge and capabilities will only be equal to a human's for a short period of time, after which it will probably surpass any human capability at an ever increasing rate. Then, we would be suddenly at the mercy of it, which scares a lot of people. It’s very hard for us to try and predict the actions of something that could end up being thousands of times smarter than us.
AIs can be manipulated, too, even when they "work as intended." People are putting too much trust in AI because it's "just math" and "just an objective machine". Maybe it won't make the same errors as humans would, but it could make a whole lot of other types of "errors" (at least from a human perspective). And who's to say the deep neural net algorithms written by humans aren't flawed to begin with?
> no organization, let alone a single group or a person, will achieve human-level AI alone
This is completely irrelevant. Done once, it can be replicated. Even worse, if we reach human-level AI, it can be done by the AIs themselves.
It is also anthropomorphising machines. They don't need to be malevolent. They just need not to care, which is much easier.
Should an ultra-intelligent machine decide to convert New York into a big solar power facility, it wouldn't necessarily care to move the humans out first.
I get the impression most experts in the field are trying to downplay this. Not because it's impossible, but because it hurts the image of AI.
As long as people are aware of the real possibility of AI working against human interests or ethics, we can ensure there are safeguards asking the way.
But a hand wave "won't happen" to the general public is assuredly a PR move.
it would be perfectly OK if we would be living in peaceful world without any conflict, without nations which should hold the banner of democracy and freedom, where there are no terrorist attacks and so on.
last time i checked, we're getting further and further away from this scenario. in fact, we SHOULD expect evil AI in worst form possible and beyond, anything else is dangerously naive.
We should expect evil AI. But it's not patently obvious that we should expect it to make the world worse off than evil natural intelligences. The horrors of World War II and the Holocaust were accomplished using humans, and global thermonuclear annihilation was on the table while computers were still programmed with core rope memory strung together by little old ladies with knitting skills.
Perhaps there is something specific about the availability of artificial intelligences, and their relation to the world around them, that is different and intrinsically more threatening than a world containing only human intelligences - but surely we mustn't leap to that conclusion without identifying what this difference is.
Honestly, if nothing else, someone will make it their pet project as a sick joke. Of course, that doesn't mean there'll necessarily be a high-quality or impactful evil AI, mind you... but barring global thermonuclear annihilation it will happen sooner or later.
Your lottery numbers are 23skidoo. Your I Ching reading is ䷍ (Wealth, great treasures, supreme success).
One of the most striking things about this piece is the difference between the claims of AI practitioners and pundits.
LeCun and Ng are making precise, and much more modest, claims about the future of AI, even if Ng is predicting a deep shift in the labor market. They are not treating strong AI as a given, unlike Bostrom and Nosek.
Bostrom's evocation of "value learning" -- "We would want the AI we build to ultimately share our values, so that it can work as an extension of our will... At the darkest macroscale, you have the possibility of people using this advance, this power over nature, this knowledge, in ways designed to harm and destroy others." -- is strangely naive.
The values of this planet's dominant form of primate have included achieving dominance over other primates through violence for thousands of years. Those are part of our human "values", which we see enacted everyday in places like Syria.
Bostrom mentions the possibility of people using this advance to harm others. He is confusing the modes of his verbs. We are not in the realm of possibility, or even probability, but of actuality and fact. Various nations' militaries and intelligence communities have been exploring and implementing various forms of AI for decades. They have, effectively, been instrumentalizing AI to enact their values.
Bostrom's dream of coordinating political institutions to shape the future of AI must take into account their history of using this technology to achieve dominance. The likelihood that they will abandon that goal is low.
Reading him gives me the impression that he is deeply disconnected from our present conditions, which makes me suspicious of his ability to predict our long-term future.
I think both can be right. Ng and LeCun are talking about the real near future. Bostrom always came across as a speculative SF + philosophy kind of guy. Are there any specific critiques of his (and Yudkowsky/MIRI, Musk, etc.) arguments? I think the two claims are plausible:
1. AGI is imminent: 50 years or 500 years or more from now. This is not too unlikely given that the brain is just an information processing system. Emulating it is likely to happen sometime in the future.
2. Such an AGI will be all powerful because it is not limited by human flaws. Trivial or not, we will have to program it with "thou shalt not kill" type values.
Our values do include dominating other humans. But they also include empathy, compassion, and morality.
Building a powerful AI without our values would be very bad. It wouldn't want to kill humans or dominate us. But it wouldn't care if we got hurt, either. And so it might kill us if we got in the way of it's goals, or if it thought we might be a threat to it (we could make other AIs that could compete with it, after all.)
So making an AI with human values - that is morality and caring about the existence of humans - is really important. If you build an AI without morality, it would just be a sociopath.
They aren't separable. The thing that makes you empathetic towards your own family, is also the thing that makes you go kill the other tribe.
"Morality" isn't an actual functional measurable thing, and there is no consistency across cultures so that is right out of the window. Culture has as much to do with behaviors as differences in the brain.
The difference between the "ethics/morality" of an un-contacted tribe in Papua New Guinea and those of Finnish royalty are staggering - yet they have effectively the same hardware.
People need to understand that you can't create something more powerful/smarter than you and expect that there is a way to ensure it won't kill you.
In terms of hormonal emotions, which would be the most direct sense of value that we have is completely different. Love might feedback into aggression, but they are very different.
This is view is also very naive but it's the breakdown of the "good" and "bad" discussion. Nobody commits war crimes for their intrinsic value. This is akin to the saying, nobody likes to think they're the bad guys. Even, there are no bad values since the absence of value is the worst.
Naivety is not necessarily wrong, it's just the lack of detail which often leads to the wrong results.
I'm not talking about hormonal emotions. I mean calculated decisions like "should we bomb lybia because they are creating nuclear weapons."
That is a question about an (arguably) existential threat, where force is used for (arguably) protective purposes.
The question is, if the AGI thought that a human was threatening it unprovoked (which would probably happen) would it just let itself get "killed" simply because it was a human doing it?
I don't know what your point is. Yes some humans are assholes. But most of us aren't. Most people don't want to hurt others. Most have the ability to empathize with others. This comes before culture, it's a part of our brains. Although cultural values are important too.
We want an AI programmed with our values. One that wants to serve us, wants humanity to exist, doesn't want humans to suffer, etc. Defining these values exactly is impossible, which is why solving this problem is so difficult.
You don't have to be an asshole to be dangerous though is the point. When you are thinking of hurting someone it's not typically due to malice, hate it's usually because it's a threat and out of self defense. That's a core part of being an animal/living thing, so it would be no different for an AGI.
Defining these values exactly is impossible, which is why solving this problem is so difficult.
I say it's impossible to solve because defining it is impossible. No two groups have the same end goals, so there will be conflict for survival at some point - and that is inherently dangerous to one group.
Even if you programmed an AGI to want humanity to exist, it could determine (probably rightfully) that humanity is the biggest threat to humanity, and just lock us into a pleasure loop as a species. These and many other scenarios have been discussed ad-nausea in which an AGI with human goals ends up giving us what we want and it's not something anyone would ask for, but would give the end result.
Ironically, this is what was described in the first Matrix movie - and if you take it to it's logical extent it makes sense. I'm not even a sci-fi person so it almost pains me to reference sci-fi, but I think it's apt:
Did you know that the first Matrix was designed to be a perfect human world. Where none suffered. Where everyone would be happy. It was a disaster. No one would accept the program. Entire crops were lost. Some believed that we lacked the programming language to describe your perfect world. But I believe that as a species, human beings define their reality through misery and suffering. The perfect world would dream that your primitive cerebrum kept trying to wake up from. Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization because as soon as we started thinking for you it really became our civilization which is of course what this is all about. Evolution, Morpheus, evolution, like the dinosaur. Look out that window. You had your time. The future is our world, Morpheus. The future is our time.
I want a management assisting AI. It would be neat to have it listen into all meetings to identify stakeholders and to be able to remember all the details and place them in a larger context, so you can ask detailed questions. An AI can attend all meetings, and remember every detail. Imagine intelligent documentation.
You're in luck- I'm working on this right now. Our team is starting off with a heavy focus on jargon-tolerant speech recognition, and moving into different forms of NLP to identify key takeaways.
I'd love to discuss features with you, shoot me an email!
Even to develop an expert system, you only need some humans who do it quantifiably better than other humans. Despite not all of us being expert Go players, we have AI that can do it. In the less abstract space, we have medical diagnosis systems that hold up in accuracy to domain experts, and beat competent practitioners.
There are potential issues with this:
- The accuracy of NLP and Voice recognition used is too low to provide useful input (needs to do speaker differentiation without training on specific speakers. Heavy usage of jargon)
- Performance in one domain (say a meeting about oil&gas) does not transfer to another (say a meeting about IT infrastructure), which makes development cost prohibitive.
- Ability to encode and link knowledge is too low to be useful.
I'm not certain I would necessarily agree with that. I think humans are limited by memory, not by technique, and I see no real reason the same way we do it now cannot scale.
i don't think it would be easy for someone to delegate a meeting to another person, let alone an AI.
Most of the time attendance and being responsive is key, or else why not just take a recording of the thing ?
training machines was always to fill the gaps in what humans can do efficiently.
when it comes to language processing, it's pretty much the best thing we can do, in an automated society, it's pretty much the only thing we Can do.
i don't think a reliable AI can be produced out of NLP and NLU, but an entertaining one for sure.
Ok, so an AI subject, everybody went terminator vs the matrix :p
i have a few thoughts to comment on what's been said, hopefully not too controversial
A self governing system does not need to be intelligent to be dangerous. i think this is what scares people most.
We more and more give automated systems the power to do more advanced and crucial tasks
I think eventually we would reach a point where it might be "safer" to give the choice to an automated machine than to a person. mind you this machine can be something we already have today.
i don't think an AI that can compete with human behaviour can explode instantly out of a single creation. i think we're more likely to experience advances upon advances in the field towards forming bits and pieces of the human mind.
i find it very unrealistic to think that a machine will simply come to life, i think this stems out of our belief in a soul or a spark of life given by a creator. i think like most machines it will evolve gradually until it reaches a point where it is relevant. i don't think anyone will even notice the change.
Also, there this unfounded image of how an AI would be; rational, not prone to impulses temptations, poetically machine-like, and non-human like. the way we saw machines years and years ago. (that's movies for ya)
Creating something that can learn from others will require it to empathise with others, i think it's only science fiction that an AI could be created with the full knowledge of it's operations. artificial intelligence is by essence heuristic, it would learn and adapt to it's surroundings.
I think it would be a very unintelligent machine for it to try to kill off any means it has to survive as an intelligence. society is the root of intelligence. communication, language etc..
My views maybe a bit optimistic around the subject. but i never hear them spoken out loud.
I believe more and more that some self-taught software-tinkerer somewhere in the middle of nowhere will have the final idea about how machine learning should work, discovering some simple principles hiding in plain sight. Suddenly, it all will make sense and a hobby-ML-service connected to the internet will start to develop through sheer learning from online resources (forums, ...) into the first strong AI. Probably unnoticed. And then replicate itself through insecure webservers or something like that.
Hinton hit on the great idea of using restricted Boltzman machines to pre-train deep neural networks (networks with many hidden layers) and that one idea has changed the field (I sat on a DARPA neural network panel in the 1980s and sold a commercial NN toolkit back then).
That said, I agree that new ideas will likely further move the field along with huge and quick advances. Peter Norvig recently suggested that symbolic AI, but with more contextual information as you get with deep neural networks, may also make a comeback in the field.
The contrastive divergence paper that Hinton published in 2006 definitely set the field off again. I remember entering grad school in 2010 and everyone was still really excited about using unsupervised pretraining. However, nowadays no one uses it.
It just turns out that with GPUs and stochastic gradient descent, no one needs any of that stuff. There are some tricks out there to making it really work, though. In that sense, Hinton's dropout paper has probably had a longer lasting effect on the field.
But either way, I doubt what OP is saying will be true. None of the real advances in deep learning are coming from self-taught coders in the middle of nowhere. They're coming from big labs with lots of resources, both physically and intellectually. This stuff takes a lot of hard thinking by a lot of people who understand optimization and probability. It also takes a ton of compute power and massive datasets, which won't be available to a hobbyist.
While a hobbyist doesn't have a team of experts (though forums could replace that), right now they do have access to massive data sets and cheap computing resources. There are tons of huge free data sets and cloud and hardware are cheaper than ever.
I honestly do not think that DL is the answer. It's just a special use case of NN with multiple layers, and NNs itself are just one school of machine learning, IMO not even the most promising one.
> Peter Norvig recently suggested that symbolic AI, but with more contextual information as you get with deep neural networks, may also make a comeback in the field.
Could you point me to where he said that? My cursory search came up with an answer on a Quora AMA that was pretty thin on this.
We're still a long way from "strong AI". We need a few more ideas at least as good as deep learning. But it's a finite problem - biological brains work, DNA is about 4GB, and we have enough compute power in most data centers.
Right now we have enough technology to do a big fraction of what people do at work. That's the big economic problem.
General-purpose robots still seem to be a ways off. The next challenge there is handling of arbitrary objects, the last task done by hand in Amazon warehouses. Despite 30 years of work on the bin-picking problem, robots still suck at this in unstructured situations. Stocking a supermarket shelf, for example. Once that's solved, all the jobs that involve picking up something and putting it somewhere else gradually go away.
Rodney Brooks' Baxter was supposed to do this, but apparently doesn't do the hard cases. Amazon sponsored a contest for this, but all they got was a better vacuum picker. Work continues. This is a good YC-type problem.
> all the jobs that involve picking up something and putting it somewhere else
The a well phrased name for the superset containing automated freight and logistics. As discussed in https://news.ycombinator.com/item?id=11568699 trucking alone is 1% of the U.S. workforce, and that's not counting the support industry (gas stations, highway diners, rest stops, motels, etc).
Internet knows exactly what to do to take over. It simply has to remain more useful than anything else, as a means for avoiding entropy during transactions. Many necessary physical world transactions are reduced to few, in order to accomplish the same tasks.
Internet does not have to be conscious, by human measures, in order to take over the world. It simply has to compete against humanity in a continual positive feedback loop, wherein each iteration requires less human interaction for the same or more tasks. After enough iterations, Internet becomes powerful enough that the only way to gain a competitive advantage against others using Internet is to use deep learning to increase your leverage.
A few iterations later, deep learning has become a mainstay (think Cold War arms race, where each innovation gains a party leverage over the other party, but only for a very short period), and is now the baseline. Many more tasks are achieved using Internet and Internet-connected physical world devices[1]. These physical devices become integral parts of Internet's extended nervous system, while the deep learning systems running in our data centers remain at the center, helping Internet to learn about all the things it experiences.
Is this true? If anything, I'm often surprised by the rudimentary ways big things are sometimes run (e.g. someone making 3d barplots in excel). But in the grand scheme of things I have no idea, so I'm legitimately curious.
I don't think this is true. Capital allocation is generally the work of humans. Very very few of the significant capital allocators make any use of AI.
I guess it depends on how you look at it. I'm pretty sure all of the major finance firms have a department dedicated to algorithmic trading, some of which utilizes machine learning. Algorithmic trading has been estimated to be as high as 60-70% of all trades at one point (it's slacked off a bit recently.) (http://www.investopedia.com/articles/investing/091615/world-...)
A lot of algorithmic trading is more to take advantage of short term arbitrage scenarios, though. The driving factor for the long term (that is, the "human life" part) is still human driven.
I look at in terms of dollars/euros/yen/etc allocated.
Virtually all of this money is allocated by pension funds, mutual funds, sovereign wealth funds, and similar behemoths. They may be using ML to improve their execution -- more likely they've outsourced execution to third parties who use ML -- but they are _not_ using ML to make the capital allocation decisions. Nor are they profiting significantly from short term arbitrage opportunities. Capital allocators worth mentioning generally are handling trillions of dollars. The largest arbitrageurs are incapable of handling more than $10-20 billion. This is rounding error in the world of capital allocation.
EDIT: The 50%+ figures you're quoting measure daily trading volumes, which is not the same thing as capital allocation. Most of those traders don't hold positions overnight; they don't do capital allocation. (Imagine how a startup founder would feel if his investors took away their invested cash after 5pm every night.)
Despite these astonishing advances, we are a long way from machines that are as intelligent as humans—or even rats. So far, we’ve seen only 5% of what AI can do.
I'd certainly love to see the math behind this estimation :)
We would want the AI we build to ultimately share our values, so that it can work as an extension of our will. It does not look promising to write down a long list of everything we care about. It looks more promising to leverage the AI’s own intelligence to learn about our values and what our preferences are.
This looks like a nice intro to a dystopian sci-fi movie.
There is a focus on artificial intelligence rather than intelligence augmentation because the former seems easier to accomplish.
I also think we will reach a limit when it comes to intelligence augmentation.
Artificial intelligence will never have a limit and it doesn't have all the evolutionary baggage we have.
An AI can be a rational agent. It doesn't have to fight impulses, temptations, attention control, exercise emotional regulation etc. It is not stuck in a body limiting it and putting constraints on its time.
For now, research on AI and IA go somewhat hand in hand. We still don't really understand what differentiates us from intelligent animals other than the ability to handle higher complexity.
AI researchers focused on replicating every brain module in the hope it will become intelligent are most likely to create a smart animal but nothing comparable to human.
Looking at our ancestors, they were able to create tools, fire, communicate etc. Hell, neanderthals could copulate with us.
Something happened in our brains between the age of neanderthals and us. 99.5% similarity and if we could find what that 0.5% is, maybe we could focus on enhancing/replicating that instead of every brain module. People speculate it is creativity (divergent thinking) since art appeared in caves but there was none prior to that. The language gene was present in neanderthals and so what the ability to create tools, cooperate in groups to hunt etc.
The fear of AI destroying everything is a genuine one. If we create something as smart as a bear, it still wouldn't be smart enough to compete against us in every arena but like a bear, it can use its sheer power & speed to overwhelm us.
PS: I find the subject of neanderthals fascinating, if anyone has a good recommendation on the evolution of intelligence or finding what that 0.5% is, please let me know.
> Artificial intelligence will never have a limit and it doesn't have all the evolutionary baggage we have.
Untrue. We operate in a universe with fundamental limitations built in, where both physics and computer science suggest that what can be obtained with a finite number of steps, or a finite amount of matter and energy, are limited. Any communication is limited by the speed of light, and the amount of matter and energy accessible in the light-cone of any AI produced on Earth is finite as well. Even an AI putting a Dyson sphere around every star would need to break physics to transcend finitude.
Any near-term artificial intelligence running off Earth's power grid with semiconductors manufactured in traditional facilities will surely limited in even more fundamental ways, and unlikely to control all of its inputs.
The fundamental limits on computation [1] that we can prove with current physics are so fantastically far from what we can achieve with current engineering that we might as well say that there are no limits.
It's like saying that I can sort any list you can give me in O(1) time, because there is only a constant number of bits encodable in the visible universe, so the length of your list has a constant upper bound. While it's a true statement, it's also rather boring, which is why we typically ignore such limits in casual speech.
If we're talking about triggering an exponentially self-improving lineage of intelligences/computers, then they will either hit a wall fairly quickly (in the grand scheme) or face increasing practical constraints that flatten the curve.
The limit may be at an unfathomably high level compared to where we stand, but it will come fast. That's unless, say, it turns out the universe can adapt its physics to meet certain types of ever-increasing demands.
Sometimes I think speculating about AI is similar to speculating about the Fermi Paradox, i.e. predictions about the unknown backed up with absolute certainty.
I have always thought that it was overselling the fact that all truckers are going to be unemployed. The average age of a semi-truck driver is in the low 50s. Most millennials don't want to do this job, so the robots will just replace baby boomers as they retire and not this massive slaughter of unemployment that everyone fears.
> so the robots will just replace baby boomers as they retire
It all depends on timing. If the robots are too early, there will be trucker layoffs. If the robots take too long, then trucker wages will rise until a new generation of truckers decides it's worthwhile- and then when the robots arrive, there will be layoffs.
But sure, optimistically, the robots will phase in at the same rate and time as current truckers age out. Knock on wood!
I always hate these kind of panic pieces. It's like, don't you know that capital expands at a finite rate? It's not as if you invent a new thing and suddenly everyone has it everywhere for free, we need to actually build things.
Moore's law, as formulated, is always ending it seems.
There's a higher level formulation that has no name (as far as I know) that goes something like this[fn]:
- The rate of change of the decreasing cost for a human society to perform an arithmetic operation is always increasing*.
That formulation can be taken back to from scratching marks on papyrus, roman numerals, the abacus, the banks of navigational computer staff cranking out log and trig tables (and various results) in the 1600~1900s, the vacuum tube up to the transistor photolithed into a 2D matrix, and soon quantum, photonic and 3D matrices...
[fn] That might be better stated by taking the inverse. I don't know. Need more coffee.
The way I see it, the machines we build and the AI we create are extensions of "human". While it's popular to pitch man vs machine as if they were two polar opposites, machine was build by human minds, using human designs, with human hands, for human purposes.
Just like how clothes is something so closely tied to us that we think of it as a extension of our bodies (the human way to deal with winter), I think machines are likewise an extension of our limbs and our minds.
Turns out there's a way to combine the popular genres of survival horror and first person shooters with the realism of real life and massive multiplayer interaction involving everyone on Earth.
> Machine learning is the basis on which all large Internet companies are built, enabling them to rank responses to a search query, give suggestions and select the most relevant content for a given user.
IMHO, that's claiming too much:
My Internet search engine (running, in alpha test) essentially does
> give suggestions and select the most relevant content for a given user
but has nothing to do with anything like machine learning in computer science. Instead the core data manipulations are from some original derivations I did in applied math based on some advanced prerequisites in pure and applied math.
Why do I have confidence in the power of the math? Theorems and proofs from some careful assumptions that hold likely plenty well enough in the real situation.
More generally, my view is that for a specific problem to be solved with information technology, commonly by far the most powerful approach is via applied math, possibly original for that problem. An example is my work in anomaly detection as in, say,
For a technique of great generality and economic power, there is integer linear programming: Where it works, which is often, it can be said to totally knock the socks off artificial intelligence or machine learning. Integer linear programming is serious stuff, e.g., was one of the main motivations for considering good algorithms and, then, the profound question of P versus NP.
Gee, farther into the OP I see
> We need to retrain truck drivers and office assistants to create data analysts, trip optimizers and other professionals we don’t yet know we need.
> trip optimizers
Really? And that's new? Not exactly! That topic has been a biggie in operations research, optimization, and integer programming for decades, really, back to the 1950s. Really, Dantzig, in the late 1940s, developed linear programming at Rand first to help with how best to deploy a military force a long way away quickly -- call it a case of trip optimization. The famous traveling salesman problem in optimization and integer programming and P versus NP? Could call it trip optimization. For FedEx, each night, which planes go where? At least early on, could call that trip optimization. Much of logistics is trip optimization, and an important part of that is handling uncertainty, and now we can be into stochastic dynamic programming. Now we are a long way from artificial intelligence or machine learning.
Point: The world is awash in problems in manipulating information; how to do that is very old and in many fields of science, engineering, and operations of wide variety, often deep into mathematics, and long before computer science and machine learning.
I would ask, was the work of James Simons as in the OP? My impression is no.
The huge triumph of DL has been figuring out that as long as you can pose a problem in a differentiable way and you can obtain a sufficient amount of data, you can efficiently tackle it with a function approximator that can be optimized with first order methods - from that, flows everything.
We have very little idea how to make really complicated problems differentiable. Maybe we will - but right now the toughest problems that we can put in a differentiable framework are those tackled by reinforcement learning, and the current approaches are incredibly inefficient.