One challenge I see in AI is that humans tend to attribute human traits to non-human entities (Anthropomorphism). This tendency leads people to expect something from AI that they can do quickly and to be shocked when we find AI to be brittle and lacking fundamental features of human understanding.
Having raised children and now playing with my grandchildren, it often amazes me at just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.
I've worked with computers for 37 years, and the progression has been amazing, but we're still a long ways away from the primary capabilities of even a house cat.
All this said I often wonder if the reason for the failure of current AI systems to wow us is the gap between power density of a human brain vs. compute systems. I've heard it said that DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.
Either way, I think AI has useful applications today and I hope we will find many useful applications of these technologies to make our lives better and make more time for us to learn, love and care for one another.
>> This tendency leads people to expect something from AI that they can do
quickly and to be shocked when we find AI to be brittle and lacking
fundamental features of human understanding.
Just to be clear- "AI" ≠ classification and even more so, AI ≠ machine
learning ≠ deep neural nets. The article above runs fast and loose with the
terminology, but machine vision and in particular object detection (or
classification of objects in images) is one area and one sub-task of AI in
general. It happens to be one of the two or three areas that have seen strong
empirical results in recent years, but it's by no means the only active area
of research (although, thanks to the funding from large technology companies,
it is probably the fastest growing one).
It's also very strange to see "brittleness" as a criticism of deep neural
networks in particular (the attacks described only work on convolutional
neural nets as far as I know). In the past, the type of AI system criticised
as "brittle" was the hand-crafted rule-based expert system type of AI. And the
reason why that kind of AI was criticised as brittle is because it did not
deal very well with the noise in real-world domains, such as in photography or
speech etc. Deep neural nets in particular are extremely robust to noise,
which is why they work so well in speech and image processing.
I think what you really mean by "brittle" is the tendency of deep nets to be,
well, a little too good in dealing with noise. Specifically, they have a
tendency to overfit to the noise, because they produce models with very high
variance. Indeed, the whole adversarial examples thing is probably best
understood as a result of overfitting.
As to "lacking fundamental features of human understanding" what you mean, I
think, is that image classifiers only do classification and nothing more- which
is true, but then that's what they 're designed to do. Nobody expects an image
classifier to have any understanding of the images it's classifying.
"Brittle" is easy to explain: they're wrong in ways that a human would not be wrong. That goes for both rule-based systems and deep neural nets.
Humans are inherently social; we have a model of how other humans behave which is essential to getting things done. It doesn't involve say mistaking a bunch of white noise for a camel.
For another example, when you're driving, don't underestimate how many of the rules are not written down and involve a model of what the other driver (human) might do. Those rules are also specific to a geographic area and evolve (slowly and imperceptibly) over time.
The question "Is this person going to be surprised by what I do?" is inherently different than "Is this object a cyclist?"
> (the attacks described only work on convolutional neural nets as far as I know).
These attacks work on basically every classifier. It works for multi-layer perceptrons, it works for linear models, it works for random forests, it works for SVMs. It works on basically anything that tries to generalize far from the points it is trained on.
There are a lot of super interesting results in this area, but it's practical significance is waaaaaay over-hyped. One cool thing I saw recently was that some researchers made a physical turtle statue with a texture on it that fooled ML systems into thinking it was a gun, with I think the implication that you could make an AI system think a gun was a turtle, but you could also just put a gun in a turtle case, which fools humans as well as AI systems.
I think part of the implication was the opposite, a turtle that was registered by AI systems as a gun, potentially triggering a automatic “swatting” response.
The adversarial example attacks work against most known machine learning models, including statistical models, decision trees, feed-forward nets, convolutional nets, recurrent nets. They not only work against classifiers, but also against regression, and RL models.
There are more attacks than just overfitting errors for visual domains. Mirrors, lasers, [transparent] masks, costumes, environmental conditions (many factors here), high speed motion, twins, polymorphic objects, optical illusions, color inversions; and that's not even getting into behavior variations. Once DNN can handle all of these things, they will be so deep and varied, they could easily have more hallucinations because of their growing complexity, trying to adapt.
Handling these oddities will most likely require a strategy of smoothing out the complexity, rather than deepening it. My guess is that solving one of these kinds of problems will solve most or all of them together.
I agree with you completely about things like a housecat and how far we are from something that advanced. The movie Blade Runner made me cherish and appreciate just how incredible animals are when I learned real animals were like the ultimate status symbol in that damaged world. I looked at my own animals differently from that point on and began to see just how complex and amazing these self-replicating, totally autonomous organic beings are and how very far away from creating that we are.
An example that stuck with me was on a windy day when I saw a seagull approach a telephone pole, coming in with the wind, turning 180 degrees and landing neatly facing the wind.
The brain of those creatures are tiny. No scientists are handholding them as they grow up.
Yet they manage to search vast areas for food, learn new ideas like stealing from shops and tourists and even as you mention - self replicate.
Yes, while AI is improving we are orders of magnitude away from replicating animals.
Is that true, and is that objectively more impressive?
To the former, I always assumed there were in flight adjustments after launch. Elon's roadster had inaccurate calculations, I heard they were off by like 2%. I assumed the moon landing was similar.
> I've heard it said that DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses.
Consider, on the other hand, the number of watt-hours AlphaZero spent to reinvent the current state of the art in Go strategy, vs. the number of cumulative watt-hours the brains of the global community of Go masters spent—over centuries—to figure out that state of the art in the first place.
ML might be kind of expensive to run right now, but its "thinking" can already be horizontally parallelized in a way human thinking just can't. For jobs that fit in one human brain, the human consumes less energy to get the result. For jobs that don't fit in one human brain, humans will have to do exponentially more redundant work to get the same results as one computer cluster.
Now, if we manage to get to "instantaneous recording and ingestion of mental mastery of a skill, like in the Matrix" before we get to "AI that builds better AI", maybe humans will (at least temporarily) be the better thinkers once again. Maybe we'll go back to the model of having offices full of human "computers" analyzing problems!
Humans can build associations with very few samples
This to me is an example of whatever the opposite of anthropomorphism is - assuming that humans sample like computers, and then extrapolating to "low" relative to computing. It's also my #1 pet peeve in DL debates.
As someone who has also raised children I can see how this conclusion (low sample rate) can be made, however as someone also deep into ML/RL I see how wrong it is.
You say "very few" samples without a metric. I've seen people in the past cite 2 or 3 presentations of a stimulus to a child, for example in the form of a toy, and then state that the child has correctly visually identified the toy with a verbal label in subsequent tests.
Assuming that these 2 or 3 presentations correlate with 2 or 3 samples is wrong because it doesn't take into account sample rate.
Every presentation batch is a 4D (continuous time + three dimensional) multi-sensory supervised labeling exercise at first (no RL until the first recitation/exploration). Using rough abstractions, at 60 "frames per second" input rate, and lets assume there was a "supervised labeler" (aka parent/guardian) which said the word "toy" multiple times across a 5 minute play period, you have up to 18,000 "labeled" pieces of training data across multiple sensory inputs for one object.
If you blindfolded the child and had them identify the object by feel you may need more batches, similarly with other senses (smell for example).
Obviously this is a gross simplification - but the constant 1:1 batch comparison at the sampling rate between humans and [linear models/MDP/differentiable programs/Neural Networks] really is way off.
The problem is that when you show a single example of "a toy" (let's say- a fire engine) to a child, they don't just learn to recognise the unique object you showed them as a "fire engine"; they learn the concept of "fire engine" that they can subsequently correctly recognise in different objects, with very different characteristics. Having learned what a fire engine is, they can then recognise fire engines of all shapes an sizes as belonging to the same category of "fire engine" as the original; a blue fire engine as a special case with a surprising colour; or a real fire engine as a different class of fire engines that is not a toy; and so on.
Machine vision classifiers can do nothing of the sort, no matter how many examples you give them and for how long they learn to look at them. If you label a fire engine toy as a "fire engine" then either the classifier will only be able to recognise toy fire engines, or it will have to mislabel real fire engines as "toy fire engine".
I agree that the difference between the sampling rate of humans and machine vision classifiers is not well defined, but it is obvious (and far as I can tell there's a strong consensus on this) that machine vision algorithms are many orders of magnitude less sample efficient than humans.
when you show a single example of "a toy" (let's say- a fire engine) to a child, they don't just learn to recognise the unique object you showed them as a "fire engine"; they learn the concept of "fire engine" that they can subsequently correctly recognise in different objects
I don't have that same experience at all. In fact if anything it's the opposite. My kids called ambulances "fire trucks" until I - the supervised labeler - corrected them.
that machine vision algorithms are many orders of magnitude less sample efficient than humans.
I don't think anyone disputes that - but they are at least in the same ballpark in terms of structure, especially if you look at the way RL works.
As someone who raised children and grandchildren, I can't find any explanation to how fast they learn the language, based on very few samples (where your 4D argument doesn't apply). Sure, children learn from conversations with adults, but those are mostly trivial, and involve trivial concepts. And children seem to be able to learn not only from the very limited number of samples, but also (in a sense) - learn more than these samples contain. BTW, did anyone try to analyze how many words/phrases the child heard, say, by the age of 7, when they develop perfect understanding of the language and ability to speak like adults? And after that age, one can spend 50 years learning foreign language and still not get it.
Intially, kids don't learn language that fast, they spend a whole year getting samples from parents where we try-and-try-and-try over and over to get them to say something, so there is a high sample rate going on for sure. However, it is also true that humans learn faster at some point by using self other tools, like consciousness. Not sure how exactly that works on a toddlers brain, but on mine, if you ask me to remember a phone number, I will repeat it in my head several times and try to make associations, those higher-level learning processes seem to be the algorithms that we are missing to discover and implement successfully.
The 4D argument is even more applicable to human language, IMO. Object recognition pretty exclusively involves sight and touch. Human language involves all the senses, frequently at once.
My Spanish is not great, but usually I can communicate pretty well despite that, partially because there are a lot of other contextual cues (body language, nonverbal vocalizations, known objects) I can use to figure things out.
It's amazing how frequently the words don't matter at all, and the meaning is almost entirely contained in tone and pacing of speech.
based on very few samples (where your 4D argument doesn't apply)
Again, define "few." Language development starts in-utero [1] and basically is a constant stream thereafter.
Children who have more consistent exposure to directed language and singing from their parents learn language faster, so there is absolutely correlation between exposure rate (sample rate) and acquisition time.
Additionally the idea that language isn't 4D is just completely missing the concept. There is no linguistic association with a "ball" if there is no physical (visual/tactile) representation of said ball. Assuming a child doesn't have a disability there are no single sense concepts that I can think of.
I often wonder how much the multi-sensory aspect plays into it. When I see an image, I don't really process it as an image, I map that visual cue into the full gamut of sensory memories (?) of that object. I could write a page worth of these descriptive meatspace 'vectors' that are invoked when i see a banana and color my interpretation of its context.
If my understanding is remotely correct, an RNN's view of a banana is basically like the face in Aphex Twin's Equation -https://youtu.be/M9xMuPWAZW8?t=5m30s (headphone users beware). No qualitative or quantitative information about the object, just a certain tone of integer triplets in a cacophony of noise.
It seems like a many-dimensional view of the world around us is going to be necessary for systems to more effectively intuit about interacting with it. It could be something we synthetically inject or we may need to give our models new senses they can use to extract their own meaning.
Well that's why I call it 4D. It's a multidimensional understanding of "banana" that crosses multiple sensory barriers.
As you more or less correctly point out, the way a DNN understands a 2D image of a banana is by basically compressing (convolving and pooling) an image into a mathematical "fingerprint" for which we provide a label. If the labeling process is homogenized then we can relatively rapidly generate inferences when testing the fingerprints on new images at a high probability.
That is to say the complexity of the "fingerprint" of a banana is several orders of magnitude greater in humans than it is for even our most advanced object detectors - if for no other reason than the mapped data is multi-sensory.
We have a decent error rate. People go searching for their car keys, or some item in the fridge, and will overlook what they're looking for multiple times, and then suddenly they notice it and feel like an idiot.
Rather than anthropomorphizing the AI, it's more like expecting superhuman qualities. People overlook objects in their rear view mirror all the time, but the expectation for the AI is that it will never make that mistake.
I think it’s related to learning. From my own experiences, it seems like our brains use the same technique to learn from all senses at once. Ie we can probably spot our equivalent of a random pixel through having multiple sensory inputs of all types processed together. We can know we don’t have a spot on our head despite seeing one in a mirror because we can touch or otherwise feel it is not there. That takes an incredible amount of cross reference across all of our combined senses.
Similarly, my mind can differentiate from the floaters in my eyes because it remembers their existence. O do that it had to build a concept of a floater, etc.
That can not be the important aspect, at least not in the general case. Show me only a few good photos of an object I have never seen - and one may be sufficient if the object is different enough from all the things I know - and it will become almost impossible for you to trick me into confusing it with anything else and certainly not be changing a few pixels in validation images. I may make mistakes if the image quality is bad or the lighting or if you show me an object that is at least visually very similar and where I am unaware of the distinguishing features. No where here are other senses than vision involved.
Human object recognition has its own common failure modes.
Take almost any photo that has dense content, mirror it, and "faces" or "creatures" will appear. We're primed to parse curvy symmetric shapes as organic, and if there's any hint at all of eyes we'll probably see a face.
Optical illusions are a catalogue of other failures. We can easily be tricked into misreading colours, shapes, and lengths.
Because the visual cortex works at a higher level of abstraction than naive pixel-reading classifiers, we're not likely to be confused by adversarial pixel flipping. But we're just as likely to misread equivalent abstractions at a higher level.
Things get even worse at higher cognitive levels. The recent study into the spread of fake news shows that we're far more likely to mis-parse "facts" than to be ruthlessly objective.
The underlying problem is that we expect AI to show the same predictability that simple digital logic does. AI won't and can't, because the problems operate on a completely different level and uncertainty is built into the domain. The best solutions will be better than humans, but it's naive to think infallibility is possible.
It's true, we see faces where there aren't any. But we usually know very quickly that it's not a real person- even if we might be confused if we catch it out of the corner of our eye, as soon as we get a chance to look at it for more than a split second we will change our mind and not believe it's a real face.
Of course there are more persistent failure modes where we can be tricked for a lot longer, and artificial illusions that persist despite knowing what they are.
I think an interesting point here is that you often need to learn the distinguishing features themselves - especially when classifying quite similar things. as a child I found it hard to distinguish plants from each other, until I learned to look at features like flowers, leaf shape, bark, branch patterns etc.
Yes you have a concept of an object based on all your memories across all senses. It takes all of your senses and their multiple sources to form concepts.
This is a very interesting point that I think has more going for it than the poster you are replying to may realize. Related, and apologies I don't have a link, but there have been a few stories circulated of adults who were blind at birth / a young age who had their sight restored at adulthood - They often had great difficulty, or even found it impossible to correlate what they saw with their eyes with the concept of objects that they knew by touch and their other senses.
I have no link at hand either, but I also read about blind people being unable to transfer their concepts of objects build from touching them to seeing them when their sight was restored. They know what a sharp edge or a pointy thing feels like but they don't know what it looks like.
But I think that actually strengthens my point, blind people are capable of learning how different object feel and they recognize them this way but they so just fine without vision and if they suddenly gain access to vision it does not help them.
In general having more senses will help to more reliably identify objects because you have access to more features to differentiate them, for example distinguishing materials from imitations just by vision can be hard or maybe even impossible. But when you can also touch them you gain a lot of new information about surface structure, hardness, thermal conductivity and so on and you can easily distinguish between, for example, real stone and plastic or wood with a stone print on it.
But just because it is advantageous in the general case to have access to more than one sense that does not imply that it is necessary for a specific task.
Apologies ahead of time. I'm typing this on a small screen phone, and trying to not ramble :-)
We might be more on the same page than I realized On the one hand, I agree that it doesn't imply that it is necessary to have access to more than one sense.
On the other hand I do think that having access to more senses may make the problem a LOT easier, especially during the training process for teaching systems to do things like try to distinguish objects by sight.
Don't get me wrong, the progress made so far has been incredible, but it isn't surprising that we are eventually running into limits given the limited amount of data these systems have to train on.
Sort of riffing off of this - In particular, my sense is that when you are training on 2-D pixel data only, it is no wonder that no matter how clever your network is to extract high-level features from the data, you are going to run into these kind of issues. We are asking the system to perform a task (describing, classifying, etc.) about a fundamentally 3-d world, using only a 2-d image, when it has never had a "concept" of the world in 3 dimensions. I think that we take for granted that when we are learning, not only do we get information by seeing, but also touching and manipulating objects and existing in the world around us. We can see things in different lighting conditions, and we can also move our heads and our bodies and around the world and see things from new angles and manipulate them to learn the rich set of correlations between what we see, and what we experience...basically the rich structure of the world around us. A system which is trained on images alone doesn't have the integrated knowledge, which is what I was hoping to get at with the anecdote of adults being cured of blindness.
When you look at some of the work we do to augment visual data sets, by scaling images, cropping them, rotating them, skewing them, etc. its basically a poor mimcry of something that humans and animals get just by being in the world...this idea of learning that things remain the same and have a certain structure, regardless of the viewpoint we see them from. (From what I understand, this idea is also part of what inspires capsule networks).
Anyways, to come to my main point. Yes, you can absolutely identify objects, even a completely novel object after only seeing it once or twice. But I would argue that this is only because you already have a rich framework of the world as 3-dimensional spatially, along with all of the other priperties that object have that you have learned. So when you see a 2-dimensional picture, your brain can form a 3-d image of the scene, identify materials of objects, etc., which you can leverage. Things that our current AI systems have no idea of... so perhaps its a miracle they made it this far, and no wonder that we find they can get fooled by flipping pixels!
(I'll end my rant here with the caveat that I am a roboticist, so perhaps I have very skewed views of intelligence and feelings about embodied intelligence, but I'm willing to learn and be disabused of my notions!)
I think we pretty much agree, identification can be done just from images, learning only from them asks for quite a lot. The issue is probably that annotating thousands of images is a comparatively easy task when compared to doing this on years of video from a camera moving through the world. And who has the time to watch a neural network ingesting videos for several years? But something like that may be necessary if you want to achieve human-level performance by learning from scratch, watching billions of video frames. I guess you can take a shortcut, sort of, and get away with a lot less training examples but only if you engineer a lot of the fundamentals about the world into the system and avoid having to learn them, too, and all in one process.
What I’m trying to say is AFAIK, AI is generally solving a problem through training something akin to a black box. It’s data -> function -> expected output. But I think we’re able to get past silly things like a pixel off or other simply “odd” images because our learning is much more involved. I’d liken it to be along the lines of data (with many data associated to our own, known input ie pushing against a car, petting a cat, looking in a mirror and moving our arms and realizing we “did that”, etc) -> contextualizing based on memories -> identifying relevant concepts (chair, car, color/blue, etc) -> thinking/doing something -> other steps -> repeat
This is also where I think robots have the best chance at really becoming self aware (far down the road). They’re not going to be passive observers.
One example can be found in Oliver Sacks' "An Anthropologist on Mars", which has a chapter about a blind man named Virgil who regained his sight in adulthood and it took him a long time to make sense of visual stimuli.
This is true insofar that the huge amount of background knowledge I have and use to build a model of what I see in an image was almost inevitably build over my lifetime with all my senses, but this does not change the fact that I performed the task at hand only visually, i.e. without fusing different senses. The AI is lacking my background knowledge, not senses other than vision. One might argue that building human-level background knowledge requires more than just vision but it seems at least not totally obvious that this is indeed the case.
I find fascinating the idea o sensations and I think there's so much more about them as a stepping stone to intelligence; maybe machines will find something better than sensations, but I doubt it.
Why are sensations necessary?
Why do I need to feel hungry in order to eat?
Certainly an automatic feeding mechanism could exist where I eat without the sensation of hunger, sex without the sensation of lust, kill without anger and so on.
Certainly an automatic feeding mechanism could exist where I eat without the sensation of hunger, sex without the sensation of lust, kill without anger and so on.
Because that's an incredibly brittle solution. Imagine yourself in an environment where you have to track your prey for hours or days at a time in order to hunt. How are you going to accomplish that if your "automatic feeding mechanism" takes over as soon as it decides you need food?
Our sensations and emotions combine to regulate our behaviour, not control it absolutely. The ability to make short-term sacrifices in order to achieve long-term goals is dependent on our ability to ignore or override these sensations.
I think emotions, impulses, short and long term desires, and the like are felt across many forms of life, to various degrees. I base this on my experiences with life directly. Thought definitely advances evolutionary success more than simple unthinking behaviors would - if only for the flexibility afforded. A state machine might flip between billions of possible states in reactions to things, whereas evolved brains flip through their past experiences, emotions (as a representative of external and internal factors), and so on to arrive at some perceived preferencial act/outcome. My bet is the organic model winds up taking fewer resources to achieve more robust results.
I follow the same line of thought as yours - that sensations could be just a hint and you utimately wheigh all your sensations and intentions; BUT just as machines don't have sensations and they can do just fine with planning, so too could animals. "Automatic" doesn't mean right now.
My point is sensations/emotions are an incredible information processing paradigm wich we are largely ignoring when we think of AI.
I think it's not entirely clear what sensations even are and, if you follow this line of reasoning more thoroughly, you arrive at the classic problem of qualia (https://en.wikipedia.org/wiki/Qualia).
A good question that should be asked is why are we focused on trying to recreate being human in a machine? There's already billions of human beings. Machines are our tools. I think Augmented Intelligence was always the better term, but the scifi stories and overhype stole the show a long time ago.
Do we really want and need a Data or HAL? They're great for story telling, but is that what humanity really needs from it's tools? I don't need to fall in love with Siri like in the movie Her. I just need it to be useful when I want to ask my device oral questions.
There was an AMA with a google team of AI coders. I asked them if any current AI was as smart as a snail (or a lizard or a fish), could it make it's way in the world and eat food and find a mate. They didn't think so, not yet.
I think these two points captures the challenges for the AI to match primitive human abilities .
>just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.
> DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.
> progression has been amazing, but we're still a long ways away from the primary capabilities of even a house cat
I'm in complete agreement with your point about how amazing things we consider "basic" are. But (using your post as a chance to soapbox) we need to remember that we're not TRYING to match the primary capabilities of a cat (or humans). In part, because we know little about how thought and instinct work on a useful level, but we know more about flowchart-style logic. And as I understand them (which isn't much), even the deeper neural work doesn't try to emulate biological thought. Instead, we focus on a _goal_ where we get a result similar to biological thought and try different (complex) ways of mixing deterministic logic to get there.
Everything is still rudimentary, but going forward I expect that AI (or any of the various related fields often lumped as AI) will do remarkable things beyond biological capability long before certain "basic" (as you say, actually amazing) things. And I don't mean just super-logic or strong memorization - we already have that - I mean they will "think" from thought A to thought B easily (e.g. 'I've read this book before' leads to 'an accountable monarchy is the best system of rule'), while we struggle to do the same, while we can make the A to C conclusion easily ('I like green' leading to 'I need to mow the lawn') that AI finds harder to accomplish or even understand enough to predict.
Biology is messy - just a few days ago we had an article on here about how social comfort makes us warmer...and in reverse, raising the temperature makes us feel more befriended. We have bizarre social rules at some level imprinted into our DNA - Tall people, pretty people, they will find success. Mob mentality, mass hallucination, zealotry, we have "work" and "fun" as mostly separate areas. Being fit is healthy, but becoming/staying fit is emotionally difficult. Our memories are ridiculously unreliable - our impressions and desires shape interpretation, and then continue to reshape the memories on each recall. We categorize thoughts down into intellectual and emotional. Change blindness, desiring comfort but pretending it is ideology, getting defensive when proven or even suggested to be wrong, conspiracy theories, all drama, fiction, gender, parody, humor itself, sleep, xenophobia, celebrities, luxury goods, clowns, desire to get intoxicated, placebos, psychological schema, "I'm not a snitch", ...the list goes on.
We (humans) tend to think there are two options: human-like thought, and cold predictable logic (witness almost every sentient computer in media...including popular science journalism). We don't think "huh, the complexity and history of our biological systems has resulted in all these surprising connections. I bet a non-biological system complex enough to "think" will have it's own surprising connections because of a totally different structure, physics, and background." They will be surprising because their way of thinking will be something we cannot wrap our brains around unable to emulate because it is literally contrary to the physical way we think. AI will not be simple and unable to process "this statement is false" - but they might find that hilarious. Or perhaps it gives a result similar to horror. Or it isn't worth much attention...but too many such statements is like dirt under your fingernails. Who knows? I only know that expecting an early AI to have the capabilities of a child is most likely self-deceptive.
I don't know when AI as generally thought of will exist. I don't know if it will prove the salvation/destruction/entertainment of humanity. I fully expect it will be very weird from a human perspective, with different strengths, weaknesses, and quirks. I also expect that humanity en masse will translate this collection as "inferior", because our brains say we should be skeptical of "different").
OP did not say he hoped to see 'biologically equivalent' behavior, I don't think that't the argument of very many people. We're talking about replicating (useful) aspects of intelligence and we use humans (or housecats) as our baselines, because that's where intelligence came from. And indeed, all forms of AI, including your latest NIPS conference showroom DL/ RL models are very brittle in replicating intelligence. As Ng said I think a year ago, the limit of AI right now is any mental task that a human can do in less than 1 second. Agents that reason about things are still in the realm of science fiction.
I really wish the press (and a subset of highly vocal Deep Learning practitioners) would stop conflating Deep Networks with the entirety of AI/ML. This hallucination problem is
1) highly pronounced in this one class of methods that has surged in popularity in the last 10 years
2) difficult to address because these methods are (so far) quite opaque to human understanding
I work on multiple in-production vision systems and in cases where we absolutely need to know why something went wrong we use much more conventional, but more transparent, learning algorithms. The performance loss is often an acceptable tradeoff for being able to understand your edge cases.
"AI hallucination" and its negative effects is not limited to self-driving cars... for example, see the latest incident with Alexa "hallucinating" a command to laugh. What makes that scary to me, is Alexa also had to "hallucinate" the trigger words as well. How many times does Alexa "hallucinate" those trigger words and send random conversations off to a third party?
For the sake of argument, suppose someone hacked your program by causing buffer overflow. Then at least you know where the problem is, you go fix it and ship a patch the next day. Now suppose someone hacked your AI by causing hallucination. Do you know where the problem is? How to fix it? How long will it take to fix it? Does the fix really fix it? Etc... Not sure how this all is supposed to work.
- you suggest that software is easy and logical, in that you can triage, locate, and patch the buffer overflow. But in any sufficiently large codebase, there are simply near-unlimited more bugs to find: hence all the memory mitigation’s that have been implemented. So it’s not a robust comparison from the get go.
- there’s constantly increasing research into being able to diagnose & analyze what CNN’s “see”: possibly not to the level and accuracy that you’d expect from lldb, but how long did it take to have really amazing debuggers for binary
Applications?
In a normal program, given an exploit, it's easy to understand the idea of attack and specific target of attack. In CNN case, because no one really understands how things work (see https://www.technologyreview.com/s/604087/the-dark-secret-at...), how can they understand why things don't work? The whole contraption is a black box.
What about the latest Intel bug Meltdown? It's not always easy to understand such attacks. Also those involving rare race conditions or things that only worked because a core part of your code depended on undefined behavior can require a major refactoring of your whole code base in order to mitigate.
The "hacking" is entirely different in that case. It's not like the person gets sudo power over your computer, it's that your car goes through a stop sign without stopping.
It's easy to find the problem because you know exactly what the car should have seen, and fix that thing.
Interestingly, the evolution of the human mind has dealt with similar challenges.
A lot of what AI is doing (pattern matching, creative problem solving, etc) could be considered “right brain” activities. Some think that many human mental problems arise out of an unchecked, overactive right brain. We still mistake shadows on the wall for something sinister, or read more into a person’s glance than is really there.
Some posit that the right brain is always hallucinating, in a sense; that psychedelic drugs simply disable the left brain, and allow the right brain to take center stage. Until the corpus callosum developed (which allows the left and right brain to send data back and forth across the divide), it’s possible that right brain insights came to us as “voices” in our heads.
See:
“Incognito: the secret lives of the brain.”
“The Dragons of Eden: speculations on the evolution of human intelligence.”
This sounds like a bunch of pseudoscience? I've taken a bunch of courses on neurosciences etc, and there's certainly some left/right splits but nothing as severe as that…
Probably more like outdated conjecture than pseudoscience (Sagan’s book, “Dragons of Eden”, was written in the 1970s), but point taken — contemporary neuroscience talks more about a modular theory of the mind than a strict Right / Left split.
I don't have anything insightful to say about this (I think it's a super interesting area of research) but am commenting anyways to point out that Nicholas Carlini, one of the researchers cited, is also responsible for engineering the best of the later Microcorruption levels. Small world!
It’s only hard to hear on a phone speaker at low volume on the 50dB example. All other examples have what sounds like some sort of static or background noise.
Granted, to an average consumer it may sound like just bad audio, but it’s not imperceptible and thus can be screened for.
Most likely we’ll end up in the same perpetual update cycle as other computer security - someone finds an exploit, that exploit is either reported for a bounty or discovered in active use in the wild, a fix is implemented and exploit is added to the test suite.
For ML that will mean adding the examples of the exploits into the learning sets/providing negative re-enforcement feedbacks.
The question will be whether we can get enough of these caught before some machinery injures someone because it saw something that wasn’t there or accepted a malicious command, and it becomes a media frenzy.
If we are to mass deploy self driving cars, we must solve this problem. No one would want to travel by cars that could be tricked so easily.
This also leads to a moral and legal questions. If a self driving car injures someone, who will be responsible, the person owning the car or the manufacturer?
I think Volvo mentioned at some point that when they released fully self driving cars the responsibility would be on them as long as the car was in autonomous mode.
Since then I think they've postponed the plans for fully automatic driving though.
That cartoon seems naive to me, in that one might be concerned about self-driving cars being vulnerable to malicious activity that scales in a way that attacks against human-driven cars don't.
The attacks that scale are unlikely to involve fooling the object detection pipeline by manipulating its input (because you need to physically modify the environment to do that), but rather something like compromising the firmware update servers, stealing the signing key and pushing a remote killswitch.
So the weak point in this case isn't the fragility of machine learning against adversarial inputs, but old-fashioned network security.
Spreading tyre piercing shrapnel out your window on a busy motorway scales pretty well I imagine. Luckily moth people have better things to do with their time!
i) The authors and commenters of the article have a strong incentive to get funding for their projects.
ii) The mentioned problems are mostly academic. The real-world implications are not tested, unknown, and likely overblown. Yes, one can create contrived examples. But that isn't same as real life.
The way spoofing examples are created is by feeding small perturbations of the same image to a time-invariant detector. It's unclear if such attacks have much practical value. E.g. a password screen won't allow you to try more than 3-5 values. Attacks must be detectable (due to repeated tries of small perturbations). Plus noise and other perturbations can be added to the detector.
When the problem of adversarial examples will be solved, AI will have leapt one more step ahead. It's clear right now that adversarial examples are the elephant in the room. It's a make or break situation for the field but I think it will come out stronger. We're grasping at the limits of our neural network technology, trying to discriminate between ghosts of perceptions.
I think the solution will come from marrying a top-down approach to the bottom-up one we're using right now. We need more prior knowledge about the world. We need to be able to simulate situations and understand their effects. Maybe what we're lacking right now is a mental simulator of the world, an imagination module. Coupling perception with imagination would reduce the sample complexity as well.
Interesting to note that the researchers that started the adversarial examples craze and invented the imagination module (GAN - generative adversarial network) are one and the same - Ian Goodfellow. He was right on the spot to identify the weak point of deep learning.
> Making subtle changes to images, text, or audio can fool these systems into perceiving things that aren’t there.
This isn't a hallucination problem. It is a robustness issue.
The outputs of modern AI lack conceptual depth and substance. These algorithms produce very shallow categorizations that are only useful in narrow, constrained contexts. Not surprisingly, it isn't hard to break or hack these fragile categorizations.
Sure, one could potentially argue that this has some similarity to human hallucinations, but I think that is a needless distraction. We know with great certainty that our AI techniques don't have the robustness and generality of animal intelligence. We are much less certain about the causes of human mental illness and any resulting hallucinations, so that analogy doesn't really lead us in a productive direction.
I think that DeepXplore [0] is on the right track. I think that perhaps some methodological improvements could likely be made, but applying an adversarial, security-like approach to deep neural networks is a way to build fault-tolerance into these methods.
I'll keep hoping that things like capsules and "smarter" network design/training and data augmentation will eventually, de novo, help add "safe" generalizability. Perhaps optimizing accuracy/minimizing loss in a more broadly, uniformly random way rather than in optimizing accuracy, weighted by frequency of observations.
"The vision systems of autonomous vehicles, voice assistants able to spend money, and machine learning systems filtering unsavory content online all need to be trustworthy."
I feel like we're building up to some horrible situation here, because these systems are never going to be worthy of trust in the same way that a human is.
As just one example: as more and more of our lives happen digitally, more evidence of real crimes will be digital evidence. So these AI systems can be easily fooled into thinking you were in the wrong place or requested something that you didn't. And there is absolutely no way to correct the record.
Might work OK for video/audio (though DeepFakes calls that into question). But what about the timestamp on it? Or what about GPS location (did you even posses the phone at the time? Was the GPS data manipulated later?)?
There will be a huge class of things that basically can't be questioned, denying you the ability to confront your accuser.
Not a lawyer, but it's my understanding that the side that's presenting any physical evidence needs to show why it probably wasn't tampered with and the other side can question that.
When gathering evidence that you want to hold up in court, you need to think about how to prove that.
> are never going to be worthy of trust in the same way that a human is.
I have this worry that the glue that holds society together is empathy. Which is we feel other peoples emotional state as if it were our own. Most of us at least. You can bet for the governments and corporations that deploy AI technologies in order to control and manipulate people that is exactly the opposite of what they're interested in.
I am a little bit bored with these GAN style attacks.
They show that the networks haven't yet generalized well enough. They don't show that the technology, or even the approach, is broken.
Personally, I think the main problem is some combination of a little bit of over claiming in research, and a metric ton of over hyping and generalizing in both media and business circles.
Robust multi model ensembles with strong generalization ability will show up within the foreseeable future, and will be no more susceptible to optical illusions than human beings.
I think the obvious answer to this is to use a generative adversarial approach, but the more I think about that the more difficult it sounds in practice. Say once you've trained in the initial weights of your classifier from your training set, you have a generator start adding distortions, occlusion, and noise to the training set images, then train the classifier to recognize those as true positives. Of course the difficult part is defining a generator architecture that can learn to generate a wide enough variety of distortions, occlusions and noise...
One method of creating adversarial examples in a "black box" setting is to create and train a local model as a stand-in for the actual model using the inputs and outputs of the actual model. [1] So, the answer is "no" but a qualified "no" since in practice this seems to work. The second part, being able to forward many images, is also a qualified "no".
It doesn't seem surprising to me that AI can be fooled by adversarial input. Human brains, as advanced as they are compared to contemporary AI, are also vulnerable to this. People figured out a long time ago that hunters, soldiers, and other military assets can be camouflaged quite effectively by painting them in certain patterns, for example.
The human visual cortex employs a combination of bottom-up and top-down processing. So before we see a dog the brain sort of generates a picture of a dog top-down, and compares that with the visual stimuli. My feeling is the problem with the "AI" described here is that it's still bottom-up only.
The difference is that a human will know it’s been tampered with and lower their confidence: the attacks against NNs make it think with 98% certainty it’s a 60mph sign, not a stop sign. Furthermore, if a human is monitoring the system, a small well-placed sticker will not likely alert them to the fact that the car is in trouble
> “People tend to trust each other in machine learning,” says Biggio. “The security mindset is exactly the opposite, you have to be always suspicious that something bad may happen.”
It's not just machine learning; far too much tech - software and hardware - has this problem. Everyday here on HN you see discussions about the benefits of a new idea without any consideration for how it could be exploited.
Notice the top comment is negative. In general, almost every top comment on HN is negative. It's hard to write well positively, just like it's hard to be a comedian and only make clean jokes.
As a negative commenter on that topic: it's not a new idea. It's not even a new tool. It's an iteration on an existing tool to make it more user friendly. It's an iteration which, by all appearances, didn't go far enough.
Tools, like ideas, get feedback when posted to a forum like this. Feedback is rarely going to be all positive, especially in the cases where a tool is still rough.
I didn't mean to imply it was a bad thing. I meant to say it's hard to write positively.
Without straying too far from the original premise, it's worth pointing out that the idea of using a graph database for traditional tasks is in fact a new idea. Most people would not consider doing it, and that makes it new.
Notice an interesting philosophical thing that happens every time philosophical debates occur: We quickly get mixed up in a debate over the meaning of words. "What do you mean by 'new'?"
Oh dear, look where we've ended up. But it was an intriguing place, I hope.
The overall point is that if you advocate something non-traditional on HN, you're in for a world of hurt. This place is nothing if not adversarial. You'd better have your I's dotted and your T's crossed. But those constraints are what make the medium so interesting.
Having raised children and now playing with my grandchildren, it often amazes me at just how much we take for granted in that comes "out of the box" with a human brain. Humans can build associations with very few samples, and we come pre-wired with all sorts of tools from primary systems like hearing, visual, sensory systems to complex capabilities like speech and communication abilities.
I've worked with computers for 37 years, and the progression has been amazing, but we're still a long ways away from the primary capabilities of even a house cat.
All this said I often wonder if the reason for the failure of current AI systems to wow us is the gap between power density of a human brain vs. compute systems. I've heard it said that DeepMind burned an order of magnitude more than the 20W/hr a typical human brain uses. When we have compute systems with that power density, we may see more emergent behaviors from our silicon-based friends.
Either way, I think AI has useful applications today and I hope we will find many useful applications of these technologies to make our lives better and make more time for us to learn, love and care for one another.