I think most people believe that the problem with robots is that we don't have the right software, and if we just knew how to program them then today's robots could be incredibly useful in everyday life. From that perspective, this move from OpenAI seems dumb.
That belief is wrong. Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
In the past the software and hardware were equally bad, but today machine learning is advancing like crazy, while the hardware is improving at a snail's pace in comparison. Solving robotics is now a hardware problem, not a software problem. When the hardware is ready, the software will be comparatively easy to develop. Without the right hardware, you can't develop the appropriate software.
OpenAI is right to ignore robotics for now. It's a job for companies with a hardware focus, for at least the next decade.
You're demonstrably wrong. I can waldo any number of commercially available arms to do work humans do today. Surgeons waldo precise robots to conduct surgeries as a matter of course. Every piece of construction machinery operated by a human today is an incredibly useful robot lacking sufficiently capable software.
"When the hardware is ready, the software will be comparatively easy to develop." I take it you've never written any software for a robot? The long tail of the real world takes years and years to handle. Probably the most advanced robotics company, at the cutting edge of the ML+Robotics, is Covariant and their entire business model rests on an understanding that the long tail can and should be handled by humans.
I agree that OpenAI is right to cut out the hardware, but all your reasoning about why is wrong.
The reason, which they state, is that data collection on physical devices is slow and modification to those devices is slow and maintenance on those devices is expensive. You want to simulate everything, not because it reproduces the real world in high fidelity, that doesn't matter, but because it gives you approximations with sufficient variety and complexity that you can continually challenge your AI, and you can do all that at 1M fps.
> I can waldo any number of commercially available arms to do work humans do today
Not for everyday tasks with anywhere near the efficiency, reliability, speed, and cost that humans have without robots. You can't waldo any robot to do the laundry or the cooking in a normal home anywhere near as well as a human can do it. (I'd love to see you try!)
Sure, you can make a robot that can do work humans can't. You can make a robot stronger, or more precise, or better suited to repetitive motion than a human. Those attributes are useful in specialized tasks. But generally not for the everyday tasks humans do today that we want robots to help us with. For everyday tasks you need a robot that is comparable in speed, efficiency, weight, reliability, durability, flexibility, sensor capability, and cost to a human. Not one of those areas, but all of them simultaneously. That's the hard part.
This is just moving the goal post of your argument.
But the "Hardware lags behind" only makes sense to Sci-fi like expectations of robot agility but the software isn't even remotely close to embody that hardware. Even In the real world robotics applications TODAY this statement falls flat by one simple demonstration:
Use existing arm + teleoperation and conduct X amount of tasks (could be a mobile robot too, or a car for that matter). Now find a software that have same versatility in task execution as the human.
Most softwares for simple robotics manipulation tasks lose out to human operating it directly, bar efficiency maybe, in an static controlled environment even using the same control and perception system. Yet human controlling these arms directly show that the hardware is capable enough to conduct those tasks.
The "hardware lags behind" statement is if anything just a convenient excuse from the software / automation developers in Robotics, (also being one of them myself) shifting the blame to others, or have a sense of false highground.
The need of Lidar on early self driving cars was the same motivation; somehow softwares couldn't just use camera but needed an additional 6th sense, that humans don't even need, and still performed quite bad.
Even if this is true, that weakens your point that hardware is the fundamental limit for robots. If there are situations where software giving human like behavior to a robot could be extremely valuable, then there's certainly a motivation for generic AI companies to be in that area.
That doesn't mean OpenAI robotics leaving wasn't a good idea. It seems like it was but for other reasons.
Giving robots human-like behavior is mostly useful for general-purpose robots. Specialized robots don't need general AI to do their jobs. OpenAI is trying to develop general human-like AI. There's no general-purpose robot for them to put it in, and developing one is a hardware problem, not a software problem.
Specialized robots don't need general AI to do their jobs.
Self-driving cars have been unable to succeed based on their lack of a broad understanding of what's happening on the road (ie, "too many corner cases"). Self-driving cars would be a huge change and their failure is very significant.
We can build perfectly good robot arms for a huge swath of assembly/warehouse/retail tasks, but there's no AI that can aim them well enough and carefully enough. An overqualified AI would still be a valid solution and extremely valuable.
Off-topic but this is the first time I've seen waldo used as a verb. It's not in the Urban Dictionary, but from the way you use it I take it that it means "to find the correct thing out of the sea of so many similar things", right?
>
Waldo Farthingwaite-Jones was born a weakling, unable even to lift his head up to drink or to hold a spoon. Far from destroying him, this channeled his intellect, and his family's money, into the development of the device patented as "Waldo F. Jones' Synchronous Reduplicating Pantograph". Wearing a glove and harness, Waldo could control a much more powerful mechanical hand simply by moving his hand and fingers.
The way I see it, the problem is that the "any number of commercially available arms" - the models that are beyond absolute toys and might be waldoed to do work humans do today - are staggeringly expensive compared to the humans they might replace. The cheap arms are ... toys, they can do a nifty demo, but they are really shit for doing any actual work, and while there are also good arms available, they're so expensive that they're tricky to afford even for experimental use, but in practical applications for the same price you can just get many outsourced workers to do that with actual hands instead.
The world will be ready for robotics revolution - creating immense demand for robotics software - when you'll be able to get a decent arm for the price of a fridge, not the price of a fancy car; just as we got the computer revolution not when we developed capable computers, but only after we developed affordable capable computers.
The way I see it, the problem is that the "any number of commercially available arms" - the models that are beyond absolute toys and might be waldoed to do work humans do today - are staggeringly expensive compared to the humans they might replace. The cheap arms are ... toys, they can do a nifty demo, but they are really shit for doing any actual work, and while there are also good arms available, they're so expensive that they're tricky to afford even for experimental use
But the expense of such is a product of the lack of demand for them. Cars (again) are produced with engines now having astoundingly good accuracy.
Of course, an accurate arm is going to always be expensive than a simple arm but cars, chips, phones and whatnot show that with large scale processes and heavy capital investment, accuracy and cheapness are compatible. But today, with software incapable of doing useful things with those arms, the people and institutions with the capital to make accurate robot arms cheap, through economies of scale, are not going to mobilize that capital.
just as we got the computer revolution not when we developed capable computers, but only after we developed affordable capable computers
Some technological advances happen through a feedback loop of commodities getting sold and producers improving those commodities (cars are an example here). Other technologies require a leap where a significant clump of capital has to be devoted to creating an advanced device for which there's no sellable (and sometimes no operable) device. (the biggest example of technological leap was the Manhattan project). It might be the case that things will happen that way with robots. But I'd also say it's an open question.
cars will be the first commercially available robots with widespread use. They will then be mutated into lawnmowers, farm equipment, commercial transport, etc. They will then get 'add-ons'- arms, legs, power tools. Because humans are mobile, widespread use of any robot will first require portability or mobility. Actually, the smartphone was our first robot-and it is portable.
I want to react to 'the long tail can and should be handled by humans' and I find this thinking counter-productive and dangerous.
Humans are excellent to handle the long-tail when they're already handling the rest. Take driving. We're already seeing cars with large cognitive assistance, taking more and more an active role in 'easy' tasks. Think Tesla's autopilot. You're supposed to be there and 'take over' in case the 'machine' fails to handle the 'long tail' or decides to give you the responsibility of whatever happens next (because you trained it to do so).
Driving is a very complex task, you need training, experience, anticipation and (very important) context. There's no easy way to scramble all the details necessary for a decision in a human brain in the time to take the decision 'correctly'. Similar problem for industrial automation where you call the 'long tail' person once in a while and that person probably doesn't have the expertise of reconstructing the context, after 3 turnover phases in your provider.
I think we're taking this problem the wrong way, and aiming for the lower fruits, and higher and higher, while handwaving the long tail and sending it over the fence to the human. We should be putting the human at the center of this, and extend their capabilities, reduce the repetitiveness, help, not take over.
The paper I like a lot on this is 'automation should be like Iron Man, not like Ultron'.
This is a very important point about shifting the calculus of automating human labor when doing it incompletely. There are still big debates in the self driving car world between people who want to get to level 2-3 (partial/conditional automation) vs level 5 (full automation).
The earlier group often says since Edge cases that they can't automate now constitute only <5% of training scenarios they encounter, they've automated 95% of the job. But with what you're saying, we can't really expect the calculus to work that way.
I don't think it's really true to say that the issue is hardware or software. There is lots of robotics in everyday life, from autonomous vacuums in our homes to autonomous factories producing the goods we consume. The reason we don't have millions of little robots buzzing around us is... there's very little need for it.
The average human spends most of their time barely engaged, our brains and bodies are operating far below what we're capable of, the romanticised sci-fi vision of a world filled with intelligent robots performing every menial task for humans builds on the idea that humans have better things to do, but do we? We already have enough knowledge and resources to end world hunger, to bring a high standard of living to every human, but we choose not to: our problem is social, not software or hardware.
As an aside, I'd dispute the claim that hardware is lagging behind software: Tesla has lots of money and lots of smart people and they haven't been able to deliver self driving cars after more than a decade of promises (because of software).
> Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
You're absolutely wrong. Anyone with basic electronics knowledge and a few hundred bucks can build a passable robot body out of hobby grade servos and 3D printed parts. If you're willing to spend $10k+ you can make something quite capable.
Programming it to then actually do anything, let alone anything useful in the real world, is still out of reach for all but a tiny fraction of companies.
Hardware still has a long way to go before it's as capable as biological systems but it's usable. Real world AI is far from that in most areas.
No, and yes. You can build something. But even if you imagine tele-operating it 24/7, there is not going to be something relevant you can do with it around the house. It's not about the programming.
And that will not even be taking into account the time-to-maintenance of such a system.
On the other hand, Boston Dynamics' manifold, where they do the control of the dozens of hydraulical parts, is an absolute marvel of technology that shows what you can achieve with 45 (?) years of dedicated focus.
You might be able to teleoperate their robot for something useful in a human environment, and I guess that would be a gamechanger. But even there I want to wait-and-see if they can escape the fate of many that came before them.
Think of literally any piece of equipment that needs a human operator to operate it. If we had the software side figured out, none of these machines would have human operators.
Robotics software is incredibly complex. Even with machinery that was a perfect replica of a human body to the most minute details, throwing some ML algorithms at it wouldn’t get us anywhere.
If it worked that way, my job would be much easier.
I'm not an expert, but I'd suggest hardware is trailing software, but there is still great progress happening in materials design, soft-robots, miniaturization, etc. Just as in the computer era, we go through phases where the software is ahead of hardware, then hardware gets ahead of software. It seems that is a pendulum that swings, and the argument many people have to the integrated pipeline Apple operates.
I think self driving car is an incredibly useful robots where massive adoption is under way (very early stage still); and in many less challenging areas, self driving capable vechles have been taking over.
One day a person will see a Spot robot and smile at its cuteness, then notice that there's no way to get past it, no way to have it not decide where you can and can't step towards. It won't have a gun, but that person will no longer have a reason to smile at them in the future.
Later you will have a Spot robot chasing a person and getting it to stop, surrendering at the machine without being threatened by it, but just by recognizing that there's no longer a point in running away.
robots do a lot today, it's just not AI and most of it isn't ML. I think the ML folks find what robots do today "tiresome and manually trained" but that doesn't stop robots from producing billions of dollars a year in goods.
Also, regular ML researchers sit at tables with laptops. Robotics people need electronics labs and electronics technicians, machine shops and machinists, test tracks and test track staff...
If you have to build stuff, and you're not in a place that builds stuff on a regular basis, it takes way too long to get stuff built.
> If you have to build stuff, and you're not in a place that builds stuff on a regular basis, it takes way too long to get stuff built.
I wonder why they don't invest in establishing the competency for robotics. The potential return might seem enormous, though their choices might signify that they don't agree.
Or maybe they just aren't willing to leave their comfort zone. 'Software will eat the world' is a convenient idea for people who want to stay in that comfort zone.
My prediction is that dropping the real-world interactions will severely slow down their progress in other areas. But then again, I'm super biased because my current work is to make AI training easier by building specialized hardware.
Reinforcement learning can work quite well if you produce the hardware, so that your simulation model perfectly matches the real-world deployment system. On the other hand, training purely on virtual data has never really worked for us because the real world is always messier/dirtier than even your most realistic CGI simulations. And nobody wants an AI that cannot deal with everyday stuff like fog, water, shiny floors, rain, and dust.
In my opinion, most recent AI breakthroughs have come from restating the problem in a way that you can brute-force it with ever-increasing compute power and ever-larger data sets. "end to end trainable" is the magic keyword here. That means the keys to the future are in better data set creation. And the cheapest way to collect lots of data about how the world works is to send a robot and let it play, just like how kids learn.
I dont think this is cynical and I don't think it's a bad thing. OpenAI is not a huge org. The truth in 2021 is that not only is robotics 'just not there yet' in terms of being a useful vehicle for general intelligence research (obviously robotics research itself is still valuable), there is also nothing really pointing at this going to be the case in the next 5-10 years.
Given that, unless they want to commercialise fruit picking or warehouse robots, it seems sensible.
Sure. Yet consensus among "brain scientists" has long been that locomotion and the ability to explore the physical world is essential (or even central) to how consciousness develops and works in humans. Which in turn would seem pretty important for an institute working on cutting edge AI?
One of the reasons ML-based AI is pretty dumb still is possibly that this autonomous exploration side of AI is largely ignored.
It all seems to tie back into what Judea Pearl talks about in his "book of Why" (how you can't model intelligence without modelling learning of causal inference) or what Jeff Hawkins explores with his "reference frames of reference frames of the world" 1000 brains theory.
> Given that, unless they want to commercialise fruit picking or warehouse robots, it seems sensible.
How successful do you think attempts to monetize this will be? Apart from Kiva at Amazon, I'm not even sure most shelf-moving robots are profitable enterprises (GreyOrange, Berkshire Grey, etcetera). I'm very skeptical of more general purpose warehouse robots such as you see from Covariance, Fetch, etcetera. I don't really know too much about fruit-picking other than grokking how hard it would be and how little it would pay.
To be clear, I'm not saying these companies make no money or have no customers. But it's not clear to me that any of them are profitable or likely will be soon, and robots are very expensive. I'm happy to learn why I'm wrong and these companies/technologies are further ahead than I realize.
Presumably they are referring to the OpenAI co-founder Wojciech Zaremba mentioned in the first few sentences of the article.
It seems madisonmay didn't read the article either, or they would have known that the podcast they were referring to was the exact source used by the article.
GYM is not exclusively for robotics - it’s for reinforcement learning in simulated environments, which I’m sure they will keep doing. Also it looks like it is still being maintained, so not really sure what you mean.
Makes sense I guess, integrating robot hardware requires an entirely different set of skills to ML research and has a much slower dev cycle.
I think OpenAI has progressively narrowed down its core competency - for a company like 3M it would be something like "applying coatings to substrates", and for OpenAI it's more like "applying transformers to different domains".
It seems like most of their high-impact stuff is basically a big transformer: GPT-x, copilot, image gpt, DALL-E, CLIP, jukebox, musenet
their RL and gan/diffusion stuff bucks the trend, but I'm sure we'll see transformers show up in those domains as well.
Is the prevailing opinion that progress in reinforcement learning is dependent on algorithmic advances, as opposed to simply scaling existing algorithms? If that is the case, I could see this decision as an acknowledgement that they are not well positioned to push the frontier of reinforcement learning - at least not compared to any other academic or industry lab. Where they have seen success, and the direction it seems they are consolidating their focus, is in scaling up existing algorithms with larger networks and larger datasets. Generative modeling and self supervised learning seem more amenable to this engineering-first approach, so it seems prudent for them to concentrate their efforts in these areas.
Yes, it feels like we have squeezed most of the performance out of current algorithms and architectures. OpenAI and deepmind have thrown tremendous compute against the problem with little overall progress (overall, alpha go is special). There was a big improvement in performance by bringing in function approximators in the form of deep networks. Which as you said can scale upwards nicely with more data and compute. In my opinion as an academic in the deep RL, it feels like we are missing some fundamental pieces to get another leap forward. I am uncertain what exactly the solution is but any improvement in areas like sample efficiency, stability, or task transfer could be quite significant. Personally I’m quite excited about the vein of learning to learn.
There is a sense in which it was: out of all the games that have ever been designed, or that it would be logically possible to design, humans selected Go as one of the relatively few to receive sustained attention, in part because it is particularly well suited to the deep neural network that is the visual cortex. So it is not a coincidence that it is also well suited to artificial deep neural networks.
I think the premise of your question actually points to the real problem. In RL, b/c your current policy and actions determine what data you see next, you can't really just "scale existing algorithms" in the sense of shoving more of the same data through them on more powerful processors. There's a sequential process of acting/observing/learning which is bottlenecked on your ability to act in your environment (ie through your robot). Off-policy learning exists, but scaling up the amount of data you process from a bad initial policy doesn't really lead anywhere good.
Nope, if you see RL as just another tool for niche industrial domains? One of the targets put forward at global level is, for example, a fully automated, closed-cycle, high-throughput lab for drug discovery. More in general, fully automated factories and networks of factories (another reason why delocalization of supply chain is not being pursued anymore).
In a nutshell it’s too wasteful in energy spent and it doesn’t even try to mimic natural cognition. As physicists say about theories hopelessly detached from reality - “it’s not even wrong”.
The achievements of RL are so dramatically oversold that it can probably be called the new snake oil.
I'm going to need you to unpack that a bit. Isn't interacting with an environment and observing the result exactly what natural cognition does? What area of machine learning do you feel is closer to how natural cognition works?
Adding to the other comment, it's quite clear that animals, and especially humans, act and learn based on many orders of magnitude less experiences than pure RL needs, especially when discussing higher order behaviors. We obviously have some systems that use inductive and deductive reasoning, heuristics, simplistic physical intuitions, agent modeling and other such mechanisms, that do not resemble ML at all.
I would say that it is likely, intuitively, that these systems were trained through things that look much like RL in the millions of years of evolution. But that process is obviously not getting repeated in each individual organism, who is born largely pre-trained.
And for any doubt, the poverty of the stimulus argument should put it to rest, especially when looking at simpler organisms than vertebrates, which can go from egg to functional sensing, moving, eating, predator avoiding in a matter of minutes or hours.
> What area of machine learning do you feel is closer to how natural cognition works?
None. The prevalent ideas in ML are a) "training" a model via supervised learning b) optimizing model parameters via function minimization/backpropagation/delta rule.
There is no evidence for trial & error iterative optimization in natural cognition. If you'd try to map it to cognition research the closest thing would be behaviorist theories by B.F. Skinner from 1930s. These theories of 'reward and punishment' as a primary mechanism of learning have been long discredited in cognitive psychology. It's a black-box, backwards looking view disregarding the complexity of the problem (the most thorough and influential critique of this approach was by Chomsky back in the 50s)
The ANN model that goes back to Mcculloch & Pitts paper is based on neurophysiological evidence available in 1943. The ML community largely ignores fundamental neuroscience findings discovered since (for a good overview see https://www.amazon.com/Brain-Computations-Edmund-T-Rolls/dp/... )
I don't know if it has to do with arrogance or ignorance (or both) but the way "AI" is currently developed is by inventing arbitrary model contraptions with complete disregard for constraints and inner workings of living intelligent systems, basically throwing things at the wall until something sticks, instead of learning from nature, like say physics. Saying "but we don't know much about the brain" is just being lazy.
If you're really interested in intelligence I'd suggest starting with representation of time and space in the hippocampus via place cells, grid cells
and time cells, which form sort of a coordinate system for navigation, in both real and abstract/conceptual spaces. This likely will have the same importance for actual AI as Cartesian coordinate system in other hard sciences. See https://www.biorxiv.org/content/10.1101/2021.02.25.432776v1
and https://www.sciencedirect.com/science/article/abs/pii/S00068...
And generally look into memory research in cogsci and neuro, learning & memory are highly intertwined in natural cognition and you can't really talk about learning before understanding lower level memory organization, formation and representational "data structures". Here are a few good memory labs to seed your firehose
The place/grid/etc cells fall generally under the topic of cognitive mapping. And people have certainly tried to use it in A.I. over the decades, including recently when the neuroscience won the Nobel prize. But in the niches where it's an obvious thing to try, if you can't even beat ancient ideas like Kalman and particle filters, people give up and move on. Jobs where you make models that don't do better at anything except to show interesting behavior are computational neuroscience jobs, not machine learning, and are probably just as rare as any other theoretical science research position.
There is a niche of people trying to combine cognitive mapping with RL, or indeed arguing that old RL methods are actually implemented in the brain. But it looks like they don't much benefit to show in applications for it. They seem to have no shortage of labor or collaborators at their disposal to attempt and test models. It certainly must be immensely simpler than rat experiments.
Having said that, yes I do believe that progress can come considering how nature accomplish the solution and what major components we are still missing. But common-sense-driven tacking them on there has certainly been tried.
For what it’s worth, I agree with this take. But I think RL isn’t completely orthogonal to the ideas here.
The missing component is memory. Once models have memory at runtime — once we get rid of the training/inference separation - they’ll be much more useful.
Maybe true if you consider policy gradient methods and Q learning the only things that exist in RL… it’s a pretty wide field that encompasses a lot more than the stuff OpenAI puts out.
This is lunacy. The first country/company to replace human labour with general bipedal robots, will reap wealth beyond imagination. The short sitedness is astonishing, if you ask me.
I genuinely believe how we as a society act once human labour is replaced is first aspect of the great filter.
We are decades away from being able to build a general bipedal robot that can snake out a plugged toilet or dig a trench or nail shingles to a roof. It's just not a rational goal yet. Aim lower.
And we're further away since nobody bought Schaft from Google, and Schaft was shut down. They had the best humanoid.
But so many of the little problems have been solved. Batteries are much better. Radio data links are totally solved. Cameras are small and cheap. 3-phase brushless motors are small and somewhat. Power electronics for 3-phase brushless motors is cheap. 3D printing for making parts is cheap.
I used to work on this stuff in the 1990s. All those things were problems back then. Way too much time spent on low-level mechanics.
You can now get a good legged dog-type robot for US$12K, and a good robot arm for US$4K. This is progress.
I'd just note that "decades away" means "an unforeseeable number of true advances away" - which could mean ten years or could mean centuries.
And private companies can't throw money indefinitely at problems others have been trying to solve and failing at. They can it once and a while but that's it.
This is correct. Right now our best and brightest can only build demos that fall apart the moment something is out of place. Humanoid or even partial humanoid (wheeled base) robots are far from ready for general purpose deployment.
There are no mechanisms in place for the generated wealth to benefit the replaced people, the wealth will go mainly to vanishingly few persons self selected to be okay with gross economic inequality.
We have been at this since at least the dawn of the industrial revolution and do not have it right yet.
Backing off and taking it slow now to let some cultural adjustments happen is a responsible step.
My cultural norms are repulsed by the thought of me not working as much as possible, it is how I expect my value to society to be gauged (and rewarded).
This line of reasoning will be (is) obsolete and we need another in its place globally.
I hope some may have better ideas of what these new cultural norms should look like than I with my too much traditional indoctrination.
I only know what I will not have it look like;
humanity as vassals of non corporeal entities or elites.
There are no mechanisms in place for the generated wealth to benefit the replaced people, the wealth will go mainly to vanishingly few persons self selected to be okay with gross economic inequality.
That hasn't stopped the march of progress so far. Conveniently (or not), humanoid robots do not appear likely for the foreseeable future. But keep worrying, the problem you list are appearing in other fashions anyway.
AI will impact productivity but not replace humans. We have the needs and wants, AI lends our goals, it has none of its own. We'll expand our desires to match the increased abilities and remain as busy as always. We can't even begin to imagine the future applications, and that's where most of the work will be.
The ability to train huge models does not belong to a single entity and many of these models get shared with everyone. So you can right now type "import transformers" and have thousands of trained models at your fingertips. All these toys are ours (thanks to important work done for free by some of us) we just need imagination to use them.
I totally agree. I worked at a robotics company about a decade ago, and I was familiar with the people at Willow garage.
Robotics research is going to be extremely binary. It's expensive and frustrating, and there's little use for it until it works as well as human labor, which is a high bar.
But, once that Rubicon is crossed, I believe there will be a sort of singularity in that space. It's related to but somewhat orthogonal to the singularity that's prognosticated for g a i.
> replace human labour with general bipedal robots
No need for bipeds, car factories employ dumb robot arms, no humans needed. Not very general purpose robots though.
The first country/company to create robots that can be instructed similar to a humans to do any job will indeed have great benefits, but how long until that happens? Not within any amount of time that an investor wants to see. I'm unsure if I will ever see that in my life (counting on ~60 years to go still maybe?)
One thing that struck me recently was that the famous imagenet competition that was won by a neural net took place in 2012. So we have made fantastic advances in ten years. But I'd still say at best robots like you describe are 20 years away, and that's a long time horizon for a small organization.
Has robotics had such an 'ImageNet moment'? Nothing springs to mind, just slow advancement over decades.
If suddenly robot manipulators could grasp any object, operate any knob/switch, tie knots, manipulate cloth, with the same manipulator, on first sight, that would be quite a feat.
But then there's still task planning which is a very different topic. And ... and .... So much still to develop for generally useful robots.
Not yet. I have a four wheel drive robot I designed with four 4k cameras feeding in to an Nvidia Jetson Xavier. [1]
Just getting it to navigate itself using vision would mean building a complex system with a lot of pieces (beyond the most basic demo anyway). You need separate neural nets doing all kinds of different tasks and you need a massive training system for it all. You can see how much work Tesla has had to do to get a robot to safely drive on public roads. [2]
From where I am sitting now, I think we are making good inroads on something like an "Imagenet moment" for robots. (Well, I should note that I am a robotics engineer but I mostly work on driver level software and hardware, not AI. Though I follow the research from the outside.)
It seems like a combination of transformers plus scale plus cross domain reasoning like CLIP [3] could begin to build a system that could mimic humans. I guess as good as transformers are we still haven't solved how to get them to learn for themselves, and that's probably a hard requirement for really being useful in the real world. Good work in RL happening there though.
Gosh, yeah, this is gonna take decades lol. Maybe we will have a spark that unites all this in one efficient system. Improving transformer efficiency and achieving big jumps in scale are a combo that will probably get interesting stuff solved. All the groundwork is a real slog.
I am a researcher on the AI/Systems side and I wanted to chime in. Transformers are amazing for language, and have broken all the SOTA is many areas (at the start of the year, some people may have wondered if CNNs are dead [they are not as I see it]). The issue with Transformer models is the insane amount of data they need. There is some amazing progress on using unsupervised methods to help, but that just saves you on data costs. You still need an insane about of GPU horsepower to train these things. I think this will be a bottleneck to progress. The average university researcher (unless from tier 1 school with large funding/donors) are going to pretty much get locked out. That basically leaves the 5-6 key corporate labs to take things forward on the transformer front.
RL, which I think this particular story is about, is an odd-duck. I have papers on this and I personally have mixed feelings. I am a very applications/solutions-oriented researcher and I am a bit skeptical about how pragmatic the state of the field is (e.g. reward function specification). The argument made by the OpenAI founder on RL not being amenable to taking advantage of large datasets is a pretty valid point.
Finally, you raise interesting points on running multiple complex DNNs. Have you tried hooking things to ROS and using that as a scaffolding (I'm not a robotics guy .. just dabble in that as a hobby so curious what the solutions are). Google has something called MediaPipe, which is intriguing but maybe not what you need. I've seen some NVIDIA frameworks but they basically do pub-sub in a sub-optimal way. Curious what your thoughts are on what makes existing solutions insufficient (I feel they are too!)
Yes unless the industry sees value in a step change in the scale on offer to regular devs, progress on massive nets will be slow.
Hooking things together is pretty much my job. I have used ROS extensively in the past but now I just hook things together using python.
But I consider what Tesla is doing to be pretty promising, and they are layering neural nets together where the output of three special purpose networks feed in to one big one etc. They call that a hydra net. No framework like ROS is required because each net was trained in situ with the other nets on the output of those nets, so I believe all compute logic is handled within the neural network processor (at some point they integrate standard logic too but a lot happens before that). Definitely watch some Karpathy talks on that.
And currently I am simply not skilled enough to compose multiple networks like that. So I could use multiple standalone networks, process them separately, and link them together using IPC of some kind, but it would be very slow compared to what's possible. That's why I say we're "not there yet". Something like Tesla's system available as an open source project would be a boon, but the method is still very labor intensive compared to a self-learning system. It does have the advantage of being modular and testable though.
I probably will hand compose a few networks (using IPC) eventually. I mean right now I am working on two networks - an RL trained trail following network trained in simulation on segmentation-like data (perhaps using Dreamer V2), and a semantic segmentation net that is trained on my hand labeled dataset with "trail/not-trail" segmentation. So far my segmentation net works okay. And a first step will actually be to hand-write an algorithm to go from segmentation data to steering. My simulation stuff is almost working. I built up a training environment using Godot video game engine and hacked the shared memory neural net training add on to accept image data, but when I run the sim in training on DreamerV2, something in the shared memory interface crashes and I have not resolved it. [1]
But all of this is a hobby and I have a huge work project [2] I am managing myself that is important to me, so the self driving off road stuff has been on pause. But I don't stress about it too much because the longer I wait, the better my options get on the neural network side. Currently my off road rover is getting some mechanical repairs, but I do want to bring it back up soon.
First off, amazing farm-bot project! I am looking forward to reading the details on your site.
Thx for the pointers on Tesla. Had not seen the Hydranet stuff. There was a Karpathy talk about 2 weeks back at a CVPR workshop .. he revealed the scale of Tesla's current generation deep learning cluster [1]. It is insane! Despite being in industrial research, I don't foresee ever being able to touch a cluster like that.
A lot of our current research involves end-to-end training (some complex stuff with transformers and other networks stitched together). There was a CVPR tutorial on autonomous driving [2], where they pretty much said autonomy 2.0 is all about end-to-end. I've spoken to a few people who actually do commercial autonomy, and they seemed more skeptical on whether end2end is the answer in the near-term.
One idea we toy with is to use existing frozen architectures (OpenAI releases some and so do other big players) and do a small bit of fine-tuning.
I thing we might get biobots faster than mechanical ones. With recent advancements it seems that reusing biological hardware is simpler with our current software capabilities.
Imagine that there only needs to be ten people to “run the world”. What is the population size going to be then? Ten? As large as possible? Somehow it seems that the way we’re headed, it’ll be ten plus some administrative overhead.
You might be right long term - but think about it short term. Labor in low cost countries is very cheap. A few thousand a year. Unless you can build these machines for low 10's of thousands and maintain them for 100's per year, the economics won't work. Construction robots might be a good counter example because you can't offshore them.
If robots are doing all the work how will people make money to buy the stuff the robots make? Is Jeff Bezos going to own the whole world or are we going to have another French revolution?
We should really endeavor to build collectively owned institutions that can purchase and operate the robots (and physical space) we depend on.
EDIT: Imagine the "credit unions" I mention in the following linked comment, but holding homes and manufacturing space to be used by members.
https://news.ycombinator.com/item?id=27860696
I think the comments are confounding shutting down the robotics research team with eliminating all RL research. Most robotics teams don't use data-hungry RL algorithms because the cost of interacting with the environment is so expensive. And even if the team has a simulator that can approximate the real world to produce infinite data, there is still the issue of the "simulator-gap" with the real world.
I don't work for openAI but I would guess they are going to keep working on RL (e.g hide and seek, gym, DoTA style Research) to push the algorithmic SoTA. But translating that into a physical robot interacting with the physical world is extremely difficult and a ways away.
With the mentioning that they can shift their focus to domains with extensive data that they can build models of action with etc... Why not try the following (If easily possible)
---
Take all the objects on the various 3D warehouses (thingiverse, and all the other 3d modeling repos out there) -- and have a system whereby an OpenAI 'Robotics' platform can virtually learn to manipulate and control a 3D model (solidworks/blender/whatever) and learn how to operate it.
It would be amazing to have an AI robotics platform where you feed it various 3D files of real/planned/designed machines, and have it understand the actual constituancy of the components involved, then learn its degrees of motion limits, or servo inputs etc... and then learn to drive the device.
Then, give it various other machines which share component types, built into any multitude of devices - and have it eval the model for familiar gears, worm-screws, servos, motors, etc... and have it figure out how to output the controller code to run an actual physically built out device.
Let it go through thousands of 3D models of things and build a library of common code that can be used to run those components when found in any design....
Then you couple that code with Copilot and allow for people to have a codebase for controlling such based on what OpenAI has already learned....
As Copilot is already built using a partnership with OpenAI...
I suspect it's because at a certain point detailed physics matters and simulating things well enough is really hard. A robotic arm might flex just a bit, a gear may not mesh quite correctly, signals may take just a bit longer to get somewhere, a grip might slip, a plastic object might break from too much force, etc, etc.
I'm sure the overhead and upkeep of a robotics lab far outweighs that of a computer lab for software research.
Are there any Open* organizations for robotics that could perhaps fill the void here? I think robotics is really important and I think the software is a big deal also, but it's important that actual physical trials of these AIs are pursued. I would think that seeing something in real space like that offers an unparalleled insight for expert observers.
I remember the first time I ever orchestrated a DB failover routine, my boss took me into the server room when it was scheduled on the testing cluster. Hearing all the machines spin up and the hard drives start humming, that was a powerful and visceral moment for me and really crystallized what seemed like importance about my job.
Designing robots to pick fruit and make coffee / pizzas cannot have a positive ROI until labor laws make the bsuiness-case for them. Majority of use cases where we can use robots for activities humans cannot perform (fast spot welding on production line, moving nuclear fuel rods, etc) have been solved. It is smart to focus on language and information processing, given that we are producing so much more of it, everyday.
How on earth would you know if a whiff was accurate, when we're talking about something which has never before been created?
I think even if you have intuitions about an approach, and have promising results, if you're trying to arrive at something new, it's really hard to know how far away you are.
That belief is wrong. Today's robots can't be made useful in everyday life no matter how advanced the software. The hardware is too inflexible, too unreliable, too fragile, too rigid, too heavy, too dangerous, too expensive, too slow.
In the past the software and hardware were equally bad, but today machine learning is advancing like crazy, while the hardware is improving at a snail's pace in comparison. Solving robotics is now a hardware problem, not a software problem. When the hardware is ready, the software will be comparatively easy to develop. Without the right hardware, you can't develop the appropriate software.
OpenAI is right to ignore robotics for now. It's a job for companies with a hardware focus, for at least the next decade.