There's an interesting section that talks about what you want from a pair programmer - questioning your assumptions, spotting errors, debating design with you, etc, and how Copilot doesn't do this, but instead just spits out the code it thinks you want. That made me think that, when using Copilot, the human is actually the copilot. You're the one spotting bugs, picking different designs, and so on.
I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.
For what it’s worth, Copilot is not being marketed as anything else than code suggestion software - an alternative to google search, perhaps more of a “I’m feeling lucky” without having to actually type the search term. Knowing how to succinctly express a problem is half the problem after all.
It's called "Copilot" and the tagline is "Your AI pair programmer." It's obviously being marketed as more than code suggestion software. It's being marketed as an AI pair programmer.
I agree that it is basically just code suggestion software, or like a big autocomplete. But it's definitely being marketed as more.
This is very true, and much like Tesla getting away with it for years (up until recently), GitHub will get away with this for a while as well. And so will the media decrying shit like "Is this the end of developers?!?"
> And so will the media decrying shit like "Is this the end of developers?!?"
Yeah, CoPilot already replaced my job. I used to be the office copypasta, going around to my coworkers desk all day long to fix their problems by googling them, then pasting in the first snippet to pop up. It didn't matter if the language matched or not, it was more the thought that counted. Now I've had to get a job as a loan rejection stamper at my local bank. It's not nearly as fun or exciting, but at least my effort makes a difference now. A big difference.
the biggest concern and reason to pump the brakes on copilot is the unresolved elephant in the room: the growing consensus that it violates FOSS licenses.
>Tests without the toil. Tests are the backbone of any robust software engineering project. Import a unit test package, and let GitHub Copilot suggest tests that match your implementation code.
Accompanied by a picture of copilot filling out the implementation of a unit test, based on the test's name.
The implication seems very clearly that it is more than a google search. According to the ads, it understands your code well enough to write the unit tests for it!
> According to the ads, it understands your code well enough to write the unit tests for it!
Yes, Github Copilot marketing has overstatements -- but it also has caveats about its limitations and creating errors.
In other words, the landing page has a mix of statements ... some hyping up the potential -- and some disclaimers to mention the reality and pitfalls.
Excerpt from that same Copilot landing page about it being wrong more often than right: https://copilot.github.com/
>How good is GitHub Copilot?
>We recently benchmarked against a set of Python functions that have good test coverage in open source repos. We blanked out the function bodies and asked GitHub Copilot to fill them in. The model got this right 43% of the time on the first try, and 57% of the time when allowed 10 attempts.
>Does GitHub Copilot write perfect code?
>No. GitHub Copilot tries to understand your intent and to generate the best code it can, but the code it suggests may not always work, or even make sense. While we are working hard to make GitHub Copilot better, code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted, like any other code.
> I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.
Right but realize how much of a marginal gain we're fighting for here. You can already do this with a separate window and some extra clicking and typing. Saving a little clicking and typing isn't a huge win, unless you're also willing to argue the most valuable work a SWE does is the clicking and typing.
This whole thing feels ultimately like a PR spot that we're all tripping over ourselves to participate with. Its the SWE equivalent of getting all excited about how self-driving is just around the corner TM. It's a premature waste of energy until real advancements are made.
I think the utility is similar to instant search on Google. I get very little by Google autocompleting the term I'm trying to type. I get quite a bit by Google suggesting things I didn't even know to search for.
The automatic SO search pane I was describing might give the same benefits. Most of the time maybe it would be useless. Sometimes it would show what I knew I wanted and save me the trivial effort of a search. The real gain would come from when it hit something that I should have asked but didn't think to. Then, I'd have a valuable insight I never would've had otherwise sitting in some pane ready for me.
Maybe it's just my imagination - but that's how I think it could be good.
I've been using TabNine for a while and found it very helpful. It really does seem like more of a hyper-intelligent autocomplete than CoPilot which feels like it wants to write it for you.
The thing I love the most about tabnine is how it learns on your own code base. So if you have particular naming schemes or weird idioms, it eventually picks up on them.
I've installed it a few weeks ago but haven't found it very useful yet. May I ask, which language(s) do you use it for and was it worth paying (I paid).
Python, works quite well, golang works super well. C++ and JS are reasonable. Rust, you might as well turn it off. The more popular the language overall, the better it does. Swift and Kotlin might have too little to have refined the model on.
Another thing: I was using the free version in beta so it was effectively the full blown version. It worked super well. I changed jobs, changed hardware, and haven't set up my key yet (hopefully still grandfathered in) but it's felt not as good - the output is a lot shorter, for one. I would likely get the paid version if I lose my beta key. But it's weird that you have the paid version and it's not useful.
It's also possible they nerfed the models over time in the name of some other tradeoff, like cpu or memory. May still be able to configure it. But I distinctly remember it spitting out entire lines or chunks of code, exactly as I would have written them.
It works best when you are writing stuff similar to your own codebase. Perhaps try vendoring some Swift or Kotlin in your project dir to get the engine to fine-tune on it?
Copilot was made from stealing code on Github, ignoring the licenses set on repos such as e.g GPLv2, using AI as a trick to license-launder code.
Copilot has announced their plans to become a paid service.
So this product that would not be possible without public, open-source code will itself be non-public, closed-source, closed-data. It is extracting value from the commons and funneling it to a private company.
The following is just my opinion, and I'm not that hard-set in case anyone has any nice arguments.
Generally, when you (for example) use GANs to learn from (copyrighted) images and generate new images, I see no reason why those new images should inherit any copyrights from the original image if the resulting images look sufficiently different. (Obviously, if you just train on 1000 images of Mickey Mouse, you'll get mostly Mickey Mouse, and you wouldn't get the copyright.)
Humans work the same way -- artists train on existing images or subjects/etc., and unless they produce something that looks similar to an existing image, they get the copyright.
In other words, I don't think training on copyrighted code violates copyright (UNLESS the license explicitly disallows that; maybe there will be licenses for that soon). However, if generated code is too similar to existing code, then that could be a copyright violation. In other words, the user is responsible to make sure they aren't violating any copyrights from their usage of Copilot. It may be useful to build plagiarism checkers for this purpose. (Or maybe they already exist.)
If you use Copilot, I recommend keeping track of things that were generated automatically, just so you can go back to change those components if necessary.
Out of the replies I got so far, I liked this one best. Unfortunately the story is off the front page already, so I doubt there will be much more discussion.
In my view, artists work the same way. If you ask them to draw Mickey Mouse, they (maybe) will. It wouldn't be fair to say that they are infringing on Disney's copyright by storing images of Mickey Mouse in their brain. But their version of Mickey Mouse won't be copyrightable (unless they add parody/significant creativity etc.).
If we perform an _exact_ simulation of a human brain such that it believes it is a human, how will copyright law work? (Maybe it will own the copyright and turning the simulation off would be robocide. Okay, enough sci-fi.) If we remove consciousness and all that from the simulation, will the copyright go to its creator?
I saw in a couple other comments saying that machine learning is "just an algorithm"... But is machine learning sufficiently different from the way some parts of the human brain work to warrant being held to different standards?
My opinion is that it's reasonably similar and should have the same privileges that humans enjoy -- learning from whatever's in sight that is not explicitly marked as "for authorized personnel only".
Thanks for the answer, but here is a bit difference though in our understanding of law, I think if I draw Mickey Mouse now, it is derivative work. Disney can sue me. I think there should be some reasonable difference like trademark cases, people will not see it as Mickey Mouse.
But I think eventually we are at level of what is the minimum required duplicate to define something as duplicate. (I think this is more of a problem of music, similar songs etc)
I think main problem is we are not yet there (exact simulation of human brain), we are more like 'convince people to you are exact simulation of brain' stage.
Also another problem here is 'self trained/directed' vs 'trained/directed by someone'. Imagine I have a human artist (never seen Mickey Mouse), and in front of me some Mickey Mouse art, if I am giving directions to draw a mouse, then saying make in cartoon style, then saying make ears bigger, etc. Till I get the Mickey Mouse reasonably similar, even maybe exactly to the pixel in front of me, is it copyright violation, I wouldn't maybe 100% yes, but I am very closer to that.
I think the catch is, a human is considered a creative being, as in it can create new content which consequently is copyrighted to that human.
An AI cannot - at least so far - not create create genuinely new content and also cannot assume copyright.
If the copilot would have been trained on open source software e.g. to discover bugs, bad code style, or other things "learned" by analyzing existing code and using the results of this analysis as a metric for judging code, it wouldn't be a copyright problem. But creating new code based on what it "learned" is a much more difficult field, especially, if it is "quoting" so literally.
> I don't think training on copyrighted code violates copyright
It might not be illegal, but for a lot of us it feels unethical. IMO licenses should list what they allow versus what they forbid. In other words, if something isn’t explicitly allowed it should be assumed to be forbidden.
We need similar protections for personal data too. Tech companies have gotten too used to ingesting data and profiting from it without asking for consent. I’m willing to bet that had GitHub asked repo owners whether GH could use their work to train a new product, most would have said no, much like what happened when Apple asked iOS users if they were ok with Facebook tracking them.
> In other words, I don't think training on copyrighted code violates copyright (UNLESS the license explicitly disallows that; maybe there will be licenses for that soon).
A new license saying "this license does not permit use in training an AI" won't have any effect, because the claim by the trainers is that they don't need to license the work in the first place. Unfortunately this is likely to be something that's only settled ad hoc in court.
Welcome to software engineering, where engineers have extremely strong and fixed opinions about all sorts of fields that are completely disconnected from software engineering.
«A derivative work is based on a work that has already been copyrighted. The new work arises—or derives—from the previous work.
If you own the copyright to a work, you need to be aware that you also have rights to derivative works. If you're considering incorporating someone else's work into your new work, you need to be aware that you may be violating the copyright to the original work.»
Regardless of the nuances of that point, GitHub copilot violates copyright because the content it was trained on still lives in it, and you can get it to spit it out verbatim.
If I were a master artist, my existence wouldn’t violate copyright, but I would certainly be violating it every time I chose to reproduce a copyrighted work for a client.
So this product that would not be possible without public, open-source code will itself be non-public, closed-source, closed-data. It is extracting value from the commons and funneling it to a private company.
This is just business as usual for tech companies though. How many Silicon Valley businesses built their products on top of open source projects without contributing much back? Heck.. how many YC companies do that? When you see vital open source projects like OpenSSL struggling to raise more than a few tens of thousands in donations (it only continues through OSF contracting work), and libraries like Svelte barely clearing $30k, when you know they're used by Apple, Google, Facebook, Microsoft, etc you can't be that surprised when companies do the same thing on a larger scale.
The entire tech industry and all the multi-trillion and multi-billion dollar unicorn businesses that have billionaire founders and millionaire developers working at them fall under the description of "would not be possible without public, open-source code" and "will itself be non-public, closed-source, closed-data."
We can't reasonably claim there's anything wrong with a company building a product on the back of open source work when literally all of us do that as well. The only difference with Copilot is scale.
> We can't reasonably claim there's anything wrong with a company building a product on the back of open source work when literally all of us do that as well.
We cannot (are not allowed to, by law, contracts etc.) open source everything we write. I think the most important contribution is not funding but to contribute in some way or another for most.
Having said that, it is disheartening that important projects lack even basic funding. Is this getting better? There are also success stories that pop up with funding and donations becoming easier.
For companies open source funding can be a major thing to improve their relationship with the wider software community. It is a signal that they are interested in sustainability of software and collaboration. And it is a form of recognition for the authors.
Edit:
I just realized we are talking about GitHub here. They are a major contributor to open and free software as they provide free hosting and tooling. I'm not saying this absolves them of everything they do with Copilot, but I'm very, very happy to have such an amazing service freely available and many others are too.
GitHub isn't good for Open Source in my opinion. Debian is right to use their own GitLab instance. Lots of things seem good in the short term and are bad in the long term.
Edit: I don't really know what things would be like without GitHub. Same with YouTube and Facebook. I don't assume any of these have a net positive impact, though.
No part of any open source license says I have to like companies that extract billions in value from open source projects without contributing back to the project. My opinion is simply that once you get to a few million in revenue putting a few thousand back in to the code that got you there is a decent thing to do, and if you don't then you're not very nice.
The fact that the license allows you to do this is great; the fact that people actually do is not.
GPL asks to release back changes to code made by a company, for which the company paid already, i.e. it's almost zero price for the company, and just look how much companies are afraid to donate zero ($0) worth of code back to the opensource project and prefer to pirate it instead, including such mega-rich companies as M$.
There's a difference between building a product using open source and not contributing back, and copying licensed code into your own codebase. One is rude and the other is straight up illegal.
The code doesn't exist in Copilot. The instructions for how to recreate the code does. On a very pedantic level those are not the same, but it probably is enough to argue that the product is 100% legal.
I don't think there's any water in that argument at a fundamental level. By that logic you could encode copyrighted information in any reversible format and it would be OK.
If I copy the code line by line with my eyes and keyboard, then the code briefly doesn't exist, but the instructions to recreate it do. Copying it is then a two-phase process: first I read the code, then I type it in. It is clear that I'm allowed to look at the code, so it's the action of typing it in that violates copyright. In the same way, distributing Copilot doesn't necessarily break copyright, but using it may.
I don't think people should look at this based on current law, but what is right.
Copyright was established (much) after the printing press made copying so easy.
We now need protection against machine learning (learnright?) because the only way to not work for Microsoft for free now is to not release your source code... so not open source. Remember when people said that Microsoft changed its mind on open source?
Yes. Models are much much smaller than compressed training sets, making it very clear that they are doing more than just compressing the entire training set.
Copilot generally (excepting rare cases where it produces snippets verbatim) does not steal code. The GPL restricts distribution, not usage. And (to my knowledge) no open-source license restricts learning from code. I cannot see anyone who doesn't want others to learn from their code ever release code as open-source.
I as an open source author absolutely do not want Microsoft to get richer from using my code, code that I contributed or published for the benefits of other developers.
They took my work, removed my name and trained an advanced pattern matching technique to try to make code like mine and then sell it. It’s so obviously ethically questionable it’s insane.
Developers are absolutely pissed about this, and rightfully so.
Not even copyleft licenses prohibit somebody from earning money from what you released, and that includes Microsoft. The idea behind free software is that it benefits all users equally, even if other developers get the biggest direct benefit.
The best question to ask yourself is if you would be annoyed as much if a company like Black Duck did a similar training or analysis with their OpenHub (openhub.net)?
I think one could even make a case for training an AI in this manner from the leaked Windows code: copyright law treats these generally as "fair use", though how you gained the copy of the code might still be illegal.
You need to comply with license first before you can use it to defense your position.
Copilot doesn't comply with opensource licenses, so authors of Copilot lost the right to use opensource licensed code permanently, until they settle the case with authors of the code.
Dual licensing with a copyleft license is common if you want to offer an ability for someone to develop a closed-source project: they can perfectly develop a GPL-licensed commercial project without paying anything.
If CC-BY-NC prohibits commercial use, it is not an open source or free software license (which have compatible definitions, but differ in motivations).
AFAIK, Creative Commons was set to create a set of licenses in the spirit of open source for creative works, and I wouldn't expect them to be open source at all.
Screwing over the little guy because he didn’t spend enough time contemplating possible legal troubles with his OS software seems, again, ethically dubious at best.
I wonder if they could generate the correct license to code copilot produces, and maybe even infer the preferred one from repo and generate code that is restricted to that?
The license for my software was written in a world where AI was not being used to replace me with my own code. Whatever license was chosen, was chosen to deal with the questions and issues known at the time.
It’s such a BS argument to say “your license didn’t anticipate the future, it’s your fault.” No, that’s not how law works.
Furthermore, law is not ethics. I said it’s ethically questionable because that’s what matters. Not if a court will find Microsoft guilty of some kind of overreach.
Anybody with even passing knowledge of law knows this, so please, stuff it somewhere.
The GPL very much restrict derivative works. It's the whole point of the GPL. "Usage" in the context of the GPL does not have the meaning you are using.
It is way more nuanced than that. For example if you never redistribute your work that was a fork from GPL code, then GPL states it's ok to never give back the source.
What we both said is compatible and consistent. The derived work is restricted by the GPL's provisions. Those restrictions just don't require you to distribute the source on demand unless and until you distribute derivative works to other users.
I'm open to new perspectives. But here's where I stand so far.
If I learn programming from a book that is copyrighted and use a small snippet (for example, how to do a particular kind of sort) from it in my own program, am I violating a copyright?
Read the book copyright statement and license. Some books have separate license for examples.
However, if you copy an example from a book into a code, then very often it's fair use, but if you copy the same example from a book into your own book, then very often it's a copyright infringement, unless explicitly allowed by the book license.
Turning the free contributions of an enthusiastic community and turning it into private, closed source wealth is a bit of a Microsoft tradition. Arguably Bill Gates did exactly the same thing with Microsoft; it was picked by IBM specifically because the community recommended it, a community despised by Gates who thought they were “stealing” from him.
Using the word “extract” seems misleading to me. It has connotations of removing something, or exploiting scarcity. When we extract water from the Earth, there is less there for others to use. But in this case nothing is being removed from the code they trained on. I don’t mean to have the argument devolve into mere semantics, but I really think the use of that word demonstrates an assumption about the issue: people are perceiving a loss to the folks whose code was trained upon.
IMO any place where Copilot is mentioned is a relevant place to put this. I don't know how anyone working in software can just turn a blind eye to shit like this. Anyone who uses Copilot is implicitly endorsing this theft.
We have a duty as practitioners in the industry to call it out when we see something wrong. If even devs aren't calling bullshit on Copilot, the media won't care, courts won't care, and it will be declared legal, and future theft will be normalized.
Its ridiculous how we all see the big tech companies doing various kinds of terrible shit and then the next new shiny thing comes along and everyone forgets all about it? Are you goldfish? What will it take to get someone to actually give a shit and stop supporting this kind of product/behavior?
> Anyone who uses Copilot is implicitly endorsing this theft.
Just like you're endorsing being sponsored and effectively stealing GitHub's bandwith? :)
> We have a duty as practitioners in the industry to call it out when we see something wrong.
It's wrong both ways. You accepted the service from GitHub, the free one where you get to open your account, host your code, share it etc.
What exactly did you expect? To be served for free for your entire lifetime?
Please, get off the moral highground. You ate the devil fruit, now you're whining about it. You should be smart enough to know that this kind of whining will get you nowhere. Just quit.
Basically you are saying any product thats free should be able to break the law arbitrarily? Google can decide to post your all email online, your location history for all time, your photos and you would be ok with that because its free? The undoubtedly free DNS server you use can leak all your requests too? You're ok with that?
Yes its free, no they did not tell me or ask for my permission before using the code in this way. Free does not give them the right to break existing laws and licenses. Its pretty simple.
However you did give them permission to use your code by the fact that you acceded to their terms and conditions[1] when you created an account. IANAL, I don't know if this section would hold to scrutiny in a court of law, but I'm pretty sure this is what their legal team considers to cover them when it comes to training Copilot on code hosted with them.
IANAL, but that is not my reading. They cover "Your Content" with the license grant, but not "any Content". The user still has the right to post "any Content" if they have the appropriate license to do so, but obviously they can't grant additional licenses to content the user doesn't own.
In my understanding your reading is that users uploading code that they don't own the copyright to, but otherwise have the right to copy through a license, are in violation with the ToS in general.
My reading is that the license grant only applies to "Your Content" as defined in the ToS, and otherwise users are free to upload code with permissive license and it _does not grant_ additional licenses to Github.
> Certainly the GitHub TOS grants them some common-sense ability to copy the code you upload so that they can usefully host it. Can you point to the portion that allows them to use it for Copilot?
> Because I'm pretty sure it doesn't. Section D4:
> > This license does not grant GitHub the right to sell Your Content. It also does not grant GitHub the right to otherwise distribute or use Your Content outside of our provision of the Service...
> You grant us and our legal successors the right to store, archive, parse, and display Your Content, and make incidental copies, as necessary to provide the Service, including improving the Service over time
> The “Service” refers to the applications, software, products, and services provided by GitHub, including any Beta Previews.
That's actually a great point. Ditto for GDocs. I've been pleasantly surprised at how good autocomplete suggestions have been in docs lately.
If I were to hazard a guess, I'd say that the vitriol around Copilot stems from five factors that distinguishes it from Google:
(1) The length of the suggestions alongside some of Copilot's marketing demonstrated that perhaps non-trivial replacement of engineers with AI might not be as far-fetched or far away as most people thought. Google's autocomplete has yet to make me feel replaceable.
(2) The content of the training data had a clearer intrinsic commercial value, making perceived license violations feel more 'real'.
(3) GitHub (historically) didn't have the same reputation as Google for training AI models on data uploaded to its free services. People likely (mis)placed some trust in GitHub when they uploaded code, and this backlash is part of the adjustment process.
(4) The indication that Copilot will eventually be a paid commercial service, effectively building a commercial service off the backs of millions of open source developers. While this is perfectly legal and common across all industries, it doesn't feel good.
(5) Copilot spitting out raw training data really doesn't help its image.
Honestly, I think all this talk about GPL vs MIT/BSD is a red herring.
It doesn't matter whether the code is GPL or MIT or BSD. If Copilot reproduces it in your codebase, you're violating the license anyway - almost all FLOSS licenses carry an attribution requirement, which Copilot does not and can not reproduce[0].
The difference between GPL and MIT is whether you have to release your source code, or just add a blurb in README. It's a big one, but it's downstream from the core problem: with Copilot, you won't even know when you're violating some license - much less what to do about it.
--
[0] - The whole point of a DNN model is to pre-mix the inputs in training, so that responses to queries are cheap and fast. This comes at the cost of making it impossible to reverse-query the model, so the only way for Copilot to give correct attribution would be to take its output and run a search on the training data, which would kill all the costs savings they won by using a neural network.
While I see how Copilot's data gathering can be considered unethical, in the end I ask myself, does it matter? Would it have been of any concrete advantage to open-source programmers if Copilot hadn't used their source code? I can't think of any
Well, if you chose a viral license like the GPL, you do not do that to have your code and knowledge being reused in a non viral (possibly closed source) licensed solution. So this is a fundamental issue at least GPL minded authors will definitely mind and fight against.
Conversely, the generous olive branch of free hosting is not a blank check that allows GitHub to use the hosted code for any purpose, especially when that purpose wasn't made clear as part of the terms of the original free hosting offer.
When I uploaded my code to GitHub, it was done so with the understanding that in exchange for the hosting and bandwidth, GitHub was permitted to use the code in a set of limited ways, as spelled out in their terms of service. I understood that I was contributing to building and establishing GitHub's brand as the go-to place for open source collaboration, a brand which they have undoubtedly benefited from.
With Copilot, GitHub has extended that use in a way that was not made clear during that initial contract. Regardless of the legality of this change, it's normal and expected for some users to be "offended". This isn't "being entitled", but a legitimate response to what many perceive as a violation of the norms of this industry.
That doesn't even get to the ambiguous legal questions involved, particularly with licenses that go beyond the typical MIT/GPL licenses. Based on GitHub's statements, it sounds like any public repo was fair game. What does this mean in the context of AGPL and other more restrictive licenses?
No I am happy to pay for it, and have paid for github in the past in fact. That doesn’t mean they can change the deal on what free hosting means with out any notice or method to opt out. Paid user’s public repos were not spared from Copilot either.
The hosting of open source code on GitHub is not some completely selfless act on their part. GitHub's value proposition to commercial users is in part contributed to by the fact it is used by a lot of open source projects and for solo or hobby projects, thus breeding familiarity with the platform.
This has been my experience as well. Some of the early promo material for Copilot showed someone writing a function signature with docs, and having Copilot write the entire body. This rarely works for me, except for fairly trivial functions. This is also not how I normally write code.
However, where Copilot has been rather good is offering intelligent "tab completion", where it makes a suggestion for the next line of code. In particular, it seems very good at figuring out the structure of my code and making sensible suggestions.
I suspect that if anyone ends up using Copilot for the long term, they will (as a human operator) need to learn and gain intuition about how to use Copilot's strengths and avoid its weaknesses.
Copilot is also almost certainly going to become better.
For example, NVIDIA's DLSS was viewed as a failure at launch, but now it's almost magic in terms of offering better results at a lower performance cost.
Better at issuing haphazardly modified boilerplate that already exists... the real costs of writing code are about carefully choosing how pieces will fit together and deciding on minutae with that in mind.
Copilot is the antithesis of designing good software.
Both parts matter. Making humans faster at writing the well isolated implementations means more time for the important stuff. Tons of productivity features could be foolishly dismissed with "well, the actually important bit is design so this is worthless".
What it means that it can't code well, so there's nothing to be excited about. That we cannot predict the future is of course true but really has no bearing on what you're saying, except to lend credence to the idea that it will be able to some day (i.e. that you can predict the future). Which is in essence the opposite of what "we cannot predict the future" should mean. That makes the entire argument kind of disingenuous.
We can't know the future, but we guess based on past trends. Technology has been pretty good at replacing humans in the past, and a lot of times we've said "a machine can't do this" and been wrong. We'll have to see, but I think it's silly to say it's not working now so it never will.
We've heard breathless claims that AI will replace people coding ever since the advent of AI in some form or another. If we truly were going on past trends, we would conclude it is not very useful and will probably stay that way.
It might also become worse. It seems to me that GitHub has opened themselves (well, this Copilot thing anyway) up to a form of DoS attack: Just imagine what might happen if a bunch of people opened up a number of public repos filled with utter crap?
Not suggesting anything, here, just musing on a possibility.
Well, that's a solvable problem. You just need a classifier to filter good code from bad. I would imagine they're already using analytics on the popularity/usage of the repos to weight the training data.
Yeah but only software engineers care about "good" code. Everyone else just wants their software to do it's job. If you can train AI to write code which is going to get the right outputs, it will win.
AI can probably also beat humans in terms of things like performance optimization. An AI can write code a human never would. We have to write short functions and well-factored code which fits in the human brain. An AI could theoretically write millions of lines of highly-specialized spaghetti which is perfectly correct and perfectly optimized.
> In this case, I felt like the Copilot approach would be worse for both experienced and beginner programmers. Experienced programmers would need to spend time studying the various options proposed, recognize that they don’t correctly solve the problem, and then would have to search online for solutions anyway. Beginner programmers would likely feel like they’ve solved the problem, wouldn’t actually learn what they need to understand about limitations and capabilities of regular expressions, and would end up with broken code without even realizing it.
This is similar to what I thought—replace "experienced" and "beginner" with "curious" and "lazy"—when I first read the tagline of GitHub Copilot on its landing page:
> Skip the docs and stop searching for examples. GitHub Copilot helps you stay focused right in your editor.
We need docs and examples for things that we aren't familiar with. Skipping them means that we blindly trust Copilot to generate correct code. And even if the generated code is correct, we still need to understand why and how it's correct.
One use case I see the best fit for Copilot is for generating boilerplates that we are already familiar with and understand.
Having used Copilot for a couple of hours on a Typescript/React codebase, I've encountered a very wide range of results.
Sometimes, the model correctly guesses exactly what I'm trying to write, and gives me non-trivial contextually correct suggestions. More than once I've been impressed enough to pause and take a screen recording of what it generated.
Other times, it's a complete mess that competes with the local language server for autocomplete corrections.
Among the misses, the most glaring issues have been when it's suggested code that violates static typing constraints. The model outputs the right contextual code, but uses variables with slightly different spelling/wording/capitalization.
This is to be expected of course, as Copilot doesn't integrate with the language server in VSCode. However it does show the limits of building the model without a native understanding of the AST it's working with.
I sure hope so, and given Copilot currently integrates with VSCode and its amazing language server APIs, I imagine this could be a (relatively) simple win to make a significant bump in model accuracy.
I do wonder if Copilot has any idea of the structure of what it's generating though. Does it know when something is a comment, a variable, or a piece of syntax?
Whenever a new generative language model is released and discussed on Hacker News, there's always the comment that "it's worthless because it doesn't always generate coherent text" which I've always disliked because it's unfairly reductive as progress in this field is iterative.
However, in the case of Copilot, there has to be a much higher signal-to-noise ratio in terms of generated text than creative language models like GPT-2/GPT-3 to use it as a pair programmer where the overhead in working with the model is worth the time saved via the model.
> it's unfairly reductive as progress in this field is iterative.
is that true though? Progress in natural(!) language processing is very iterative, but coding and code correctness is a very precise thing. Both semantically as well as literally. Code that's a 'little bit' wrong technically does not compile, and code that's a 'little bit' wrong semantically might produce catastrophic results.
Engineering doesn't become a natural language problem just because you throw a natural language model at it.
Natural language syntax is precise too. It's not like it's personal idiosyncratic preference. People will mostly agree whether something is syntactically correct or incorrect.
Agree with your sentiment. But there is a solution:
You: “Great, that might be good. Please stay here while we write and run the tests for that. (Our engineering handbook emphasizes how important tests are, as you know.)”
How does copilot fair in a language where correctness or at least well soundness is more transparent, for example Haskell (eg at least making sure all the types are correct, actual logic on the other hand seems harder, but maybe the declarative style helps)? I imagine in a less strict language you can generate garbage more easily.
Some of the examples given wouldn't even run, as far as I know. Copilot writing Haskell/Rust/<insert your intricate type system lang here> would probably just fail to compile.
I know very, very little about AI, but, like the OP said, it'll reproduce the traits of its training distribution. If the traits don't map on to the domain-specifics of the type ssytem, the traits don't map on, and the code fails.
Generating garbage the compiles is harder, but generating garbage in general is just as easy.
Building on this idea, if you could design a type-level program with Haskell (including some liquid Haskell to add further type invariants) it'd be awesome for something like Copilot to fill in the blanks. Seems easier than program synthesis!
That's why everybody programs in base notepad, with no syntax highlighting or auto complete or live error notifications or linting or any other quality of life feature.
What you say here is true. Writing code is not remotely the hardest part of software engineering. But that does not mean that there is zero value in making it easier.
Shifting gears is not remotely the hardest part of safely driving a car. Yet automatic transmissions are a nice feature for tons of people.
I do not understand this dismissal based on the fact that copilot does not completely revolutionize software engineering in a way that no other product has ever come close to doing.
The challenging part of coding is converting domain knowledge into correct code.
An AI has no high-level understanding of the domain and desired solution. It just guesses at what you might want and glosses over all the edge cases and exceptions that actually need to be considered and resolved. The issues that only come to light after a real intelligence studies the problem, and works through correct steps to implement a solution.
It's an iterative process, because the solution is arrived at only after attempting a solution, learning what you don't know or haven't considered about the problem, then resolving those ambiguities, and producing a revised and more correct solution.
When responsible programmers realize that they don't know what they don't know, they ask questions and apply the answers.
AI never realizes when it makes mistakes because it has no domain knowledge. It just pulls a best guess out it's ass and says, "Here you go. Check my work."
So far it's been worse than useless for me, the suggestions it offers don't even make sense. The existing code completion suggestions in VSCode for TypeScript are better.
The article mentions TabNine as an alternative. I think TabNine’s approach of providing one-line autocomplete suggestions is a nice middle ground where you’re still in charge of writing code but can do it faster with AI assistance.
Unlike Copilot you don’t have to give it instructions - it guesses what you’re trying to do from context. And since it works on a single line of code at a time, providing multiple completions which you can glance at and decide whether to use or ignore, you the programmer are still in charge of planning the function and making choices.
I have found that the cognitive load required to evaluate its suggestions is more of a distraction from what I'm doing--in sharp contrast to conventional autosuggest which is predictable.
That said, I haven't turned it off yet. It's very fun on those occasions when it suggests correctly.
It's mentioned towards the end of the blogpost, but I think the real blessing in disguise might be the ability to identify similar code from elsewhere on Github and linking to it. Too many libraries have you wading through pages of documentation in search of a single usage example.
Some other ideas:
- Look for libraries with similar code and suggest them
- Deobfuscating minified variable names? Or just suggesting better ones for your source code
- Speculatively running things. If the model predicts importing a certain library, start indexing it before I even type it
- Estimate how long it thinks a block of code will take to write, based on the predicted code + past usage patterns (like the kindle reading progress estimate)
Just spitballing but all of these things feel more helpful and pair programming-y than trying to fabricate code out of the blue.
I would be happy of my IDE could throw me to the relevant topic on cppreference.com from a type or variable. Or still show the function signature after I started typing the function arguments.
Feels like its a magical thing for me honestly. although I'm never waiting for suggestion while coding, sometimes the suggestions just slipped through and there is the OMG moment when it completes your thought.
1 thing though, tab for auto complete still does not work for me.. ever
Same, I've enjoyed using it in a couple of one off scripts and honestly it made writing them more enjoyable because I was kind of excited to see what copilot would do. I asked it to write a js function to clean special characters from a string and it worked perfectly, I think, well it worked perfectly enough for the script to complete.
I can't speak to whether or not using copilot in long lasting projects would be a good idea, but even if you just tagged the copilot generated code with a comment that said '#TODO: Verify this is correct' I still think it would help you get a lot of work done that you would normally have to switch to stackoverflow to do.
One thing that I found very interesting from the article was the idea of anchoring bias, I definitely felt this a lot. When I used copilot to generate some nontrivial functions that were directly related to my problem, if the code did not work and I ended up erasing the generated code completely, I am just realising now that when I reimplemented it myself I was still essentially using the algorithm copilot had come up with.
Copilot is going to really rock the numbers on lucrative unhose-your-sucky-codebase consulting gigs. I'd feel gleeful about that but those gigs are the definition of soul-sucking so I guess still no free lunch.
Well if they ever get it right I guess I could always become an electrician (this is not a dig on electricians, it seems like a fun trade, that or under water welding)
Yeah, the real curse is on the programmers who are going to be out of jobs much sooner than people anticipate.
All the talk about the low quality of code got me thinking: if humans aren’t reading the code, then the only thing that matters is correctness. Correctness may be an easier problem to solve than readability and refactorability.
Media synthesis means artists of all sorts will take a hit. Gpt means writers, programmers, journalists, and others will take a hit. Prompt engineering and larger models could mean any procedural use of knowledge in any context could be subject to automation.
Consumer level robotics, drones, and battery tech is close to a level that manual labor, delivery, and such tasks can be automated.
Maybe we should be preparing for post-scarcity. At the very least, the idea of "jobs" is going to have to transform. The 40 hour work week isn't going to make much sense in the near future.
Copilot (and all other AI tools, and non-AI tools) will lead to more developer jobs, not less. The thing holding most industries back at the moment is the lack of available developers at a cost-effective price. Literally every business has an opportunity to automate some process better than it does now, or to make an app to assist a business function, or to drive sales and growth through an application of new tech. The reason why they don't do that is cost and availability of skilled people to do the work. If a new AI tool removes the need to write code entirely and a developer just needs to work out the specifications and requirements then whole industries will start employing more developers.
The part of dev that you get paid to do is not writing the code. It's the part where you define what the code should do. That's where all the value in development resides. Getting AI to write the syntax is 100% upside for the tech industry.
> The part of dev that you get paid to do is not writing the code. It's the part where you define what the code should do. That's where all the value in development resides. Getting AI to write the syntax is 100% upside for the tech industry.
I guess they felt the same ways when the first compilers were introduced. Or interpreters.
> the lack of available developers at a cost-effective price.
There's already developers on the market at an incredibly low price. Good luck getting code that compiles out of them, much less that is correct. I'm afraid we'll see more and more of these, since now they'll be able to cycle through completions and try until it seems to pass all tests.
There's already developers on the market at an incredibly low price. Good luck getting code that compiles out of them, much less that is correct.
I've been a dev for almost 25 years, and in my experience how much someone is paid has very little correlation to how good their code is. I've worked with outsourced developers in India, Poland, and Vietnam who are brilliant at writing clean, robust, well-designed code and earn the equivalent of low-two-digits thousands of dollars per year, as well as former FAANG engineers who earn three-digit thousands of dollars and write untested, untestable spaghetti code.
I'm afraid we'll see more and more of these, since now they'll be able to cycle through completions and try until it seems to pass all tests.
In which case the value will lie in defining the tests, and the people who do that will earn the most. That's fine.
> I've been a dev for almost 25 years, and in my experience how much someone is paid has very little correlation to how good their code is. I've worked with outsourced developers in India, Poland, and Vietnam who are brilliant at writing clean, robust, well-designed code and earn the equivalent of low-two-digits thousands of dollars per year, as well as former FAANG engineers who earn three-digit thousands of dollars and write untested, untestable spaghetti code.
That has not been my experience at all.
There's this myth of the genius dropout in our industry, or that FAANG is basically luck, and yet it always fails to materialize.
It's not like gpt models are some sort of random walk or Markov bot. They produce nuanced text with deep semantic relationships between large segments of text. There's a direct relationship between the algorithms human brains used to produce the training data and the algorithms being approximated by these models. No, gpt-3 isn't human level generally, but it is human level competent in some domains.
These tools are effectively less then a year old in production, but we're already seeing the potential for huge disruption in lots of markets based on relatively straightforward uses of the tech.
I can't wait to see what a skillful and artfully sophisticated use will be. I don't think we've even scratched the surface.
> There's a direct relationship between the algorithms human brains used to produce the training data and the algorithms being approximated by these models.
Eh. That deep learning networks is like the brain is like saying that cars are like cheetahs. Sure, they go really fast by converting some kind of fuel into kinetic energy, and they move by exerting force on the ground, but that's about it.
Brains don't have ReLU units. Brains have lots of different types of topology, not just an uniform network, and can handle some of that topology arbitrarily being shut off due to damage. Brains use global chemical changes for (otherwise) out-of-band signaling purposes. Brains don't use gradient descent. Etc...
This seems like a gross simplification. We want factorable code because that helps with DRY. DRY code is more portable and the end product is smaller in size. Also, there's performance considerations.
I'll be surprised if there's a tool like copilot anytime soon that can identify need for memoization, for example, and implement it.
But if a human isn't messing with the code, the code doesn't matter - what the bot does to the code matters. And if the bot can handle bad code in a way humans can't, that means something. Code quality will become less important if automated tools, the ones reading and writing the code, don't care about it.
In some ways this has been the case for a long time. Tools that generate code, such as Visual Studio's form designer as a simple example, often generate really bad code. But no one ever reads it, and if you need to make a change you just run the tool again. So the fact the code is bad isn't very relevant.
> if humans aren’t reading the code, then the only thing that matters is correctness. Correctness may be an easier problem to solve than readability and refactorability.
I suspect it's the opposite: It's rather easy to write code that looks good but isn't exactly correct.
Underwater welding is pretty rough--all of the danger (and expertise required) of deep sea diving plus all of the danger of welding. I have a strong suspicion unless you started from a young age and got trained in the Navy or other industry as an apprentice no one is going to spend the time and money to train you up when you'll only get a few years of good work in for them.
Electricians are good but you're going to get zapped quite a few times in your career. From what I've heard plumbing is where it's at for finding a trade that will let you retire without destroying your body. All of my friends/family who got into construction eventually tried to find their way to plumbing because they saw it was something they could do well into their 40s and 50s without destroying themselves with overly physical work.
Get into a union early if you want to become a union electrician. The low number unions usually pay the most.
That is if you can get in. The test is a high school equivelency exam, at least for local 6, but hundreds of guys are taking the exam. You can't miss one question.
1/2 of the hiring process is the interview. (They used to accept only electrician's sons, like the police/fire departments; but that has changed.)
You will make a good middle class living, have retirement, medical, and will never be homeless if you show up. Being around guys all day long can get tedious.
Get in early if you can. To become vested for retirement you need to work for 25 years. Union electricians seem to die at 65. Too many guys retire--collect three paychecks, and gone. Whatever you do in life don't expect to be happy in retirement. Sorry guys--tonight is my Journal night. I am sad.
You can bypass the test, and the interview, if you know your trade. You will be referred to as White Paper. You will be treated like chit, but you are making exactly the same as the other guys.
(I worked as an electrician, but didn't like it. The pay was good, but it just wasen't for me. I have had too many jobs that, "weren't for me"? I was becoming my father? If you had my father, you would understand. Oh yea, you can always set up shop yourself if you get your C-10 license. Non-union construction work should be avoided. Why--terrible job conditions, and pay.)
Hi all, Jeremy Howard here, author of the article. I'm happy to answer any questions you have about language models and program synthesis, Copilot, etc.
Thanks Jeremy! Where have language model(s) had the most useful for practical applications in terms of generated text? In terms of practical applications, I specifically mean the inclusion of a language model(s) has made life net better.
I'm actually most interested in language models that are fine tuned for other purposes, such as classification. IMO that's where the biggest immediate opportunities are. I've seen them used quite widely in industry, such as for legal discovery.
Hi Jeremy. Can you describe your knowledge of program synthesis? How long have you been studying the subject? Can you point to some interesting program synthesis work you have read lately?
The value of copilot is in ideating solutions not code correctness.
Given it's far from getting syntax and semantics right, it should probably be best to position this product as a tool that promotes creativity not productivity.
> Given this, why does Copilot write such crappy code?
> The reason is because of how language models work. They show how, on average, most people write. They don’t have any sense of what’s correct or what’s good. Most code on GitHub is (by software standards) pretty old, and (by definition) written by average programmers.
Does Copilot apply any project quality filters before training? A small minority of public repos on GitHub are awesome projects that are popular and maintained, while the rest are forgotten experiments, side projects and interview answers.
I signed up for Copilot’s waiting list yesterday and I am looking forward to trying it, with some skepticism.
That said, I find OpenAI’s beta APIs based on GPT-3 to be very useful (so useful that I immediately released new versions of my Common Lisp and Clojure books with examples). These APIs are so inexpensive to use and useful, that they really should be part of many developers’ standard stack: different than Copilot that seems like a gimmick (but to be fair, I haven’t tried it yet).
what i don't get, is how a copy of (parts of) whatever's code, (or worse, a concoction of them), without any context, would solve anything but import the (unfound yet) errors as well, and introduce new errors.
Hah, just make it autocommit, and will be much like with self-driving cars - human takes the blame for computer's mistakes (= algorithm/ programmer/ knowledgebase), as well as hir own, and can't/won't do anything about it..
I’m not into copyleft, but is there already an analogue to AGPL that explicitly prohibits the usage of the code to train AI models? Shouldn’t be that hard to do, right?
There's a need to answer the question (precedent, new technology, huge potential / market interest, and ethical extremes in one way or another) but there's an alternative to giving in to that urge. Perhaps we're better off leaving it as-is. We can evaluate about this in some later stage of the process.
Neither of both. It's completely irrelevant, as you cannot and may not use it. You cannot use it for production code as it's merely a toy for managers.
And you may not use it as the license of the generated code is unclear at best, GPL at worst.
I hope they release a fine tuner for Codex. Jeremy is right that whole problem solving is likely overreach right now. But can I train the model to do a Rails version upgrade?
It will depend on the companies and the price of Copilot.
Then they will replace expensive experienced programmers with cheaper inexperienced ones and Copilot.
I haven't used Copilot yet (I'm not coding in any of the languages it covers)
but I've tried to understand it from the few articles (most critical) posted
about it on the internet and from my understanding of program synthesis and
languae modelling. I don't fully trust lay users to be able to exercise a
machine learning model comprehensively enough to precisely understand its pros
and cons, but I think I've formed some intuition about how Copilot works from
the demonstrations I've seen. Here's my thoughts:
a) As TFA points out, Copilot was trained on _all_ code on github (for select
languages). "All" includes good code and bad code, code with bugs and code with,
er, fewer bugs, code that does what it should and code that doesn't. The result
is that Copilot models _all_ that code, regardless of whether it's got bugs or
not.
b) Copilot is (based on) a language model that predicts the next sequence of
tokens, not a discriminator of good/bad code. That means it can generate code
but it can't generate "correct" code. It doesn't have a concept of
correct/incorrect code.
b.1) More generally, it's impossible to discriminate between "correct" and
"incorrect" code from features (tokens, in the case of language models) alone.
The correctness of code depends on the programer's intent. In program synthesis,
the programmer's intent is encoded as a specification and the correctness of a
synthesised program must be considered in the context of that specification. A
language model could be trained to include a specification as a feature, or as
an additional modality (similar to training on images and text together to
generate captions for images, or images from captions). However, this is not how
Copilot was trained. The upshot of all this is that Copilot can't generate
"correct programs" because it doesn't know what "correct programs" are.
b.2) But wait! Copilot can generate code from a specification! From a natural
language specification, in the form of inline comments given as part of a
prompt. This seems to be a favourite mode of testing Copilot and its results are
impressive, but there are two problems: i) natural language specifications are
imprecise and get more so the longer they grow; and, ii) Copilot still has no
way to determine whether the code it generates satisfies a natural language
specification, or not.
b.3) For program synthesis with guarantees of correctness, a specification is
not enough: a verification procedure is also necessary -and those tend to be
very tricky because Halting Problem.
c) Copilot has memorised at least part of its training set. This is evident in
the demonstrations of verbatim generation of known code, e.g. the Inverse Square
Root function (from Quake... III Arena I think?) etc.
c.1) This observation is helpful in understanding language models in general.
The assumption is often stated that a model can't possibly be memorising its
training set because models are smaller on disk than their training sets. And
yet it seems that large language models are memorising parts of their
training sets- most likely, the ones that are more common, and therefore most
likely.
c.2) This in turn means that the programs that Copilot is most likely to generate
are variations of programs included in its training set and that have been
memorised. Why is this most likely? Because those programs are themselves "most
likely" - they are programs with the highest prior probability in the space of
programs represented by Copilot's model.
c.3.1) That said, Copilot doesn't always generate memorised programs verbatim. It
clearly has the ability to "mix and match" and generate programs that are not
included in its training set. How does Copilot (or language models in general)
achieve that? A language model is essentially a representation of a dense region
of Cartesian space bounded by the elements (tokens, expressions, whole programs)
of the training set. Anything within that dense geometric region, a language
model can recognise and generate. Anything outside it, is down to luck.
c.3.2) To generate programs not included in its training set, Copilot
interpolates between the programs in its training set and other programs in
its model ("interpolates" as in "finds a point on a gradient between two
programs' representations"). However, this interpolation is stochastic.
Fudging very, very much, we can imagine Copilot traversing the AST of a
memorised program and applying slight peturbations to that AST, at random,
though according to its learned prior probabilities. This is why Copilot will
generate different code each time it's run even with the same prompt, and the
generated code is closer or farther away from memorised code according to its
"temperature" parameter (that controls the randomness of "perturbations").
c.4) Suppose you take some program P and walk down its AST, perturbing the nodes
at random, without having any way to know whether a perturbation results in
correct code, incorrect code, or pure garbage. What is the expected result? The
expected result is correct code, incorrect code, or garbage. Remember: Copilot
can't tell which is which. So we don't know which we'll get.
d) All that said, I think that i) Copilot can make a great boilerplate generator
but ii) only if people learn to use it right. So far I think most demonstrations
I've seen are "holding it wrong".
d.1) As to ii) above, I think the best use-case of Copilot is to generate short
code snippets, spanning at most a couple of lines, rather than entire functions.
I think it's also going to work a lot better if it's allowed to complete code
rather than comments (with code). See point b.2 above. The demonstrations I've
seen so far seem to try to get Copilot to act as an Autopilot (sorry, couldn't
resist). In that, it fails in the ways it should be expected to fail . I suspect
that as users get more familiar with Copilot's abilities the best way to use it
will become evident and Copilot will take its rightful place as a useful tool in
the programmer's toolbox that can simplify, but not replace, day-to-day
programming work.
e) Rumours of the imminent demise of the programming profession have, of course,
been greately exaggerated. Wait another couple of generations, maybe. Human
generations.
f) If you're interested in program synthesis and the progress that has been
achieved so far in this old, but still active field of research, the following
is a good recent survey of the field:
It is tempting to apply Betteridge's Law and answer no. But, perhaps the answer is yes. It probably can be a blessing, once you're used to it. And it certainly can be a curse, writing code that is incorrect, buggy, or just not understood by the programmer. It would be nice if it better could explain its reasons and add references relating to the snippets. Maybe one day. Until then, I think I'll just google stackexchange as I've been doing for so long...
I think Betteridge's Law works just fine here. The answer to the article title is 'no' because Copilot is not just one thing used in one scenario. Depending on what situation you're in and how you use it, it could be a blessing or a curse.
A brief tangent - the only useful bit of voice recognition for me has been telling Siri to remind me about something at a day and time. Calendar events don't work well (gets inserted on the wrong calendar, can't invite attendees properly), and every other use case for voice control has been a gimmick at best that I have shut off or not used. Whenever I call any phone number, I slam the 0 button and say "operator" to bypass the horrible menus, which usually accept digits anyway as the voice control is a piece of shit. The number of "Chat Bot AI" startups that have been funded, found no usage, pivoted to something else has been astronomical, and chat bots should seemingly be one of the first use cases of useful AI.
With that said, I have found all "creative" types of AI to be similarly gimmicky and shallow. Co-pilot is like IntelliJ code-complete in more generic circumstances. It can't architect systems, interview and recruit good engineers, or do any of the things that separate true success from failure. It's yet another cool demo but ultimately useless for me.
I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.