Copilot has so far been pretty useful to me as a "sometimes smarter intellisense". It'll frequently correctly guess the arguments I want for a function (and their types), and every once in a while I'll type `a.map(` and it'll auto-fill the transformation code I was planning on writing.
The simpler the task I'm trying to do, the better chance it has of being correct, but that's also the part where I feel I get the most benefit from it, because I already thoroughly understand exactly what I'm writing, why I'm writing it, and what it needs to look like, and Copilot sometimes saves me the 5-30s it takes to write it. Over a day, that adds up and I can move marginally faster.
It's definitely not a 100x improvement (or even a 10x improvement), but I'm glad to have it.
If this works as well, locally, to escape the privacy issue, I'll be thrilled. Checking it out.
Yes, I went into all of this thinking the tools need to handle complex code in order to really be useful. But that's the code I want to be most careful about, so it's the code I really want to architect myself, and understand thoroughly.
Making the simpler code much faster to develop leaves me a lot more time and focus for the complex work.
These tools really are amazing, even with their limitations.
Copilot also helps a lot with verbose, RSI-inducing languages like C++. On days where my hands are cold or my hands are just not feeling as physically limber as usual, it's a big relief when it autocompletes a long series of symbols for me.
I know regular old LSP/Intellisense helps here but you are still often constrained to only autocompleting one token at a time, and have to type it at least partially in most cases.
I wonder if this will guide language design toward more verbose RSI-inducing syntax over time, sort of like how faster CPUs make software bloat more tolerable.
For example, explicitly declared type information helps the model with predictions, so a language with more of this kind of syntax will be a better fit for the model.
Gradually, will we see languages that are only meant to be used within the context of an AI assisted IDE?
>Gradually, will we see languages that are only meant to be used within the context of an AI assisted IDE?
Some languages basically require an ide to be productive. I spent the first 3 years of college using vim to code java, until a kindly professor pulled me aside and told me to get with the 21st century, lol. After I got used to it I couldn't believe I didn't switch sooner.
Some old arcane 'languages' are already things I wouldn't touch without gpt. When is the last time you had to write awk? By god I will never manually write a line of awk.
Tried it in hobby projects: saved a lot of time in java, less in python. In my scala project i have something like an internal dsl, and it was not much of an improvement over intellisense. I just realized how much i think and overthink, while typing. When it completed the entire block, I’d have to stop and think over it regardless.
Local models are a different game though. I want not an autocomplete, i want to tell it “go do this, similarly how we did it there” and it does it. Basically, a high level dsl, that magically appears after a model looks at your codebase.
Did you python code have thorough types usage? I find that strongly-typed languages seem to do better for code suggestions based on existing code in your codebase. Now, why that is, I don't want to speculate, but for example it seems to generate better suggestions for Typescript than it does for Javascript. Python's type system is... well, _there_ and that's about all you can say for it, and I find that most Python I come across, even the professional stuff, is much less... "rigid" than code in typed languages. I wonder if that makes a difference in terms of usefulness for suggestions.
Having been using Copilot for a few weeks I would substantially agree with this.
However, just yesterday in a module with quite a lot of context and types it autocompleted a function I was pondering for converting between different schema languages.
It was subtly wrong but the basic structure was correct. I had wanted to implement this function for a while but, having not really thought it through, imagined it would be much more complicated and would have required reading the ABI spec for the target. I never would have attempted it unless it had shown me the way and that it was relatively simple.
This is probably a piece of code that saves significant manual code. Copilot doesn't need to nudge me towards writing that many high leverage pieces of code to start becoming a big multiplier.
Genuine question, do you not use GitHub for things other than copilot? It seems to me either the privacy issues of copilot are overblown or the privacy issues of GitHub itself are underblown, because they both end up with basically the same data.
I have files open in VSCode that aren't committed to Git, and that I wouldn't be allowed to commit there even in private repos (customer data covered by GDPR).
In this context I think it's important to note the distinction between "Copilot for Individuals" and "Copilot for Business", because for twice the money you potentially get a lot more privacy:
> What data does Copilot for Individuals collect?
> [...] Depending on your preferred telemetry settings, GitHub Copilot may also collect and retain the following, collectively referred to as “code snippets”: source code that you are editing, related files and other files open in the same IDE or editor, URLs of repositories and files path.
> What data does Copilot for Business collect?
> [...] GitHub Copilot transmits snippets of your code from your IDE to GitHub to provide Suggestions to you. Code snippets data is only transmitted in real-time to return Suggestions, and is discarded once a Suggestion is returned. Copilot for Business does not retain any Code Snippets Data.
This is frightening. I tested copilot when it first came out in beta, while I had client-owned docs, possibly containing API keys, open in the same IDE.
I wonder how long it will be until people's baseline assumption is that their shit is being used in this way... I genuinely don't understand how it isn't already.
My baseline assumption would be that an IDE running on my own box wouldn't be uploading my code somewhere unless it was to my own repositories. Understanding that for Copilot to work, it needs to write prompts to GPT, I'd assume it wasn't uploading anything other than the tab it was working in. Not just, like, scanning my whole drive and uploading everything. That's insane.
Maybe I rely too much on the community to red flag these things. I don't install questionable extensions. I wouldn't have tested copilot at all if it hadn't received such enthusiastic support (here, among other places). The fact that it isn't sandboxed to the document you're working on should make it an absolute malware pariah, and this is the first I'm hearing of it.
unless that's directly in the contract, I'd not trust Microsoft to discard anything that might be of value in a training dataset at some point in the future
I would hope they’re training it to recognize and not suggest API keys. I would emphasize that as an explicitly important part of its initial definition-of-self/prompt.
“privacy” sits in a broader sea that includes “control”, “customization” and “lock-in”. i’ve been using the GPT family of tools less than otherwise because they’re constantly changing in ways that do or will disrupt any workflow i integrate them into: new guard-rails are added such that new versions explicitly don’t support use-cases the old versions do, etc.
self-hosting solves those problems: if done right, it gives me a thing which is static until i explicitly choose to upgrade it, so i can actually integrate it into more longterm workflows. if enough others also do this, there’s some expectation that when i do upgrade it i’ll have a few options (models) if the mainline one degrades in some way from my ideal.
LLMs are one of the more difficult things to self-host here due to the hardware requirements. there’s enough interest from enthusiasts right now that i’m hopeful we’ll see ways to overcome that though: pooling resources, clever forms of caching, etc.
though that too comes with its own set of privacy issues, especially when done at a larger scale with strangers on the internet as opposed to just with your personally known and trusted buddies… and rare are the people for whom the latter happen to contain enough people with strong enough hardware that that pool would be sufficient.
It's not too weird a position that some people are mistrustful of Microsoft and use other version control solutions, but want to be able to take advantage of these new classes of tools where they help.
Personally, yes, I do host with Github, bit previously that was a very well-defined boundary - only what I deliberately committed and pushed went to them. Now, it's potentially anything in my editor, so I make sure to use another program for scratch files. I probably don't have anything I'd really care about them getting their hands on, but still, it's better not to get in a bad habit and then slip up someday, pasting in private or sensitive details without thinking.
If I can eliminate that as a potential source of mistakes, so much the better.
API keys, passwords, email addresses, etc. We write those data with our editor but we don't commit them to vc and github never see them. Copilot must not see them too.
No, you're not, unless you never edit any code that you don't commit. I write scratch code all the time, and I can think of at least a few times where I hard-coded things into these files that probably should never leave my work computer.
Agreed, would be good if you could whitelist only source accessible from the git repo (eg. won't work on gitignored files unless explicitly activated).
Neither do I. For that matter, I don't use it for anything where privacy is a requirement (be it for personal or for business/work reasons).
That being said, I've way too often encountered people for whom git oddly only exists in the form of Github, to the point of those being synonymous to them. The concept of a self-hosted repo seems totally foreign to them, and the only local git trees they have is what their IDE created for them in the directory of their checked out or created source project. It's weird, but such people exist… and lots of them.
Do you worry that if you use it too much you will no longer be able to bang out those simple tasks without it? It seems to me like it will make you weaker at some skills. Like how I take around 30 seconds to read a mechanical clockface.
I was working on a personal project last night, and I wanted to extract out a title between title=" and S0, I asked bing and it instantly gave me back a regex with the proper look ahead and look behind grouping.
Now theoretically, I know about those things, but even I'll admit I would have messed up the syntax and probably spent 15m on regex 101 tweaking things. The ai assist is great if you take it as a suggestion of something you should look at, rather than blindly trust it
Yeah, I think it's pretty good at this. I've also done "write a test like the one above but for Bar instead of Foo" and it got the right answer. It would be better to not duplicate code like that in most cases, but sometimes it's OK.
I recently needed a script to grab all my github repos from the github api and produce a CSV with the columns URL, name, description, and it could not do that at all. Just hallucinated libraries and APIs that did not exist.
I completely agree. Lately, I've been doing lots of green-field code development, and it shines in that area because there can often be lots and lots of boilerplate. I think Copilot makes me perhaps 20-30% more productive in that phase.
That's a lot less than 100x, but it's way more than nothing! I don't think I'd want to give it up.
i feel the same way, but the `a.map(` autofill is hit or miss for me. it'll interrupt my train of thought and it'll take me a minute to remember what i was trying to do, then i'll have to review the code to make sure it's right. and often there's a better way to do it, so i end up refactoring.
I was frustrated with it after a week and about quit it but am developing a more nuanced perspective.
Tldr; Can't imagine using it with a language that doesn't have great type checking to catch its BS, and they really need to tune how it interacts with traditional intelligence.
Been using it in VSCode with C# and Typescript. It gets in the way of the normal intellisense forcing you to use a shortcut to get out and back to intellisense.
For me this was really getting in the way because working one line at a time when you know what you need isn't its strong suite.
Stubbing out methods or even classes, or offering a "possibly correct" solution in areas you are fuzzy about is a strong point. Even stubbing out objects, methods, and etc using your existing code as a reference point...
But possible is the key. It's a bullshitter so it could randomly invent parameters or interfaces that are plausible, but not real. It also trips up on casing regularly.
All this to say; you gotta review everything it does and be on the lookout. Without the tooling help of a typed lang like c# or typescript this would be much harder.
I thought I was reaching a productive flow with this but then I loaded my proj in Rider, which didn't have copilot installed, and banged out some code and man it was frictionless in comparison.
Feel like copilot should be opt in via a shortcut, not opt out as it is. They really need to work out how to reduce the friction of using intellisense when it's the best option which it often is. But the copilot creators seem dead set on it constantly back-seat driving.
This is bang-on. You absolutely have to review everything it produces. Unfortunately I think a lot of junior and intermediate developers are going to let Copilot-generated errors into their codebases.
I was dealing with timezone stuff yesterday, which I have a lot of experience with, and I got Copilot to generate a bunch of code to deal with data in various timezones. I reviewed it and discovered that mixed in with all the multiplying of various numbers like 36000 and so on, it was basing Eastern time on UTC - 4, Pacific on UTC - 7, and so on. Simply by putting numbers like 4 and 7 in the code.
But of course that doesn’t take into account the difference between EST and EDT: what we call “Eastern time” is different depending on the season. Not to mention that there is a legislative push to eliminate daylight savings time, which means hardcoding these differences as integers is not a good idea.
This is a prime example of bullshit: an unambiguous solution, presented confidently, that is technically incorrect and also completely shorn of nuance.
The simpler the task I'm trying to do, the better chance it has of being correct, but that's also the part where I feel I get the most benefit from it, because I already thoroughly understand exactly what I'm writing, why I'm writing it, and what it needs to look like, and Copilot sometimes saves me the 5-30s it takes to write it. Over a day, that adds up and I can move marginally faster.
It's definitely not a 100x improvement (or even a 10x improvement), but I'm glad to have it.
If this works as well, locally, to escape the privacy issue, I'll be thrilled. Checking it out.