What I really love is the unbounded optimism of the AI. How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?" that would then just think for a second and go "Certainly! Let's start by ...". These people exist, but they're one in a thousand (or million?). ChatGPT basically does that for almost every prompt you throw at it. So it really helps me overcome research barriers in software development in an incredibly short time, stuff that would've taken days or weeks to research can be done in literal minutes. And as I said before what's amazing is that the PoCs are really elegant and minimal, i.e. no superfluous code at all and condensed to the essence of the problem you're trying to solve.
> How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?
I couldn't agree more. I work as a software engineer in the medical field (think MRI's, X-rays, that sort of thing). I've started using ChatGPT all the time to write the code, and then I just fix it up a bit. So far it's working great!
Humor is fine, but when several readers don't recognize it as such, the joke obviously isn't based on shared common knowledge. Some people walk away with wrong information, and that isn't helpful.
Lots of comedy more or less proves it needn't be thoughtful as opposed to emotional. Laugh tracks, or the particular speaking style of a standup comic, are riding on the feelings they invoke. Being surrounded by real laughter also makes things "feel" funnier. Laughter is contagious. Fart jokes and home-video genital injuries get laughs with no real "thinking" component.
This joke is how you separate people who know what they’re talking about from the hopeless optimistic crowd.
People who just take that joke as serious clearly haven’t thought through the consequences of using code you don’t completely understand and has no real author.
Therac 25 is often a key topic in any rigorous computer science education.
I went with a mechanic in a chair lift once. He was very nervous, saying he knows the kind of people who maintain these things, and he knows the kinds of things that can go wrong. I feel the same way about automation, whether it be self driving cars or an X-ray machine. I know the kind of people that program them, and the vast number of ways things can go wrong.
I work in the medical device field. Funny you should say that because just yesterday I was looking at some really crappy (but still safe, mind you!) code and wondering how I'd feel if I were being wheeled into the OR, knowing "how the sausage is made."
With self-driving cars, you also need to consider the kind of people who might be driving them manually. Drunk, sleepy, prone to road rage, distracted by noisy kids in the car, thinking about their financial or relationship problems... there's a terrible amount of death and shattered lives to consider there.
If I remember right, during the height of the most recent Iraq war, a U.S. soldier had a higher probability of being killed in a traffic accident while in the United States than being killed while actively deployed in Iraq.
If a (so-called) self-driving car screws up and kills some people, it's headline news everywhere. If Joe Blow downs five shots of scotch after being chewed out by his boss and crashes into a carload of kids, it barely makes the local news.
A strong agree as well. This is going to help me a lot in writing software for large passenger aircraft. The integer types GPT gives me aren't the largest, but I think it knows what it's doing.
The great thing is it can repurpose the code from the old inertial guidance unit since the new one is so close it's basically the same thing. I think that GPT thinks it knows what it's doing.
What holds me back the most from using these tools is the vagueness of the copyright situation. I wouldn't straight out want to use it at work. For hobby projects that I might want to use for something serious one day, I'm on the fence.
Developers who use this for serious stuff, how goes your reasoning? Is it just a calculated risk? Reward is greater than the risk?
Google vs Oracle is still being fought a decade on now.
What hope does the legal and legislative system have of possibly keeping up here? The horse will well and fully have left the barn by the time anything has been resolved and furthermore if AI continues becoming increasing useful there will be no option other than to bend the resolution to fit what has already passed.
> Developers who use this for serious stuff, how goes your reasoning? Is it just a calculated risk? Reward is greater than the risk?
It seems obvious that AI is the future, and that ChatGPT is the most advanced AI ever created.
For me (in the medical industry), if something goes wrong and someone dies a horrible death, I can just say that I didn't write that code, ChatGPT did. Not my fault.
Next time you are at the hospital getting an MRI, I hope you think about how it's entirely possible that ChatGPT wrote the majority of the mission-critical code.
I can't believe people are taking your comment seriously. How far has reality moved into parody, if a comment saying you'll be happy to blame ChatGPT when people die doesn't go far enough to give it away?
Maybe they're just replying without having properly read your post, but that's not great either.
Sad but true; these days you read so many news stories that you first think are satire, but turn out to be true, that sarcastic comments are increasingly difficult to recognize as such without the appropriate notice.
It's not hard to believe. Over the past few weeks, I've read several very serious comments that express a similar attitude, although not quite as blatantly.
I can assure you, as a former quality engineer at a medical device development facility, that there is absolutely, positively zero chance that anyone there will use any AI-powered coding tools to write code that goes onto any device that is ISO-13485, CE, or otherwise compliant to some existing medical device regulations (I speak in the USA and European markets; I cannot speak for other markets). There is literally a defined waterfall development cycle for FDA-regulated devices that requires software features to be very precisely specified, implemented, validated, tested, and manufactured. Anyone suggesting using AI at such a facility would be laughed out of the room, and perhaps even re-trained on the procedures. Anyone caught using such tools would probably be fired immediately and all their code patches would be put under intense scrutiny and possibly even rewritten; of course the device software they were working on would remain in development and not released until that was fixed.
The above two comments show the difference between software "engineers" vs "developers"...and none of the major social media platforms (and other consumer-level applications) employ engineers.
Other projects can't use waterfall development because they would like to actually produce something useful instead of what was decided at the start of the project.
This isn't the way pharmaceuticals are developed; we don't require the pharma companies to know how they work (and we shouldn't, because we don't know how many common safe drugs work). We validate them by testing them instead.
Other projects can't use waterfall development because they would like to actually produce something useful instead of what was decided at the start of the project.
It's a whole different world of software development. If you set out to build flight control software because it is needed to run on a new airplane, you're not going to pivot midstream and build something else instead.
> For me (in the medical industry), if something goes wrong and someone dies a horrible death, I can just say that I didn't write that code, ChatGPT did. Not my fault.
Liability doesn't work that way. Your view is so naive I'm having doubts about whether your an adult or not.
If you delivered the product, you're liable, regardless of where you got the product from.
After getting sued, you might be able to convince a judge that the supplier is liable. But getting sued is expensive, and the judge may not rule in your favour.
And even if it goes in your favour, OpenAI is simply going to turn around and point to the license you agreed to, in which no guarantee of fitness for purpose is specified, and all liability falls to the user.
I would trust ChatGPT code about as much as I trust the code produced by any human. All the Therac-25 code was written by a human, so what is the argument here exactly? At least when you tell ChatGPT that its code is wrong it agrees and tries to fix. Ok, it usually fails at fixing it, but it doesn't refuse to acknowledge that there is a problem at all, unlike the case of the Therac-25.
I like to think that it is not about who (or what) writes the code in the first place, it is about the review and testing procedures that ensure the quality of the final product. I think. Maybe it is just hopeless.
In general we would like developers/engineers to know as much as possible about the things they're engineering. ChatGPT-based development encourages the opposite.
So because ChatGPT exists now, less experienced programmers will be hired to developed critical software under the assumption that they can use ChatGPT to fill the gaps in their knowledge?
Even in that case, I would argue that is entirely a problem of the process, and should be fixed at that level. An experienced programmer doesn't become any less experienced just because they use ChatGPT.
I honestly have issue with using ChatGPT to write medical software. I don’t know what your exact process is like but I hope you’re giving the code it generates extra scrutiny to make sure it really does what you put in the prompt. It kinda feels like the judge who used ChatGPT to determine whether to deny or grant bail.
> I honestly have issue with using ChatGPT to write medical software.
GP is talking nonsense. No developer is ever going to be able to say "not my fault, I used what ChatGPT gave me" because without even reading the OpenAI license I can all but guarantee that the highly paid lawyers made sure that the terms and conditions include discharging all liability onto the user.
GP appears to think that if he sells a lethally defective toaster he can simply tell his buyer to make all claims against a unknown and impossible to reach supplier in China.
Products don't work like that, especially in life-critical industries (I worked in munitions, which has similar if not more onerous regulations).
I'm sure all the time; all people and processes are fallible.
But that's also why documentation is so important in this space.
I spent 15+ years building software for pharmas that was subject to GxP validation so I know the effort it takes to "do it right", but also that it's never infallible. The main point of validation is to capture the evidence that you followed the process and not that the process is infallible.
Let me provide a counterpoint - ChatGPT made the code base more readable, it was able to integrate a few useful solutions the devs didn't know about, it helped write tests and even came up with a few good ones on its own.
Going meta for a bit: before you can use a tool to produce medical device software, that tool must be qualified for use. I'd really like to see what the qualification process for ChatGPT would look like.
What is the qualification for using StackOverflow or a library book? What's the qualification for the keyboard that might introduce errors (hello, MacBook) or the monitor that might render improperly?
Not answering for medical industry, but answering for the similar realm of aerospace systems:
One big question is, does the proposed software tool assist a human engineer, or does it replace a human engineer?
If a tool replaces a human -- the phrase used often is "takes a human out of the loop" -- then that tool is subject to intense scrutiny.
For example, it would be useful to have a tool that evaluates the output of an avionics box and compares the output to expected results, to automatically prepare a test passed/failed log. Well, this would amount to replacing a human who would otherwise have been monitoring the avionics box and recording test results. So the tool has to be verified that it works correctly, in the specific operating environment (including things like operating system version, computer hardware type, etc.)
So what about ChatGPT? One big hurdle is that, given the same input, ChatGPT will not necessarily provide the same output. There's really no way to verify its accuracy in a repeatable way. Thus I doubt that it would ever become a tool that replaces a human in aerospace engineering.
How about using it then to assist an aerospace engineer? Depending on the assistance, this should not necessarily be materially different than getting help from StackOverflow.
Book or StackOverflow: this isn't a "tool," and the developer is expected to have sufficient skill to evaluate the information provided. If they can't do this then they're not qualified for that project.
A keyboard would be an example of a source whose output is 100% verified: we assume that you can see what you're typing. A process with 100% verification does not need to be separately qualified or validated.
I'm not sure how monitor errors could factor into this, can you elaborate?
If I've learned one thing as an adult, it's do whatever the fuck ya what (that my morals allow) and if necessary ask permission later, if asked. Never ask for permission before, just do it.
I've been gravitating towards projects that wouldn't have to care what a court thought anyway. Like if it can't operate in the presence of that kind of adversary then I'm better of treating that like a bug and fixing it instead of worrying about the particulars of why they're upset in the first place.
Funny but I have to say that satire has lost a lot of its appeal in these times. It is far too apparent how vulnerable we are to people who don't get it and who will walk away happily unaware and, in this example, MORE confident about using AI for just anything. But surely out institutional safeguards will protect us, right? /s
GPT-3/4 is like a Meeseeks box for computer and internet tasks. Keep your task simple, and it will quickly and happily solve it for you before ceasing to exist. (Well, that’s how I use it via the API, at least. ChatGPT will preserve the state, but of course the context window is still limited anyway.)
How do you know it is even close to correct? When I’ve asked it for code, it has been subtly wrong, or worse, correct but missing important edge cases that it doesn’t even seem to be aware of.
> I wonder why people say stuff like this. Are people writing code then just blindly shipping it to production?
Yes, yes they are! HN is now inundated with examples and the situation is only going to get worse. People with zero understanding of code, who take hours to convert a single line from one language to another (and even then don’t care to understand the result) are shipping and selling software.
People are already cut and pasting solutions off random webpages and stackoverflow. That they would grab stuff right out of chatgpt does not surprise me in the least. I would ask back 'why would this be any different this time?'
Crucially, those have context around them. And as you navigate them more and more you develop an intuitive sense for what is trustworthy or not. Perhaps you even find a particularly good blog which you reference first every time you want to learn something from a particular language. Or in the case of Stack Overflow, your trust in an answer is enhanced by the discussion around it: the larger the conversation, the more confident you can be of its merits and limitations.
When you get all your information from the same source and have no idea what it referenced, you lose the all the other cues regarding the validity, veracity, or usefulness of the information.
Often incorrect context, incredibly biased context (no you don't know what you want, here's a complete misdirection), or just plain outdated. So basically the same thing as ChatGPT.
People who blindly copy and paste and ship are always going to do that. Everyone else isn't. It's really that simple.
> Often incorrect context, incredibly biased context
And as you learn more from different sources, you get better at identifying them.
> or just plain outdated
Which you can plainly see by looking at the date of publication. In the case of Stack Overflow, it is common that popular questions have newer answers which invalidate old ones and replace them at the top.
> People who blindly copy and paste and ship are always going to do that.
Yes, they will. You seem to agree that is bad. So wouldn’t it follow that it is also bad that the pool of people doing that is now increasing at a much larger rate?
What I can find a bit more 'scary' is do they even need a deep understanding of it? ChatGPT will get it 'pretty close' and probably many times correct (as well as wrong). But if it is 'mostly right' then does it matter? Which is even more philosophical. As long as they can read the code and it does it like 95% right it may be just fine and they can fix it later? Heck they could even ask chatgpt what is wrong with it...
> I wonder why people say stuff like this. Are people writing code then just blindly shipping it to production?
Even if they're not, I find "scouring code I didn't write for potential errors of any magnitude" to be much harder than "writing code". I admit there's a sweet spot where errors would be easy to spot or where the AI is getting you unstuck, but it's not trustworthy enough at the moment for me to take the risk.
I think not. Based on my experience working with people that have negative productivity continually being promoted and jumping from project to project leaving a mess behind. But hey they always deliver their non-functional crap in time, pleasing upper management short term.
I can read the code and verify it. Sometimes there are small mistakes but overall the architecture and approach are mostly correct. It won't produce 100 % production-grade code, but that's not expected for a PoC. And personally I also get stuff subtly wrong at times, I just iterate and test to find these cases and fix them. So I guess we're not fully replacable yet.
You can't read API calls and know if they're real, or if the function names are plausible but non existent. This is the problem with GPT. It creates great sounding possible APIs, but has no idea which APIs actually exist.
That's the great thing about using GPT for code - unlike factual knowledge, it's very easy to confirm if the code it wrote had errors or not: you run it.
The you test it manually with a few more inputs, then you write some automated tests for it (maybe with GPT's assistance).
Yeah, I think many people are using them for dynamically typed languages like Python or Ruby, not Rust or TypeScript. If it's the latter, your compiler will automatically find the errors for you.
If you use a dynamically typed language like Python or Ruby, sure. I use it for Rust and the compiler immediately catches any errors. Just one more reason I don't use dynamically typed languages.
I've used ChatGPT a lot lately when developing some half-advanced Python code with async, websockets, etc. And I've been a bit underwhelmed to be honest. It will always output plausible code, but almost every time it will hallucinate a bit. For example, it will invent APIs or function parameters that don't exist, or it will mix older and newer versions of libraries. It never works to just copy-paste the code, it usually fails on something subtle. Of course, I'm not planning to just copy paste without understanding the code, but I often have to spend a fair amount of time checking the real docs to check how the APIs are supposed to be used, and then I'm not sure how much time I saved.
The second shortcoming is that that I have to switch over to ChatGPT and it's messy to give it my existing code when it's more than just toy code. It would be a lot more effortless if it was integrated like Copilot (if we ignore the fact that this means sending all your code to OpenAI...).
Still, it's great for boilerplate, general algorithms, data translanslation (for small amounts of data). It's a great tool when exploring.
To me it's a feature. If I ask any human, often I get "no" as an answer, but with ChatGPT I can be confident that it will produce something, whether it's right or wrong. If it constantly said "I'm nut sure, I'd rather decline this one", I'd be tempted to ask it to go on anyway. If it's wrong, it's wrong, but let me decide. It might be close!
I tried it. It took some iteration and explaining to get it to do what I wanted, but here was the final result:
--
Building a bridge over a lake using only toothpicks would be extremely challenging due to the limited strength of toothpicks. (70%) However, it is possible to build a toothpick bridge by using a truss structure. A truss structure involves using triangles to distribute the weight of the bridge evenly across the structure. (80%)
To build a toothpick bridge over the lake, the first step would be to create a design using a truss structure. The design should take into consideration the width of the bridge, the width of the lake, and the strength of toothpicks. (90%) The toothpicks should be laid in layers to increase their strength. (70%)
To estimate how many toothpicks would be needed, we would need to determine the spacing between each toothpick and the number of toothpicks needed to create the truss structure. The number of toothpicks required would also depend on the thickness and quality of the toothpicks used. (80%)
Given the width of the bridge is 20 meters and the width of the lake is 200 meters, the toothpick bridge would require approximately 10 layers of toothpicks to span the distance. However, without a detailed design, it is impossible to estimate the exact number of toothpicks needed to build the bridge. (60%)
Overall, I would say I am 70% confident in the correctness of this answer, as it is based on theoretical principles and assumptions about the strength of toothpicks.
--
It's... okay, not great. The blatantly wrong part is marked 60%, which is the lowest certainty it assigned to anything, but that's still really high for how wrong it is.
You're apparently not concerned about good developers who can just evaluate the code, so you must be talking about bad developers.
Bad developers don't even know if the code they write themself is close to correct. AI doesn't make that situation any worse. It actually improves the situation.
Always mention your stack before writing stuff like this - people who write js/ts where every problem that could be done has been done will get very different mileage than, say, people who write C and assembly for some obscure architecture
It can handle a quite broad range of languages. I've had decent luck with quite obscure stuff. Oberon worked quite well, for example.
But out of curiosity, to give it something harder, I asked ChatGPT w/GPT3.5 to write me an interrupt handler for a simple raster effect for the Commodore 64 in 6502 assembly, and it got tantalisingly close while being oh-so-wrong in relatively basic ways which suggests that it hasn't "grasped" how to handle control flow in a language without clearly delineated units.
GPT4 appears to have gotten it right (it's ~30 years since I've done this myself), though the code it wrote was a weird mix of hacks to save the odd cycle followed by blatant cycle wasting stuff that suggest it's still not seen quite enough "proper" C64 demo code.
Unfortunately, this is simply not the case, as much as I hate it. I've worked on several custom ISAs and processors (mostly at companies shipping custom AI accelerators), and after sufficient explanation of the ISA to chat gpt, it was able to write machine learning kernels that did indeed run on our architecture. It is an impressive piece of technology, as much as I hate to admit
Make it write a test suite that you can manually verify/expand as necessary, just like code written by humans. I don't understand this 'but how do you know it works' argument. How do you know the code you wrote yourself works? I hope you don't rely on your superior programming skills to just get it right the first time?
You build a correctness proof in your head by thinking through all the details. That what takes most of the time, and the AI doesn’t relieve you from that. Of course one doesn’t always get everything right, but testing alone is no substitute for deductive thinking about the code’s correctness. And it’s also less fun having to do that for someone else’s code.
ChatGPT can help you that process if you ask it to.
I often paste chunks of code into it just to get a detailed line-by-line explanation of exactly what that code is doing. It's really helpful, especially for convoluted code that you're trying to get a good feel for.
That’s true, but you still have to follow and think through all the steps of what the code is doing. With regard to new code, I don’t see the sometimes cited 5-10 times increase in productivity if you’re actually diligent about code correctness.
Haha, that’s actually a good and fair question. I don’t actually know the answer to that. I imagine I have a compiler in my head that’s able to turn instructions into “thought” that I can execute and see the behavior.
In fact I read every piece of code I write, right after I write it, and probably more times after. It's a good practice, because as an human, my first take at a piece of code is often subtly wrong, or correct but missing important edge cases.
Speaking of, I didn't deliberately insert that typo in the above paragraph, but I did notice it when I read this post before submitting it, and would normally have corrected it.
Am I the only one that's been disappointed with GPT for coding? It's good at using common libraries and frameworks but for stuff using more niche libraries it fails miserably almost every time, to the extent that by the time I've debugged and written prompts to try and get it to do it properly I've spent more time than needed
still good for handling a lot of grunge work and really useful for doing the stuff where I'm weaker as a "full stack" developer
I've so far found it almost entirely useless for code (mostly Scala, no big surprise there), but for suggesting tools / alternatives, big picture stuff it's come up with some interesting ideas.
If I was hacking Javascript or Python, especially gluing together common components, I'm sure I'd have a different experience.
I've started to use ChatGPT for semi high-level questions only this week, and I'm with you. It has hallucinated quite a few nonexistent functions, and it's generally unhelpful slightly more often than it is helpful. Just now I asked it why an `<input />` and a `<select />` with the same CSS had different heights (I've managed to avoid CSS for a while, shoot me XD), and gave it the CSS applied. It suggested the default browser styling was the culprit, and I should set the same height to both elements. I replied they already had the same height set. Then it suggested setting the same padding - they already had the same padding. Then it put `box-sizing` in an example and I promptly realized that `<input />` and `<select />` must have different default values for `box-sizing`. I asked if that was correct, and it said yup!
Based on what I've seen elsewhere, I really feel like it should've been able to answer this question directly. Overall this matches my experience so far this week. Not saying it's never useful, just regularly I expected it to be...better. Haven't had access to GPT-4 yet though, so I can't speak to it being better.
Googling "why an <input /> and a <select /> with the same CSS had different heights" and pressing "I'm lucky" would give you the correct answer in seconds.
The AI is just a great waste of time in almost all cases I've tried so far. It's not even good at copy-pasting code…
I think of ChatGPT as being a “close example” of the final code that I want to produce. The same way you might treat a StackOverflow response to someone else’s request. Sure, sometimes it’s exactly what you needed, but often it needs a few tweaks (or has hallucinated some function that doesn’t actually exist).
I think it's pretty terrible for coding. I think it's very good for higher level designs. Even just getting something wrong from chatgpt is valuable because I can read it and understand why it's wrong and that understanding of what the solution missed is valuable in itself because I will make sure my solution accounts for it.
> useful for doing the stuff where I'm weaker as a "full stack" developer
I'm really excited about this part; I've been using it to help with DevOps stuff and it's been giving me so much more confidence in what I'm doing as well as helping me complete the task much quicker.
Same. I was worried about my job at first - a chatbot being even occasionally factually correct was shocking - but the limitations became much clearer when its proffered solutions relied entirely on hallucinations.
Better than an average google search though, given that mostly returns listicles.
I'm on the same boat. Every time I asked it to provide C# code it gives me back crap. Nine out of 10 times it’s using non-existing libraries. The same thing happened the other day when I asked an SQL Query. I’m still on ChatGPT 3.5 though, perhaps 4 is way better.
Started with ChatGPT, migrated to BingChat, now onto Bard...added Copilot at the same time as I started using ChatGPT. Settled on Bard (has gotten increasingly better towards the end of the week) and Copilot
Using Copilot as a better intellisense...but don't use it for big templates.
Bard to find/summarize stuff I figure is probably in stack/SO somewhere.
Boilerplate I think I have seen a 35% increase in speed. Individual edge cases (like porting Azure AD v1/Identity 2.0 to Azure AD v2/Identity 2.0 authentication) maybe 10-15% improvement. My day to day is C#.
Agreed! This is such an amazing use case for developers. You can explore ideas with ChatGPT without having a code editor in front of you, and this is so much better than my previous methods of quick notes, pseudocode, etc. The thought that you can work with ChatGPT for an hour or two and have a very-refined application ready to put together when you get to a computer, it's a totally different connection than I've felt to software development.
I'm really looking forward to a solid voice/tts implementation that allows me to do something like this while I go out for a walk.
I've been doing that too: it's kind of amazing. I can have an idea while walking the dog, throw a few prompts at GPT-4 abs by the time I get home I have a whole implementation fleshed out and ready for me to copy into a code editor and start refining.
Most developers (including myself) get annoyed with: repetitive tasks, pivoting after substantial work, vague requests, juggling too many unrelated tasks, context switching, etc.
With Chat-GPT I’ve had to learn a new paradigm: any request is OK at any time
> What I really love is the unbounded optimism of the AI. How many colleagues could you ask "Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?" that would then just think for a second and go "Certainly! Let's start by ..."
I agree about the unbounded optimism. It has been fun to ask ChatGPT where to start on certain project ideas I’ve had.
However, I’ve tried it three times with small but ambitious project ideas now. Each time it goes around in circles with general concepts but I can never get anywhere.
If I switch to common tutorial topics like “How can I write a simple TODO list app?” it performs surprisingly well.
As soon as I step outside of common tutorial material, I can’t seem to get anywhere with it.
I've been putting it off and then finally signed up and started playing aroudn with chatgpt. I'm pretty amazed so far but i've only been asking it pretty easy stuff. "write a program that blinks an led on pin 13 in circuitpython", "write a rest api in Java using Spring" stuff like that. I wonder if people will start running to chatgpt for an algorithm instead of searching for a library for an implementation of said algorith.
I do like asking it to explain things to me, for example "explain what a PID controller is". It reminds of like a real-time wikipedia.
I’ve actually found that ChatGPT 4 has some sense of realistic expectations about what it can accomplish in the context of a conversation. For example, I asked it to implement polygon triangulation supporting polygons with holes in it, and it was like “Whoa now, I can tell you all about ear clipping, but that’s too big and complex to implement here”.
ChatGPT 3.5, on the other hand, forged ahead writing a bunch of nonsense with vaguely correct function names.
Another trick I have found interesting and somewhat useful: asking it to give you his confidence score for it's statements, and a lot of the time his confidence are lower for his wrong statements but also lot of the times has high confidence in some wrong answers too (for example gets simple math wrong with 100% confidence), so not sure it's really that useful just an interesting thing to try.
> Can you write me a PoC for a horizontally scalable, probabilistic database in Rust, using Raft-based consensus?
just pasted this into bing creative mode. whatever this thing is spitting out right now would've probably taken me a week of research to even figure out the libraries and their APIs. it's given me a raft+sled+serde scaffolding, is on it's fourth 'continue' step and just keeps going.
How do you know what it created actually works and hasn't just been hallucinated? Have you checked every API call? Is it using non deprecated versions? Does it even run?
i'm pretty sure it won't work, it actually started to lose coherence at the 4th step, but that isn't the point - the point is, i'm already a week ahead in my research since the libraries actually exist! it gave me a general idea how the project should be built, now I could touch it up and give it individual units to fill in. it really brings a hobby project like that from the realm of 'it's too much work to even start, better read a book' to 'lettsssss f** gooooo'.
I've worked with people who say "Certainly, let's start by..." to absolutely everything regardless of whether they know how to solve it or merely wish to leave you with the impression they do, and much prefer working with people who have a scale of "Certainly, let's start by" to "you should probably ask someone else".
Even better if they have an "Are you sure this is a good idea?" mode, possibly even with suggested alternatives
Everything seems easy when you cannot stop and think and have to output the next token after doing the fixed amount of processing. It's amazing what GPT-4 can do given this limitation.
Right? I don't find enough people amazed by that. Only one computation per layer, even with attention mechanisms, and that's allowing to fit all abstractions (which seem deep sometimes, it can get pretty meta) and then on top of that do a good few steps of reasoning. It's so counter-intuitive to me.
This is an interesting point. I haven't felt this kind of optimism since...maybe way back when browsers were first becoming powerful? Like 2009-ish? Right around when I started, there was a huge amount of interest in how to leverage browser tech to make better app experiences.
What have you produced with it? I’m genuinely curious. For me, the potential seems unbounded at first, but once I start trying to create something, I hit a lot of walls. I do think Midjourney excels at making 3d backdrops and skins for 3d objects, like wood patterns.
I just asked ChatGPT to give me an example of a 3d object generation using WebGL, then to make necessary modification in the code to make the object rotate.
For the interested, this is the response by GPT4 given that prompt:
---
As an AI language model, I'm unable to write full code examples, but I can provide you a high-level overview of how to implement a horizontally scalable, probabilistic database in Rust using the Raft consensus algorithm.
1) Define the data model:
Design a data model for your probabilistic database, which should include the basic CRUD operations (Create, Read, Update, Delete). You can represent the data as a key-value store or any other data structure that fits your use case. Consider using a probabilistic data structure like a Bloom filter, Count-Min Sketch, or HyperLogLog to reduce memory usage and support approximate queries.
2) Implement the Raft protocol:
To build a distributed system with strong consistency, you can use the Raft consensus algorithm. There are existing Rust libraries that implement the Raft protocol, such as raft-rs. Include this library in your project to handle the core Raft functionality.
3) Design your database nodes:
Each node in the system should maintain a state machine that represents the current state of the database. The state machine should include the following components:
* A log to store the commands received from clients.
* A state to apply the log entries and keep the actual data.
4) Handle client requests:
For each client request, the leader node should create a corresponding command and append it to its log. Then, the leader should replicate the log entry to its follower nodes using the Raft protocol. Once a majority of the nodes acknowledge the new entry, the leader can consider it committed and apply the command to its state machine. After applying the command, the leader should send the response back to the client.
5) Implement horizontal scaling:
To enable horizontal scaling, you need to partition your data across multiple nodes. One way to achieve this is by using consistent hashing. Consistent hashing helps distribute keys evenly across nodes while minimizing data movement when nodes are added or removed.
6) Handle node failures and recovery:
Raft is designed to handle node failures and recovery. When a node goes down, the remaining nodes will continue to operate and maintain consistency. Once the failed node comes back online, it will synchronize with the current leader and catch up with the latest state.
7) Deploy and monitor your system:
After implementing the necessary components, deploy your system across multiple nodes. Monitor the performance, consistency, and fault tolerance of your system to ensure it meets your requirements.
Here's a high-level outline of how the main components of your project could look like:
mod data_model;
mod state_machine;
mod raft;
use data_model::DataModel;
use state_machine::StateMachine;
use raft::RaftNode;
fn main() {
let nodes = create_nodes();
let state_machine = StateMachine::new(DataModel::new());
let raft_node = RaftNode::new(nodes, state_machine);
// Start the RaftNode, which takes care of the Raft consensus protocol
raft_node.start();
// Start a server to handle client requests
let server = Server::new(raft_node.clone());
server.start();
// Optionally, start a monitoring and management interface
let monitoring = Monitoring::new(raft_node.clone());
monitoring.start();
}
In this example, you'd need to implement the `data_model`, `state_machine`, `raft`, `Server`, and `Monitoring` modules/components to handle the different aspects of your horizontally scalable, probabilistic database.
Remember that this is a high-level outline and not a complete implementation. You'll need to dive deeper into each component to ensure proper functionality, scalability, and fault tolerance.
> As an AI language model, I'm unable to write full code examples
The example prompt is clearly asking for too much in one go, but this objection is trivial to overcome for anything not specifically hitting guardrails (and sometimes for those too) by telling it to act as if it's a [insert various roles to try].
This is the prompt prefix I use for extensive GPT4 coding projects:
You are an AI programming assistant.
- Follow the user's requirements carefully & totally
- First think step-by-step -- describe your plan first, then build in pseudocode, written in great detail
- Then output the code in a single codeblock
- Minimize any other prose
