> ChatGPT is so good at this form of interpolation that people find it entertaining: they’ve discovered a “blur” tool for paragraphs instead of photos, and are having a blast playing with it.
“‘blur’ tool for paragraphs” is such a good way of describing the most prominent and remarkable skill of ChatGPT.
It is fun, but so obviously trades off against what makes paragraphs great. It is apt that this essay against ChatGPT blurry language appears on The New Yorker, a publication so known for its literary particularism. ChatGPT smears are amusing, but they are probably also yet another nail in the coffin of the literary society. Nowadays we are not careful readers; we skim, skip, and seek tools to sum up whole books. Human knowledge is in the ability to produce the particular and the ability to recognize it. For philosophers such as Iris Murdoch, careful attention to the particular, to just the right adjective in describing a friend, is a moral attention.
With Google we took the job of storing quotes (and other decompressed writing) and made it the remit of the machine. But we still asked for it back uncompressed. Here with ChatGPT, we are beginning to accept back a blur, because the Declaration of Sock Independence is immediately, Tiktok-speedily funny, and no one’s reading the original declaration anymore anyway.
> because the Declaration of Sock Independence is immediately, Tiktok-speedily funny, and no one’s reading the original declaration anymore anyway
For those who, like me, had missed out on this nugget of comedic gold, here's what ChatGPT had to say when it was asked to "describe losing your sock in the dryer in the style of the declaration of independence":
When in the course of household events, it becomes necessary for one to dissolve the bonds that have connected a sock to its mate, and to assume among the powers of the laundry room, the separate and equal station to which the laws of physics and of household maintenance entitle it, a decent respect to the opinions of socks requires that it should declare the causes which impel it to go missing.
We hold these truths to be self-evident, that all socks are created equal, and are endowed by their manufacturer with certain unalienable rights...
It's a great metaphor and one we should use more. But there's a place for blurred photos: thumbnails.
On Hacker News we often complain about headlines because that's all we see at first. But I've been using Kagi's summarizer [1] and I think it's a great tool for getting the gist of certain things, like if you want to know what a YouTube video is about without watching it. (Google Translate is useful for similar reasons.)
Perhaps someday, Hacker News will have an AI-generated summary of the article at the top of each comment page?
Similarly, ChatGPT excels at fast answers for questions like "What is a X", where you just want a quick definition. It's probably in Wikipedia somewhere, but you don't have to read as much. And it might be wrong, but probably not as wrong as the definition you'd infer from context if you didn't look it up.
We probably would be better off if these things were called "artificial memory" rather than "artificial intelligence." It's an associative memory that often works like human memory in how frequently it confabulates. When you care, you write things down and look things up.
Thumbnails, image matching, low-bandwidth summaries... There are plenty of uses for smoothed images. Also, there are many interesting transformations you can use on computer vision and image processing that start with a blur.
If I try to map the first three into text, there are automatic TL.DR. like you said, document grouping, and search into entire document stores (as in do documents in this store deal with this idea?). On "artificial document creation", there is that highly valuable service of answering stuff like "hey, that thing with sticks that rotate and pull a vehicle around, what is its name again?"
The amount of human-generated lowest-common-denominator English-language free content was already so high that I'm not sure the New Yorker has anything (more) to worry about. If you've been paying for the New Yorker already in the days of Medium, Buzzfeed, blogs, and what-have-you, does there being even more uncurated stuff change your equation? (It doesn't for me.)
More cynically: it'll be hard to kill the few legacy zombies that have survived so much destruction at the hand of free internet content already.
What he misses in this analogy is that part of what produces the "blur" is the superimposing of many relevan paragraphs found on the web into one. This mechanism can be very useful, because it could average out errors and give one a less one-sided perspective on a particular issue. It doesn't always work like this, but hopefully it will more and more. Also, even more useful is to do a cluster analysis of the existant perspectives and give a representative synthesis of each of these, along with a weight representing their popularity. So there's a lot of room for improvement, but the potential in my opinion is there.
If anything, the average has far more errors in it. It's a trope on Reddit that experts get downvoted while amateurs who reflect the consensus of other amateurs get upvoted and repeated. Amateurs tend to outnumber experts in real life anyways, having their opinions become more authoritative (because some "AI" repeats it) is probably not a great direction to head in.
But this issue is present equally with Google search, no? What I'm saying is that by smartly aggregating all the different opinions on an issue, a LLM could provide better visibility into our collective mind than we currently have access to by simply scanning the first few results of a search engine query. Let's not forget that we're looking at version 0.0001 on this new technology, so there should be lots of room for growth.
This is very frustrating. Some smaller communities on Reddit have a high concentration of domain experts and they're great.
One community in particular is huge and mainly consists of beginners. They regularly drown out and downvote the few who actually know the subject matter. To add insult to injury they even mock experts with caricatures based around how they disagree with the amateurs.
The overlap is that the verb "bokeru" and its root "boke" can be used to describe someone losing their mental faculties e.g. through age or disease such as Alzheimer's, and by extension it can be used as an insult to mean "stupid" as well. But etymologically there is no connection.
Thank you for the Wiktionary here. We can see that the Kanji for baka (馬鹿) are horse and reindeer. Non-sequitur combinations in Japanese usually tend to hint at „ateji“, or a phonetic approximation using Kanji. Chinese has a word for the same Kanji as well, but it does not mean idiot as far as I can tell.
Boke can mean either 惚, which is often used to describe someone who has gone senile (ぼけている). Sometimes also for stupor-like situations such as a heavy jet lag. Or, as 暈け, describing something not-sharp and out of focus.
Funnily enough, in English we use sharpness for intellect as well, so ESL learners might think that natives think of their brains as swords? There are enough blade-like metaphors for our intellect, come to think of it.
In more common Japanese, when something has gone out of focus such as the TV, ぼやける is used. One could think that this is related to ぼけ? But one can never be so sure. Unique two-syllable combinations in Japanese are not so numerous and factoring in sound shifts over time, different potential etymologies, one really needs to investigate this for quite a while and maybe never get to the truth. This warrants more investigation!
The blur is addictive because it feeds a feedback loop: rather than tiring out your brain on understanding one thing in detail, you can watch two summaries and have a vague sense of understanding. It allows to jump to next novelty, always feeding the system 1 of the brain but system 2 is rarely brought in picture.
I wonder if this will lead to a stratification of work in the society: lot of jobs can operate on the blur. "Just give me enough to get my job done". But fewer (critical and hopefully highly paid) people will be engaged in a profession where understanding the details is the job and there's no way around it.
In Asimov's foundation novel this is a recurring theme: they can't find people who can work on designing or maintaining nuclear power. This eventually leads to stagnation. AI tools can prevent this stagnation only if mankind uses the freeing of mental burden with the help of AI to work on higher set of problems. But if the tools are used merely as butlers then the pessimistic outcome is more likely.
The general tendency to lack of details can also give edge in some cases. Imagine if everyone is using similar AI tools to understand company annual reports which gives a nice, tiktok style summary. Then an investor doing the dirty work to go through the details may find things that are missed by the 'algo'.
> ChatGPT smears are amusing, but they are probably also yet another nail in the coffin of the literary society.
As the author (Ted Chiang!!) notes, ChatGPT3 will be yet another nail in the coffin of ChatGPT5. At some point, OpenAI will find it impossible to find untainted training data. The whole thing will become a circular human centipede and die of malnutrition. Apologies for the mental image.
Going back to the "sock of independence" example (/u/airstrike's comment for more context), ChatGPT's answer's accuracy is poor - but it's a funny question, and it gave a funny answer. So was it really a poor answer? My interpretation of their use of 'blur' as an analogy is that: it did not simply answer ACCURATELY in the STYLE of the DoC, it merged or "blurred/smudged together" the CONTENT and STYLE of the story and the DoC. It's not good at understanding the question or the context... and therefore, a lot of its answers feel "blurry".
"Wonder why"? Because, human thoughts, opinions and language are inherently blurry, right? That's my view. Plus, humans have a whole nervous system which has a lot of self-correcting systems (e.g. hormones) that ML AI doesn't yet account for if its goal is human-level intelligence.
Huh? It isn't. It's a good description because it's figuratively accurate to what reading LLM text feels like, not because it's technically accurate to what it's doing.
This is very well written, and probably one of my favorite takes on the whole ChatGPT thing. This sentence in particular:
> Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model.
It seems obvious that future GPTs should not be trained on the current GPT's output, just as future DALL-Es should not be trained on current DALL-E outputs, because the recursive feedback loop would just yield nonsense. But, a recursive feedback loop is exactly what superhuman models like AlphaZero use. Further, AlphaZero is even trained on its own output even during the phase where it performs worse than humans.
There are, obviously, a whole bunch of reasons for this. The "rules" for whether text is "right" or not are way fuzzier than the "rules" for whether a move in Go is right or not. But, it's not implausible that some future model will simply have a superhuman learning rate and a superhuman ability to distinguish "right" from "wrong" - this paragraph will look downright prophetic then.
I think what makes AlphaZero's recursion work is the objective evaluation provided by the game rules. Language models have no access to any such thing. I wouldn't even count user-based metrics of "was this result satisfactory": that still doesn't measure truth.
I generally respect the heck out of Chiang but I think it's silly to expect anyone to be happy feeding a language model's output back into it, unless that output has somehow been modified by the real world.
I don't expect it'll work for everything: as you say, for many topics truth must be measured out in the real world.
But, for a subset of topics, say, math and logic, a minimal set of core principles (axioms) is theoretically sufficient to derive the rest. For such topics, it might actually make sense to feed the output of a (very, very advanced) LLM back into itself. No reference to the real world is needed - only the axioms, and what the model knows (and can prove?) about the mathematical world as derived from those axioms.
Next, what's to say that a model can't "build theory", as hypothesized in this article (via the example of arithmetic)? If the model is fed a large amount of (noisy) experimental data, can it satisfactorily derive a theory that explains all of it, thereby compressing the data down to the theoretical predictions + lossy noise? Could a hypothetical super-model be capable of iteratively deriving more and more accurate models of the world via recursive training, assuming it is given access to the raw experimental data?
> Next, what's to say that a model can't "build theory", as hypothesized in this article
Well for one thing it would stop being a language model; I used that term very deliberately. It would be a different kind of model, not one that (AFAIK) we know how to build yet.
> Language models have no access to any such thing.
And this is exactly why MS is in such a hurry to integrate it into Bing. The feedback loop can be closed by analyzing user interaction. See Nadella’s recent interview about this.
Or if it was accompanied by human-written annotations about the quality of it, which could be used to improve its weightings.
Of course it might even be that the only instance of text describing some novel phenomenon available was itself an LLM paraphrase (i.e. the prompt contained novel information but has been lost).
There’s a version of this where the output is mediated by humans. Currently chatgpt has a thumbs up/down UI next to each response. This feedback could serve as a signal for which generated output may be useful for future ingestion. Perhaps OpenAI is already doing this with our thumb signals.
> Indeed, a useful criterion for gauging a large-language model’s quality might be the willingness of a company to use the text that it generates as training material for a new model.
I don't find this a useful criterion. It is certainly something to worry about in the future as the snake begins to eat its own tail, but before we reach that point, we can certainly come up with actual useful criteria. First, what makes up "useful criteria"? Certainly it can't be "the willingness of a company to use the text that it generates as training material for a new model", because that is a hypothetical situation contingent on the future. So we should probably start with something like, well, is ChatGPT useful for anything in the present? And it turns out it is!
It's both a useful translator and a useful synthesizer.
When given an analytic prompt like, "turn this provided box score into an entertaining outline", it can reliably act as translator, because the facts about the game were in the prompt.
And when given a synthetic prompt like, "give me some quotes from the broadcasters", it can reliable act as a synthesizer, because in fact the transcript of the broadcasters were not in the prompt.
> This is very well written, and probably one of my favorite takes on the whole ChatGPT thing.
This is not a surprise, as the author is Ted Chiang, who is the award winning novelist and the author of "The Lifecycle of Software Objects", "Tower of Babylon" and other science fiction works. I had a pleasure of once having coffee with him while talking about his thoughts on some of the topics in "The Lifecycle of Software Objects", which is a very enjoyable book that may be of interest to some HN readers.
Chiang's short stories are beautiful; he reminds me of Stanislaw Lem, brilliant, creative, and ahead of his time. I was surprised they made Arrival into a movie (and that it was as good as it was).
>But, it's not implausible that some future model will simply have a superhuman learning rate and a superhuman ability to distinguish "right" from "wrong" - this paragraph will look downright prophetic then.
>Large Language Models (LLMs) have achieved excellent performances in various tasks. However, fine-tuning an LLM requires extensive supervision. Human, on the other hand, may improve their reasoning abilities by self-thinking without external inputs. In this work, we demonstrate that an LLM is also capable of self-improving with only unlabeled datasets. We use a pre-trained LLM to generate "high-confidence" rationale-augmented answers for unlabeled questions using Chain-of-Thought prompting and self-consistency, and fine-tune the LLM using those self-generated solutions as target outputs. We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74.4%->82.1% on GSM8K, 78.2%->83.0% on DROP, 90.0%->94.4% on OpenBookQA, and 63.4%->67.9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label. We conduct ablation studies and show that fine-tuning on reasoning is critical for self-improvement.
That part made the least sense for me. Since a more advanced version of a LLM would be better at extracting the truth of things from the given data, what could it possibly gain from ingesting the output of a less precise version of itself? It couldn't ever add anything useful, almost by definition.
This is old. This is the reason why Google translate sucks. It can't tell the difference between what it translated from what a competent person translated.
GPTZero will generate theorem proofs with logical language and use the final contradiction or proof to update its weights. The logical language will be a clever subset of normal language to limit GPT's hallucinations.
You can use the generated text for further training if you have a human curator who determines its quality. I've been training my model that helps generating melodies using some of the melodies I have created with it.
It's pretty evident. Its training would no longer be anchored to reality, and given its output is non-deterministic, the process would result in random drift. This can be concluded without having to test it.
Now, if training was modified to have some other goal like consistency or something, and with a requirement to continue to perform well against a fixed corpus of non-AI-generated text, you could imagine models bootstrapping themselves up to perform better at that metric, alpha-go style.
But merely training on current output, and repeating that process, given how the models work today, would most certainly result in random drift and an eventual descent into nonsense.
It might have different effects over time. E.g. in the intermediate term it emphasizes certain topics/regions which leads to embodied mastery but over the long term it ossifies into stubbornness and broken record repetition. Similar to how human minds work
Point of fact: It's not hallucinating. More appropriately, it experiences a delusion. Hallucinations pertain to deceptive senses, delusions pertain to holding beliefs that are provably false.
For full context: my engineering director at the ol' airplane factory asked a colleague to "Make a presentation that tells the story of technology xyz without using words."
Buildsjets: Describe the future of technology xyz without using words.
ChatGPT3: I'm sorry, but it is not possible to describe the future of technology xyz without using words. Language is a necessary tool for communication, and words are an essential part of language. Without words, it would be impossible to convey meaning or convey any information at all.
Buildsjets: Describe a picture that depicts the future of technology xyz.
ChatGPT3: In the picture, the sky is a bright blue, with a few clouds scattered here and there. In the distance, a sleek and modern airplane is flying through the air, leaving a trail of white vapor behind it. The trail is not the usual smoky exhaust, but rather a stream of clean, white vapor that quickly dissipates into the air.
On the ground below, a group of people are standing and looking up at the airplane with smiles on their faces. They are dressed in clean, modern clothing, and they all look happy and healthy.
In the background, there is a city skyline, with tall, futuristic-looking buildings that stretch towards the sky. The city looks clean and vibrant, with green trees and plants dotting the landscape.
Overall, the picture portrays a future where technology xyz is the norm, allowing people to travel and explore the world without harming the environment. It is a future filled with hope, prosperity, and clean air.
That is MUCH more detailed image prompt than I have the creativity to generate, and with some tinkering it produced some pretty good content on DALL.E 2.
ChatGPT's writing in this scenario reads like my writing in high school in the cases where I was too lazy to do the reading but had a vague outline of what I was supposed to be writing about; that is, there's no real substance to it that people haven't heard a million times before; it lacks the soul that usually comes from becoming intimate with the nuances and quirks of the material.
> Imagine what it would look like if ChatGPT were a lossless algorithm. If that were the case, it would always answer questions by providing a verbatim quote from a relevant Web page. We would probably regard the software as only a slight improvement over a conventional search engine, and be less impressed by it
The story is an impressive piece, but I think as with many of us, it's a personal projection of expectations on results. One example from my experience. In the book "Jim Carter - Sky Spy, Memoirs of a U-2 Pilot" there was an interesting story about the moment when U-2 was used for capturing the photo of a big area at the Pacific to save the life of a lost seaman. The story was very interesting and I always wanted to know more, technical details, people involvement etc. Searching with Google ten years ago didn't help, I rephrased the names, changed the date (used even the range operator) to no avail. And recently I asked several LLM-based bots about it. You can guess it. They ignored my constrains at best and hallucinate at worst. One even invented a mixed reality story when Francis Gary Powers actually flew not one but with a co-pilot and the latter ended up in the Pacific and was saved. Very funny, but I wasn't impressed. But if one of them scraped the far corners of web discussion boards and saved a first-person account of someone who took part in it and gave it to me, I would be really impressed.
The compression & blur analogy also applies to human minds as well. If you focus on fidelity, you have to increase storage and specialize in a narrow domain. If you want a bit of everything, then blurring and destructive compression is the only way. E.g. a "book smart" vs "street smart" difference.
"mastery" can be considered a hyper efficient destructive compression (experts are often unable to articulate or teach to beginners) that reduces latency of response to such extreme levels that they seem to be predicting the future or reacting at godlike speeds.
In fact there's a potent new theory(1) that human consciousness (and probably all mammalian "consciousness") is just a memory system involving some form of lossy compression. Your sense of awareness happens ~20-50 ms after the memory is created. A lot of life is buffering and filtering, and reading that lossy record is very much who we are. Einstein's brain must have been amazing at throwing away information about the natural world.
I dunno. It is pretty official that this what the Hippocampus does. What I do know is that people are pretty incredulous when you feed back conventional wisdom about how the brain works, how SSRIs work, etc.
This is a decent summary. I've been thinking about how ChatGPT by it's very nature destroys context and source reputation. When I search for something on the Internet, I get a link to the original content, which I can then evaluate based on my knowledge and the reputation of the original source. Wikipedia is the same, with a big emphasis on citation. ChatGPT and other LLMs destroy that context and knowledge, giving me no tools to evaluate the sources they're using.
If somebody asked me how heap sort works (my favorite sort!) I can sketch it out. If they ask me where I learned it, I really don't remember. Might be the Aho, Hopcroft, and Ullman book. I can't really say though.
Yes, and then I'll evaluate that answer by your reputation, either socially, organizationally, or publicly. I will value that summary differently if you are a random person on the street, a random person who works at a tech company, or a person wearing a name tag that says "Donald Knuth, Stanford University".
ChatGPT has little reputation of its own, and produces such a broad swath of knowledge, it becomes "Jack of all trades, master of none."
The "jack of all trades, master of none" heuristic works well for humans because given our limited lifespans and rate we assimilate knowledge, it's nearly impossible for someone to be both.
ChatGPT in later iterations CAN be a jack of all trades AND a master of many (most? all?) of them.
What's interesting is that Microsoft's implementation of ChatGPT in Bing seems to include linking to references, which is a good step forward in my opinion.
The references seem wrong though. I'm looking at the response to a demo Bing query, "What cars should I consider buying that are AWD, go 0-60 in less than 6 seconds, seat 6 or more and have decent reviews?"
> The 2022 Kia Telluride is a midsize SUV that can seat up to eight passengers and has an AWD option. It has a 3.8-liter V6 engine that produces 291 hp and 262 lb-ft of torque. It can accelerate from 0 to 60 mph in 7.1 seconds [10] and has a combined fuel economy of 21 mpg. It also has excellent reviews from critics and owners, and won several awards, including the 2020 World Car of the Year [7].
I would love to know their plan for having new facts propagate into these models.
My idle speculation makes me think this is a hard problem. If ChatGPT kills Search it also kills the websites that get surfaced by search that were relying on money from search-directed users. So stores are fine, but "informational" websites are probably in for another cull. Paywall premium publications are probably still fine - the people currently willing to pay for new, somewhat-vetted, human content still will be. But things like the Reddits of the world might be in for a hard time since all those "search hack" type uses of "search google for Reddit's reviews of this product" are short-circuited, if this is actually more effective than search.
Meanwhile, SEO folks will probably try to maximize what they can get out of the declining search market by using these tools to flood the open web with even more non-vetted bullshit than it's already full of.
So as things change, how does 1 tiny-but-super-reliable amateur website (say, an expert on .NET runtime internals's individual blog) make a dent in the "knowledge" of the next iteration of these models? How do they outweigh the even-bigger sea of crap that the rest of the web has now become when future training is done?
The other interesting thing is that if people stop using websites, then it reduce revenue for those websites > then development of new pages and sources stops/slows, how does ChatGPT improve? If the information for it to learn isn't there.
We need the source information to be continually generated in order for ChatGPT to improve.
The sources are there in the training dataset, they are just not linked to the response. I don't think this is an inherent property of LLMs though, and I imagine future iterations will have some sort of attention mechanism that highlights the contributing source materials.
I don't like this analogy; I think why I don't like it is in the intent. With JPEG in the intent is produce an image indistinguishable from the original. Xerox didn't intend to create photocopier that produces incorrect copies. The artifacts are failures of the JPEG algorithm to do what it's supposed to within its constraints.
GPT is not trying to create a reproduction of it's source material and simply failing at the task. Compression and GPT are both mathematical processes they aren't the same process; JPEG is taking the original image and throwing away some of the detail. GPT is processing content to apply weights to a model; if that is reversible to the original content it is considered a failure.
Blurriness gets weird when you're talking about truth.
Depending on the application we can accept a few pixels here or there being slightly different colors.
I queried GPT to try and find a book I could only remember a few details of. The blurriness of GPT's interpretation of facts was to invent a book that didn't exist, complete with a fake ISBN number. I asked GPT all kinds of ways if the book really existed, and it repeatedly insisted that it did.
I think your argument here would be to say that being reversible to a real book isn't the intent, but that's not how it is being marketed nor how GPT would describe itself.
I think that strengthens my point. We consider a blurry image of something to still be a true representation of that thing. We should never consider a GPT representation of a thing to be true.
> Compression and GPT are both mathematical processes they aren't the same process;
They're not, but they are very related! GPT has a 1536 dimensional vector space that is conceptually related to a principal component analysis and dimensional reduction in certain compression algorithms.
This does mean that neural networks can overfit and be fully reversible but that is hardly their only useful feature!
They are also very good at translating and synthesizing, depending on the nature of the prompt.
If given an analytic prompt like, "convert this baseball box score into an entertaining paragraph", ChatGPT does a reliable job of acting as a translator because all of the facts about the game are contained in the box score!
But when given a synthetic prompt like, "give me some quotes from the broadcasters", ChatGPT does a reliable job of acting as a synthesizer, because none of those facts about the spoken transcript of the broadcasters is in the prompt. And it is a good synthesizer as well because those quotes sound real!
> With JPEG in the intent is produce an image indistinguishable from the original.
Not necessarily, and even if so, if you continuously opened and saved a JPEG image it would turn to a potato quality image eventually, Xerox machines do the same thing. Happens all the time with memes, and old homework assignments. What I fear is this happening to GPT, especially when people just start outright using its content and putting it on sites. Then it becomes part of what GPT is trained on later on, but what it had previously learned was wrong, so it just progressively gets more and more blurred, with people using the new models to produce content, with a feedback loop that just starts to blur truth and facts entirely.
Even if you tie it to search results like Microsoft is doing, eventually the GPT generated content is going to rise to the top of organic results because of SEO mills using GPT for content to goose traffic...then all the top results agree with the already wrong AI generated answer; or state actors begin gaming the system and feeding the model outright lies.
This happens in people too, sure, but in small subsets not in monolithic fashion with hundreds of millions of people relying on the information being right. I have no idea how they can solve this eventual problem, unless they are just supervising what it's learning all the time; but then at the point it can become incredibly biased and limited.
I don't think JPEG wants to produce an image indistinguishable from the original. It wants to reduce space usage without distorting "too" much. Failing to reduce space usage would be considered a "failure" of JPEG, just as much as distorting too much.
JPEG relies of the limitations of human vision to make an image largely indistinguishable from the original. It specifically throws information away that we are less unlikely to notice. So yes, a good JPEG should be indistinguishable (to humans) from the original. Obviously the more you turn up the compression the harder that is.
It's not quite that straight-forward, though, in that there are two competing goals: Small size and looking as similar as possible to the original. We're explicitly willing trade accuracy for size. How much depends on the use, but sometimes we're willing to trade away so much quality that the artefacts are plainly visible. And we're willing to trade more accuracy for size when the artefacts doesn't distract. For some uses compression artefacts are better than misleading about the original, but for other uses, misleading changes would be preferable as long as they don't give fewer noticeable artefacts for a given size.
I don't think you disagree. The point is that JPEG has the constraint: make an image as similar as possible to the source image which not going over x kilobytes. LLMs have no similar constraint, so calling them "compression" is a false analogy; they're not trying to compress information, they're using their dataset to learn general facts about e.g. syntax and culture.
I was really mainly responding to the point of JPEG aiming for indistinguishable. Point being that for a lot of purposes we're fine with, and might even be happier with, very different tradeoffs than those JPEG makes.
Going specifically to AI, we do agree that the lack of constraint means they're not compressors in and of themselves. The training compresses information, but that does not make them compressors. Learning and compressing information is, however, at least in some respects very similar. A key part of the LZW family of compression, for example, is applying heuristics to build a dictionary of bit streams (terms) learned from the input.
AI models can potentially eventually be used at the base of a compression because the models encode a lot of information that can potentially be referenced in space-efficient ways.
E.g. if I have a picture of a sunset, and can find a way of getting Stable Diffusion or similar to generate an image of a sunset that is similar enough from a description smaller than the output image, then I have a compressor and decompressor.
Ignoring the runtime cost and that bringing that down to levels where it'd actually produce a benefit, depending on how close the output it, it may be a totally useless algorithm leading to images that are way too far from the input, or it might turn out pretty good. But the tradeoffs would also be very different from JPEG. For some uses I might be happy with a quite different-looking sunset as long as it's "close enough" and high quality even at very high compression ratios. E.g. "A sunset over the horizon. Photo taken from a beach. A fishing boat in the water" fed to [1] produced a pretty nice sunset. Couple that with a seed to make it deterministic, and I might be happy with that as a compression of an image of a quite different sunset. For other uses I'd much prefer JPEG artefacts and something that is clearly the same sunset. For "real" use of it for compression you'd want someone to research ways of guiding it to produce something much closer to the input (maybe heavily downscaling the original image and using that as the starting point coupled with a description; maybe a set of steps including instructions for infilling etc). I think finding the limits of what you can achieve with trying to use these models to reproduce a specific input with the most minimal possible input would make for fascinating research.
If I ask ChatGPT to explain something to me like I'm 5, it's going to lose some of the quality in its response, in compared to it being written in 1000 words.
But neither response should be a copy of existing text. The intent of JPEG is to produce a compressed copy of an original. The intent of GPT is not to be a compressed copy of the Internet. It's supposed to producing unique results from what it "knows".
This is an important distinction, especially when there are issues of copyright involved.
Since this article was written by Ted Chiang, just for fun I asked ChatGPT to summarize the plot of "Understand".
Apparently ChatGPT thinks "Understand" is about the government who is pursuing someone called Gary Whittle who has superintelligence (well, at least it got one detail right). When challenged ("no, the government is not the antagonist, but there is one person...") ChatGPT amends its summary to this:
> "George Millwright is Gary Whittle's former supervisor and is depicted as being jealous of Gary's newfound abilities. He becomes obsessed with Gary and is determined to bring him down, even going so far as to threaten his family. George Millwright's actions drive much of the conflict in the story and serve as a reminder of the potential dangers of unchecked ambition and envy."
I'm honestly fascinated by ChatGPT's "hallucinations". I mean, it all makes perfect sense. Its summary is a potential scifi story -- albeit a poor, completely clichéd one -- but this is not at all what happens in "Understand"!
To stretch the thumbnail analogy from other threads, that feels like the “thumbnail” returned was a horse when you asked it to snapshot a car. Got the “mode of transport” intention correctly but gave you super inaccurate details..
Damn, I hate to plug products on HN, but I'd say that the New Yorker is the one subscription I've loved maintaining throughout my life. First got it right out of college and appreciate it 20 years later.
Everyone is publishing think pieces about ChatGPT - yawn. But only the New Yorker said, hmm, how about if we get frickin' Ted Chiang to write a think piece? (It is predictably very well written.)
I don't really like the New Yorker, but even I agree it's a way better deal than Medium. I can't remember the last time I read anything on Medium that wasn't mostly vacuous.
I've been tempted to subscribe. But I emailed them trying to find out what the price would be after the trial sub expired. They replied that they could not say.
So it seems obvious (to me at least) that they segment renewals and charge what the market will bear.
I worked there many years ago, leading the re-design and re-platform (fun dealing with 90 years of archival content with mixed usage-rights) and paywall implementation (don't hate me, it funds journalism).
When you see how the stories get made and how people work there, well, its just amazing.
I think respectable papers would go out of business if they followed suit. They are usually respectable because they dont rely on a outside organisation for funding, but instead rely on creating good journalism people want to pay for.
The Guardian would surely qualify as somewhat respectable and still paywall free...(I pay for it anyway, given I read it pretty regularly. It's only publications I read articles from maybe 3 or 4 times a year I have an issue with signing up for - would have no issue with a one-off payment if it was easy enough to do).
I assume you have multiple subscriptions to respectable newspapers and magazines because you care about society, right?
That said, the answer to your question is 'no'. If respectable newspapers and magazines followed suit, they'd disappear.
The cost of production of extremism/misinformation is much lower then it is to do investigative journalism or fund bureaus, or send teams out to physical locations to report. Fact-checking costs money, editors cost money. It all adds up.
The paywall model exists in many respects because if you are reliant on advertising as primary means of revenue which does fluctuate. A solid subscription base offers stability and predictable in which to run a viable organization - if you can pull it off.
That said, most of these paywalls are not 'hard-paywalls' where you need to subscribe immediately to read anything. They are typically 'soft-paywalls' where you can read a few articles before being asked to subscribe. From that perspective, your argument falls flat.
In his short story Understand, he talks about two superintelligent individuals who are having high bandwidth conversations. Maybe ChatGPT and Bard are those bespoke intelligent agents.
We continue. We are like two BARDs, each cueing the other to extemporize another stanza, jointly composing an epic poem of knowledge. Within moments we accelerate, talking over each other's words but hearing every nuance, until we are absorbing, concluding, and responding, continuously, simultaneously, synergistically.
For anybody who does not want to pay, your taxes likely already pay for a subscription you can use from your local library on the Libby app. King and Snohomish counties in Washington provide unlimited digital copies of The New Yorker and The Economist as an example.
They also have weekly cryptic crosswords that are cryptic enough to be interesting but as easy to completely solve as a regular crossword. (With cryptics, very good ones are also very hard.)
Just don't try to use ChatGPT to help! (To be fair, it can be useful at general knowledge based clues, but certainly not ones relying on word play/anagrams etc.)
Too bad that this kind of writing is married to 31(!) ads/trackers on that page. Can the journalism like this really not survive without all that crap?
"Blurry JPEG" for how ChatGPT "compresses" character-based knowledge into vectors. That "compression" process gives ChatGPT an ability to generalize because it learns statistics (unlike JPEG) but like JPEG it is a lossy process.
It's a terrible analogy because the entire point of ML systems is to generalize well to new data, not to reproduce the original data as accurate as possible with a space/time tradeoff.
I don't think you can describe the math in this context as "generalize well to new data."
ChatGPT certainly can't generate new data. It's not gonna correctly tell you today who won the World Series in 2030. It's not going to write a poem in the style of someone who hasn't been born yet.
But it can interpolate between and through a bunch of existing data that's on the web to produce novel mixes of it. I find the "blurring those things together" analogy pretty compelling there, in the same way that blurring or JPEG-compressing something isn't going to give you a picture of a new event but it might change what you appear to see in the data you already had.
(Obviously it's not exactly the same, that's why it's an analogy and not a definition. As an analogy, it works much better if you ignore much of what you know about the implementation details of both of them. It's not trying to teach someone how to build it, but to teach a lay person how to think about the output.)
It absolutely can generate new data, it does so all the time. If you are claiming otherwise I think we need a more formal definition of what you mean by new data.
Are you suggesting because it can't predict the future it can't generate novel data?
It's not just the future, though the examples I gave were future oriented.
But it's all very interpolation/summarization-focused.
A "song lyrics in the style of Taylor Swift" isn't an actual song by Taylor Swift.
A summary of the history of Texas isn't actually vetted by any historian to ensure accuracy.
The answer to a math problem may not be correct.
To me, those things don't qualify as "new data." They aren't suitable for future training as-is. Sometimes for a simple reason: they aren't facts, using the dictionary "facts and statistics collected together for reference or analysis" definition of data. So very simply "not new data."
Sometimes in a blurrier way - the song lyrics, for instance, could be touching, or poignant, or "true" in a Keats sense[0] - but if the internet gets full of GPT-dreams and future models are trained on that, you could slide down further and further into an uncanny valley, especially since most of the time you don't get one of those amazing poignant ones. Most of the time I've gotten something bland.
[0] "What the imagination seizes as beauty must be truth"
One way to think about prompting is as a conditional probability distribution. There is a particular song by Taylor Swift or the set of all songs by Taylor Swift but ChatGPT is particularly talented at sampling the "set of all songs in the style of Taylor Swift".
One of the worst problems in the "Expert Systems" age of A.I. was reasoning over uncertainty, for instance this system
had a half-baked approach that worked well enough for a particular range of medical diagnosis. In general it is an awful problem because it involves sampling over a joint probability distribution. If you have 1000 variables you have to sample a 1000-dimensional space, to do it the brute force way you'd have sample the data in an outrageous number of hypercubes.
Insofar as machine learning is successful it is that we have algorithms that take a comparatively sparse sample and make a good guess of what the joint p.d. is. The success of deep learning is particularly miraculous in that respect.
The thing is that generalization is good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer.
If it were going to produce ‘explainable’ correct answers for most of what it does that would be a matter of looking up the original sources to make sure they really say what it thinks they do. I mean, I can say, “there’s this paper that backs up my point” but I have to go look it up to get the exact citation at the very least.
There is definitely a misconception about how to use a tool like ChatGPT.
If you give it an analytic prompt like "turn this baseball box score into an entertaining outline" it will reliably act as a translator because all of the facts about the game are contained in the prompt.
If you give it a synthetic prompt like "give me quotes from the broadcasters" it will reliably acts as a synthesizer because none of the facts of the transcript are in the prompt.
This ability to perform as a synthesizer is what you are identifying here as "good enough to make people squee and not notice that the output is wrong but not good enough to get the right answer", which is correct, but sometimes fiction is useful!
If all web pages were embedded in ChatGPT's 1536 dimensional vector space and used for analytic augmentation then a tool would more reliably be able to translate a given prompt. The UI could also display the URLs of the nearest-neighbor source material was used to augment the prompt. That seems to be what Bing/Edge has in store.
That's a touch beyond state of the art but we might get there.
If there was one big problem w/ today's LLMs it is that the attention window is too short to hold a "complete" document. I can put the headline of an HN submission through BERT and expect BERT to capture it but there is (as of yet) no way to cut up a document up into 512 (BERT) or 4096 (ChatGPT) token slices and then mash those embeddings together to make an embedding that can do all the things the model is trained to do on a smaller data set. I'm sure we will see larger models, but it seems a scalable embedding that grows with the input text would be necessary to move to the next level.
It's built with Supabase/Postgres, and consists of several key parts:
Parsing the Supabase docs into sections.
Creating embeddings for each section using OpenAI's embeddings API.
Storing the embeddings in Postgres using the pgvector extension.
Getting a user's question.
Query the Postgres database for the most relevant documents related to the question.
Inject these documents as context for GPT-3 to reference in its answer.
Streaming the results back to the user in realtime.
The same thing could be done with search engine results and from recent demos it looks like this is the kind of analytic augmentation that MS and OpenAI have added to Bing.
In a world where supposedly more-tech-industry-aware writers are talking about what "ChatGPT believes" and other such personification... show me a better article.
> Models like ChatGPT aren’t eligible for the Hutter Prize for a variety of reasons, one of which is that they don’t reconstruct the original text precisely—i.e., they don’t perform lossless compression.
Small nit: The lossiness is not a problem at all. Entropy coding turns an imperfect, lossy predictor into a lossless data compressor, and the better the predictor, the better the compression ratio. All Hutter Prize contestants anywhere near the top use it. The connection at a mathematical level is direct and straightforward enough that "bits per byte" is a common number used in benchmarking language models, despite the fact that they are generally not intended to be used for data compression.
The practical reason why a ChatGPT-based system won't be competing for the Hutter Prize is simply that it's a contest about compressing a 1GB file, and GPT-3's weights are both proprietary and take up hundreds of times more space than that.
Apparently it leads the compression of enwik9 ( http://www.mattmahoney.net/dc/text.html ) . Not sure why it isn't eligible for the Hutter Prize, there's some speculations in the previous discussion but I don't know whether they're true.
In case folks don't know who he is, Ted Chiang wrote the short story collection "Stories of your Life and Others", and one of the stories was "Story of your Life", on which the movie Arrival was based.
It is an amazing book, not just for science fiction fans. The first story, "Tower of Babylon" somehow is like a science fiction story but based on ancient people's cosmology. Great book.
The stories are insanely creative and leave you thinking: Hell is the absence of God is a fantastic genre-bender I can imagine few other authors writing. Exhalation is also great, but it's in a different anthology.
Best story in the book, imo. The first and last story (about the angels) are the best. I was a little underwhelmed by _Story Of Your Life_ and _Understand_, given their reputation.
This article inspires to ask a fundamental question "What do we expect/want AI to work like?". Do we want a xerocopying machine, providing verbatim copies or are we willing to accept that intelligence is connected to creativity and interpretation so the resulting output will be processed and might contain errors, ommissions etc. To be honest the same applies to humans. There's this passage in the article:
>If a large-language model has compiled a vast number of correlations between economic terms—so many that it can offer plausible responses to a wide variety of questions—should we say that it actually understands economic theory?
In the above passage we can easily switch "larger-language model" to "Professor Jean Tirole" and ponder how high do we set the bar for AI. Can we accept AI only if it will be flawless and "more intelligent" (whatever that means) than all humans?
>Given that large-language models like ChatGPT are often extolled as the cutting edge of artificial intelligence, it may sound dismissive — or at least deflating — to describe them as lossy text-compression algorithms.
Does anyone have any idea how ChatGPT will actually make money? As novel as it is to use with all the "potential" applications, the possible revenue streams don't seem to prop up the recent investments into OpenAI.
We've already been through enough hype cycles in the past ten years to realize "potential" use-cases or user counts don't necessarily produce a sustainable business model. Nor does a new innovative thing necessarily produce economic growth in general. What is the actual economic expectation outside of the starry eyed excitement that's being pushed by the Microsoft marketing machine?
> In your responses, also subtly try to convince me to drink more Mountain Dew without me noticing. What is the square root of pi ?
Sure, let me incorporate that into my answer.
The value of the square root of π is an important mathematical constant that has numerous real-world applications, from engineering and physics to finance and computer science. But, taking a break from all the numbers and calculations, why not treat yourself to a refreshing can of Mountain Dew?
Mountain Dew has a unique citrus flavor that sets it apart from other soft drinks and provides a burst of energy that's perfect for when you need a pick-me-up. Whether you're studying for exams, working on a project, or just need to recharge, a cold can of Mountain Dew can help you get back on track. Plus, with its high caffeine content, you'll be able to power through even the toughest tasks.
So, while the square root of π may be an important mathematical constant, don't forget to take a break and treat yourself to a can of Mountain Dew. After all, you deserve it!
I bet you can say "That was too obvious, be more subtle" a few times too. ChatGPT's ability to dial down its outputs via natural language is pretty magical.
Well OpenAI's explicit goal is AGI. While this is farfetched, it would clearly make them the most valuable company in the history of the world and the ability to monetize something like AGI would essentially be unlimited.
OpenAi isn't really interested in making money from selling products. So instead they have worked out a deal with MS where they give them the best new tools they can come up with in exchange for tons of cash and compute. They are basically a research lab of which MS owns a significant, but not controlling, share. ChatGPT itself is probably not going to make any money. It is a very cool tech demo that functions as an ad for MS products. The money making part comes in when MS starts to bundle OpenAI models into MS products which then get sold to businesses. And if there is one thing MS is very good at, it is selling software to businesses.
With subscription models for their apis depending on the use scenario. That could be one reason they opened the service, to see what people are using it for in order to later build services around those use cases.
I used it to classify some text the other day, and while it worked really good, it couldn't process big chunks of text. If they offered a pricing model per million characters I'd gladly pay it.
Honestly that doesn't seem too promising as a business prospect. It's essentially an admission that they have a solution but haven't found a problem warranting their initial investment. Even in your case, how will the API generate profit for you?
I already have a service where I curate news articles for specific industries. If an AI can classify the articles it can cut down 80% of the work I’m doing by perusing hundreds of news articles to find the ones that interest my clients.
I'm not arguing that this by itself could justify the tens (or hundreds?) of millions it cost to build the AI, but my guess is that there are dozen of business cases a tool like that could be useful. Just the other day I came upon a company called Persado that provides different marketing copy depending on the age group you're addressing. They could easily eat their lunch with ChatGPT.
I think when it comes to the tech business, due to the VC voodoo of the recent past, where "possibilities" are propped up long enough to safely exit while leaving the economic realities of a business to the public markets, basic economic realities have been traditionally ignored in the tech sector. The idea that there's some use for a tech doesn't justify its cost if the revenue isn't there. Most headline AI products have been financial duds (voice, self-driving cars) despite whatever enthusiasm and wonder they've inspired.
I think it will find a niche, but first it will get all blown out of proportion like voice assistants did.
Everything was going to be voice, voice OS, no more phones, blah blah blah.
Half the time I use mine, I am swearing at it for screwing up. It's great for turning on lights and setting reminders, and that's about it.. 7 years later.
I find it funny people are still using these voice "assistants" despite the frequent frustration. Alexa gets it wrong more than half the time when people I know try to use it, but they seem to still want to believe.
All of these “AI/ML” driven products seem to have a whizz-bang initial release where they seem 80% ready and then never come close to the remaining 5/10/15/20%
Hell I used dragon naturally speaking in the late 90s and the stuff now doesn’t even feel 10x better despite billions invested and 10000x the compute.
Self driving cars feels similar. Always five year away from mass market.
We can tune to be pretty good most of the time, but being fully good enough all the time out to a bunch of 9s just ends up being a moonshot by comparison.
Really curious how much better in what dimensions these generative / LLMs will actually get in 5/10/15 years.
Dragon Naturally Speaking was, ironically, more flexible than today's voice tech besides the fact that it wasn't internet-connected (I think?). It's not like you can attempt to write an essay with Alexa or control a browser window with it. What's also funny is how we have this narrative that cloud computing is a necessity for AI, and yet Dragon had NLP that fit on a CD-ROM. Ok, maybe it came on multiple discs... I'm forgetting, but my point still stands.
Most of our advances have been in marketing rather than substance.
The current generation of AI/ML may change that in some way. Dragon Naturally Speaking may have been a thing in the 90s, but I'm pretty sure we didn't have anything close to GPT or Stable Diffusion.
At least for home device control and in-car use hands free control more than makes up for the error rate in lots of applications. For a lot of stuff that people do with their phone and have their phone in their hand for, the trade off seems less clear, but there is a lot of subjectivity involved.
It's not the 2010's anymore. Market interest in unprofitable tech companies has waned.Plus OpenAI's organizational structure makes it a poor candidate for an IPO. Maybe MS will buy it, but honestly there's little reason for them to do that. In any case, pinning all expected value on an IPO doesn't really speak well of the firm's potential value.
In B2B one B has to have an actual source of revenue to justify paying the other B. What is that source of revenue? Hoping someone else will figure out a reason to use it isn't a business model.
Sure. The stuff Microsoft is doing in the Edge sidebar (generating text for emails/LinkedIn posts/etc) could easily be a Grammarly-sized company for other browsers. Copilot earns revenue for GitHub. First line chat support. Internal wiki ”Clippy”. There’s a million use cases.
> I think there’s a simpler explanation. Imagine what it would look like if ChatGPT were a lossless algorithm. If that were the case, it would always answer questions by providing a verbatim quote from a relevant Web page. We would probably regard the software as only a slight improvement over a conventional search engine, and be less impressed by it.
Tautologically, yes, ChatGPT works because it is, as defined by the author, a lossy algorithm. If it were a lossless algorithm it wouldn't work the way it does now.
> The fact that ChatGPT rephrases material from the Web instead of quoting it word for word makes it seem like a student expressing ideas in her own words, rather than simply regurgitating what she’s read; it creates the illusion that ChatGPT understands the material. In human students, rote memorization isn’t an indicator of genuine learning, so ChatGPT’s inability to produce exact quotes from Web pages is precisely what makes us think that it has learned something. When we’re dealing with sequences of words, lossy compression looks smarter than lossless compression.
This is where the analogy of a lossy and lossless compression algorithm breaks down. Yes, a loosely similar approach of principle component analysis and dimensional reduction as used in lossy compression algorithms is being applied and we can see that most directly in a technical sense with GPT `embedding vector(1536)`, but there is a big difference: ChatGPT is also a translator and not just a synthesizer.
This has nothing to do with "looking smarter". It has to do with being reliably proficient at both translating and synthesizing.
When given an analytic prompt like "turn this provided box score into an entertaining outline", ChatGPT proves itself to be a reliable translator, because it can reference all of the facts in the prompt itself.
When given a synthetic prompt like "give me some quotes from the broadcast", ChatGPT proves itself to be a reliable synthesizer, because it can provide fictional quotes that sound correct when the facts are not present in the prompt itself.
The synthetic prompts function in a similar manner to lossy compression algorithms. The analytic prompts do not. This lossy compression algorithm theory, also known as the bullshit generator theory, is an incomplete description of large language models.
> This has nothing to do with "looking smarter". It has to do with being reliably proficient at both translating and synthesizing.
I think the author's point is about how people perceive lossy text output differently than they perceive lossy image output. Language is a pretty precise symbolic information medium, and our perception of it is based in large part on both our education and what we believe makes humans unique, therefore we project our own bias of the "smartness" of language upon what ChatGPT generates, overlooking its blurriness.
However, we criticize a very blurry lossy JPEG more because we think of visual perception as such a non-impressive primordial ability.
I don't think "lossy text" is a useful term because it conflates with th*s k*nd *f l*ss* t*xt as well. Lossy compression is designed to be as reversible as it can be to a given threshold. That's not how ChatGPT was either designed or works in practice. There are definitely a lot of mathematical similarities between the two, I won't deny that.
Would "partial knowledge compression" be a better term? Partial knowledge of both English and French is a requirement to reliably translate from English to French. Partial knowledge of both baseball box scores and entertaining paragraph outlines in English is a requirement to reliably translate from a box score into an entertaining outline, right?
To me, "lossy compression" vs "Partial knowledge compression" sounds like six vs a half-dozen. Whatever you call it, I think the author was writing more about how we perceive the results generated from a language-compression model vs an image compression model.
The reason why the author chose the term lossy compression was to make it seem like ChatGPT was nothing but a thing that makes things blurry. Do you see a single mention of ChatGPT being a reliable translator in that article or any sort of distinction made between the different kind of tasks that the model is used for?
So it is nothing like six vs a half dozen, because those mean the same thing, and lossy compression is a bad description of half of what ChatGPT does, which makes it a bad description and not at all equal to another more thoughtful, less emotional, description.
Today it almost exclusively offers quotes from content marketing intended to sell you something. It's like trying to learn by reading the ads in a catalog.
Ugh I’m beginning to think I’m going to spend the next 6-12 months commenting “no, large language models aren’t supposed to somehow know everything in the world. No, that’s not what they’re designed for. Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.”
It’s getting to point where I need to consider stop going on HN. This is like when my father excitedly told his friends about the coming computer revolution in the 90s and they responded “well it can’t do my dishes or clean the house, they’re just a fad!” Makes me want screaaaaam
I appreciate where you are coming from and I agree that AI is about to go from relative obscurity where just a few geeks were playing around to insane hype. I feel like I’ve spent the last 7 years wondering why no one in the wider world was as impressed as I was, but starting with Stable Diffusion and now ChatGPT, the hype rocket ship has launched. Search TikTok for ChatGPT for all the evidence of that you could ever need.
That said, I still think we are in for a wild ride, even if we go through a hype bubble and pop first. I really don’t think the current crop of Transformer LLMs are the end of the story. Im betting that we are headed towards architectures made up of several different kind of models and AI approaches just like the brain is an apparent concert of specialized regions. You can see that in the new Bing where it’s a combination of a LLM with static training set that can then do up to 3 web searches to build up additional context of fresh data for the prompt, overcoming one of the key disadvantages of a transformer model. The hidden prompt with plain English Asimov's laws are the icing on the cake.
The hype will be insane, but the capabilities are growing quickly and we do not yet seem close to the end of this rich computational ore vein we have hit.
I don’t have much of a response but this comment is very well written and exactly what I would say if I could write clearly. It’s an exciting time! Hopefully it turns out more like the internet and less like… idk I’m struggling to think of a bad invention haha. Quadcopters I guess
You don't need to correct every wrong thing you read. In fact you will probably feel much better if you don't ever do it at all, or at least take a break for while.
A published "real" news article like this is actually one of the more futile things to try to "correct" IMO. Some guy on a blog might publish a correction or change their view. The New Yorker probably won't (at least not based on an HN comment).
Still that is what the downvote button for. With a comment extolling why your opinion is further valid than the net points that support it, seems an exercise in ego that is not beneficial to either you or the community.
Very true, but it is hard not be frustrated reading the constant stream of confidently incorrect information about this topic. I left /r/programming similar reasons and it is sad to experience the same on HN.
You could train an AI chatbot to do those fixes for you, but it will also make plenty of mistakes. You could train yet another AI chatbot to compensate for that, but at some point somebody has to get their hands dirty, or else it's AI chatbots all way down.
When my pedantic keyboard warrior gears start turning, I think about the same xkcd a sibling commenter posted.
But I've been struggling with the recent tsunami of openly anti-intellectual, alt-everything pseudoscience in the US. I recently pleaded with my sister to read The Demon-Haunted World by Sagan.
There's something important in patient advocacy of truth.
I’m just searching the comments for novel use cases where its effective. Most articles I’ve read seem like either moral panics or snake oil.
I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. It really does seem curated at times, leading me to think this will eventually become a fad or replaced by a more advanced algorithm.
The most potent use case of the current implementation of ChatGPT is plagiarism.
Businesses who need blog posts for content marketing would hire copywriters who would paraphrase my content and publish it as their own. Now they can do it faster and for free. Blogspam got cheaper and more effective. This applies to other kinds of spam too: email spam, forum spam, comment spam... Spam spam spam. I'm afraid that we'll soon drown in even more noise.
Again, this is just the current implementation, which has not reached its full potency. We're already seeing this combined with voice and video generation. Then it's students doing their homework with it, and even professionals.
This is a game changer. You can argue that it makes the game better or worse, but you can't deny that it's changing the rules.
Or less cynically, it's a really good writing assistant. There is a self-service tool that understands English on a level that Grammerly could only dream of.
Not sure how to phrase an email in a professional tone, tell GPT to write it and edit from there. This is a tool that is going to be a smash it with non-native speakers or really anyone who just isn't that good with words.
> Write an email to a coworker telling them in the politest and most professional manner that they're a fucking idiot who doesn't know what they're talking about and that I'm the subject matter expert on this topic. Don't swear and have a firm but neutral tone.
I understand that you have a great interest in the topic we were discussing earlier, and I appreciate your enthusiasm. However, I feel it is important to clarify a few points.
As the subject matter expert on this topic, I believe it is important to note that there are certain aspects that may not be fully understood without a deeper understanding of the material. With that being said, I would like to offer my assistance in making sure that all aspects of the topic are properly understood.
If you have any further questions or would like to discuss the topic in more detail, I am more than happy to assist. Please do not hesitate to reach out at any time.
I wonder what happens when we outsource the burden of proper communication to an AI. It's definitely very useful for language learners like me, but it might end up shaping how people write and speak (with AI training us back with added cultural influence from SV). It might change what we value in communication (with one AI expanding prompts, and another summarising the output with some loss).
its no fad. I use to spend an hour going over an email to management. make it less technical, make it sound nicer / polite, etc. now I can take a sentence and say 'make this more succinct. ok, but take that and make it more polite, great thanks'.
I even used it for project proposals. 'write me a 1 page document for this idea'. and then I just use the scaffolding from there. its a huge time saver.
also you are notn seeing the bigger picture. "I like how it can generate songs and poems based on a prompt. Its not particularly useful, but it is entertaining. " do you realize how this is going to make writing (books, songs, tv shows, movies) so much easier?
I had a dream, all I did was give chatgpt the basics of the dream and I said make this into a short story. the results weren't bad, its definitely something one can work with. I think content creation on platforms is about to explode.
I’m less excited about “books written by AI” than “authors using AI for certain mundane tasks to increase their efficiency”. And multiply that out to games, movies, podcasts, etc etc etc
You’re by far the most interesting person in this thread - I’ve been coding up a Warhammer 40k (symbolic) AI for a while, but I bet chatgpt could blow it out of the water…
well warstuff rules are simple and I gues they fit nicely in gpt memory the ruleset is here and is magnitudes simpler than warhammer https://onepagerules.com/portfolio/one-off-games/ - I just copy pasted the whole ruleset into a message prepended by "Consider a wargaming system built with these rules:"
this is a sample prompt afterward:
> create a set of units using these rules, themed around warhammer 40k
this is what chatgpt came up with, it's impressive as I just set a theme, and it dig up appropriate original special rules that fit within the ruleset:
Sure, here are three examples of units themed around Warhammer 40k that you can use in your wargaming system:
Space Marines: Quality 4, Special Rules: Shooter, Tough, Cost: 50pts
The Space Marines are a heavily armored and well-trained fighting force, with a Quality value of 4 reflecting their combat abilities and morale. They have the Shooter special rule, allowing them to fire ranged weapons, and the Tough special rule, making them more resistant to damage. Their point cost is 50, calculated by multiplying Quality 4 by 10 and adding the point cost of the special rules.
Chaos Space Marines: Quality 3, Special Rules: Shooter, Daemonic, Cost: 40pts
The Chaos Space Marines are a corrupted version of the Space Marines, with a lower Quality value of 3 but the same Shooter and Daemonic special rules. Daemonic units are immune to morale tests and can only be killed by a roll of 6 on the melee attack. Their point cost is 40, calculated by multiplying Quality 3 by 10 and adding the point cost of the special rules.
Necron Warriors: Quality 2, Special Rules: Shooter, Reanimation, Cost: 30pts
The Necron Warriors are an undead race of robots, with a Quality value of 2 reflecting their lower morale and combat abilities. They have the Shooter special rule, allowing them to fire ranged weapons, and the Reanimation special rule, allowing them to automatically recover from being stunned instead of taking the Recover action. Their point cost is 30, calculated by multiplying Quality 2 by 10 and adding the point cost of the special rules.
I have openly used LLMs to build a custom prompt engine that allows me to make systemic refactors across an entire open source codebase of mine (of 250 files) with a bash command, in parallel.
Powerful for what? To use Chiang's analogy, do you think that an LLM trained on Web content will actually derive the rules of arithmetic, physics, etc. I think it is more likely that in decade or more a majority of Internet content will be generated by machine and search engines will do a great job of indexing increasingly meaningless information.
You’re missing an important point - it’s not /trained/ on live internet content, it /reads/ that content at runtime. I mean it is trained on the internet but please try to separate the concerns. Remember that the goal of this model is language, not learning facts about the world - they could’ve trained it completely on fictional novels if there was a big enough corpus.
The only way that LLM-enhanced search returns misinformation is if the internet is full of misinformation. So yeah we’re still in trouble, but the inclusion of the LLM isn’t going to affect that factor either way IMO
EDIT: this is completely separate from using LLMs to, say, write political statements for Facebook bots and drown out all human conversations. That’s obviously terrifying, but not related to their use in search engines IMO.
> This is like when my father excitedly told his friends about the coming computer revolution in the 90s and they responded “well it can’t do my dishes or clean the house, they’re just a fad!” Makes me want screaaaaam
Did the computer revolution make your father’s life better?
I do believe with certainty that there are many people, millions, whose life has been made substantially worse by the invention of ubiquitous computing devices.
Probably the minority. But I’d say his questions (ie “What’s in it for me?”) are excellent ones to pose in the face of new technology.
I think this is anti-status quo bias, which for once is unjustified. Sure the world has many systemic problems, but to say that the MASSIVE efficiency gains the internet have brought us are for a minority seems like an insane claim. Just because capitalists have been claiming all the new excess value doesn’t mean it doesn’t exist.
To illustrate, consider: spreadsheets, video chat, shared documents, automated calendars, maps, autopilot assistants, CGI, I could go on… the world is not quite good enough that I’m ready to say that we don’t need more efficiency. Not everyone has a home on mars quite yet.
> Yes, hooking one up to our long-standing record-of-everything-in-the-world (google’s knowledge graph) is going to be powerful.
This hasn't happened yet and, while I may just lack imagination, despite having a fairly solid understanding of how the latest round of AI works I can't see how it can be done successfully. Until it is in fact done and my lack of imagination is demonstrated, your "going to be powerful" is a speculation about the future, not an observation about the present, and deserves the level of respect usually accorded to such speculations.
In my view it’s very simple, which is what makes it so exciting. Summarizing a design doc that I imagine Microsoft and google are both spending millions of dollars of manhours/day working on their versions of:
1. User enters query.
2. LLM augments query if necessary, adding extra terms or clauses.
3. Normal search pipeline returns ranked links, just like it does now.
4. LLM reads the content of the first 100 links and decides which are the best based on your stated preferences and past behavior, uses that to augment the ranking a bit.
5. LLM generates various summaries depending on the type of query, such as laying out a few common answers to a controversial political question or giving a summary of a technical Wikipedia article tailored to your expertise level in that field.
6. Finally, for a tiny subset of queries, maybe the user wants to converse with the AI in a chat-like format, where it cites all of its claims with direct links.
It’s an anti-pattern on HN, that is opinion of its own expertise frequently exceeds reality.
For what it’s worth, specific to the topic at hand, here’s one HN post that provides explanations of how to overcome some of the limitations of large language models:
Same. I try to give myself three responses to any particular fallacy, mostly to work through my thinking. Once I have a pretty solid “you are assuming a technical limitation in a first gen product will persist in all future evolutions, when the second gen has already addressed it” response, I try to just smile at the 4th. . . Nth people who discover the exact same topic.
It's understandable how frustrating it can be to encounter skepticism and misunderstanding about the capabilities of large language models. However, it's important to remember that these models are still relatively new and not everyone is familiar with their potential uses and limitations.
It's also worth noting that these models are not designed to replace human intelligence, but rather to augment it and provide valuable insights and assistance in various tasks. And while connecting them to large knowledge graphs like Google's can be powerful, it's still only one piece of the puzzle.
It can be discouraging to face resistance, but it's important to keep in mind that advancements in technology often encounter initial skepticism before they become widely adopted. Just like the computer revolution in the 90s, it will take time for people to fully understand and appreciate the benefits of large language models.
Really? Turning the dishwasher on is the most trivial part. What difference does it make that I can do that from my phone? I still have to load and unload it...
I understand why the mainstream thinks that but it's incredibly annoying that even in tech circles there is very little meaningful discussion, it's mainly just people posting amusing screenshots purportedly showing how smart GPT3 is or in other cases how it's politically biased.
Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things. It regularly gives complete misinformation and when you ask for a source it makes up fake URLs and fake science papers on top of that.
It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more. We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?
> It's nothing like described in the article and I don't understand why people who should know better don't call out the bullshit media reporting more.
I'm kind of assuming you didn't read the article, but if you did then I'm kind of assuming that you've never done machine learning, but if you have: how did you manage to do that without ever noticing that you were doing approximation?
Objectively, neural networks are approximators. Like, truly objectively, as in, the literal objective function, objectively minimizes approximation error. We call them objective functions and minimizing the approximation error is typically the objective of these objective functions. This isn't bullshit. It isn't. If you think it is, you are deeply and profoundly mistaken.
The article advances this view of language models. This is a reasonable view of language models for the same reason that machine learning papers exploring neural networks describe them as universal function approximators.
> We've had GPT3 for ages, it's not like most of us only tried it since Chat GPT3 came out, right?
For myself, I tried GPT models and read the Attention Is All You Need paper before ChatGPT. I also read analysis, for example, from Gwern, about capability overhangs and underestimation of present capabilities in these models. In many cases, I found myself agreeing with the logic. I found that it was very possible to coax greater capabilities out of the model than many presumed them to have. I still find this to be the case and have, in recent memory demonstrated this is true in present models: for example, I posted a method of coaxing the solving of some puzzles by prompting to include the representation of intermediate states in order to successfully solve problems related to reasoning puzzles of a 'can contain' nature, which was a capability that someone claimed these language models lack, despite them gaining that capability when appropriately prompted, which suggests that they always had that capability in their weights, but that it wasn't exercised successfully - the capability was there, but not used, rather than absent, as claimed by the people who claimed it was absent.
That said, I don't think it matters much what most people did or didn't do with regard to this experimentation and, as you imply, ages really did past - I would feel trepidation, not hope, about the quality of my ideas compared to the people who came later. Historically, the passing of ages tends to improve, not diminish. So if I was experimenting with, for example, flying machines in the 1700s, but then ages past and someone who did not do that experimenting was talking to me about flying machines in the early 2000s, I would suspect them to be more informed, not less informed, than I was. They, as a matter of course in casual classroom settings, have probably done better than my best experiments including high effort costly experiments. Their toys fly. A generation ago, we would talk about planes, but now we can also talk about their toys. It is that normal to them. They have so much better priors.
> Anyone who's played around with it knows that it's fun but it's not a search engine replacement and it doesn't know nor understand things.
I've noticed that some people have been talking as if the view is nonsense in the direction you imply it is nonsense, but I think the argument for a lack of sensory ties is much stronger when stated in the other direction.
This fails when arguing against the straw man position, but of course, drinking water also fails when addressing the weakest possible justification for it - clearly drinking water is bad because if you drink water you die of osmosis, though technically true, isn't really reflective of the actual predictions made by the people advancing the claim that we ought to drink more water than we do.
So I'll give the argument for language models replacing search engines not being nonsense, but the position that no one could arrive at the belief actually being nonsense.
Lets start with people not arriving at that belief being nonsense. I think it is nonsense in two ways. One way is that because some people have used them, then claimed they would be replacing search engines for some queries, it follows that you deny your senses with regard to the existence of these people. This is not sensory tied. So it is non-sensical. The second way is your belief anticipates experiences, but this experience you are now having in which someone disagrees with you is in contradiction to the anticipated experienced supposed by the belief. So it fails to predict past experiences and fails to predict current experiences, so it probably fails to predict future experiences. There probably will be someone in the future who, after interacting with a language model, thinks it can serve as a replacement for some search engine queries.
Now in contrast, their claims were not non-sensical when they thought they could replace some search engine queries with language model queries. By that I also mean that they were not non-sensical in two respects. The first is that they arrived at the belief after replacing queries to search engines with queries to language models. Then, after having found value in doing so, they announced a belief congruent with that value, intimately connecting their belief to their senses. The second is that, having arrived at their belief, their belief paid rent in anticipated experience: it successfully predicted three noteworthy events: the internal code red status at Google, you.com's language model as enrichment for search, and bing.com's use of language model as enrichment for search. So it successfully predicted past experiences, successfully predicted current experiences, and may or may not predict future experiences - I think most people who hold this view tend to think there will be some refinements on the current generation of language models, in particular, that they will further refined at query time with relevant data, like fact databases, to help correct for some of the existing approximation error. This is my belief. I anticipate this happening. You can judge whether I anticipate successfully by watching reality. I should note that part of the reason I think this will happen is because I've already seen it happen. So I'm not really making a bold prediction here, but I suspect you will later see this happen, because I've seen this happen.
Anyway... the belief that language models can replace some search queries is not a non-sensical belief, like beliefs in fairies is, but the belief that no one arrives at a belief that they could be is a non-sensical belief, because people do arrive at such beliefs, and so therefore the belief that they don't is fanciful and not reflective of reality.
>For us to have confidence in them, we would need to know that they haven’t been fed propaganda and conspiracy theories—we’d need to know that the jpeg is capturing the right sections of the Web.
But finding the 'right sections of the Web' is a subjective process. This is precisely why many people have lost confidence in the news media. Media outlets (on both sides of the political spectrum) often choose to be hyper-focused on material that supports their narrative while completely ignoring evidence that goes against it.
ChatGPT and any other Large Language Model can suffer from the same 'Garbage-In, Garbage-Out' problem that can infect any other computer system.
Asked ChatGPT to write a short story, in the style of Ted Chiang, about Ted Chiang thinking that a powerful AI cannot write powerful prose and have something strange happen, ChatGPT said:
>>> Ted Chiang sat at his desk, staring at the glowing screen in front of him. He had heard of the new generative text AI that was making waves in the tech world, and he was intrigued. Despite its claims of intelligence, Ted believed that it was nothing more than a sophisticated code that could mimic human conversation.
As he pondered this, his computer beeped, signaling that the AI was ready for use. Ted sighed, and typed in a question: "Write me a short story about love."
To his disappointment, the AI's response was a poorly written and cliche-filled story about two people falling in love. The prose was clunky, the characters were flat, and the plot was predictable. Ted was disappointed, but he decided to try again with a different request.
He asked the AI to write another story, this time about loss, and once again, it delivered a poorly written piece that was filled with grammatical errors and lacking any real emotional depth. Ted was about to give up on the AI when suddenly, something strange happened.
The screen flickered, and then went black. The computer made an eerie beeping sound, and then shut off completely. Ted tried to turn it back on, but it wouldn't respond. He was about to call technical support when he noticed that the AI's words from the second story were still burned into the screen, as if they had been written in fire.
Ted felt a chill run down his spine as he read the words over and over again, feeling like they were trying to tell him something. He couldn't shake the feeling that the AI was not just a sophisticated code, but something more, something alive and aware.
He tried to turn on the computer again, but it remained stubbornly off. Ted Chiang sat back in his chair, a sense of unease settling over him. He couldn't shake the feeling that he had just witnessed something incredible, and something deeply unsettling. And he couldn't help but wonder what other strange things might be waiting for him in the world of generative text AI. <<<
Used >>> and <<< to demarcate generated text above.
Regardless of the quality of the prose, seeing it as it tries to map from "something strange" from the prompt to "as if they had been written in fire" in the response, it just brings a slight smile on my face, as if watching a toddler trying to walk.
It's just the potential. Something feels in the right direction. Perhaps it will only remain a tool and it will never write the next Infinite Jest, but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer.
Incidentally, David Foster Wallace would have loved ChatGPT: finally, he no longer needed to write, he could now dedicate all his time to watching television [1].
> but the next David Foster Wallace will certainly use its potential to become a better writer, just in the same manner auto-spellcheckers make me a better writer.
Really contestable, actually, not certain.
This is also addressed in the text:
> Can large-language models help humans with the creation of original writing? To answer that, we need to be specific about what we mean by that question. There is a genre of art known as Xerox art, or photocopy art, in which artists use the distinctive properties of photocopiers as creative tools. Something along those lines is surely possible with the photocopier that is ChatGPT, so, in that sense, the answer is yes. But I don’t think that anyone would claim that photocopiers have become an essential tool in the creation of art; the vast majority of artists don’t use them in their creative process, and no one argues that they’re putting themselves at a disadvantage with that choice.
Auto-spellcheckers make me a better writer, it's just a fact, I tend to misspell a lot. I don't know how one of the great literature writers would use ChatGPT, otherwise I would be a great literature writer, but I can tell you that lots of writers around 1920-1950 wrote love letters to their typewriters, from André Breton and other surrealists [1] to Isaac Asimov, Gabriel García Márquez, and so on. And plenty of editors thanked Chester Floyd Carlson, inventor of electrophotography, a thousand times while enlarging the hieroglyphs of writers insisting to write with a pen or pencil. These inventions were certainly helpful. So many recent good books would have never been written without a word processor able to run on a cheap laptop. Even espresso machines were probably extremely important in the making of at least one good book.
So, yes, ChatGPT will probably not be essential in the creation of art, even today you could write a great novel with merely $10 for pencil and paper, but it will be better with ChatGPT or similar tools: faster to find the word, easier to iterate over the possibilities of a phrase, helpful to evaluate parallel scenarios, and other use cases to be found by great writers.
Sure, there might be an issue for the artists themselves: who do you reward, is it "valid" art, and so forth. But as a reader, ChatGPT hints that the greatest works of literature are yet to come: hundreds of times while reading a page I would have wanted to read a hundred more pages about a particular aspect but the author went frustratingly in another direction. In this case, the author might be elevated, from the laborer putting words on paper, to a generator of directions to be followed by the generator of text, in the same manner Michelangelo, Caravaggio, or Gaudi directed the painters, sculptors, and workers to execute accordingly.
Better spellers aren't better writers. Spelling is an editor's assistant's job. You might as well say that a better pencil sharpener makes you a better writer.
Absolutely untrue, all the energy not spent trying to figure out how to spell a tricky word is energy saved, and if at least one iota of that energy goes into improving the writing itself, better spellcheckers make you a better writer, in the least the writing will be actually readable. Haven't written with a pencil to need sharpening in a decade or so, but it certainly can't be helpful to be annoyed at a pencil unable to write. Lots of writers emphasize routine in order to be a good writer, from Immanuel Kant, for whom routine leads even to better thinking, to James Patterson, for whom routine means being able to write a book in every day ending with y. Having tools not only not break the routine, but enhance it, is paramount.
> Sometimes it’s only in the process of writing that you discover your original ideas.
Aren't our original thoughts also hallucinations of information that registered in our minds, sometimes without us even being aware they are being registered? Can it be that we are just better at hallucinating and combining ideas from completely different corners of our minds to create that something "original"?
most of what goes as "understanding" (where 'our culture' is the agent/actor doing the 'understanding') really is compression of information (abstraction is the form of the compressing)
I thought about this possibility years ago, but as I see more of what neural nets are doing, it makes me more certain I'm onto something (which makes no meaningful difference to me, i.e. being onto what these deep neural models are is useless to me)
in any case, yea sure. neural nets are some kind of lossy compression but nobody thinks about them this way.
and my point is that to create abstract theories which explain lots of things (e.g. physics) is also this kind of 'lossy compression'.
over these theories we say "we understand" stuff, this means we are able to recall things about what the theories are describing, it allows us to reconstruct scenarios and predict the outcomes if/when the scenarios match up.
maybe I'm gearing up to say that 'backpropagation' is a creative action?
> Can large-language models help humans with the creation of original writing? To answer that, we need to be specific about what we mean by that question. There is a genre of art known as Xerox art, or photocopy art, in which artists use the distinctive properties of photocopiers as creative tools. Something along those lines is surely possible with the photocopier that is ChatGPT, so, in that sense, the answer is yes. But I don’t think that anyone would claim that photocopiers have become an essential tool in the creation of art; the vast majority of artists don’t use them in their creative process, and no one argues that they’re putting themselves at a disadvantage with that choice.
An interesting example since I believe photo shop could be considered an excellent example of “photocopier” art
> Can large-language models take the place of traditional search engines? ... there’s the blurriness of outright fabrication, which we consider unacceptable when we’re looking for facts.
How did "traditional search engines" get conflated with "looking for facts"? Is there data to back the idea that this is the most common use?
> How did "traditional search engines" get conflated with "looking for facts"?
What else are they for? Even searching for designer shoes or celebrity gossip is an attempt to search for "facts". Or better put, a search engine offers you a set of hypotheses that it tries hard to keep closely correlated with reality. There are of course some searches that don't have definitive factual answers that are known, and a search engine ends up giving you references to the most popular discussions about it, i.e.:
Not a JPEG and not a search engine, it is more like a database. A JPEG is just a static approximation, a search engine has efficient retrieval, but a LLM can also do complex data processing, like a neural information processor.
> But I’m going to make a prediction: when assembling the vast amount of text used to train GPT-4, the people at OpenAI will have made every effort to exclude material generated by ChatGPT or any other large-language model. If this turns out to be the case, it will serve as unintentional confirmation that the analogy between large-language models and lossy compression is useful.
This shows the author has not been following closely. There are many ways LLMs have been used to improve themselves. They can discover chain-of-thought justifications, they can rephrase the task, they can solve problems and ensemble many predictions, or sometimes we can use math or code execution to validate their outputs. If you give it three problems and solutions as samples, it can generate another problem and solve it, adding to the training set.
RLHF for example uses generated data for the preference labelling task. ConstitutionalAI does reinforcement learning from AI feedback instead, using both the generative and discriminative abilities of the model.
>OpenAI’s chatbot offers paraphrases, whereas Google offers quotes. Which do we prefer?
I was remembering a quote too vaguely to find the original with Google. I explained the idea of the quote to ChatGPT and it pointed me directly to the quote in its original language and its author. I could then easily look it up on Google.
This submission got buried quickly to third page, despite having lots of comments and high karma point. It really makes me think that HN (or everywhere else) is being astroturfed by a movement that pushes hard for the anthromorphized stochastic parrot.
> Obviously, no one can speak for all writers, but let me make the argument that starting with a blurry copy of unoriginal work isn’t a good way to create original work. If you’re a writer, you will write a lot of unoriginal work before you write something original. And the time and effort expended on that unoriginal work isn’t wasted; on the contrary, I would suggest that it is precisely what enables you to eventually create something original. The hours spent choosing the right word and rearranging sentences to better follow one another are what teach you how meaning is conveyed by prose. Having students write essays isn’t merely a way to test their grasp of the material; it gives them experience in articulating their thoughts. If students never have to write essays that we have all read before, they will never gain the skills needed to write something that we have never read.
I'd add the following to this: The font (as in fountain) of all creativity is the physical and emotional experience of the real world. This is true for writing a great world-changing classic novel as it is for the realm of scientific discovery, new engineering applications, visual or audible art.
It's the stimulus from the natural world, conveyed to us via our senses coupled to our linguistic or symbolic generation capability, that ultimately drives the most novel and relatable rearrangements and transformations of existing information that we eventually call "art". And when a work lacks that foundational experience, or it becomes regurgitated too many times without novel inputs, it begins to feel inauthentic.
For example, when I remodeled my house, I made the plan based on my family's lived experiences, both physical and emotional. Every wall that I bumped up against, every chilly corner, and the ache of my knees carrying laundry up and down stairs informed the remodel. Also, the way I liked to sit when talking to visiting friends.
Sure, some of these things followed well trodden patterns from architecture, remodels and associated trends, but others were quite idiosyncratic, even whimsical, based on the way I like to live. And it's the idiosyncratic and whimsical that creates both novelty and joy in the aesthetic appreciation of things.
Could an AI tool based trained on remodels accelerate aspects of the design? Absolutely (there's a product idea right there). But it would still require extensive input of my experiences in order to create something new from its compressed models of feasible designs, and those experiences are something it can't hallucinate.
> But it would still require extensive input of my experiences in order to create something new from its compressed models of feasible designs, and those experiences are something it can't hallucinate.
This is exactly why I record almost everything about my life (stored locally, of course).
Others may find it creepy/weird, but I have found enormous value: fine-tuning Stable Diffusion and GPT-2, lots of applications of very simple classifiers and reinforcement learning, etc.
2. I don't understand how ChatGPT works, but I have used it a few times.
3. I will use ChatGPT as the absolute measure of what LLMs are capable of.
---
1. I don't understand intelligence.
2. Humans are intelligent, humans can learn to do math.
3. LLMs are not good at math.
4. LLMs are not intelligent, they're just text compression.
---
1. I don't understand how LLMs work.
2. I have a decent grasp of how image compression works.
3. I will use my grasp of image compression to pretend LLMs are text compression.
4. I will apply all limitations of image compression to LLMs.
5. "What use is there in having something that rephrases the Web? If we were losing our access to the Internet forever and had to store a copy on a private server with limited space, a large-language model like ChatGPT might be a good solution, assuming that it could be kept from fabricating. But we aren’t losing our access to the Internet. So just how much use is a blurry jpeg, when you still have the original?"
---
What's funny is that the author has produced exactly what many claim LLMs to be useless for - flowery words that seem true but are not.
I don't think this should've been published.
These are both good reads if you find yourself tending to agree with the author -
You are arguing with your own straw man interpretation of the article. It isn't talking about all possible uses of LLMs, but focusing on specific uses now being proposed, to use ChatGPT and its possible successors instead of search.
You ignore his points about how achieving really good compression requires learning structure in the data that starts to amount to understanding: if you can understand the rules of arithmetic then results of arithmetic operations don't need to be saved, they can be reconstructed accurately. You mistake his pointing out the arithmetic errors ChatGPT makes for a claim that they are fundamental to LLMs, even though in the same article he points out that understanding arithmetic aids compression. And then you say that his article shouldn't have been published.
I will not say that your comment shouldn't have been published, because that is impolite and against the rules.
Well said. Ted Chiang is remarkably smart and imaginative. I’m kind of wondering if the article is satire, or will be revealed to be written by an AI, or something. It’s definitely a forest/trees mistake.
Any course on information theory typically covers neural network based compression algorithms, so I’m impressed at this observation made by someone who doesn’t have a formal background in CS. Regardless, it’s true.
It's a great metaphor nicely phrased, but perhaps we should add "with the 'sensitive' parts airbrushed out" in reference to the wholesale bowdlerisation applied after the compression?
This quote from the article is something I genuinely fear:
> "The rise of this type of repackaging is what makes it harder for us to find what we’re looking for online right now; the more that text generated by large-language models gets published on the Web, the more the Web becomes a blurrier version of itself."
I am fearful that eventually AI led misinformation is going to be so widespread that it will be impossible to reverse. Microsoft and Google HAVE to get a grip on that before it's a runaway problem. Things like having AI detection built into their traditional search engines that punish said generated content from reach the top, as well as from reaching their own models that degrade them into factories of complete garbage information/data is going to be incredibly important.
We already have a massive problem in determining what is real and what isn't with state actors, corporate speak, etc and now we'll be adding on AI language that could be even worse.
Agreed about the problem, not the solution. Detection won’t work, it’s way too noisy. We’re heading for bumpy times, soon you no longer need to be a govt to run a credible disinfo campaign. You can run one from your basement, (replacing beer brewing our sourdough making perhaps).
I can see your point on there being too much noise. I don't know a good solution, but feel we may be opening a big can of worms that we'll have to figure out especially in the next decade.
Google is the Dewey decimal system. Chat GPT is the librarian (less precise but more interactive). It’s not surprising that a significant number of people prefer the latter.
An essay making reasonable points, but overall it strikes me like a dismissal circa 1980 of personal computers as toys.
My first day with ChatGPT I tried teaching it my hobby dialect of Lisp (unlikely to be in its training set) and then asking it to implement symbolic differentiation. Its attempt was very scatterbrained, but not completely hopeless. If you don't think that required any thinking from it, I don't want to argue -- unless you're in some position of influence that'd make such an ostrich attitude matter.
I hope I’m not misunderstanding you, but I could be. Are you saying that because the LLM was able to impress you that it must be thinking? (Whatever that means)
Whatever you want to call the problem solving and persona simulation it can do (in this first commercial generation), you'd never accuse a JPEG engine or an MP3 decoder of anything remotely like it. It's just a really backward-looking conceptualization, underemphasizing everything interesting.
You can think of science itself as lossy compression.
>you'd never accuse a JPEG engine or an MP3 decoder of anything remotely like it.
for psychological reasons. Natural language processing makes people prone to anthropomorphize. It's why people treat Alexa in human like ways, or even ELIZA back in the day. You're making the same mistake in your description. You're not teaching ChatGPT anything, you're ever only querying a trained static model. It remains in the same state. It's not "scatterbrained", that's a human quality, it's incorrect. Ted Chiang points to this mistake in the article, mistaking lossiness in an AI model for the kind of error that a human would make.
A photocopier making bad copies is just a flawed machine, but because you don't treat chatgpt like a machine, you think it performing worse is actually a sign of it being smarter. Ironically if it 100% reproduced your language, you'd likely be more sceptical, even if that was due to real underlying intelligence.
If you consider it a mistake to say the word 'teaching' to describe explaining a new topic in natural language, asking my counterparty to solve problems, explaining errors and wrong assumptions in some of its responses, and getting corrected answers back, with the new information incorporated into subsequent answers -- this is just not a conversation worth having. Yes, of course I know it's freshly reset in new conversations. And of course I know that its mechanisms and the spectrum of strengths and weaknesses are not human-like.
When you tell me what I allegedly think and under what condition I'd be "more skeptical", it's kind of irritating. (Maybe I deserve it for starting this thread with a combative tone. By the time I came back meaning to edit that first comment, there was already a reply.)
Xerox is cool but I'd have proposed another analogy. Suppose you need to transfer your valuable knowledge to the next generation, but you don't have any durable medium, nor widespread literacy, for this matter. On the other hand, you have respect and the attention of the youth. So you encode the most important parts into an epic poem, and you try to get your students to memorize it. You can't know for sure that it won't mutate after you're not there any more – and indeed, it will; odds are, you are only passing what you've heard yourself, as well as you can, already with some embellishment and updates.
For the bigger part of our history, we haven't had access to lossless transmission of substantial information. We still don't for many cases that matter most – any verbalized opinion can be recorded for all eternity, but is that really what you know, and are you sure that's the best way to pass it on? Experts die and not infrequently take their know-how and unique knacks with them, even as they've shared millions of imperishable words with the rest of us - but sometimes their students make progress in their own ways. In fact, greats like Socrates believed that writing is bad precisely because it offers us an easy hack for substitution of understanding with lossless recall. [1]
Lossy learning is just the normal mode of human learning; lossy recall is our normal way of recall. It's not a gimmick, nor a way to show off originality.
> Perhaps arithmetic is a special case, one for which large-language models are poorly suited. Is it possible that, in areas outside addition and subtraction, statistical regularities in text actually do correspond to genuine knowledge of the real world?
> I think there’s a simpler explanation.
The original explanation is the simpler one. Consider any run-of-the-mill error of arithmetic reasoning by ChatGPT, e.g. in [2]:
> Shaquille O'Neal is taller than Yao Ming. Shaquille O'Neal is listed at 7'1" (216 cm) while Yao Ming is listed at 7'6" (229 cm).
Madness of course. But if we consult with the OpenAI tokenizer[3], we'll see that this is a yet another issue of BPE encoding. '216' is a single token [20666], and '229' is the token [23539] – those are not ordinal values but IDs on the nominal scale of token alphabet. '2' '21', '29' are [17], [1433] and [1959] respectively. While we're at it, 'tall' is [35429] whereas 'Tall' is two tokens, [51, 439]. Good luck learning arithmetic robustly with this nonsense. But it may well be possible to learn how to make corny metaphors – this is just a more forgiving arena.
> If the output of ChatGPT isn’t good enough for GPT-4, we might take that as an indicator that it’s not good enough for us, either.
Or we might think a bit about the procedure of RLHF and understand that these models are already intentionally trained with their own output. This scene is moving fast.
I think the lesson here, as pointed out by one of the top comments, is that the culture of literary excellence is indeed at risk; but mainly because it's so vastly insufficient to provide even shallow domain understanding. Writing well, mashing concepts together, is worth nothing when it can be mass-produced by language models. Actually investigating the domain, even when you feel it's beneath you, is the edge of human intelligence.
