I checked out your site (excellent content and very, VERY useful), and I came back comment with what I think is similar feedback that other commenters have.
I would suggest a few things -
1. Less resource intensive site - I’m on my phone, and it took probably 5 seconds to load the front page, and then to actually get to the list, it took a few seconds more. I do not have a current gen phone, but I’d hazard a guess that my phone is about average in age as far as the potential user-base for this site goes! Do not discount mobile users, especially for a site focusing on CLI applications.
2. Maybe I care what language a CLI utility is written in sometimes (Albeit, I can’t recall a time when I have). I’d add a search in the front page, if well thought out. Or, honestly, I’d just have the front page be the list. People are great at lists, and generally quite good at reading, and your list has very well-written descriptions.
3. Kinda related to the above, but I think the categories view is effectively useless. The categories already appear in the description (and if they don’t, they probably should) - so why not just skip straight to the list? You could add a filter for the list if you want. Or, even better - a simple search box!
I’m only giving these criticisms as a means to (subjectively) improve your site, because overall I think it is very well done and thoughtful, without a lot of fluff. There are hundreds more I’d not have criticized, but only because they felt like a waste of time and dead weight. Keep up the good work!
Only for memoized boilerplate, and that’s the way I’ve used them since I realized I could do it. “AI” hasn’t supplanted the practice for me at all for me, and I’m still surprised that apparently it had for most people. I can insert a template faster than a round-trip to an LLM.
That’s not to say that “AI” assisted coding isn’t useful -in fact it can be indispensable. For me, the value is mostly it being a seemingly perpetually slightly drunk, yet prescriptive, pair coding partner.
This story just keeps getting more and more bizarre. Reading the charges and supporting affidavits, the whole thing is reading more and more like some sort of Yorgos Lanthimos film. The rationalist connection - a literal sequel to the 2022 events (in turn a sequel to the 2019 CFAR stuff) - is already weird enough. But I can't get over the ridiculousness of the VT situation. I have spent time in that area of VT, and the charged parties must have been acting quite bizarre for the clerk to alert the police. Checking into a motel wearing all black, open carrying does NOT cut it. The phones wrapped in foil is comical, and the fact that they were surveilled over several days is interesting, especially because it reads like the FBI only became aware of their presence after the stop and shootout?
The arresting agent seems pretty interesting, a former risk adjuster who recently successfully led the case against a large inter-state fraud scheme. This may just be the plot of Fargo season 10. Looking forward to the season arc of the FBI trying to understand the "rationalist" community. The episode titled "Roko's Basilisk", with no thematically tied elements, but they manage to turn Yudkowsky into a rat.
Yeah, the CN is cdn.media.ccc.de, and SANs on the cert are DNS Name: berlin-ak.cdn.media.ccc.de, berlin-ak.ftp.media.ccc.de, cdn.media.ccc.de, ftp.media.ccc.de, static.media.ccc.de. Unfortunate submission typo! I've done similar a few times.
Maybe @dang or one of the moderators can swap to the link in your edit!
It's an open congress put on by a German club, and people giving talks aren't being _told_ to give talks in German. If someone is more comfortable presenting in their native language, why shouldn't they?
communication is a 2 way process. Languages are just a tool. One might be proficient in one language but it may still not be the best tool for reaching a wider audience
I've been to CCC (and hope to continue doing so again when life ceases to get in the way), and it's never bothered me in the slightest. The Congress is organized by the Chaos Computer Club, founded in Germany and consists primarily of decentralized clubs/associations that themselves are German-speaking. The talks (often) live-dubbed English, and post-talk the talks are translated as well (both the live translation and the post-recording subtitling are done by volunteers, BTW). The majority of congress-goers speak German as a first language, and frankly many of those who don't speak German still attend German talks - thanks to the translators (you can even get a sense of this by watching the talks on the website; there are many instances of English being used for questions and even answers in the Q&A sections of German talks). P
ersonally, I believe the world would be less interesting if everything of interest was in the same language - I think all (major, at least - those with a budget for translators/enough volunteers to translate) conferences should allow the speakers to give their talks in the language they are comfortable in.
Anecdotally, I've never had passable German conversationally, but have studied the language a fair bit, and watching so many German talks with translation (both remotely and in-person) actually passively brought up my understanding of the language that I could understand most of what was being said; to the point that I felt comfortable over the years passively understanding the German language outside of the congress in most situations. Sure, being able to to speak in another language comfortably is ideal, but being able to listen, even just passively, in another language really feels like a superpower.
is Moxie actually sailing derelict sailboats "as far as he can take them"? While I'm admiring this, I'm also worried he's going to be marooned somewhere...
I can't help but feel that this talk was a lot of...fluff?
The synopsis, as far as my tired brain can remember:
- Here's a brief summary of the last 10 years
- We're reaching the limit of our scaling laws, because we've trained on all the data we have available on the limit
- Some things that may be next are "agents", "synthetic data", and improving compute
- Some "ANNs are like biological NNs" rehash that would feel questionable if there was a thesis (which there wasn't? something about how body mass vs. brain mass are positively correlated?)
- 3 questions, the first was something about "hallucinations" and whether a model be able to understand if it is hallucinating? Then something that involved cryptocurrencies, and then a _slightly_ interesting question about multi-hop reasoning
I attended this talk in person and some context is needed. He was invited for the “test of time” talk series. This explains the historical part of the talk. I think his general persona and association with ai led to the fluffy speculation at the end.
I notice with Ilya he wants to talk about these out there speculative topics but defends himself with statements like “I’m not saying when or how just that it will happen” which makes his arguments impossible to address. Stuff like this openly invites the crazies to to interact with him, as seen with the cryptocurrency question at the end.
Right before this was a talk reviewing the impact of GANs that stayed on topic for the conference session throughout.
I mean he repeateadly gave some hints (even if just for the lulz and not seriously) that the audience is at least partially composed of people with little technical background or AI bros. An example is when he mentioned LSTMs and said "many of you may never seen before". Even if he didn't mean it, ironically it ended being spot on when the crypto question came.
As someone who is at NeurIPS right now with a main conference paper, I was shocked at how many NeurIPS attendees had no paper. At ACL conferences, almost every person attending has a paper (even if it's only at a workshop)
NeurIPS is "ruined" by the money and thus attracts huge amounts of people who are all trying to get rich. It's a bloody academic conference people!
A person with little or no technical background, that neither knows nor cares about AI (or other scientific/mathematical advancements) other than their potential to make the AI bro rich. There is a big overlap with crypto bros, and in fact many AI bros are just grifters who moved on after crypto tanked with the recent fed funds rate hikes.
Well, it looks like the entire point was "you can no longer expect a capability gain from a model with a bigger ndim trained on a bigger internet dump".
That's just one sentence, but it's pretty important. And while many people already know this, it's important to hear Sutskever say this. So people know it's a common knowledge.
But they are self-aware, in fact it's impossible to make a good AI assistant which isn't: it has to know that it's an AI assistant, it has to be aware of its capabilities, limitations, etc.
I guess you're interpreting "self-awareness" in some mythical way, like a soul. But in a trivial sense, they are. Perhaps not to same extent as humans: models do not experience time in a continuous way. But given that it can maintain a dialogue (voice mode, etc), it seems to be phenomenologically equivalent.
Would you say that a calculator is also self-aware, and that it must know its limitations so that it doesn't attempt to do calculations it isn't capable of doing, for instance?
Alright. Suppose "meaning" (or "understanding") is something which exists in human head.
It might seem as a complete black box, but we can get some information about it by observing human interactions.
E.g. suppose Алиса does not know English, but she has a bunch of cards with instructions "Turn left", "Bring me an apple", etc. If she shows these cards to Bob and Bob wants to help her, Bob can carry out instructions in a card. If they play this game, the meaning which card induces in Bob's head will be understood by Алиса, thus she will be able to map these cards to meaning in her head.
So there's a way to map meaning which is mediated by language.
Now from math perspective, if we are able to estimate semantic similarity between utterances we might be able to embed them into a latent "semantic" space.
If you accept that the process of LLM training captures some aspects of meaning of the language, you can also see how it leads to some degree of self-awareness. If you believe that meaning cannot be modeled with math then there's no way anyone can convince you.
How does math encode meaning if there is no Alice and Bob? You should quickly realize the absurdity of your argument once you take people out of the equation.
Not sure what you mean... A NN training process can extract semantics from observations. That semantics can be then subsequently applied e.g. to robots. So it doesn't depend on humans beyond production of observations.
The function/mathematics in an NN (neural network) is meaningless unless there is an outside observer to attribute meaning to it. There is no such thing as a meaningful mathematical expression without a conscious observer to give it meaning. Fundamentally there is no objective difference between one instance of a NN with one parameter, f(θ), evaluated on some input, f(θ)(x), and another instance of the same network with a small perturbation of the parameter, f(θ+ε), evaluated on the same input, f(θ+ε)(x), unless a conscious observer perceives the output and attributes meaning to the differences because the arithmetic operations performed by the network are the same in both networks in terms of their objective complexity and energy utilization.
How does the universe encode meaning if there is no Alice and Bob?
One common answer is: it doesn't.
And yet, here we are, creating meaning for ourselves despite being a state of the quantum wave functions for the relevant fermion and boson fields, evolving over time according to a mathematical equation.
(Philosophical question: if the time-evolution of the wave functions couldn't be described by some mathematical equation, what would that imply?)
The universe does not have a finite symbolic description. Whatever meaning you attribute to the symbols has no objective reality beyond how people interpret those symbols. Same is true for the arithmetic performed by neural networks to flash lights on the screen which people interpret as meaningful messages.
> The universe does not have a finite symbolic description
Why do you believe that? Have you mixed up the universe with Gödel's incompleteness theorems?
Your past light cone is finite in current standard models of cosmology, and according to the best available models of quantum mechanics a finite light cone has a finite representation — in a quantised sense, even, with a maximum number of bits, not just a finite number of real-valued dimensions.
Even if the universe outside your past light cone is infinite, that's unobservable.
> Same is true for the arithmetic performed by neural networks to flash lights on the screen which people interpret as meaningful messages.
This statement is fully compatible with the proposition that an artificial neural network itself is capable of attributing meaning in the same way as a biological neural network.
It does not say anything, one way or the other, about what is needed to make a difference between what can and cannot have (or give) meaning.
I don’t understand what you consider self-awareness to be in this trivial sense. Is Eliza self-aware for example? Eliza maintains a dialogue albeit not obviously as coherent as a modern LLM.
From my perspective, we know what the benchmark is for our own self-awareness, because Decartes gave it to us: Cogito ergo sum. I think therefore I am. We know we think, so we know we exist. That is the root of our self-awareness.
There is a great deal of controversy about the question of whether any of the existing models think at all (and of course that’s the whole point of Turing’s amazing paper[1], and the Chinese room thought experiment[2]) and the best you could say is the burden at the moment is on the people who say models can think to prove that.
Given that, I really don’t see how you can say models have self-awareness at the moment. Models may hypothetically be able to convince themselves they are self-aware via Decartes’ method but notice that Decartes’ proof doesn’t work for us - he was able to pull himself up by his own bootstraps because he knew there was a thought so there must be an “I” who was doing the thinking. We have to observe models from the outside and determine whether or not thought is present, and that’s where the concept behind the Chinese room shows how tricky this is.
The only kind of intelligence we know of since childhood is meat-based intelligence. Brain made out of flesh.
So our intuition tells us that flesh is essential. "Chinese room" appeals to this intuition.
But it's a circular argument.
Anyway, I believe they are self-aware, to some extent. Not like a human. But I have no arsenal to convince people who believe intelligence has to be made out of meat.
I certainly don’t think intelligence has to be made out of meat but I do think it’s a complex set of questions and we owe it to ourselves to actually try to tackle the underlying issue of what intelligence and self-awareness etc really are and how we might know whether a model did or did not exhibit these, because we are apt to fool ourselves on both sides of this argument depending on our prior beliefs.
You can see the same thing when people talk about whether animals exhibit self-awareness. There are experiments with dolphins and mirrors for example that definitely suggest that dolphins recognise and might even be amused by their reflection when they see it, but some people find it very hard to reconcile themselves to the idea tgat a dolphin might have a sense of self. I personally find it harder to believe that any particular characteristic would be uniquely human.
Self-awareness is a huge rabbit hole to go down. Its one if the many concepts we think make humans unique, at least in degree, but we never really found a clear way to define or identify it.
I have watched dogs, cats, cows, and chickens pretty extensively. I still couldn't tell you if they are really self-aware, and it ultimately it comes down to a definitional challenge of not having a clear line to draw and identify.
What makes you say LLMs are already self-aware, and how do you define it? And as long as an LLM is functionally a black box, how do you know it comprehends the idea that it is an LLM rather than having simply been trained on that token pattern or given that context as an instruction?
Is it possible to tell intelligence from a simple lookup-table based script by asking questions? I think so.
If you ask one question, there's a chance it was in a lookup table.
If you ask multiple questions from an immense set of questions (like trillions of trillions of trillions..., sampled uniformly), and it answers all correctly, then it's either true intelligence or a lookup table which covers this whole immense space. (I'd argue there's no difference, as a process which makes this nearly-infinite table has to be intelligent.)
Same with self awareness - you can ask questions where it applies...
In either case you are only looking at inputs and outputs. The results may correlate with what you'd consider self-awareness or intelligence, but can you really say that's what you are set ng without understanding how the system works internally.
LLMs are trained on a massive dataset and the resulting model is effectively a compressed representation of that dataset. On the surface you'd have no way of knowing whether the algorithm answering is in fact just a lookup table.
This issue feels very similar to scientific modelling vs controlled studies. Modelling may show correlation, but it will never be able to show causation. Asking a system a bunch of questions and getting the right answers is the same, you're just coming up with a sample set of modelling data and attempting to interpret how the system likely worked only by looking at inputs and outputs.
When I talk with a person I can tell that they are intelligent and self-aware.
Claiming that self-awareness is unknowable concept is inherently unproductive.
People have been using "theory of mind" in practice for millenia so we have to assume it's good for something, otherwise we won't go anywhere. I don't think that knowing internals is important - I don't reach for a scalpel to get what a person means.
When you're talking to a person, though, you also have an understanding of what a human is and what your own experience is.
Its reasonable to interact with another human and expect that they are roughly similar to you, especially when your interactions match what you'd expect.
That doesn't extend as well to other species, let along non-living things that are entirely different from us. They could seem intelligent from the outside but internally function like a lookup table. They also could externally seem like a lookup table while internally matching much better what wed consider intelligence. We don't have context of first hand experience that applies and we don't know what's going on inside the black box.
With all that said, I'm phrasing this way more certain than I mean to. I wouldn't claim to know whether a box is intelligent or not, I'm just trying to point out how hard or impossible it would be today without knowing more about the box.
> Its reasonable to interact with another human and expect that they are roughly similar to you, especially when your interactions match what you'd expect.
It is a default belief that most of us have. The more I learn, the less I think it is true.
Some people have no autobiographical memory, some are aphantasic, others are autistic; motivations can be based on community or individualism, power-seeking, achievements, etc.; some are trapped by fawning into saying yes to things when they want to say no; some are sadistic or masochistic; myself I am unusual for many reasons, including having zero interest in spectator sports and that I will choose to listen to music only rarely.
I have no idea if any AI today (including but not limited to LLMs) are conscious by most of the 40 different meanings of that word, but I do suspect that LLMs are self-aware because when you get two of them talking to each other, they act as if they know they're talking to another thing like themselves.
But that's only "I suspect", not even "I believe" or "I'm confident that", because I am absolutely certain that LLMs are fantastic mimics and thus I may only be seeing a cargo-cult version of self-awareness, a Clever Hans version, something that has the outward appearance but no depth.
> It is a default belief that most of us have. The more I learn, the less I think it is true.
Sure, that's totally reasonable! It all depends on context - I think I'm safe to assume another human is more similar to me than an ant, but that doesn't mean all humans are roughly equivalent in experience. Even more important, then, that we can't assume a machine or an algorithm has developed similar experiences to us simply because they seem to act similarly on the surface.
I'm on the opposite side of the fence as you, I don't think or suspect that any LLMs or ML in general have developed self-awareness. That comes with the same big caveat that its just what I suspect though, and could be totally wrong.
as evidence that GPT-4 can understand Python, based on assumptions:
1. You cannot execute non-trivial programs without understanding computation/programming language
2. It's extremely unlikely that these kind of programs or outputs are available anywhere on the internet - so at very least GPT-4 was able to adapt extremely complex patterns in a way which nobody can comprehend
3. Nobody explicitly coded this, this capability have arisen from SGD-based training process
That's an interesting one, I'll have to think through that a bit more.
Just first thoughts here, but I don't think (2) is off the table. The model wouldn't necessarily have to have been trained on the exact algorithm and outputs. Forcing the model to work a step at a time and show each step may push the model into a spot where it doesn't comprehend the entire algorithm but it has broken the work down to small enough steps that it looks similar enough to python code it was trained on that it can accurately predict the output.
I'm also assuming here that the person posting it didn't try a number of times before GPT got it right, but they could have cherry picked.
More importantly, though, we still have to assume this output would require python comprehension. We can't inspect the model as it works and don't know what is going on internally, it just appears to be a problem hard enough to require comprehension.
2. This was the original ChatGPT, i.e. the GPT3.5 model, pre-GPT4, pre-turbo, etc
3. This capability was present as early as GPT3, just the base model —- you'd prompt it like "<python program> Program Output:" and it would predict the output
The tendency of your type to immediately accuse anyone who acknowledges that they have subjective awareness of themselves and that this is meaningful with "aha, evidently you believe in magical souls" is very telling, I think.
What looks like “self awareness” is baked in during instruction tuning:
Basically turning any input into a document completion task, giving lots of examples where the completion contains phases like “I am an AI assistant”. This way, if GPT-3 would have completed your question with more questions that are similar, “assistants” will complete it with an answer, and one that sounds like it was spoken by someone who claims to be an AI assistant.
Thank you to summarise the video. No trolling here: I am surprised that no one asked an LLM to summarise the video, then post the result here as a comment (with LLM "warning" of course).
For me the questions were a big red flag. Fluff questions about crypto, human rights for AI and then "autocorrect" for AI. Obviously people who ask questions at conference talks are a special type of person but these topics scream to me that there's so many grifters in the AI space right now it might as well drown authentic research.
By now, most of the fundamental contributors are multi-millionaires with cushy contracts. Various labs & departments have their big fat funding for AI research topics. They will be able to spend next 10 years on synthetic data, or "agents", or ensuring that no breasts are in auto-generated images; but somehow it doesn't feel to me like there'll be a lot of fundamental progress.
I would suggest a few things -
1. Less resource intensive site - I’m on my phone, and it took probably 5 seconds to load the front page, and then to actually get to the list, it took a few seconds more. I do not have a current gen phone, but I’d hazard a guess that my phone is about average in age as far as the potential user-base for this site goes! Do not discount mobile users, especially for a site focusing on CLI applications.
2. Maybe I care what language a CLI utility is written in sometimes (Albeit, I can’t recall a time when I have). I’d add a search in the front page, if well thought out. Or, honestly, I’d just have the front page be the list. People are great at lists, and generally quite good at reading, and your list has very well-written descriptions.
3. Kinda related to the above, but I think the categories view is effectively useless. The categories already appear in the description (and if they don’t, they probably should) - so why not just skip straight to the list? You could add a filter for the list if you want. Or, even better - a simple search box!
I’m only giving these criticisms as a means to (subjectively) improve your site, because overall I think it is very well done and thoughtful, without a lot of fluff. There are hundreds more I’d not have criticized, but only because they felt like a waste of time and dead weight. Keep up the good work!
reply