On: Llama and ChatGPT Are Not Open-Source

[littlestymaar]

Can we please stop using terminology related to code for something that's not code?

[Havoc → littlestymaar]

Agreed it's annoying, but on a certain level also not entirely wrong. In a software 2.0 [0] world the weights are functionally the code in that it is what gets you from input to output.

Open weights or something similar would be better though

[0] https://karpathy.medium.com/software-2-0-a64152b37c35

[charcircuit → Havoc]

Weights are configuration.

[alanbernstein → charcircuit]

Weights almost entirely encapsulate the behavior of the model. Whatever structural analogy you want to use, they are closest in spirit to the core functional algorithm of traditional software.

[dns_snek → charcircuit]

Are symbols in your source file merely a configuration file for the compiler/interpreter?

[charcircuit → dns_snek]

No, but I would consider the command line flags to be configuration.

[raincole → littlestymaar]

Except it's code. Weight decides how the model works so it's code. Code is just you telling the computer to work.

var a = b * 3 + 2;

You're telling me the * 3 + 2 part isn't code and only var a = b is code?

[choxi → littlestymaar]

There’s always been fuzziness between data and code, it’s kind of the core concept of Turing machines.

[aaur0 → littlestymaar]

Exactly - I wrote something on similar lines here : https://medium.com/@anandandbeyond/rethinking-open-source-fo...

[sashb → littlestymaar]

What is "code" anyways?

https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...

[hprotagonist → littlestymaar]

sometimes it’s hard to see a difference.