On: GPT Best Practices

[hammock]

All of these best practices are great for managers dealing with their staff as well:

1. Write clear instructions

2. Provide reference text

3. Split complex tasks into simpler subtasks

4. Give time to "think"

5. Use external tools

6. Test changes systematically

[f0e4c2f7 → hammock]

This is a good observation. I find that working with LLMs feels closer to the skills of managing a team than to coding itself. Intuitions about how to divide work and understanding strengths and limitations seem to go far.

[archon → f0e4c2f7]

I saw a web comic the other day that I think was right on the nose.

Something along the lines of:

"AI will eat all of the developer jobs!!!"

"Nah. AI expects exact, well-reasoned requirements from management? We're safe."

[valgaze → f0e4c2f7]

That's a really cool insight-- it's not coding. It's dispatching tasks to a team

[Tostino → valgaze]

I could see a Jira plugin that does this by looking through all people working issues and figuring out how would be best to handle this by looking at prior tasks completed, and notes associated with them, along with workload among the team.

[hammock → f0e4c2f7]

After all, GPT is a non deterministic function that requests and returns human output (albeit second-order human output).

Far from a deterministic programming function

[baobabKoodaa → hammock]

It's deterministic. If you use the same seed or set temperature to 0, you get reproducible results.

[acover → baobabKoodaa]

Does temperature 0 guarantee reproducible results?

[baobabKoodaa → acover]

"Guarantee" is a strong word, but I'm leaning towards a "yes".

[fnordpiglet → hammock]

Revision for generality:

All of these best practices are great for humans dealing with their humans as well:

[tikkun → hammock]

Yes, it goes in the other direction too. A few books that I've read about delegation have been quite helpful for prompt writing.

[justanotheratom → hammock]

RE: Give time to "think"

"Transformers need tokens to think" - @karpathy on Chain of Thought prompting.

[tomduncalf → justanotheratom]

Yeah I liked the way he explained this in his State of GPT talk (even if it might not be 100% literally accurate), that each token has an equal amount of “computation” behind it so if you want to do something more computationally complex, letting it use more tokens (“show your working” etc) yields better results as it can “do more computation” in a sense