On: Prompt Caching

[brylie]

I’m building an app with OpenAI, using structured outputs. Does OpenAI also support prompt caching?

[cma → brylie]

I'm sure internally they use it for the system prompt at least, probably since launch. And maybe for common initial user queries that exactly match.

[Onavo → cma]

They are certainly not passing the savings on to the users.

[minimaxir → Onavo]

Yet. I suspect OpenAI will release a similar offering soon. (hooray, free market competition!)

[HeatrayEnjoyer → Onavo]

That $100 billion data center has to get paid for somehow.

[minimaxir → brylie]

Not currently.