Thanks, I'm glad you like it! That makes me very happy!
You're correct, I'm not caching the results right now. I determined that caching whole queries would not make much difference in the aggregate, since the vast majority of queries are unique. (However I also just saw that OpenAI added their own caching layer with lower prices for cached results, which is nice!)
However - the new Replace function was my first step toward fetching recommendations one-at-a-time - I agree that potentially opens up interesting new possibilities for caching and other things as well!
You're correct, I'm not caching the results right now. I determined that caching whole queries would not make much difference in the aggregate, since the vast majority of queries are unique. (However I also just saw that OpenAI added their own caching layer with lower prices for cached results, which is nice!)
However - the new Replace function was my first step toward fetching recommendations one-at-a-time - I agree that potentially opens up interesting new possibilities for caching and other things as well!