On: OpenAI: Streaming is now available in the Assistan...

[athyuttamre]

Hi there, I help design the OpenAI APIs. Would you be able to share more?

You can reply here or email me at atty@openai.com.

(Please don't hold back; we would love to hear the pain points so we can fix them.)

[willsmith72 → athyuttamre]

does your team do usability tests on the apis before launching them?

if you got 3-5 developers to try and use one of the sdks to build something, i bet you'd see common trends.

e.g. we recently had to update an assistant with new data everyday and get 1 response, and this is what the engineer came up with. probably it could be improved, but this is really ugly

``` const file = await openai.files.create({ file: fs.createReadStream(fileName), purpose: 'assistants', }) await openai.beta.assistants.update(assistantId, { file_ids: [file.id], })

  const { id: threadId } = await openai.beta.threads.create({
   messages: [
    {
     role: 'user',
     content:
      'Create PostSuggestions from the file. Remember to keep the style fun and engaging, not just regurgitating the headlines. Read the WHOLE article.',
    },
   ],
  })
  const getSuggestions = async (runIdArg: string) => {
   return new Promise<PostSuggestions>(resolve => {
    const checkStatus = async () => {
     const { status, last_error, required_action } = await openai.beta.threads.runs.retrieve(threadId, runIdArg)

     console.log({ status })
     if (status === 'requires_action') {
      if (required_action?.type === 'submit_tool_outputs') {
       required_action?.submit_tool_outputs?.tool_calls?.forEach(async toolOutput => {
        const parsed = PostSuggestions.safeParse(JSON.parse(toolOutput.function.arguments))
        if (parsed.success) {
         await openai.beta.threads.runs.cancel(threadId, runIdArg)
         resolve(parsed.data)
        } else {
         console.error(`failed to parse args from openai to my type (errors=${parsed.error.errors}`)
        }
       })
      } else {
       console.error(`requires_action, but not submit_tool_outputs (type=${required_action?.type})`)
      }
     } else if (status === 'completed') {
      throw new Error(`status is completed, but no data. supposed to go to requires_action`)
     } else if (status === 'failed') {
      throw new Error(`message=${last_error?.message}, code=${last_error?.code}`)
     } else {
      setTimeout(checkStatus, 500)
     }
    }

    checkStatus()
   })
  }
  const { id: runId } = await openai.beta.threads.runs.create(threadId, {
   assistant_id: assistantId,
  })
  console.time('openai create thread')
  const newsSuggestions = await getSuggestions(runId)
  console.timeEnd('openai create thread')

```

[willsmith72 → willsmith72]

just to add to this, it's not helped by the docs. either they don't exist, or the seo isn't working right.

e.g. search term for me "openai assistant service function call node". The first 2 results are community forums, not what i'm looking for. The 3rd is seemingly the official one but doesn't actually answer the question (how to use the assistance service with node and function calling) with an example. The 4th is in python.

https://community.openai.com/t/how-does-function-calling-act...

https://community.openai.com/t/how-assistant-api-function-ca...

https://platform.openai.com/docs/guides/function-calling

https://learn.microsoft.com/en-us/azure/ai-services/openai/h...

[rattray → willsmith72]

I'm sorry for your experience, and thanks very much for sharing the code snippet - that's helpful!

We did indeed code up some sample apps and highlighted this exact concern. We have some helpers planned to make it smoother, which we hope to launch before Assistants GA. For streaming beta, we were focused just on the streaming part of these helpers.

[msp26 → athyuttamre]

Hey, random question.

Is there a technical reason why log probs aren't available when using function calling? It's not a problem, I've already found a workaround. I was just curious haha.

In general I feel like the function calling/tool use is a bit cumbersome and restrictive so I prefer to write the typescript in the functions namespace myself and just use json_mode.

[rattray → msp26]

Have you seen/tried the `.runTools()` helper?

Docs: https://github.com/openai/openai-node?tab=readme-ov-file#aut...

Example: https://github.com/openai/openai-node/blob/bb4bce30ff1bfb06d...

(if what you're fundamentally trying to do is really just get JSON out, then I can see how json_mode is still easier).

[lobsterthief → athyuttamre]

Who can I reach out to for feedback on the web UI? Specifically, the chat.openai.com interface.

Web developer/designer for 24 years so I have a lot of ideas