> are ways to transform natural language into structured data
I've seen quite the opposite, because at their core they are next word predictors. Telling it to structure an output biases the context into data that it saw in a structured format. From what I've seen, this always leads to lower quality reasoning/output.
I get much better results if I leave the data unstructured, to maximize reasoning, then add a later step to structure that output.
3. make that output useful to my non-english-speaking software by structuring it.
Forcing step three to happen in step 2 means you're in a very unusual/unrealistic "head space"/context for next word completion.
Leaving structuring as a later step means the human like output/reasoning can be maintained for step 2, with the inhuman like structuring being more of a reformatting than reasoning+reformatting.
Leaving the "context" as small and precise as possible (which might me large for a given task) gets the best output, from what I've seen. Being a next word predictor, this makes sense to me.
I suppose this is just "chain of thought".
This is pretty easy to benchmark. Have it put the answer into json format, after producing the answer. Then, have it come up with an answer in json format, as one step. It's a night and day difference, at least for my tasks.
I've seen quite the opposite, because at their core they are next word predictors. Telling it to structure an output biases the context into data that it saw in a structured format. From what I've seen, this always leads to lower quality reasoning/output.
I get much better results if I leave the data unstructured, to maximize reasoning, then add a later step to structure that output.