Replies: 2 comments 2 replies
-
Some of my findings are:
The more energy the AI has to spend processing the prompt / writing the response, the less it will have to do the job. Once I get a working prompt, I run experiments trimming it (just like with neural networks) to find how much I can remove without affecting the results.
Dummy example ahead
which can get very costly because it may have many instances
The second example may have longer text on the input (json schema), but it will get N times shorter on the output. As of 2024-03-08, openai output tokens are 3 times the cost of input tokens. But more important than that is the 'employ your energy on the task more than on writing the results' . I'd like to see if someone has found the opposite to be true (maybe it does in some cases). |
Beta Was this translation helpful? Give feedback.
-
One rule I learnt interacting with the LLM is this:
You may think, But it is easy to forget (for me) when creating prompt+output_templates. What happens if you forget the rule? The LLM will respond, with rubbish / overthinking / hallucination As far as I know you cannot include conditions while creating the class.
However you can build the Example:
Which produces
And then, conditionally increase it:
Which produces:
|
Beta Was this translation helpful? Give feedback.
-
There are lots of literature about Prompts, but this is (in my opinion) a slightly different game.
With Instructor, you have the prompt and you have the class definition that you pass to the LLM/AI/openai call.
Both need to be aligned and one needs to consider the extra cost (in tokens) of processing the JSON schema together with generating the output that fulfills that class definition.
I am sure each one has found some tricks/caveats/quirks and sharing them would/may provide some light to others.
Beta Was this translation helpful? Give feedback.
All reactions