This kind of A/B testing, or A/B/C/D.. testing is going to burn tokens. It may be justified in some regulated domains where you need to show consistency.
The ideology is that even if you express your intent well, the LLM underneath may not produce the best output with your input. This is because (it is said) "being clear" is not the same as "being optimal" for a generative model.
AutoPrompt is one option for what it refers to as "prompt tuning".
DSpy (a Python framework for building AI systems) has also been used for the same use case.
A video on DSpy can be found here.
No comments:
Post a Comment