Ability to add more models + Compare outputs

[arnavmehta7]

Proposal

• Compare results of a single prompt among many models
• Ability to add more models especially custom one. Something like a plain simple API request to a hosted endpoint.

Use-Case

New prompt engineers and companies have to iterate and switch regularly between models to test outputs of similar prompts. Instead a single interface can be developed in which new models can be added and their results could be compared in one go.

Is this a feature you are interested in implementing yourself?

No

[arielweinberger]

Thanks for following up @arnavmehta7. As discussed, I'll leave this open to see if the community shows appetite for this feature.

My bet is that we'll see more users interested in this, with AI models popping all the time (including self-hosted ones).