You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks! I don't have any plans currently, as it's not something I can think of a way to benchmark this. Fine-tuning, prompt engineering and quantization all affect the results drastically, and this project only constrains the generation, not affecting the overall quality. I'm not aware of any function calling benchmarks either. If you know of any, I'd love to hear about it.
Any plans on providing benchmarks w/ the top OSS models like Mistral 7b using this as well as benchmarks against fine-tuned models.
The text was updated successfully, but these errors were encountered: