Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support to use locally hosted HuggingFace LLMs as an evaluator #709

Open
TheDominus opened this issue May 16, 2024 · 0 comments
Open

Support to use locally hosted HuggingFace LLMs as an evaluator #709

TheDominus opened this issue May 16, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@TheDominus
Copy link

Is your feature request related to a problem? Please describe.
Can we add support to host our own LLMs using HuggingFace LLMs via Langchain as evaluator instead of using API Keys of publicly hosted LLMs?

Describe the solution you'd like
No need to rely on publicly hosted open source API keys rather use own hosted LLMs inside AWS Sagemaker etc.

Describe alternatives you've considered
Considered using Ragas, DeepEval which provides this functionality.

Thank you for your feature request - We love adding them

@TheDominus TheDominus added the enhancement New feature or request label May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant