Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Use modelfiles with external API instead of ollama #667

Closed
wants to merge 1 commit into from

Conversation

keriati
Copy link

@keriati keriati commented Feb 6, 2024

With this change a modelfile from the ollama hub can be imported in a way that it uses an external model from openAI compatible API instead of ollama.

A pre requirement is to provide an API key and refresh the page so the models are loaded.

In my testing to achieve this we have to copy an existing model's name to two places:

  • the Tag name
  • the FROM part of the content

This works in my local instance.

I am rather not so sure about this changes as I never touched svelte before.

Probably a checkbox called: use external and a dropdown list to select from external models would be a more complete solution.

Should relate to #665

@keriati keriati marked this pull request as ready for review February 7, 2024 08:47
@explorigin
Copy link
Contributor

I haven't reviewed the frontend code enough to have an opinion on the correctness of this, but I posted some thoughts as to why we should probably do this whole thing differently in the related issue.

@tjbck
Copy link
Contributor

tjbck commented Feb 9, 2024

I agree with @explorigin, we might want to have our own "modelfile" format that's extensible and compatible with other LLM runners. I'll close this for now, let's continue our discussion here: #665

@tjbck tjbck closed this Feb 9, 2024
@spammenotinoz
Copy link

Can we please revisit merging this PR, it works really well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants