New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
llama3 template #2291
Comments
I tested out newer Llama 3s made with the latest llama.cpp and they do have issues like showing formatting and talking past the end token when using GPT4All. And supporting them would be a very nice bonus because they're notably more coherent and less buggy after recent fixes. For example, they can solve 3333+777, rather than respond with 33 + 77 = 101. This is the answer GPT4All v2.7.4 with the including L38 Instruct Q4_0 gives. "Let me calculate the sum for you... 33 + 33 = 66 So, the answer is: 143. Is there anything else I can help you with?" |
these issues have been fixed in llama.cpp but the lama.cpp fork of gpt4all has not been updated so far. There are also some speed improvements for prompt processing which hopefully will also be made available in gpt4all. |
@Phil209 about formatting issues, have you encountered the following problem:
|
@agilebean No, I've never seen anything like "ERROR: byte not found in vocab:" before. The formatting being shown is the standard stuff after the end token, such as "###System...", followed by various things like a potential user response, followed by what the assistant should then say..., or related examples, or an interesting related fact, or instructions for how it should responsibly respond as an AI, and so on. |
ok all is working fine
if i dowload your llama3 model the prompt template is ok.
is it possible to handle all models that have in there names: "llama-3" or "llama3" or "llama 3" that these prompt is ready to use ?
The text was updated successfully, but these errors were encountered: