We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have read the document below. https://llama-node.vercel.app/docs/start
So, I looked for a model to use in the source code at the site below. https://huggingface.co/models?search=ggml
Then an error like this appears. What files should I get?
llama.cpp: loading model from model/llama-2-7b-chat.ggmlv3.q2_K.bin error loading model: unrecognized tensor type 10 llama_init_from_file: failed to load model node:internal/process/promises:289 triggerUncaughtException(err, true /* fromPromise */); ^ [Error: Failed to initialize LLama context from file: model/llama-2-7b-chat.ggmlv3.q2_K.bin] { code: 'GenericFailure' }
code:
import { LLM } from "llama-node"; import { LLamaCpp } from "llama-node/dist/llm/llama-cpp.js"; const model = "model/llama-2-7b-chat.ggmlv3.q2_K.bin"; const llama = new LLM(LLamaCpp); const config = { modelPath: model, enableLogging: true, nCtx: 1024, seed: 0, f16Kv: false, logitsAll: false, vocabOnly: false, useMlock: false, embedding: false, useMmap: true, nGpuLayers: 0 }; const template = `How are you?`; const prompt = `A chat between a user and an assistant. USER: ${template} ASSISTANT:`; const run = async () => { await llama.load(config); await llama.createCompletion({ nThreads: 4, nTokPredict: 2048, topK: 40, topP: 0.1, temp: 0.2, repeatPenalty: 1, prompt, }, (response) => { process.stdout.write(response.token); }); } run();
The text was updated successfully, but these errors were encountered:
No branches or pull requests
I have read the document below.
https://llama-node.vercel.app/docs/start
So, I looked for a model to use in the source code at the site below.
https://huggingface.co/models?search=ggml
Then an error like this appears. What files should I get?
code:
The text was updated successfully, but these errors were encountered: