Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Engine not instantiating for WebWorker #394

Open
kitzj opened this issue May 13, 2024 · 0 comments
Open

Engine not instantiating for WebWorker #394

kitzj opened this issue May 13, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@kitzj
Copy link

kitzj commented May 13, 2024

I got web-llm successfully installed in Rails no problem, and porting the "get-started" example into my Rails app works perfectly. However, when I try to emulate the "get-started-web-worker" example, the engine fails to instantiate. (both I just removed types to make it javascript)

The thing is the "get-started-web-worker" example works correctly out of the box using the official example, so it's strange that the exact same code in Rails only works for the non-web-worker.

As you can see in the "all_logs" screenshot, the "Before G" with the WebWorkerEngine gets logged. The other log is "End of promise" indicating that getPromise successfully (?) returns B, but for some reason the log after yield this.getPromise(g) ("End of promise") never logs. Which seems weird if getPromise seems to run fine.

It does seem strange that there isn't any error that was returned. The only thing I could see as a potential issue is within the WebWorkerEngine (see "pending_promise.png" screenshot) where the value of the caller gets logged as a TypeError (below). This is also strange to see as Rails compiled the web-llm package as a .js file so I think there shouldn't be any type errors.

TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them
at Function.invokeGetter (:3:28)

Any ideas what the issue might be here?

Here is the respective code:
`// Worker.js
import { EngineWorkerHandler, Engine } from "@mlc-ai/web-llm";

// Hookup an engine to a worker handler
const engine = new Engine();
const handler = new EngineWorkerHandler(engine);
self.onmessage = (msg) => {
handler.onmessage(msg);
};

// Main.js
const engine = await webllm.CreateWebWorkerEngine(
/worker=/ new Worker(new URL("./worker.js", import.meta.url), {
type: "module",
}),
/modelId=/ selectedModel,
/engineConfig=/ { initProgressCallback: initProgressCallback }
);

// @mlc-ai--web-llm.js
/ function CreateWebWorkerEngine(A, Q, B) {
return __awaiter(this, void 0, void 0, function
() {
const g = new WebWorkerEngine(A);

g.setInitProgressCallback(
  B === null || B === void 0 ? void 0 : B.initProgressCallback
);

console.log("BEFORE G", g);

yield g.reload(
  Q,
  B === null || B === void 0 ? void 0 : B.chatOpts,
  B === null || B === void 0 ? void 0 : B.appConfig
);

return g;

});
}

getPromise(A) {
const Q = A.uuid;
const executor = (A, B) => {
const cb = (Q) => {
Q.kind == "return"
? A(Q.content)
: Q.kind != "throw"
? B("Uknown msg kind " + Q.kind)
: B(Q.content);
};
this.pendingPromise.set(Q, cb);
};
const B = new Promise(executor);
this.worker.postMessage(A);
console.log("End of promise");
return B;
}
reload(A, Q, B) {
return __awaiter(this, void 0, void 0, function* () {
const g = {
kind: "reload",
uuid: crypto.randomUUID(),
content: { modelId: A, chatOpts: Q, appConfig: B },
};
yield this.getPromise(g);
console.log("End of reload");
});
}`

pending_promise
all_logs

@kitzj kitzj changed the title WebWorker not working Engine not instantiating for WebWorker May 13, 2024
@Neet-Nestor Neet-Nestor added the bug Something isn't working label May 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants