-
Notifications
You must be signed in to change notification settings - Fork 715
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bug: [WASI-NN] RPC client is not implemented for LoadByNameWithConfig
#3288
Comments
It is weird that even e78fedf (Jan 9, Looks like second-state/WasmEdge-WASINN-examples@f3b00f6 (Nov 10) seems the last version to work. second-state/WasmEdge-WASINN-examples@476417c (Nov 10) does not. |
Hi @AkihiroSuda We extended the WASI-NN spec with a new function called So, we now have two ways to fix this issue:
cc @dm4, WDYT? cc @q82419, let's hold the 0.14.0 release until we can reach a consensus on the NNRPC. |
I believe we should implement the |
Let's do this first. Let's ensure the previous behavior works. |
I have added a new example, wasmedge-ggml/nnrpc, which avoids using Additionally, in #3296, I implemented the |
PR #3296 is merge. This issue should be fixed after the next release. |
Thank you @dm4 ! |
@q82419 |
Summary
WASI-NN RPC currently doesn't work due to
[WASI-NN] RPC client is not implemented for LoadByNameWithConfig
.Something seems to have changed since:
Current State
Seems currently broken:
Expected State
It should work
Reproduction steps
WASMEDGE_BUILD_WASI_NN_RPC=ON
/opt/wasmedge/bin/wasi_nn_rpcserver --nn-rpc-uri=unix:///tmp/wasi_nn_rpc.sock --nn-preload default:GGML:AUTO:llama-2-7b-chat.Q5_K_M.gguf
/opt/wasmedge/bin/wasmedge --nn-rpc-uri unix:///tmp/wasi_nn_rpc.sock ~/gopath/src/github.com/second-state/WasmEdge-WASINN-examples/wasmedge-ggml/llama/wasmedge-ggml-llama.wasm default
(with https://github.com/second-state/WasmEdge-WASINN-examples/tree/c828a39783e2bd1389dfc0ec3ff31b56e7fc5b41 )
Screenshots
Any logs you want to share for showing the specific issue
No response
Components
CLI
WasmEdge Version or Commit you used
0.14.0-rc.1
Operating system information
macOS 14.4
Hardware Architecture
x86_64
Compiler flags and options
Homebrew clang version 17.0.6
The text was updated successfully, but these errors were encountered: