bug: with Wasi-nn-ggml plugin: b2715, when I using curl to send a API request, it responses the stop sign of the model

[alabulei1]

Summary

After upgrading the wasi-nn-ggml plugin to b 2715, the API response is not correct.

Example 1: <|eot_id|> from Llama-3-8b

curl -X POST http://localhost:8080/v1/chat/completions \
  -H 'accept:application/json' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system", "content": "You are a helpful, respectful and honest assistant"}, {"role":"user", "content": "Hello"}], "model":"Meta-Llama-3-8B-Instruct-Q5_K_M"}'
{"id":"chatcmpl-910f1f25-66c8-4e7b-89e3-f00c545a6b73","object":"chat.completion","created":1713947997,"model":"Meta-Llama-3-8B-Instruct-Q5_K_M","choices":[{"index":0,"message":{"role":"assistant","content":"I'm here to help with any questions you have. What would you like to know?<|eot_id|>"},"finish_reason":"stop"}],"usage":{"prompt_tokens":622,"completion_tokens":20,"total_tokens":642}}%   

Example 2: </s> from Llama-2-7b and Llama-2-13b

curl -X POST http://localhost:8080/v1/chat/completions \
  -H 'accept:application/json' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system", "content": "You are a helpful, respectful and honest assistant"}, {"role":"user", "content": "Hello"}], "model":"llama-2"}'
{"id":"chatcmpl-129c334b-20b5-4555-91b9-74af02bd447c","object":"chat.completion","created":1713949744,"model":"llama-2","choices":[{"index":0,"message":{"role":"assistant","content":"Hello there! *adjusts glasses* It's a pleasure to make your acquaintance. Is there anything I can help you with or would you like to chat? I'm here to assist you in any way I can, so feel free to ask me anything.</s>"},"finish_reason":"stop"}],"usage":{"prompt_tokens":32,"completion_tokens":59,"total_tokens":91}}

Current State

No response

Expected State

No response

Reproduction steps

See above

Screenshots

Any logs you want to share for showing the specific issue

No response

Components

Others

WasmEdge Version or Commit you used

0.13.5

Operating system information

Ubuntu 22.04

Hardware Architecture

Arm

Compiler flags and options

No response

[hydai]

Is this fixed with latest plugin?