[Info] We built a mobile 📱 AND WINDOWS 🪟 interactive version using flutter and it runs super well on a simple Oneplus 7 with 8GB ram.

[ThibautLEAUX]

Have fun

Hi, it is not a real issue, but i want to share here that we made a running app with interactive mode on mobiles using your repo.
You can find our repo here : https://github.com/Bip-Rep/sherpa

Unfortunately it doesnt have your latest commit because there is an error during runtime but we made a fork here https://github.com/Bip-Rep/llama.cpp You need to be on the "for_mobile" branch to build the libraries. We are using an older working commit on this branch.

We translated the main functions of llama.cpp in dart.

Working demo

Click on the image to view the video on YouTube.

Hope this helps.

[anzz1]

Looks really cool ! 👍
This definitely captures the essence of "inference on the edge", what this project is all about. 😄

This would be otherwise moved to a discussion but if there is a runtime error on the newer commits, well that definitely sounds like a issue. Could you elaborate on that?

[ThibautLEAUX]

Thanks !!

We have a hard crash without log.. we are trying to have a starcktrace but we dont have it for now, thats why i did not make a templated issue..
I hope someone finds a way

[gjmulder]

If there is an issue please log an issue with the issue template

[ThibautLEAUX]

As soon as we can have a log well do it, for now the crash doesnt provide a stack trace..

[ThibautLEAUX]

btw, we just released a new version of our flutter code that includes Windows build !!
https://github.com/Bip-Rep/sherpa/releases/tag/2.0