Small bugs/unexpected behavior and feedback with suggestions

[FrostyMisa]

Hello Artem. First, I want to thank you for the great app! Exactly what I'm searching for last week! It's cool we can import gguf models and you let us customize some settings!

My device: iPhone 15PM, latest iOS.

For bugs,

I found in the last beta 0.9.0 there is problem with closing keyboard. When you write and model answers, the keyboard doesn't close and you can't even close it with swipe down gesture.

When I create few chats and after delete them with swipe to the left, they appear after go to settings and back. I test it with only one model/chat. Create it, after swipe it to delete and open settings and it will appear back. But when you force close the app, it will be clean without the deleted chat.

And now some suggestion.

It will be nice to have settings to turn off text scrolling, so you can read the beginning of the text when the app still generating.

I think it will be nice to see (or have option to turn on) memory usage on screen when generating, you know to see if we reach maximum device memory.

In the setting template, add more settings, for example ChatML, Alpaca, Vicuna, CodeLlama (I only check few from TheBloke.

Maybe have some description for all the settings(like click small question mark to get what the settings do), you know, for inexperienced users.

The clean chat button can be in the chat interface, for easy access.

And now few suggestions. I get inspiration from other apps (not in Apple Store.

It will be nice when there will be toggle to switch between Instruct mode, story mode, chat mode and maybe some adventure mode. I get lot of inspiration from
https://github.com/LostRuins/koboldcpp
https://github.com/SillyTavern/SillyTavern

It will be nice when we can customize the AI bot answers. You can check for inspiration
https://github.com/KoboldAI/KoboldAI-Client/wiki/Memory,-Author's-Note-and-World-Info
Here you can set these things to teach the model how to speak with you. You can select different scenarios, so he can act like chatbot or other, like I wrote above (instruct, chat, story, adventure). Check the memory, authors note etc. So you can create character like in SillyTavern. It will be super cool and you already provide most of the settings there in your app.

You can check their horde website, look for scenarios under sandwich menu and memory above the chat input. This works great and you can customize the AI for whatever you want!
https://lite.koboldai.net/#

I want to thank you for your app, it’s already great, I like the customization and custom model import!

And sorry for long message.

And if you are curious about image generation, there is IOS/MacOS app called Draw Things, where I’m helping the developer with his discord server. It’s free and have lot of customization, model importing, Lora training, controlNET, infinite canvas etc. I wish there will be app like this for LLM text generation for iOS! And your isn’t so far!

https://apps.apple.com/cz/app/draw-things-ai-generation/id6444050820?l=cs

I'm sorry I don't add labels🙄. Bug, enhancement

[guinmoon]

Hi. Thanks for your feedback.
I know about the keyboard closing bug, but it was not a quick fix, so I've postponed solving it for later.
I didn't know about the bug with deleting chats. Thank you.
I will try to add the setting to disable scrolling in the new version.
The memory usage is a good idea, I've been planning to add warnings to users that the model may not work on their device for a long time, but I like your idea better.
The button to clear chat will be moved to the toolbox in the new version.
Regarding customization templates, I'm thinking of creating a resource where users can share their templates. Add more templates available out of the box, I think it is pointless.
About your suggestions. Thank you very much for giving me ideas. Hopefully someday I will realize them all in some form.

[FrostyMisa]

Thank you for your answer. Yeah, I don’t realize there is custom prompt, so I think it’s enough. But I find something strange behavior. I use this model:

https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GGUF

And the prompt template is:
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

So i use:

<|im_start|>system
{Please meticulously review the provided text for any grammatical errors or inaccuracies, correcting them with your expertise as a professional English language corrector. Additionally, if you believe there are opportunities to enhance the text for better clarity or expression, feel free to make thoughtful adjustments to improve its overall quality.}<|im_end|>
<|im_start|>You
{prompt}<|im_end|>
<|im_start|>LLM

But it doesn’t work. After I try use double {{text}} and it works. But on the hugging face there is the prompt template only with one {text}. Do your app change something internally?

Is there any setting to limit the models answer length? I think I didn’t find any.

And one small suggestion. It will be nice to have settings to don’t turn off display while the app generate text.

Have a nice weekend!

[FrostyMisa]

And one small suggestion. I tried use model for coding. It use often triple backticks (```) before and after the text. But in your app it doesn’t put the text on block without formatting.

And small bug, copy text from the chat window doesn’t copy anything.

[FrostyMisa]

I only want to report, copy generated text still doesn't work in the latest beta. And the text scrolling and you just wait until end if you want to read the beginning. It will be nice, if we can control the maximum text model can generate, like now it's nice it's super long, but maybe sometimes I want to limit it. Because now, when the answer is long, you need to wait longer until you can read it from beginning.

[guinmoon]

Could you please clarify what you mean by copying? The function itself should be active, selection is not working yet.

[FrostyMisa]

It's active, but doesn't work. https://github.com/guinmoon/LLMFarm/assets/59890223/389e8603-41d8-4a49-8192-836264e11973
And the keyboard still cannot be hidden.