-
-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Small bugs/unexpected behavior and feedback with suggestions #39
Comments
Hi. Thanks for your feedback. |
Thank you for your answer. Yeah, I don’t realize there is custom prompt, so I think it’s enough. But I find something strange behavior. I use this model: https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GGUF And the prompt template is: So i use: <|im_start|>system But it doesn’t work. After I try use double {{text}} and it works. But on the hugging face there is the prompt template only with one {text}. Do your app change something internally? Is there any setting to limit the models answer length? I think I didn’t find any. And one small suggestion. It will be nice to have settings to don’t turn off display while the app generate text. Have a nice weekend! |
And one small suggestion. I tried use model for coding. It use often triple backticks (```) before and after the text. But in your app it doesn’t put the text on block without formatting. And small bug, copy text from the chat window doesn’t copy anything. |
I only want to report, copy generated text still doesn't work in the latest beta. And the text scrolling and you just wait until end if you want to read the beginning. It will be nice, if we can control the maximum text model can generate, like now it's nice it's super long, but maybe sometimes I want to limit it. Because now, when the answer is long, you need to wait longer until you can read it from beginning. |
It's active, but doesn't work. https://github.com/guinmoon/LLMFarm/assets/59890223/389e8603-41d8-4a49-8192-836264e11973 |
Hello Artem. First, I want to thank you for the great app! Exactly what I'm searching for last week! It's cool we can import gguf models and you let us customize some settings!
My device: iPhone 15PM, latest iOS.
For bugs,
I found in the last beta 0.9.0 there is problem with closing keyboard. When you write and model answers, the keyboard doesn't close and you can't even close it with swipe down gesture.
When I create few chats and after delete them with swipe to the left, they appear after go to settings and back. I test it with only one model/chat. Create it, after swipe it to delete and open settings and it will appear back. But when you force close the app, it will be clean without the deleted chat.
And now some suggestion.
It will be nice to have settings to turn off text scrolling, so you can read the beginning of the text when the app still generating.
I think it will be nice to see (or have option to turn on) memory usage on screen when generating, you know to see if we reach maximum device memory.
In the setting template, add more settings, for example ChatML, Alpaca, Vicuna, CodeLlama (I only check few from TheBloke.
Maybe have some description for all the settings(like click small question mark to get what the settings do), you know, for inexperienced users.
The clean chat button can be in the chat interface, for easy access.
And now few suggestions. I get inspiration from other apps (not in Apple Store.
It will be nice when there will be toggle to switch between Instruct mode, story mode, chat mode and maybe some adventure mode. I get lot of inspiration from
https://github.com/LostRuins/koboldcpp
https://github.com/SillyTavern/SillyTavern
It will be nice when we can customize the AI bot answers. You can check for inspiration
https://github.com/KoboldAI/KoboldAI-Client/wiki/Memory,-Author's-Note-and-World-Info
Here you can set these things to teach the model how to speak with you. You can select different scenarios, so he can act like chatbot or other, like I wrote above (instruct, chat, story, adventure). Check the memory, authors note etc. So you can create character like in SillyTavern. It will be super cool and you already provide most of the settings there in your app.
You can check their horde website, look for scenarios under sandwich menu and memory above the chat input. This works great and you can customize the AI for whatever you want!
https://lite.koboldai.net/#
I want to thank you for your app, it’s already great, I like the customization and custom model import!
And sorry for long message.
And if you are curious about image generation, there is IOS/MacOS app called Draw Things, where I’m helping the developer with his discord server. It’s free and have lot of customization, model importing, Lora training, controlNET, infinite canvas etc. I wish there will be app like this for LLM text generation for iOS! And your isn’t so far!
https://apps.apple.com/cz/app/draw-things-ai-generation/id6444050820?l=cs
I'm sorry I don't add labels🙄. Bug, enhancement
The text was updated successfully, but these errors were encountered: