-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix embedding by adding fixes from llama.cpp upstream #4399
Conversation
I can confirm this PR indeed fixes very obvious issues I had when doing embedding and queries with |
Desperately waiting for this fix to be integrated into next version. Having to stay on 0.1.31 is kind of a pain ... |
@multiduplikator are you on a Mac? If so, it's very easy to download, compile and run this branch on your computer.
|
@fredrik-smedberg Not on a Mac. I still have time to wait ... would like to avoid getting into custom building. But thanks for the input. Will come in handy when the time comes :) |
GJ @deadbeef84 🚀🚀 |
@jmorganca please merge this! Thanks 😁😁 |
@jmorganca Hi, I can confirm this is important fix for any application running in prod. |
I'd also love to see this get merged assuming there aren't any major conflicts or regressions. Currently, using ollama for embeddings is a nonstarter since the results are unpredictable/inaccurate, so I'm using the sentence transformers python package as a workaround. |
@deadbeef84 thank you for figuring this out, using ollama for embeddings broke for me at some point and it had been driving me totally nuts trying to figure out what was going on... cc: @jmorganca |
Thanks all, reviewing this weekend and will make sure we get this fixed. |
I will rebase your commit here @deadbeef84: #4941 Thank you for fixing this |
Embedding appears broken since v0.1.32
See #3777 #4207 for details.
This PR applies fixes based on ggerganov/llama.cpp@1b67731#diff-87355a1a297a9f0fdc86af5e2a59cae153290f58d68822cd10c30fee4f7f7076.
I've tested it and embedding vectors looks correct after applying this patch.