Christian Kastner <c...@debian.org> writes: > Hi, > > On 2025-01-31 17:33, M. Zhou wrote: >> @ckk is planning to package llama.cpp within debian deep learning >> team (debian-ai@l.d.o). Maybe you want to discuss with the team >> whether you want to deal with the embedded copy of llama.cpp inside >> ollama source tree? > > llama.cpp will land this weekend. I'm almost finished; I still need to > tweak some corner cases. But I also need to run some final tests. > > Funnily enough, this package is trickier than one might initially > believe. The library needs to be private; multiple versions need to be > build (one or more for each backend); the library abstraction isn't that > strict; and it requires non-free data to properly test and benchmark. > > I've solved most of these problems, so perhaps my approach can help once > it's done.
Hello. That is really good to know! Do you have any general thoughts around packaging ollama? I started looking at the llama.cpp code that is inside ollama and I have many concerns. Just writing the debian/copyright file didn't feel fun. Ollama intentionally patch upstream llama before use, see: https://salsa.debian.org/jas/ollama/-/blob/debian/sid/llama/README.md?ref_type=heads#vendoring https://salsa.debian.org/jas/ollama/-/tree/debian/sid/llama/patches?ref_type=heads I didn't review any of them yet. Any thoughts? Could some be upstreamed, or made tunable somehow so that ollama could get what it wants from llama.cpp but not disturb other users? Do you think it is possible for ollama to use a llama.cpp packaged outside of ollama? The entire package is complex and I haven't familiarized myself with it. My perception is that it builds llama.cpp in a way that fits ollama and integrate it into the ollama Go build. Finding some way to revert or parametrize that integration would be nice, so that ollama could use the Debian-packaged llama.cpp code. But I'm not sure how to do that, given the patches above and generally how this is all integrated together. And different release schedules. I am very happy if you or others from the debian-ai team can help on ollama, or join the debian-ai to maintain it there. I'm certainly no llama expert and the more I look into this package, the more I think such knowledge will be useful. I put it into the Go team because that's what I'm familiar with, and because ollama is written in Go. But the llama relationship has to be resolved somehow. /Simon
signature.asc
Description: PGP signature
