Hi Christian and Petter,
On Sat, 9 Mar 2024 10:20:32 +0100 Christian Kastner <c...@debian.org> wrote:
> I've discarded the simple package and now plan another approach: a
> package that ships a helper to rebuild the utility when needed, similar
> to DKMS. Rationale:
> * Continuously developed upstream, no build suited for stable
> * Build optimized for the current host's hardware, which is a key
> feature. Building for our amd64 ISA standard would be absurd.
> I'm open for better ideas, though.
Perhaps we are letting the perfect be the enemy of the good?
There are lots of fast-moving projects that get frozen at some version
for stable. While that can be annoying for maintenance, it is also
something that provides value. It's hard to build on top of something
that keeps changing.
I would also argue that you're taking on too much responsibility trying
to enable -march=native optimizations. It's true that you can get
significantly more performance using AVX instructions available on most
modern computers, but if llama.cpp really wanted they could implement
dynamic dispatch themselves. The CPU instruction set is also irrelevant
for the GPU-accelerated version of the package.
Why not deliver the basics before we try to do something fancy? In the
time that passed between the creation of this issue and now, Fedora
created their own llama.cpp package [1]. I think they had the right
idea. There's value in providing a working package to users today, even
if it's imperfect.
Sincerely,
Cory Bloor
[1]: https://packages.fedoraproject.org/pkgs/llama-cpp/llama-cpp/
