On Thursday, November 30th, 2023 at 8:53 PM, Loren Burkholder <computersemiexp...@outlook.com> wrote: > > > Howdy, everyone! > > You are all undoubtedly aware of the buzz around LLMs for the past year. Of > course, there are many opinions on LLMs, ranging from "AI is the > future/endgame for web search or programming or even running your OS" to "AI > should be avoided like the plague because it hallucinates and isn't > fundamentally intelligent" to "AI is evil because it was trained on massive > datasets that were scraped without permission and regurgitates that data > without a license". I personally am of the opinion that while output from > LLMs should be taken with a grain of salt and cross-examined against > trustworthy sources, they can be quite useful for tasks like programming. > > KDE obviously is not out to sell cloud services; that's why going to > https://kde.org doesn't show you a banner "Special offer! Get 1 TB of cloud > storage for $25 per month!" Therefore, I'm not here to talk about hosting a > (paywalled) cloud LLM. However, I do think that it is worthwhile opening > discussion about a KDE-built LLM frontend app for local, self-hosted, or > third-party-hosted models. > > From a technical standpoint, such an app would be fairly easy to implement. > It could rely on Ollama[0] (or llama.cpp[1], although llama.cpp isn't focused > on a server mode) to host the actual LLM; either of those backends support a > wide variety of hardware (including running on CPU; no fancy GPU required), > as well as many open-source LLM models like Llama 2. Additionally, using > Ollama could allow users to easily interact with remote Ollama instances, > making this an appealing path for users who wished to offload LLM work to a > home server or even offload from a laptop to a more powerful desktop. > > From an ideological standpoint, things get a little more nuanced. Does KDE > condone or condemn the abstract concept of an LLM? What about actual models > we have available (i.e. are there no models today that were trained in a way > we view as morally OK)? Should we limit support to open models like Llama 2 > or would we be OK with adding API support for proprietary models like GPT-4? > Should we be joining the mainstream push to put AI into everything or should > we stand apart and let Microsoft have its fun focusing on AI instead of > potentially more useful features? I don't recall seeing any discussion about > this before (at least not here), so I think those are all questions that > should be fairly considered before development on a KDE LLM frontend begins. > > I think it's also worth pointing out that while we can sit behind our screens > and spout out our ideals about AI, there are many users who aren't really > concerned about that and just like having a chatbot that responds in what at > least appears to be an intelligent manner about whatever they ask it. I have > personally made use of AI while programming to help me understand APIs, and > I'm sure that other people here have also had positive experiences with AI > and plan to continue using it. > > I fully understand that by sending this email I will likely be setting off a > firestorm of arguments about the morality of AI, but I'd like to remind > everyone to (obviously) keep it civil. And for the record, if public opinion > comes down in favor of building a client, I will happily assume the > responsibility of kicking off and potentially maintaining development of said > client. > > Cheers, > Loren Burkholder > > P.S. If development of such an app goes through, you can get internet points > by adding support for Stable Diffusion and/or DALL-E :) > > [0]: https://github.com/jmorganca/ollama > [1]: https://github.com/ggerganov/llama.cpp
I am anti-LLM on the grounds that the training sets were created without the original authors' consent. I see no issue with a libre/ethical LLM, if there is one, though. If a developer or team of developers wants to implement a Qt and KDE-integrated LLM app, I have no problem with that, but I believe KDE as an organization should probably steer clear of such a thorny subject. It's sure to upset a lot of users no matter what position is taken. On the other hand, for those people who do make use of AI tools, a native interface would be nice, especially one as feature-ful as you're describing... Regards, Ethan B.
