On Thu, 12 Dec 2024 00:53:20 +0000 Andy Smith <a...@strugglers.net> wrote:
> Hi,
>
> On Wed, Dec 11, 2024 at 01:35:12PM +0800, Don Xu wrote:
> > it is enhanced by machine learning. Functions with llm involved will
> > be patched as a seperate module, and pay-respects could run without
> > this llm ai module.
>
> That may be a good idea; am I right in thinking that if this
> "request-ai" module is enabled then all commands and their error message
> outputs are submitted to the upstream author's personal site and then
> presumably proxied to some commercial LLM?
>
>
https://github.com/iffse/pay-respects/blob/main/module-request-ai/src/requests.rs#L68
>
> Unless I have misunderstood, that would probably require quite a robust
> privacy policy for general purpose use.
>
> Thanks,
> Andy
>
>
Hi Andy,
I am the author of `pay-respects`. The `request-ai` module is a fallback
module, so it only gets executed when all other rules failed to make a
suggestion.
Formerly the requests are send directly to a LLM provider with my api
key directly embedded in the source. I know it is not a good idea but I
wanted to give a battery included out-of-the-box experience regardless
of the installation method.
However, the key seems to be used by someone (bots?) for other purposes.
I reached this conclusion as I have received periodical requests with
non-default model and small input tokens (smaller than the prompt I
provided), with only 2 output tokens.
Therefore, the requests are now being sent first to `pay-respects`
server, being filtered to avoid abuse usages like above (as I will be
paying for these usages), and then being forwarded to the LLM provider:
https://github.com/iffse/pay-respects/commit/d7a2dc4e6e2fa880bf0fb8be9017e441f70da9d7
Current LLM provider is GroqClould that does not use data for training.
In any case, if this default behavior is not expected, then one can set
these environment variables (even empty) when compiling, and Rust
compiler will supposedly remove all default values I have included in
the source without the need of a patch.
- `_DEF_PR_AI_API_KEY`
- `_DEF_PR_AI_URL`
- `_DEF_PR_AI_MODEL`
Best regards,
iff.
