> Date: Sat, 8 Mar 2025 20:31:44 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: van...@sdf.org, emacs-tangents@gnu.org
>
> * Eli Zaretskii <e...@gnu.org> [2025-03-08 13:09]:
> > > Date: Sat, 8 Mar 2025 00:51:23 +0300
> > > From: Jean Louis <bugs@gnu.support>
> > > Cc: emacs-tangents@gnu.org
> > >
>
> > When Large Language Model (LLM) analyzes 130,000 documents it
> > means it is getting the information, thus uses that information. If
> > you call it learning or absorbing, whatever, it took the information,
> > it did not invent new information.
>
> > That is inaccurate. Machine learning does produce information not
> > explicitly included in the learning set.
>
> > You mentioned "new original insight in the data" -- that is matter
> > of viewpoint and consciousness of the observer.
>
> > No, it isn't. The model data includes data generated by analyzing the
> > learning set, and that fits the definition of "new insight".
>
> Let us then ask Microsoft Phi-4, fully free "smart" one:
> https://pageassist.xyz/share/cm80d751k00718c5qil4pwyo0
>
> And look what it says, surprisingly matches my view point:
No, it says that both your view and my view are possible
interpretations ("from a strict data-centric view ... however, from a
functional ...").
> In essence, whether machine learning produces "new information" can
> depend on how one defines newness and insight. From a strict
> data-centric view, all information is derived from the input data.
> However, from a functional or application-oriented perspective, the
> insights and predictions generated by the model can be seen as new
> information, as they provide value and understanding beyond the
> explicit content of the training set.
>
> I didn't ask the LLM before I made my own opinion.
No, you just read half of its answer and ignored the rest.
> You said machine learning produce information not included in the
> learning it. And I asked for example. I can understand people's
> opinions, but it is not an opinion I am looking for, I am genuinely
> interested to find such examples, which are beyond personal biased
> perspective of how it may look like.
If you ever used ML in actual applications, you would have known that
learning generates new information not explicitly present in the
learning set. Claiming that this information was in the data is like
saying that a solution of an equation was in the equation. If you
never had any practical experience with ML, then I understand how you
could be so wrong about this.
> If you have examples, some links, send me.
Sorry, I don't have the time, too busy working on Emacs on my free
time. But on my day job, among other duties, I lead a group of
engineers and algorithm developers who actually develop ML-based
solutions to hard problems, so I know this stuff first-hand.
> > Is theorem proof "innovation"?
>
> It could be, if there is some new mathematical theorem generated by
> the LLM model that doesn't exist in our society as such, then I would
> call that innovation derived from the known rules.
I said "theorem proof", not "theorem generation".
> > Is a decision which alternative out of several to prefer
> > "innovation"?
>
> Not to me.
>
> > All of the above are creative activities that humans are engaged in
> > quite a lot, where LLM can help to a great extent or even replace
> > humans.
>
> There is nothing that LLM creates. It computes.
And you can describe in enough detail how the creative process in a
human brain works, and how it is different from "computation"?
> I understand the excitement as it is like audience sitting and
> watching magician take rabbit out of the hat.
I'm not "excited" about ML, I'm using it to solve practical problems
that are quite hard or even impossible to solve by other methods.
Instead of asking LLMs for answers to the questions, try to actually
reading up on the subject and understanding how LLMs are learning and
what they produce as results of the learning.
---
via emacs-tangents mailing list
(https://lists.gnu.org/mailman/listinfo/emacs-tangents)
