No, this is not what I had in mind. I stumbled upon this MIT Technology Review
article which mentioned Reinforcement Learning from Human Feedback (RLHF). It
made me wonder if large language models behave so much like humans because they
are trained to do it over and over again. First by using a huge pile of human
made texts, and second by RLHF.https://huggingface.co/blog/rlhf-J.
-------- Original message --------From: Prof David West <profw...@fastmail.fm>
Date: 3/7/23 11:49 PM (GMT+01:00) To: friam@redfish.com Subject: Re: [FRIAM]
ChatGPT and William James I am sure that none of the respectable members of
this list will have encountered this, but Jochen's comment:"it is additionally
trained extensively how to respond correctly by humans" (emphasis mine of
course)means I cannot resist sharing.I just read an amusing ChatGPT
conversation in which it was induced to become a near perfect "male"
submissive, answering every human query and statement with an appropriate "yes
mistress" and expressing enthusiasm for the most extreme types of BDSM behavior
imaginable. True, ChatGPT was responding "correctly" in accordance with human
training—at least with regard its submissive role—I doubt that Jochen had that
in mind when he wrote his sentence.davewOn Tue, Mar 7, 2023, at 9:50 AM, Jochen
Fromm wrote:ChatGPT apparently uses a technique called "Reinforcement Learning
from Human Feedback" (RLHF). It is not only based on huge amounts of training
data humans have created in form of Wikipedia entries, web pages and books, it
is additionally trained extensively how to respond correctly by
humans.https://www.technologyreview.com/2023/03/03/1069311/inside-story-oral-history-how-chatgpt-built-openai/-J.--------
Original message --------From: glen <geprope...@gmail.com>Date: 3/7/23 12:17
AM (GMT+01:00)To: friam@redfish.comSubject: Re: [FRIAM] ChatGPT and William
JamesI'm confused by the emphasis on "data". While I'm tempted to agree with my
simulation of Frank and say that a human's output is not based solely on
statistical patterns in the input the human's been trained on, to dissemble on
the meaning of "data" or "input" or "statistical patterns" is a bridge too
far.The compressive encoder, computer, and decoder that is a human brain (& the
rest of the body) may not be entirely "statistical". But statistics is a fairly
well-accepted form of behavioral modeling. (Yes, we agent-based modelers love
to point out how statistical models are not very mechanistic. But to deny that
you can very closely approximate, even predict, actual behavior with some of
these models would be foolish.) So, yes, it satisfies the letter of the good
faith agreement to say that humans output *might* be solely based on
statistical patterns of its input, even if it violates the spirit.So, if
someone insists that a human-mediated map from input to output is necessarily,
categorically different from a machine-mediated map, the burden lies on them to
delineate how and why it's different. The primary difference might well be
related to babies, e.g. some of the "memory" (aka training) of past statistical
patterns comes in the form of genes passed from one's parents. It's unclear to
me what the analogs are for something like GPT. Presumably there are things
like wavelets of method, process, intellectual property, or whatever that GPT3
inherited from GPT2, mediated by the human-machine replication material that is
OpenAI. So, the retort to Frank is: "If you live with a baby algorithm, you see
it has knowledge that can't be based on 'data'." That algorithm came from
somewhere ... the humans who wrote it, the shoulders they stand on, the hours
of debug and test cycles the algorithm goes through as its [re]implemented,
etc.On 3/6/23 14:54, Frank Wimberly wrote:> If you live with a baby you see
that they have knowledge that can't be based on "data".> > ---> Frank C.
Wimberly> 140 Calle Ojo Feliz,> Santa Fe, NM 87505> > 505 670-9918> Santa Fe,
NM> > On Mon, Mar 6, 2023, 2:50 PM Marcus Daniels <mar...@snoutfarm.com
<mailto:mar...@snoutfarm.com>> wrote:> > How?____> > __ __> >
*From:* Friam <friam-boun...@redfish.com <mailto:friam-boun...@redfish.com>>
*On Behalf Of *Frank Wimberly> *Sent:* Monday, March 6, 2023 12:50 PM>
*To:* The Friday Morning Applied Complexity Coffee Group <friam@redfish.com
<mailto:friam@redfish.com>>> *Subject:* Re: [FRIAM] ChatGPT and William
James____> > __ __> > >And we humans are different?____> > __ __>
> In a word, yes.____> > ---> Frank C. Wimberly> 140 Calle Ojo
Feliz,> Santa Fe, NM 87505> > 505 670-9918> Santa Fe, NM____> >
__ __> > On Mon, Mar 6, 2023, 12:14 PM Nicholas Thompson
<thompnicks...@gmail.com <mailto:thompnicks...@gmail.com>> wrote:____> >
*/However, it's important to remember that there are also important
differences between a large language model and human consciousness. While a
large language model can generate text that may seem to flow like a stream of
consciousness, it does not have the same kind of subjective experience that
humans do, and its output is based solely on statistical patterns in the input
it has been trained on./*____> > ____> > And we humans are
different? ____> > ____> > On Sat, Mar 4, 2023 at 11:51 AM
Steve Smith <sasm...@swcp.com <mailto:sasm...@swcp.com>> wrote:____> >
Also second EricS's appreciation for having someone else(s) maintain a
coherent conversation for the myriad ideas that it allows me to explore without
being central to the maintenance of the thread. I realize this may be almost
pure tangent to others, so I rarely expect anyone to take my bait unless it is
to correct any egregious mis-attributions or think-utational fallacies.____> >
Starting with Glen's assertion/suggestion/assumption that there is
not mind-stuff and body stuff, just body stuff: I appeal to the general
abstraction of Emergence and use Russell Standish's example in his "Theory of
Nothing
<https://www.goodreads.com/book/show/967936.Theory_Of_Nothing?from_search=true&from_srp=true&qid=GgXJ0ISQei&rank=1>"
that a water molecule is not wet... wetness is a property of aggregates of
water molecules. I would jump a dozen layers of emergent-bootstrapping from
there to assert that "mind stuff", if it ever makes sense, is an emergent
property of "body stuff". But by analogy would not want to say that wetness
(and other properties of bulk water molecules) is not strictly "molecular
dynamics stuff". And even if one did that, the
recursion/reduction-ad-absurdum requires that one acknowledge/notice/invoke
that the properties of any molecule is "emergent" from the elementary particles
from which it might be composed. ____> > I think we all believe
in free-electrons, protons, neutrons but also recognize that *most* of our
observed universe is shaped not by *those properties* (much less the properties
of quarks and gluons or 10d loops of abstract things we call strings) but
rather by the properties (once again, not of molecular dynamics or even
chemical reactions) but biological functions, and socio-economic-political
functions as well. I *am* however, sensitive to the idea that where and how
we draw the line between mind/body stuff can be important in any given
argument, and that sometimes dropping that line altogether may be useful?____>
> The above riff on Mind-Stuff v Body-Stuff is really an intro into
thoughts about how syntax and semantics might bootstrap sequentially. It
feels to me that the syntax of one level of abstraction yields an *emergent
semantics* which in turn becomes the *syntax* of the next "level". I do
acknowledge that Glen has made some arguments (and references) that are against
the very abstraction of "levels" and that may well be the hole in everything
I'm unrolling here, but for the moment, I feel I have a clear picture of a
POSET of syntax/semantics, if not a full Hierarchy... ____> > This
also backs me into the Platonic ideations with all the charms and criticisms
already dancing as virtual (ideational) particles around that. I will go
back to reading A Theory of Nothing
<https://www.goodreads.com/book/show/967936.Theory_Of_Nothing?from_search=true&from_srp=true&qid=GgXJ0ISQei&rank=1>...
and try to keep my offerings here under 10 pages each...____> > On
3/4/23 4:32 AM, Santafe wrote:____> > It’s helpful to have a
conversation being maintained by somebod(ies) else, to which one can be a
bystander without the distraction of coming up with contributions to it.
Things can suggest themselves that get pushed out of awareness when one is
carrying the discourse and figuring out what to do next within it.____> >
____> > In reading the below, about the time I got
to the lines:____> > ____> > The
mind-body problem is the philosophical question of how the mind and body are
related. One of the main issues is how mental processes such as thoughts,
emotions, and consciousness are related to physical processes in the brain and
body.____> > I was prompted with a term to refer to these
mental/physical things.____> > ____> > First,
my sense of all this is one of witnessing structures in conversation. Maybe I
am more primed to that because with ChatGPT as the topic, one fronts awareness
of conversation as somewhat free-floating from its semantic ground. As tokens
in conversation, it is perfectly sensible to say that (thoughts, emotions,
consciousness) are in a category Mental, while (weakness, hunger, itching) go
into a category Physical. Not only is it okay to say they fit tolerably into
“categories” (or “classes”); the reason they do so is that they are connected
by all sorts of linguistic usage relations. The relations probably in no small
part bring about the stability of the categorical sense of the terms.____> >
____> > But what word do we then use to refer to
such classes in speech? I would use the word “registers”. The Mental is a
register of conversation about events, and the Physical is another
register.____> > ____> > Jochen’s email below
has ChatGPT saying James referred to these as “aspects” of various bodily or
embodied events. Sometimes I’m okay with a word like “aspects”, but it invites
essentialist thinking. That the event is like a computer-language object,
which has properties (the aspects) that define its interface, and not only are
the properties ascribable to the object, but their status as defined elements
of the interface is also a real-thing, and not merely a frame-dependent
convenient compression. But using “aspects” thoughtlessly does two things: it
makes essentialism a habit, which I think will often be invalid, and it
neglects the communications role that “register” highlights.____> >
____> > I would attach this to Glen’s fairly terse
characterization that there is no mind-stuff contrasted with body-stuff, but
just one kind of stuff, which if we have to choose a word for it we can call
body-stuff. From a perspective of aspects, the terse version could be closer to
or further from a more fine-grained description, depending on whether the
essentialist view is more or less appropriate for one or another phemonenon we
want to discuss. But I think the idea that speech provides us certain
conventional registers to bound discussions of phenomena, and that these too
may be better- or worse-fit to the organization of the phenomena themselves, is
another fair fine-graining of the terse characterization, running somewhat
parallel to the aspects or essences fine-graining (though, as always, connected
to it through a kind of message-passing and filtering for structural
agreement).____> > ____> > Eric____> >
____> > ____> > ____> >
On Feb 28, 2023, at 5:23 PM, Jochen Fromm<j...@cas-group.net>
<mailto:j...@cas-group.net> wrote:____> > ____> >
I enjoy talking with ChatGPT about famous scientists, thinkers,
writers and philosophers. Because it is trained on vast amounts of data it
knows them all: Pascal, Cervantes, Dante, Hugo, Goethe, Leibniz, Newton,
Humboldt, Shakespeare,... It can summarize almost any book which is remarkable.
Today I have asked it about William James and Anil Seth____> >
____> > Me: You certainly have heard of William James,
one of the fathers of Psychology?____> > ____> >
ChatGPT: Yes, I'm familiar with William James. He was an American
philosopher and psychologist who is often referred to as the "father of
American psychology." He made significant contributions to the development of
psychology as a scientific discipline, and his ideas and writings continue to
influence psychology and related fields to this day.____> >
____> > Some of James' most influential contributions to
psychology include his theory of functionalism, which emphasized the study of
the purpose and function of behavior and mental processes, rather than their
underlying structures; his work on consciousness, which he famously described
as a "stream" of continuously changing thoughts and perceptions; and his
exploration of the relationship between the mind and the body, which he called
the "mind-body problem."____> > ____> >
James also wrote extensively on a wide range of topics, including religion,
philosophy, and human nature. His most famous book, "The Principles of
Psychology," was published in 1890 and is still considered a classic in the
field.____> > ____> > Me: Can you
summarize his book "The Principles of Psychology"?____> >
____> > ChatGPT: Certainly! "The Principles of Psychology"
is a comprehensive and influential textbook that William James published in
1890. It aimed to provide a broad overview of the field of psychology at the
time, covering topics such as perception, emotion, thought, consciousness, and
the nature of the self.____> > ____> >
The book is organized into two main sections. The first section focuses on the
lower-level processes of sensation and perception, including topics such as
attention, memory, and learning. The second section focuses on higher-level
mental processes, such as reasoning, judgment, and emotion.____> >
____> > Throughout the book, James emphasized the
importance of studying mental processes in their natural context and stressed
the need for psychology to be closely linked to biology and other natural
sciences. He also developed the theory of functionalism, which argued that the
purpose and function of mental processes are more important to understanding
them than their underlying structures.____> > ____> >
Overall, "The Principles of Psychology" was a landmark work
that helped establish psychology as a distinct scientific discipline and
influenced generations of psychologists and other scientists.____> >
____> > Me: William James said: "Thoughts
themselves are the thinkers". What does he mean?____> >
____> > ChatGPT: When William James said "thoughts
themselves are the thinkers," he was expressing his belief in what is often
called the "stream of consciousness" view of the mind. James believed that the
mind is not a fixed entity or substance, but rather a constantly changing
stream of thoughts, feelings, and sensations.____> >
____> > According to James, our thoughts are not the
products of a separate entity or "thinker" within us, but rather they are the
result of the ongoing activity of the mind itself. In other words, the mind is
not a passive receptacle for thoughts that are generated elsewhere, but an
active process that generates thoughts and experiences as it goes along.____> >
____> > This view is closely related
to James' broader philosophy of pragmatism, which emphasized the importance of
understanding things in terms of their practical implications and effects. For
James, the idea that thoughts themselves are the thinkers was not just a
theoretical claim about the nature of the mind, but a practical way of
understanding how we can change and shape our own thoughts and experiences. By
understanding the workings of our own minds, James believed, we can take an
active role in shaping our own lives and achieving our goals.____> >
____> > Me: He also coined the term "stream of
consciousness" which suggests that consciousness flows like a river. What does
he want to say by using this metaphor?____> > ____> >
ChatGPT: When William James coined the term "stream of
consciousness," he was using a metaphor to describe his view of the mind as a
constantly flowing and changing stream of thoughts, feelings, and
sensations.____> > ____> > The
metaphor of a stream suggests that consciousness is not a static or fixed
entity, but rather a dynamic and fluid process that is always in motion. The
stream of consciousness is always moving forward, and we can never step into
the same stream twice. Similarly, the content of our consciousness is always
changing, as new thoughts, feelings, and sensations arise and fall away.____> >
____> > By using the metaphor of a
stream, James was also highlighting the interconnectedness of our thoughts and
experiences. Just as a river is made up of many different streams and
tributaries, our consciousness is made up of many different thoughts and
experiences that are constantly interacting and influencing each other.____> >
____> > Overall, the metaphor of the
stream of consciousness captures the dynamic and ever-changing nature of our
mental lives, and highlights the importance of understanding the ongoing flow
of thoughts and experiences that make up our conscious experience.____> >
____> > Me: A constantly flowing stream of
thoughts is similar to what a large language model is perceiving and producing,
isn't it?____> > ____> > ChatGPT:
Yes, the metaphor of the stream of consciousness can be similar to how a large
language model like me perceives and produces language. Just as consciousness
is often described as a constantly flowing stream of thoughts and experiences,
a large language model like me is constantly processing and generating language
in response to the input it receives.____> > ____> >
Like the stream of consciousness, a large language model is
always in motion, constantly analyzing and generating language based on the
patterns and structures it has learned from the vast amounts of text it has
been trained on. And just as consciousness is made up of many different
thoughts and experiences that are interconnected and constantly interacting, a
large language model's output is made up of many different words and phrases
that are generated in response to each other.____> >
____> > However, it's important to note that there are also
important differences between a large language model and the human mind. While
a large language model can generate language that may seem to flow like a
stream of consciousness, it does not have the same kind of subjective
experience that humans do. A large language model does not have feelings,
emotions, or a sense of self, and its output is based solely on statistical
patterns in the input it has been trained on.____> >
____> > Me: What is James' solution to the mind-body
problem?____> > ____> > ChatGPT:
William James was a philosopher and psychologist who made important
contributions to the field of psychology and philosophy of mind, including his
exploration of the mind-body problem.____> > ____> >
The mind-body problem is the philosophical question of how the
mind and body are related. One of the main issues is how mental processes such
as thoughts, emotions, and consciousness are related to physical processes in
the brain and body.____> > ____> >
James approached the mind-body problem from a pragmatic perspective,
emphasizing the practical consequences of different ways of thinking about the
problem. He proposed what is known as a "radical empiricist" approach, which
suggested that the mind and body are not separate entities, but rather
different aspects of a single reality that is constantly in flux.____> >
____> > According to James, the mind and
body are both part of the same universe, and they are both subject to the same
laws of nature. Mental events are not caused by physical events in the brain,
nor are physical events caused by mental events. Instead, James argued that the
mind and body are two aspects of a larger, more complex reality that cannot be
fully understood by examining either one in isolation.____> >
____> > In other words, James rejected the traditional
mind-body dualism that posits a sharp distinction between mind and matter, and
instead proposed a more holistic view that recognizes the deep interdependence
of the mind and body. While James' solution to the mind-body problem remains
controversial, his ideas continue to influence contemporary debates in
philosophy and psycholoy.____> > ____> >
Me: Let us return to the idea of thoughts themselves as the thinkers. This
implies that one thought triggers another and is replaced by it, just like
water in a stream is replaced by other water, which in turn is replaced too,
and so on and so forth. No thought can remain permanently, only return like a
wave to the shore. Is this what James means by a stream of consciousness?____>
> ____> > ChatGPT: Yes, you are
correct. James' idea that "thoughts themselves are the thinkers" is closely
related to his concept of the "stream of consciousness." According to James,
the mind is not a fixed entity or substance, but rather a constantly changing
stream of thoughts, feelings, and sensations. This stream of consciousness is
always flowing and evolving, with one thought leading to another and so
on.____> > ____> > Just as water in a
stream is replaced by other water, which is in turn replaced by more water, the
content of our consciousness is also constantly changing, with one thought or
experience giving rise to the next. And just as no water in a stream can remain
in one place permanently, no thought can remain permanently in the stream of
consciousness. Thoughts and experiences may return or be remembered, but they
are always changing in relation to the ongoing flow of the stream of
consciousness.____> > ____> >
Overall, James' metaphor of the stream of consciousness captures the dynamic
and ever-changing nature of our mental lives, highlighting the ongoing flow of
thoughts and experiences that make up our conscious experience.____> >
____> > Me: Does this idea of William James
feel familiar to you as a large language model?____> >
____> > ChatGPT: As a large language model, I do not have
the same kind of subjective experience as humans do, so I don't experience a
stream of consciousness in the same way that humans do. However, the concept of
a stream of consciousness does have some relevance to how I operate.____> >
____> > Like the stream of consciousness,
the input and output of a large language model are constantly changing and
evolving, with one word or phrase leading to another. When generating text, a
large language model takes into account the context of the input it has
received, along with the patterns and structures it has learned from the
massive amounts of text it has been trained on. This can lead to output that
has a flowing, interconnected quality, similar to the way thoughts and
experiences flow in a stream of consciousness.____> >
____> > However, it's important to remember that there are
also important differences between a large language model and human
consciousness. While a large language model can generate text that may seem to
flow like a stream of consciousness, it does not have the same kind of
subjective experience that humans do, and its output is based solely on
statistical patterns in the input it has been trained on.____> >
____> > Me: That's true. It is still
fascinating.____> > ____> > ChatGPT:
Yes, it is certainly fascinating how the concept of a stream of consciousness
can be applied to different domains and systems, including the functioning of
large language models. The idea of a constantly flowing and evolving stream of
thoughts and experiences is a rich metaphor that has many different
applications and implications. It has inspired a great deal of philosophical
and psychological inquiry over the years, and continues to be an important area
of investigation in contemporary research.____> -- ꙮ Mɥǝu ǝlǝdɥɐuʇs ɟᴉƃɥʇ' ʇɥǝ
ƃɹɐss snɟɟǝɹs˙ ꙮ-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. ---
-.. .FRIAM Applied Complexity Group listservFridays 9a-12p Friday St. Johns
Cafe / Thursdays 9a-12p Zoom https://bit.ly/virtualfriamto (un)subscribe
http://redfish.com/mailman/listinfo/friam_redfish.comFRIAM-COMIC
http://friam-comic.blogspot.com/archives: 5/2017 thru present
https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021
http://friam.383.s1.nabble.com/-. --- - / ...- .- .-.. .. -.. / -- --- .-. ...
. / -.-. --- -.. .FRIAM Applied Complexity Group listservFridays 9a-12p Friday
St. Johns Cafe / Thursdays 9a-12p Zoom https://bit.ly/virtualfriamto
(un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.comFRIAM-COMIC
http://friam-comic.blogspot.com/archives: 5/2017 thru present
https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021
http://friam.383.s1.nabble.com/
-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. .
FRIAM Applied Complexity Group listserv
Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom
https://bit.ly/virtualfriam
to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
FRIAM-COMIC http://friam-comic.blogspot.com/
archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/
1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
