YKY, are you doing any experiments with transformers or neural networks, or still pursuing an old school symbolic approach to knowledge representation? Have you written any code to test your ideas?
LLMs today have a huge imbalance in computation power requirements for training over prediction because each session creates a private copy. You shouldn't need a building full of GPUs and your own power plant and cooling towers to run the kind of experiments you need to do. Training and prediction should run at the same speed. You only need a few GB of text to reproduce human level text prediction. -- Matt Mahoney, mattmahone...@gmail.com On Sat, May 10, 2025, 3:03 PM YKY (Yan King Yin, 甄景贤) < generic.intellige...@gmail.com> wrote: > On Sun, May 11, 2025 at 12:58 AM Mike Archbold <jazzbo...@gmail.com> > wrote: > >> MeTTa has a means of calling Python objects so conceivably anyway you >> could mix in neural/ differentiable that way. >> > > > It's about efficiency. Deep learning is currently the most efficient > learning algorithm. So we should use DL as the learner for the graph > rewriting rules of the AGI (ie, the rules for the AGI's internal > thinking). Currently the best candidate for this part is Transformer. > > But we suspect the Transformer may be doing its job in a roundabout way > that is very inefficient. So we want to improve the Transformer, ie, > rewrite its string diagram. This is the "functorial" part, ie, rewriting > the AGI system which is itself a rewriting machine. If we know how to do > the latter, we can "gradient descend" to the most efficient AGI. > > Unfortunately I don't know how to figure out the functorial details. > Maybe someone more adept at category theory can do that... > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Tdc5c19d0f38aacd6-M6383c9ab81728d48dbf24612> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Tdc5c19d0f38aacd6-M9094dd05699f976e201e6946 Delivery options: https://agi.topicbox.com/groups/agi/subscription
