A bunch o' background inf0z:

https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
https://huggingface.co/leafspark/Llama-3.2-11B-Vision-Instruct-GGUF
https://github.com/etched-ai/open-oasis
https://altera.al/    (Their substack is better: https://substack.com/@digitalhumanity )


I feel that if I took these, threw them on the floor (breaking them into pieces), then reassembling them in the way I think they should be assembled with maybe 15% new code, I could present an agent that has something resembling consciousness, basic reasoning, and language understanding. My experience with LLama 3.2 3B is that it was definitely punching above its weight class in terms of stability but was lacking in reasoning prowess, but I feel that the deficit in reasoning should be addressed by applying a CoT architecture. OpenOasis is definitely benefitting from the simplified minecraft environment, but a general purpose perception system should be in-line with general SD models. Remember: Consciousness is the process of making predictions of the immediate future and then checking them against the environment, so where Open-Oasis appears to be a minecraft generator, I would use it as the heart of a perception system.

I think all of this is quite feasible for a high-end gaming rig.

I don't know how hard it would be to graft OpenOasis rigged in perception-mode to Llama 3.2, and / or re-rig the vision module provided as a generator module so that it will be useful as perception mode, then link in a few more subsystems for operating the avatar...

I would really like to try to put this together if I could work through all of the compiler errors my computers love spewing. =((((

Remaining issues:

- On-line learning.
- token learning  (the ability to learn new tokens, optimize token dictionary...)
- Episodic memory.

The bottom line though is that the technology family mentioned at the top can run on consumer-grade hardware which means we have no excuse for not trying to get it working...

--
You can't out-crazy a Democrat.
#EggCrisis  #BlackWinter
White is the new Kulak.
Powers are not rights.


------------------------------------------
Artificial General Intelligence List: AGI
Permalink: 
https://agi.topicbox.com/groups/agi/Tc9e2f1f0565ee7db-M581128115d37ecc9d3385415
Delivery options: https://agi.topicbox.com/groups/agi/subscription

Reply via email to