I remember your definition when I did my 2021 survey: "You understand a sequence of symbols if you can predict or compress them. If I wanted to test if you understand Chinese, I would show you some Chinese text and test how many characters you could guess next."
Link to whole survey if interested: https://drive.google.com/file/d/1XfRZudUomqTuuSmMQDBhlcTmZRrezBl2/view?usp=sharing To me, this is a bit superficial of a definition, however I think it is still one plausible level of understanding. On Wed, May 14, 2025 at 3:01 PM Matt Mahoney <mattmahone...@gmail.com> wrote: > Researchers in China demonstrate using LLMs to compress text, images, > audio, and video. In the abstract of the linked paper they claim 3x better > text compression than zpaq. In the preprint they also claim to improve on > lossless images, video, and audio by the simple method of converting pixels > or audio samples to characters in 2K chunks to fit LLaMA-8B's context > window. > > > https://techxplore.com/news/2025-05-algorithm-based-llms-lossless-compression.html > > Of course this does not include the size of the model. Still, they make > the important point that compression = understanding. We test understanding > using prediction, and we measure prediction using compression. > > -- Matt Mahoney, mattmahone...@gmail.com > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T3fa66c5feeacc892-Mae55b5bee5c5c380de9943ba> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T3fa66c5feeacc892-Mbd7e8740a9da72ae12cd2228 Delivery options: https://agi.topicbox.com/groups/agi/subscription
