The reason I use the decompressor size in my compression benchmarks is that it is otherwise trivial to compress any file to one byte by including a copy of the file in the program. Some programs don't cheat so obviously, for example they may have an English dictionary built in, but it is still misleading.
However I am happy to see exactly the kind of research that my benchmarks were intended to encourage. It would seem to make sense to model lexical, semantic, and grammatical knowledge with neural networks. Now we have the experimental results to back it up. On Sat, Mar 2, 2019, 7:43 AM ducis <[email protected]> wrote: > I searched a bit and found that the number 0.99 came from here > https://arxiv.org/abs/1901.02860 > also > https://encode.ru/threads/3059-How-much-further-can-the-best-compression-go > And it actually stated 1.06 -> 0.99. And 1.06 was from > https://arxiv.org/abs/1808.04444, which did mention cmix. > Not exactly sure if these are compressed sizes without model sizes though. > If so, after the record being broken a few more times ignoring the model > size will likely become the new standard. > Maybe the best way to end this would be an algorithm that appears to be > learning but actually remembers the input exactly? > > At 2019-02-17 00:32:06, "Matt Mahoney" <[email protected]> wrote: > > The paper mentioned improving compression of enwik8 from 0.99 to 0.93 bits > per character but gives no details or citation. enwik8 is from my large > text benchmark and is the test file for the Hutter prize. The current > record is actually 1.22 bits per character and I haven't received an entry > from them. I am on the prize committee. > > text8 is a clean version of enwik8 with only lowercase letters and spaces. > enwik8 is 100 MB of Wikipedia text with some XML formatting. > > On Thu, Feb 14, 2019, 5:28 PM Robert Levy <[email protected] wrote: > >> https://blog.openai.com/better-language-models/ >> >> Impressive work. They're use the technique introduced in the "Attention >> Is All You Need" paper called "transformers". See also: >> http://jalammar.github.io/illustrated-transformer/ >> > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + delivery > options <https://agi.topicbox.com/groups/agi/subscription> Permalink > <https://agi.topicbox.com/groups/agi/T709a492ffd52fb84-Mdc9fc355877ce2639ce9837b> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T709a492ffd52fb84-M15f048b5299dc167d21a0247 Delivery options: https://agi.topicbox.com/groups/agi/subscription
