On Tue, Feb 5, 2019 at 12:01 PM Matt Mahoney <[email protected]> wrote:
> It is easy to come up with ideas and to say what approaches to AGI > should be obvious. It is much harder to test them, and when we do we > are sometimes surprised that our ideas don't work. Linas's > dissertation length paper on the equivalence of symbolic and neural > language systems would be a good review of dozens of different > approaches Well, its a review of *two* approaches, not dozens: vector-space approaches, and monoidal category approaches. Vector spaces are a special case of monoidal categories. I'm trying to indicate that they are a straight-jacket, as a result. They also fail to align with traditional linguistics theory. (The earliest reference I know that states that "traditional lingusitics" is a monoidal category dates to 1967, and is in the form of an entire book devoted to the topic.) The current obsession with neural nets appears to be due to the fact that the practioners are unaware of the category-theoretic formulations for vector spaces. I figure its a matter of time, before there is a sea-change. > if there were an experimental results section that told us > which ones were worth pursuing. There's this: https://github.com/opencog/opencog/raw/master/opencog/nlp/learn/learn-lang-diary/connector-sets-revised.pdf https://github.com/opencog/opencog/raw/master/opencog/nlp/learn/learn-lang-diary/learn-lang-diary.pdf There's a team that has taken over the project; unfortunately, this still are getting up to speed, and haven't made any progress yet. > Rule based grammars seem to make sense because it works on artificial > languages using very little computing power. Rule-based grammars are monoidal categories. Neural nets are vector-space algorithms, which are *SPECIAL CASES* of monoidal categories. That's the key take-away message. The reason my "dissertation" is so long is that I need to build a lot of vocabulary and terminology to explain how a vector space is "the same thing" as a "rule". This is mostly because most readers in linguistics don't know what a tensor algebra is, and most readers in neural nets don't know what a tensor algebra is .. despite products with names like "TensorFlow". A tensor is a rule. Full stop. > You parse the sentence > and analyze the tree for semantics. You can cover about half of all > English sentences with a few dozen rules. But English doesn't work > that way. Nobody knows how many rules you need to parse the other > half. The current link-grammar dict contains about 2.2K hand-edited, hand-curated rules. I believe that it is the worlds most-accurate English-language parser. > > The standard measure of language model performance is word perplexity, > or equivalently, text prediction or compression. Its a truly shitty measure, created by people who do not understand what linguistics has been up to for the last 70 years. Don't waste your time on pseudo-science. I can't respond to the rest, because it seems to be based on this key mis-understanding. -- Linas -- cassette tapes - analog TV - film cameras - you ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Ta6fce6a7b640886a-M011f878ccde9ddb9602e47f7 Delivery options: https://agi.topicbox.com/groups/agi/subscription
