On 2020-09-27 08:50:AM, Matt Mahoney wrote:
On Sat, Sep 26, 2020, 10:32 PM TimTyler <[email protected]
<mailto:[email protected]>> wrote:
On 2020-09-22 12:45:PM, Matt Mahoney wrote:
> The no free lunch theorem is based on the false premise that it is
> possible to have a uniform probability distribution over an
infinite
> set. The converse proves Occam's Razor.
I don't think that's right. I looked here:
https://en.wikipedia.org/wiki/No_free_lunch_theorem#Original_NFL_theorems
It plainly says it is talking about a "finite set".
Exactly. And our universe is finite. This, all sets of real objects
must be finite too.
So why doesn't the no free lunch theorem work in practice. Why are
some search algorithms faster than others in practice?
Here is what the Wiki page says about that:
"others argue that NFL is of little relevance to machine learning research.
IfOccam's razor <https://en.wikipedia.org/wiki/Occam%27s_razor>is
correct, for example if sequences of lowerKolmogorov
<https://en.wikipedia.org/wiki/Kolmogorov_complexity>
complexity <https://en.wikipedia.org/wiki/Kolmogorov_complexity>are more
probable than sequences of higher complexity, then
(as is observed in real life) some algorithms, such as cross-validation,
perform better on average on practical problems (when compared with
random choice or with anti-cross-validation).^"
I don't think it is necessary to invoke infinite sets to explain Occam's
razor.
Simple, finite physical systems - such as Conway's Game of Life on a
torus -
^exhibit much the same dynamics.
--
__________
|im |yler http://timtyler.org/
------------------------------------------
Artificial General Intelligence List: AGI
Permalink:
https://agi.topicbox.com/groups/agi/Ta433301e9ac5fb42-Mc3b4dc6ae74903df44bd2695
Delivery options: https://agi.topicbox.com/groups/agi/subscription
