Thanks Gilles, I can provide the same sort of stats across a clustering example across commons-math (KMeans) vs Apache Ignite, Apache Spark and Rheem/Apache Wayang (incubating) if anyone would find that useful. It would no doubt lead to similar conclusions.
Cheers, Paul. On Sun, Apr 25, 2021 at 8:15 AM Gilles Sadowski <gillese...@gmail.com> wrote: > > Hello Paul. > > Le sam. 24 avr. 2021 à 04:42, Paul King <paul.king.as...@gmail.com> a écrit : > > > > I added some more comments relevant to if the proposed algorithm > > belongs somewhere in the commons "math" area back in the Jira: > > > > https://issues.apache.org/jira/browse/MATH-1563 > > Thanks for a "real" user's testimony. > > As the ML is still the official forum for such a discussion, I'm quoting > part of your post on JIRA: > ---CUT--- > For linear regression, taking just one example dataset, commons-math > is a couple of library calls for a single 2M library and solves the > problem in 240ms. Both Ignite and Spark involve "firing up the > platform" and the code is more complex for simple scenarios. Spark has > a 181M footprint across 210 jars and solves the problem in about 20s. > Ignite has a 87M footprint across 85 jars and solves the problem in > > 40s. But I can also find more complex scenarios which need to scale > where Ignite and Spark really come into their own. > ---CUT--- > > A similar rationale was behind my developing/using the SOFM > functionality in the "o.a.c.m.ml.neuralnet" package: I needed a > proof of concept, and taking the "lightweight" path seemed more > effective than experimenting with those platforms. > Admittingly, at that epoch, there were people around, who were > maintaining the clustering and GA codes; hence, the prototyping > of a machine-learning library didn't look strange to anyone. > > Regards, > Gilles > > >>> [...] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
