Just a side comment, Dragan, if you don't want to be compared against some other tech, it might be wise to not make the subtitle of a release "X times faster than Y". Make the defining feature of a release that it's better than some other tech, and the proponents of that tech will probably start to get a bit irritated.
And I think the distinction made here about floats vs doubles is very important. We can all construct benchmarks that show tech X being faster than tech Y. But if I am evaluating those things I really want to know why X is faster than Y. I want to know what tradeoffs were made, what restrictions I have in using the faster tech, etc. Free performance gains are very rare, so the moment I see "we are X times faster" I immediately start looking for a caveats section or a rationale on why something is faster. Not seeing that, or benchmark documentation, I almost immediately consider the perf numbers to be hyperbole. I know practically nothing about either one of these projects, so I'll stop commenting here, but comparisons of tech as tag-lines is rarely a good idea. Timothy On Mon, Mar 14, 2016 at 9:19 AM, Dragan Djuric <draga...@gmail.com> wrote: > >> 2) I disagree with. Most real world data science is about data >> manipulation and transformation, not raw computation. 1% of people need to >> optimise the hell out of a specific algorithm for a specific use case, 99% >> just want convenient tools and the ability to get an answer "fast enough". >> > > And most of those people are quite happy with R and Python and don't care > about Clojure. I, on the other hand, am perhaps in the other 1%, but my > needs are also valid (IMHO), especially when I am prepared to do the > required work to satisfy them myself. > > >> But if you only offer native and GPU dependencies then you aren't really >> offering an agnostic API. Won't even work on my machine..... could you add >> Windows support? Maybe you want to add pure JVM implementations as well? >> And for Clojure data structures? What about ClojureScript support? Sparse >> arrays? Oh, could you maybe support n-dimensional arrays? Datasets? >> > > Native and GPU are already available *now* and work well. Windows support > is there, you (or someone else) need to compile the Windows binaries and > contribute them. Pure JVM implementation is quite easy to add, since the > majority of Neanderthal is not only pure Java, but Clojure; the only thing > I haven't implement (yet) is a pure Java BLAS engine. I already wrote a > (much harder) GPU BLAS implementation from scratch, so compared to that > pure Clojure BLAS is like a walk in the park. If/when I need those, the > infrastructure is already there to support it quite easily, without API > changes. ClojureScript support is more complex, but mostly a matter of > putting work weeks into it, rather than as a technical challenge, since by > now I know what causes most problems and how to solve them. Of course, I > won't do that just as an exercise, but only when the need arises (if ever). > Of course, I'll welcome quality PRs. > > >> Interesting. I don't recall you posting such benchmarks, apologies if I >> missed these. >> >> I'd be happy to benchmark myself if I could get Neanderthal to build. I >> don't believe 9x though... this operation is usually bound by memory >> bandwidth IIRC >> > > OK, to be more precise it is around 8.5x with floats. With double > precision (which I do not need, and probably neither do you in your work on > deep learning) it is "just" 4x :) > > >> >> >>> >>> >>>> If anyone has other ideas / a better strategy I'd love to hear, and I >>>> welcome a good constructive debate. I'm not precious about any of my own >>>> contributions. But I do genuinely think this is the best way forward for >>>> Clojure data science overall, based on where we are right now. >>>> >>> >>> I would like to propose a strategy where more love is given to the >>> actual libraries (incanter is rather indisposed and stagnant IMO) that >>> solve actual problems instead of trying to unify what does not exist >>> (yet!). Then, people will use what works best, and what does not work will >>> not be important. That's how things go in open-source... >>> >>> >> >> core.matrix already exists, is widely used and already unifies several >> different implementations that cover a wide variety of use cases. It >> provides an extensible toolkit that can be used either directly or by >> library / tool implementers. It's really very powerful, and it's solving >> real problems for a lot of people right now. It has the potential to make >> Clojure one of the best languages for data science. >> > > I agree that this sounds great, but, come on... It sounds like a marketing > pitch. Clojure currently doesn't offer a single data science library that > would be even a distant match to the state of the art. Nice toys - sure. I > hope you didn't mean Incanter?... > > >> Don't get me wrong, I think Neanderthal is a great implementation with a >> lot of good ideas. I'd just like to see it work well *with* the core.matrix >> API, not be presented as an alternative. The Clojure data science ecosystem >> as a whole will benefit if we can make that work. >> > > That is up to people that would find that useful. I support them. > > BTW, each Neanderthal structure is a sequence, so, technically, it already > supports core.matrix. > > BBTW, I wish only the best to core.matrix, and think you do a great work > with it. Also, I suppose we are leading a technical discussion here, so I > am not getting you wrong and I appreciate your appreciation of Neanderthal > technical aspects. > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
