> > > 2) I disagree with. Most real world data science is about data > manipulation and transformation, not raw computation. 1% of people need to > optimise the hell out of a specific algorithm for a specific use case, 99% > just want convenient tools and the ability to get an answer "fast enough". >
And most of those people are quite happy with R and Python and don't care about Clojure. I, on the other hand, am perhaps in the other 1%, but my needs are also valid (IMHO), especially when I am prepared to do the required work to satisfy them myself. > But if you only offer native and GPU dependencies then you aren't really > offering an agnostic API. Won't even work on my machine..... could you add > Windows support? Maybe you want to add pure JVM implementations as well? > And for Clojure data structures? What about ClojureScript support? Sparse > arrays? Oh, could you maybe support n-dimensional arrays? Datasets? > Native and GPU are already available *now* and work well. Windows support is there, you (or someone else) need to compile the Windows binaries and contribute them. Pure JVM implementation is quite easy to add, since the majority of Neanderthal is not only pure Java, but Clojure; the only thing I haven't implement (yet) is a pure Java BLAS engine. I already wrote a (much harder) GPU BLAS implementation from scratch, so compared to that pure Clojure BLAS is like a walk in the park. If/when I need those, the infrastructure is already there to support it quite easily, without API changes. ClojureScript support is more complex, but mostly a matter of putting work weeks into it, rather than as a technical challenge, since by now I know what causes most problems and how to solve them. Of course, I won't do that just as an exercise, but only when the need arises (if ever). Of course, I'll welcome quality PRs. > Interesting. I don't recall you posting such benchmarks, apologies if I > missed these. > > I'd be happy to benchmark myself if I could get Neanderthal to build. I > don't believe 9x though... this operation is usually bound by memory > bandwidth IIRC > OK, to be more precise it is around 8.5x with floats. With double precision (which I do not need, and probably neither do you in your work on deep learning) it is "just" 4x :) > > >> >> >>> If anyone has other ideas / a better strategy I'd love to hear, and I >>> welcome a good constructive debate. I'm not precious about any of my own >>> contributions. But I do genuinely think this is the best way forward for >>> Clojure data science overall, based on where we are right now. >>> >> >> I would like to propose a strategy where more love is given to the actual >> libraries (incanter is rather indisposed and stagnant IMO) that solve >> actual problems instead of trying to unify what does not exist (yet!). >> Then, people will use what works best, and what does not work will not be >> important. That's how things go in open-source... >> >> > > core.matrix already exists, is widely used and already unifies several > different implementations that cover a wide variety of use cases. It > provides an extensible toolkit that can be used either directly or by > library / tool implementers. It's really very powerful, and it's solving > real problems for a lot of people right now. It has the potential to make > Clojure one of the best languages for data science. > I agree that this sounds great, but, come on... It sounds like a marketing pitch. Clojure currently doesn't offer a single data science library that would be even a distant match to the state of the art. Nice toys - sure. I hope you didn't mean Incanter?... > Don't get me wrong, I think Neanderthal is a great implementation with a > lot of good ideas. I'd just like to see it work well *with* the core.matrix > API, not be presented as an alternative. The Clojure data science ecosystem > as a whole will benefit if we can make that work. > That is up to people that would find that useful. I support them. BTW, each Neanderthal structure is a sequence, so, technically, it already supports core.matrix. BBTW, I wish only the best to core.matrix, and think you do a great work with it. Also, I suppose we are leading a technical discussion here, so I am not getting you wrong and I appreciate your appreciation of Neanderthal technical aspects. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
