On Monday, 14 March 2016 18:21:15 UTC+8, Dragan Djuric wrote: > > >>> >> There is a set of BLAS-like API functions in core.matrix already. See: >> https://github.com/mikera/core.matrix/blob/develop/src/main/clojure/clojure/core/matrix/blas.cljc >> > > GitHub history says they were added 7 days ago. Nevermind that they just > delegate, so the only BLAS-y thing is the 4 method names taken out of > Neanderthal (BLAS has a bit more stuff than that), but why you reinvented > the wheel instead just creating core.matrix (or vectorz) implementation of > Neanderthal's API? >
The core.matrix API has been around a lot longer than Neanderthal, and presents an abstract API rather than being tied to any specific implementations. If there is any wheel-reinventing going on, it is Neanderthal :-) The correct way to build these things is IMHO: - User / library code depends on an implementation-agnostic abstraction (i.e. the core.matrix API) - The user optionally specifies an implementation (or makes do with the default) - The underlying implementation handles how that call gets executed (Neanderthal, vectorz-clj, Clatrix etc.) It wouldn't make any sense to have core.matrix (the abstraction) depend on Neanderthal (the implementation) because that defeats the whole purpose of an implementation-agnostic API. And a hard dependency would rule the whole library out for any users who can't even run that particular implementation (including myself, due to lack of Windows support). > > >> Having said that, I don't personally think the BLAS API is a particularly >> good fit for Clojure (it depends on mutability, and I think it is a pretty >> clumsy design by modern API standards). But if you simply want to copy the >> syntax, it's certainly trivial to do in core.matrix. >> > > If you look at Neanderthal's API you'll see that I took a great care to > make it fit into Clojure, which I think I succeeded. > Regarding mutability: > 1) Neanderthal provides both mutable and pure functions > 2) Trying to do numeric computing without mutability (and primitives) for > anything than toy problems is... well, sometimes it is better to plant a > Sequoia seed, wait for the tree to grow, cut it, make an abacus and compute > with it... > Right, I agree you need both mutable and immutable options. That's why core.matrix provides both. But why are you pushing a new API rather than adopting the core.matrix one (or giving me PRs if you think it could be improved?) 2) I disagree with. Most real world data science is about data manipulation and transformation, not raw computation. 1% of people need to optimise the hell out of a specific algorithm for a specific use case, 99% just want convenient tools and the ability to get an answer "fast enough". > > >> An important point to note is that they don't do the same thing at all: >> core.matrix is an API providing an general purpose array programming >> abstraction with pluggable implementation support. Neanderthal is a >> specific implementation tied to native BLAS/ATLAS. They should ideally work >> in harmony, not be seen as alternatives. >> > > * Neanderthal has an agnostic api and it is not in any way tied to > BLAS/ATLAS * > Neanderthal also has pluggable implementation support - and it already > provides two high-performance implementations that elegantly unify two very > different *hardware* platforms: CPU and GPU. And it does it quite > transparently (more about that can be read here: > http://neanderthal.uncomplicate.org/articles/tutorial_opencl.html) > OK, so you agree you want an abstract API with pluggable implementations... but that is exactly what the core.matrix API does! But if you only offer native and GPU dependencies then you aren't really offering an agnostic API. Won't even work on my machine..... could you add Windows support? Maybe you want to add pure JVM implementations as well? And for Clojure data structures? What about ClojureScript support? Sparse arrays? Oh, could you maybe support n-dimensional arrays? Datasets? If you go down that path you will simply be reinventing core.matrix, and you will eventually come to realise why a lot of the core.matrix design decisions actually make sense. Believe me, I've been down that road :-) > >> Neanderthal is more closely comparable to Vectorz, which *is* a matrix >> implementation (and I think it matches or beats Neanderthal in performance >> for virtually every operation *apart* from large matrix multiplication for >> which ATLAS is obviously fantastic for). >> > > You think without having tried that. I tried that, and *Neanderthal is > faster for virtually *ALL* operations, even 1D. Yesterday I did a quick > measure of asum (1D vector operation), for example, and neanderthal was, if > I remember correctly, * 9x faster than Vectorz in that simple summing * > > . I even pointed to you that Neanderthal is faster even in ALL those cases > when you raised that argument the last time, but you seem to ignore it. > Interesting. I don't recall you posting such benchmarks, apologies if I missed these. I'd be happy to benchmark myself if I could get Neanderthal to build. I don't believe 9x though... this operation is usually bound by memory bandwidth IIRC > > >> If anyone has other ideas / a better strategy I'd love to hear, and I >> welcome a good constructive debate. I'm not precious about any of my own >> contributions. But I do genuinely think this is the best way forward for >> Clojure data science overall, based on where we are right now. >> > > I would like to propose a strategy where more love is given to the actual > libraries (incanter is rather indisposed and stagnant IMO) that solve > actual problems instead of trying to unify what does not exist (yet!). > Then, people will use what works best, and what does not work will not be > important. That's how things go in open-source... > > core.matrix already exists, is widely used and already unifies several different implementations that cover a wide variety of use cases. It provides an extensible toolkit that can be used either directly or by library / tool implementers. It's really very powerful, and it's solving real problems for a lot of people right now. It has the potential to make Clojure one of the best languages for data science. Don't get me wrong, I think Neanderthal is a great implementation with a lot of good ideas. I'd just like to see it work well *with* the core.matrix API, not be presented as an alternative. The Clojure data science ecosystem as a whole will benefit if we can make that work. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
