Christopher, Bobby, Mars0i, I agree with you! core.matrix works fine for what you need and it is quite sane and wise to continue happily using it.
Whether you want to touch Neanderthal or not is up to you, and it is same to me one way or the other - I earn exactly $0 if you do and lose the same amount if you don't. I haven't ever said anywhere that you (or anyone else) should switch from core.matrix to Neanderthal, so I am quite puzzled by your insistence on telling me why you wouldn't. I created Neanderthal because I needed state of the art capabilities for my other projects, and open-sourced it because I know that there are other people (no matter if only 1%, 10%, or 100%) who need exactly that. My main goal is not to convert everyone (or even anyone) to it, but to attract like minded people *who need the same stuff* to use it, and hopefully contribute to it, or build other useful open-source libraries on top of it that *I* can use too. It seems to me that your main argument is that, whether good or not, core.matrix is a unification point that somehow makes the ecosystem and community thrive, while you'd like to avoid the "lisp curse" of fragmented libraries. That's a plausable hypothesis (and quite logical), but if we look around we could see it is simply not true. Often it is the opposite, even in Clojure land. How many competing web libraries, sql bindings, network libs, and various other infrastructure projects there are for Clojure? Tons! There are even several competing wrappers for React! And - all of them thrive! They build useful stuff on top of those that often match what is available elsewhere. And no one is going around scolding them for their heresy. We can see the similar situation even in the state of the art tools elsewhere. BLAS itself has at least 2 competitive, mutually incompatible, state of the art open source implementations, and tons of proprietary ones. In Java, there are tons of competitive libraries, ditto for Python, R, etc. And all of them have several tools in each area that have arisen of thousands experiments and thrived. Clojure's "matrix" (or NDarray, or whatever we should call it) space seems to be unified, or so I've her. The reasoning is that it will produce a thriving ecosystem of data analysis, machine learning, numerical computing, and other tools. But what are the results by now? Here and there, there is some upstart project or a library that is a good learning exercise for their author(s), but otherwise completely unusable for anything beyond that. Even the tools that had received some love from their authors are far from anything available in other languages. I asked a few times to be pointed to any competitive data-science/ML library in Clojure, and no one stepped in with examples. The most telling example of this state is maybe the most serious of those libraries - Incanter. 6 years ago, it looked as an interesting project that was actively maintained and somehow usable. If not comparable to the alternatives in other languages, at least actively developed and with a potential and bits "Clojure's awesomeness will solve everything" promises. But, the author left for greener pastures long ago, and Incanter started stalling. A few maintainers helped it staying afloat for some time, until the main focus of the project became the porting to the one true lord and savior core.matrix. That took a lot of time and resources, and did it help? Judging by the commit history (master and develop), the main thing that is now happening for Incanter are occasional bumps of core.matrix dependency to the new version. I am glad that Incanter, however dead it is, is still usable for (many?) people, or that core.matrix is usable and used in closed-source projects. But that is obviously not what I need, and I know better than to follow the peer pressure to accept the dogma. I do not mind being reminded every time to hear about our true lord and savior core.matrix, but please understand me when I reserve my right to not believe in that. PS. I really think that this discussion, however heated it may be is still a technical discussion, so whatever I say about various technologies does not mean to say anything bad against their authors and users. I think these are amazing people who give they work for free so we can all freely use it and learn from it. On Wednesday, March 16, 2016 at 12:55:40 AM UTC+1, Christopher Small wrote: > > Ditto; same boat as Mars0i. > > I think the work Dragan has done is wonderful, and that he bears no > responsibility to implement the core.matrix api himself. I'm also > personally not interested in touching it without core.matrix support, > because similarly for me, the performance/flexibility tradeoff isn't worth > it. I wish I had time to work on this myself, since it would be an awesome > contribution to the community, but I just don't have the time right now > with everything else on my plate (including a 1 year old (figurative plate, > obviously)). So I would highly encourage anyone interested in numerical > computing and clojure looking for a project to sharpen their teeth on to > put some time here. I think it would be a real asset to the community. > > The one thing I *would* like to as from you, Dragan, is that you help us > (well, Mike perhaps more than anyone) understand what you really don't like > about the core.matrix architecture. In previous threads where we've > discussed this, I felt like nothing concrete ever came up. You seem to not > like that the fact that the API doesn't mirror BLAS, and I get your > thinking here (even if I don't necessarily agree). Is that really the > extent of it, or did I miss something? > > It was mentioned that core.matrix has a BLAS-style API ns, but you still > seem uninterested in this, since those functions delegate to a non-BLAS > API. If I'm right, your concern is that this would lead to leaky > abstractions, and affect performance, yes? I haven't looked at this API or > it's implementation, but I have an idea that might satisfy everyone. What > if it were possible to specify a core.matrix implementation directly > through the BLAS API, with the rest of the "standard" core.matrix API based > off these functions (by default; custom implementations could still be > provided I'm sure...)? (Does this seem doable Mike?) Could this get us the > best of both worlds and make core.matrix look more attractive to you Dragan > (even if you don't feel personally interested in working on the > Neanderthal/core.matrix implementation yourself)? Those who want and know > the BLAS API can use it, without any overhead or leakiness, and everything > would remain interoperable with other libraries. If this still wouldn't do > it, what would? You clearly have very deep knowledge in this subject, and I > think I speak for the Clojure numerical computing community (and > core.matrix community in particular) in expressing that we'd greatly > appreciate any insights you can give which would help make the ecosystem as > a whole better. > > With gratitude > > Chris > > > > > On Tuesday, March 15, 2016 at 6:42:45 AM UTC-7, Bobby Bobble wrote: >> >> I'm in the same boat as Mars0i in that I've written a load of code with >> core.matrix, which I'm happy with and which I will never have time to >> rewrite for Neanderthal. If I had a new project that was >> performance-critical I might bother to learn Neanderthal but for my >> existing project, speed is secondary to maintainability now. If my boss >> asked me to speed it up by 60x I'd love to be able to just switch to >> implementation :Neanderthal with core.matrix! One keyword and boom. >> >> On Tuesday, March 15, 2016 at 3:54:03 AM UTC, Mars0i wrote: >>> >>> Dragan, I still support you doing whatever you want with your time, and >>> I'm grateful that you've produced was I gather is a great library. >>> Wonderful. I don't feel that you have to implement the core.matrix >>> api--but I really, really wish someone would produce a core.matrix >>> interface to Neanderthal. (I don't know how difficult that will be; I just >>> know that I'm not the right person to do it.) >>> >>> Some of the following has been said before, but some the points below >>> seem to have been lost in the current discussion. >>> >>> I want to try Neanderthal, but I may never do so until someone produces >>> a core.matrix mapping. That needn't be a problem for you, Dragan, but my >>> time is too precious to reimplement my existing code for a new api. Even >>> with new code, I'm always going to prefer core.matrix in the foreseeable >>> future unless I have very special needs, because being able to swap out >>> implementations is just too convenient for me. core.matrix allows me to >>> easily figure out which library is most efficient in my application. I >>> also want there to be a standard api for matrix operations, and at this >>> point, core.matrix is the standard imo. >>> >>> I came to Clojure from Common Lisp, where there are 5-10 matrix >>> libraries with different APIs and different efficiency profiles, several >>> seemingly best for different purposes (the Curse of Lisp mentioned by >>> Mikera, applied to our case). I am just very grateful that, instead of >>> CL's mess of libraries and APIs, in the Clojure world Mikera and others >>> decided to construct a standard API for matrix implementations. What he's >>> promoting is not marketing, nor religion--it's just convenience and >>> coherence. It's part of what Clojure is all about. >>> >>> Again, Dragan, I certainly don't think that you have to support >>> core.matrix, and I am still grateful for Neanderthal even no one ever gives >>> it a core.matrix front end. From my point of view, everyone in the Clojure >>> community can be grateful for Neanderthal--even those of us who might never >>> end up using it. It's a good resource for those who know in advance that >>> it will be best for their application, or for those with time to benchmark >>> their application with variant code. I personally don't want to negotiate >>> different matrix APIs, however. That's what's important to me. >>> >>> I don't have any opinion about the relative benefits of the core.matrix >>> and Neanderthal APIs as such. I just want to write to standard API. If >>> the Neanderthal API became the standard, with multiple back ends, and was >>> overall better than core.matrix in that respect, I'd use it. At this >>> point, going that route seems unlikely to be a good option, because >>> core.matrix already exists, with a lot of work having gone into it by a >>> reasonably large community. For me the API seems just fine, and it's still >>> being enhanced. (If it can be made better through conversation with you or >>> others familiar with Neanderthal, that's even better.) >>> >>> So ... I understand why Mikera keeps mentioning core.matrix whenever >>> Neanderthal is promoted here. I support him doing that, because I want the >>> matrix libraries for Clojure to stay away from the curse of Common Lisp. I >>> also understand why core.matrix isn't important for you, personally. But >>> core.matrix is important for others besides Mikera. It's not just a pet >>> project; it's a Good Thing, because well thought out common APIs for >>> libraries with similar functionality is a Good Thing. I think you probably >>> agree, at least *in the abstract,* even if in this particular context >>> you don't think that core.matrix is the best API for all matrix >>> applications. (Maybe it's not. Still ... a common API, that's gotta be a >>> good thing. And if someone ever does produce a core.matrix interface to >>> Neanderthal, the regular Neanderthal interface will still be available, >>> presumably. Great! Best of both worlds.) >>> >>> I don't think this discussion has to get as heated as it does. Perhaps >>> I'll get some of the heat, now, but it's not necessary, imo. >>> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
