Yes, I think so. Though they should be the "raw" functionality only.... e.g. I know some of the Incanter code adds quite a bit of syntactic sugar which is absolutely fine but I don't think belongs in core.matrix itself. Hence a lot of the Incanter code should ideally have the strategy "handle syntactic sugar as necessary, then delegate to the underlying core.matrix operation(s)".
I've looked briefly at the spark RDDs / datasets and I think they are fine targets for core.matrix dataset implementations. I'd favour integrating the implementation code into existing Spark wrapper libraries, if the maintainers are agreeable. This would be a great project for somebody, and would also really help us to develop the core.matrix dataset API (which is a bit spartan right now). On Friday, 5 February 2016 19:18:14 UTC+8, Bruce Durling wrote: > > Mike, > > That sounds really useful. So things like mapping and joining should > be in core.matrix then? > > Just contemplating some PRs and where we should be looking. > > I know there was talk about hooking up core.matrix to Spark RDDs. Has > anyone made any progress on that? It would be nice to pitch in there > too. > > cheers, > Bruce > > On Fri, Feb 5, 2016 at 10:26 AM, Mikera <mike.r.an...@gmail.com > <javascript:>> wrote: > > Hi Bruce, > > > > My view is that the following things should be in core.matrix > > - Fundamental dataset and array programming operations (slicing, access, > > reshaping, dimension labelling etc.) > > - Basic numerical operations (add, matrix multiply etc.) > > - Matrix operations that may have different underlying implementations > (SVD > > etc.) > > - Protocol based support for standard implementations (Java arrays, > Clojure > > vectors etc.) > > > > The following belong in other libraries: > > - More sophisticated algorithms building on the fundamental operations > > (statistical techniques, machine learning etc.) > > - Anything to do with user interaction (graphing, GUIs etc.) > > - Tools for IO / data access etc. > > > > This is just a guideline though - happy to consider any specific > function on > > a case by case base. Probably best done via issues, so please file if > you > > have any! > > > > On Friday, 5 February 2016 03:27:47 UTC+8, Bruce Durling wrote: > >> > >> Mike, > >> > >> I've had some of my team start using incanter 1.9.0. We've found and > >> reported some issues and would like to contribute. Is there a good > >> road map somewhere of what functions should stay in incanter and what > >> ones should live in clojure.core.matrix.dataset? I'd like to know that > >> any fixes we pursue would move towards the future design of incanter. > >> > >> cheers, > >> Bruce > >> > >> On Thu, Feb 4, 2016 at 10:01 AM, Mikera <mike.r.an...@gmail.com> > wrote: > >> > There is the start of a dataframe-like implementation in > >> > clojure.core.matrix.dataset > >> > > >> > Contributions in this area (or even just vigorous testing / issue > >> > reporting) > >> > would be very welcome! > >> > > >> > On Tuesday, 2 February 2016 17:44:51 UTC+8, Gregg Reynolds wrote: > >> >> > >> >> > >> >> > >> >> On Tue, Feb 2, 2016 at 2:22 AM, Mikera <mike.r.an...@gmail.com> > wrote: > >> >>> > >> >>> If you are interested in data science, help with core.matrix and > the > >> >>> associated libraries is always appreciated, and we are very > >> >>> contributor-friendly in the numerical Clojure community. > >> >> > >> >> > >> >> Speaking of which, the topic of Clojure came up on the Julia > mailing > >> >> list > >> >> recently. One comment was "I suspect the biggest issue is the lack > of > >> >> a > >> >> good dataframes library for Clojure,..." So something along that > line > >> >> would > >> >> be a good project. > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "London Clojurians" group. > >> > To unsubscribe from this group and stop receiving emails from it, > send > >> > an > >> > email to london-clojuri...@googlegroups.com. > >> > To post to this group, send email to london-c...@googlegroups.com. > >> > Visit this group at https://groups.google.com/group/london-clojurians. > > >> > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "London Clojurians" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to london-clojuri...@googlegroups.com <javascript:>. > > To post to this group, send email to london-c...@googlegroups.com > <javascript:>. > > Visit this group at https://groups.google.com/group/london-clojurians. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
