Hi Dominic thank you! Are you maintainer/contrib to datasplash, I would be happy to swap notes, synthesize ideas.
My org looked at datasplash. The biggest dealbreaker for us was datasplash's AOT-orientation; its AOT packaging meant we couldn't float its dependencies, and didn't like the requirement to AOT compile our own code. With thurber I started w/ this goal to avoid AOT, be highly dynamic in the repl, but was also able to focus on certain performance areas from the bottom-up. thurber also eschews sugared/dsl-ish api for more direct/explicit interop w the Beam SDK leaving this implementation concern to layers above (though I may implement if interest)-- just a different 'opinionated' take here. My org had been using Onyx for streaming use cases but its original developers have moved on and we were concerned with its long-term viability. Many of the ideals of thurber are consistent with Onyx's and reaching previous Onyx users like ourselves was another line of sight for thurber. We'd also looked at clj-headlights <https://github.com/logrhythm-oss/clj-headlights> - this was the other clojure Beam lib we'd surveyed in this space. On Wednesday, January 22, 2020 at 7:30:17 AM UTC-6, Dominic Parry wrote: > > Hi! > > Congratulations on the library! It makes me super happy when people build > clojure libraries for the Google cloud ecosystem. I wanted to draw your > attention to datasplash (https://github.com/ngrunwald/datasplash) which > has made a start on this. I thought perhaps you could leverage some of it. > > Hope you have a great day! > On 21 Jan 2020, 23:10 +0200, atdixon <atd...@gmail.com <javascript:>>, > wrote: > > Here is thurber (https://github.com/atdixon/thurber) (at early alpha > release) that enables Clojure on Apache Beam platforms like Google Dataflow. > > thurber's goals include: > > - Full support for Beam capabilities > - AOT-less (AOT not required; full dynamic support for serializing > functions, including inlined functions, and proxies) > - Macro-less (very few, always optional, macros) > - Performance focus (core optimized for large volume data streaming) > - Idiomatic Clojure focus (Clojure functions are automatically > distributable functional transforms, lazy sequences over iterative output, > ..) > > When coming to Apache Beam and wanting to use Clojure there are a few > hurdles to overcome, some discussed here in the past. Clojure's Java > interop commonly falls short in the domain of distributed big data Java > platforms (proxies and functions not serializable, no support for > generation of generic type signatures, minimal/insufficient support for > method annotations, suboptimal dynamic binding performance, etc) > > thurber bridges these issues internally, giving a full dynamic/Clojure > experience on top of Apache Beam. > > (For Onyx users, thurber + Beam meet the same ideals > <http://www.onyxplatform.org/docs/user-guide/0.14.x/#what-does-onyx-offer> as > Onyx on a well-backed platform.) > > This is early alpha release and feedback on the API & facilities are > welcome. > > For the curious, the walkthrough covers most of thurber capability: > https://github.com/atdixon/thurber/blob/master/demo/walkthrough.clj > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clo...@googlegroups.com <javascript:> > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clo...@googlegroups.com <javascript:> > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clo...@googlegroups.com <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/clojure/c18cc8e1-01c9-4688-bff3-6d50f128d0e4%40googlegroups.com > > <https://groups.google.com/d/msgid/clojure/c18cc8e1-01c9-4688-bff3-6d50f128d0e4%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/b7cda0e2-2f70-46f4-947e-511bfee09faf%40googlegroups.com.
