EDN/Transit on the backend might be nice, but a little much for me. I'll take it if you feel like a lot of C. jsonb in 9.4 will have to do for now.
Transparent encoding/decoding is already built-in to the client though, even with array support to encode a clojure set to postgres text[] and back. See: https://github.com/thheller/shadow-pgsql/blob/master/src/examples/repl.clj https://github.com/thheller/shadow-pgsql/blob/90639eabc9f70ab9f8898b47088e7c07baa4f9d7/test/shadow/pgsql_test.clj#L45-L63 edn-type is in there too, just need to start writing proper tests and documentation. Cheers, /thomas On Saturday, August 23, 2014 11:24:14 PM UTC+2, Jason Jackson wrote: > > This makes me wonder if it would make sense to create a Postgresql > plugin that adds a keyword type and other similar types, that would > allow for a more precise roundtrip between Clojure and Postgres. > > Jason > > On Sat, Aug 23, 2014 at 11:23 AM, Thomas Heller <th.h...@gmail.com > <javascript:>> wrote: > > Hey Kyle, > > > > thanks for the Feedback. Appreciate it. > > > > I think you misunderstood the meaning of a "type" in shadow-psql. A > "type" > > is merely the format of how a given value is represented on the wire > since > > the backend needs to understand what we send it. Postgres supports 2 > > different Wire Formats: Text and Binary. While Text is considered the > > "default", binary is usually a lot more efficient. pgjdbc for example > only > > supports the text format. I try to be binary first, which works for most > > types so far. (Numeric is giving me trouble, but I'll eventually figure > that > > out). I allow overwriting the "types" cause not everything I store in > > postgres is understood by it (EDN, Keywords, ...). By hooking directly > into > > the encode/decode code I can efficiently do the transformation > on-the-fly. > > In my completely unscientific preliminary benchmark I cut the query time > > from pgjdbc 650ms to shadow-pgsql 200ms and that is for very simple > types > > (50k rows) with no optimizations done yet. I expect the difference to be > > much larger if you use a timestamp, timestamptz or bytea for example, as > the > > text format for those types carries a bit more overhead. But once > everything > > is stable I will do some real benchmarks. Better performance was not the > > reason I wrote this, just a pleasant side-effect. > > > > As for the amount of work: its pretty much done. Some more exotic > features > > need to be implemented, but those were never available via JDBC anyways > (eg. > > COPY). I think its stable enough that I will begin moving my projects > > "soon", when I release everything to production I'll probably release a > > 1.0.0-RC. shadow-pgsql can not be layered on top of JDBC, well > technically > > thats what I did for the last 2 years > > (https://gist.github.com/thheller/de7ecce0db58130ae6b7) BUT it required > some > > ugly reflection calls since the PGJDBC does not expose all the > information I > > needed. In the end I decided that I'd feel better to rewrite everything > from > > scratch as the documentation is quite good and the protocol is simple. > > > > Since the non-Java world does just fine without JDBC, I think we can do > too. > > ;) Also, the "Illusion" JDBC provides that you can just switch databases > if > > you feel like it only holds until you start using postgres-specific > > features. Not all databases have arrays, hstore or json types. > > > > Regards, > > /thomas > > > > > > On Saturday, August 23, 2014 5:12:30 PM UTC+2, Kyle Cordes wrote: > >> > >> On Thursday, August 21, 2014 at 1:00 PM, Thomas Heller wrote: > >> > Hey Clojure Folk, > >> > > >> > I'm close to releasing the first alpha version of > >> > https://github.com/thheller/shadow-pgsql a "native" interface to > PostgreSQL > >> > I wrote. > >> > > >> > Its an implementation of the native binary protocol without any > intent > >> > to ever support JDBC. Mostly because that provides a bunch of > features I > >> > never use, but no support for features I wanted. It is mostly Java > but I > >> > will probably only use it from Clojure so that is my primary goal > going > >> > forward. I think the Java bits are close to stable. > >> > > >> > I'm looking for interested beta testers and feedback. I'm bad at > writing > >> > docs cause I never know where to start since there are so many > features and > >> > differences to JDBC. > >> > > >> > >> > >> As a user of both Postgres and Clojure, I find this very interesting. > It’s > >> helps with a couple of pain points around JDBC, such the fact that any > >> nonstandard feature ends up hidden behind a untyped interface passing > >> strings around. But I also have a couple of bits of feedback that are a > >> little more skeptical: > >> > >> First, the amount of work it will take to get this to a complete enough > >> state that large projects could safely switch to it, could be > substantial. > >> It makes me wonder if, instead, this could be built as a layer up on > top of > >> the Postgres JDBC driver. This would not be as elegant because it would > not > >> strip out as much unnecessary code, but it may be quite a lot less > work. > >> > >> Second, it seems to most effectively target people who are both very > type > >> oriented, yet are using Java or Clojure. It seems to me that folks who > are > >> so concerned with types that they would step away from the standard way > of > >> talking to databases generically, might be found over in the community > of > >> people using more rigidly typed languages like Haskell etc. > >> > >> Third, although I like the idea of leveraging the features of the tool > you > >> are using (like Postgres), at the same time experiences taught me that, > the > >> more firmly a project seems destined to never switch to a different > brand of > >> database, the more likely some future unexpected opportunity will come > up > >> where that is exactly what is needed. I suppose this is just Murphy’s > Law. > >> > >> I don’t want to sound discouraging though, I really like this idea. > >> > >> -- > >> Kyle Cordes > >> http://kylecordes.com > >> > >> > >> > > -- > > You received this message because you are subscribed to the Google > > Groups "Clojure" group. > > To post to this group, send email to clo...@googlegroups.com > <javascript:> > > Note that posts from new members are moderated - please be patient with > your > > first post. > > To unsubscribe from this group, send email to > > clojure+u...@googlegroups.com <javascript:> > > For more options, visit this group at > > http://groups.google.com/group/clojure?hl=en > > --- > > You received this message because you are subscribed to the Google > Groups > > "Clojure" group. > > To unsubscribe from this group and stop receiving emails from it, send > an > > email to clojure+u...@googlegroups.com <javascript:>. > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
