This makes me wonder if it would make sense to create a Postgresql plugin that adds a keyword type and other similar types, that would allow for a more precise roundtrip between Clojure and Postgres.
Jason On Sat, Aug 23, 2014 at 11:23 AM, Thomas Heller <th.hel...@gmail.com> wrote: > Hey Kyle, > > thanks for the Feedback. Appreciate it. > > I think you misunderstood the meaning of a "type" in shadow-psql. A "type" > is merely the format of how a given value is represented on the wire since > the backend needs to understand what we send it. Postgres supports 2 > different Wire Formats: Text and Binary. While Text is considered the > "default", binary is usually a lot more efficient. pgjdbc for example only > supports the text format. I try to be binary first, which works for most > types so far. (Numeric is giving me trouble, but I'll eventually figure that > out). I allow overwriting the "types" cause not everything I store in > postgres is understood by it (EDN, Keywords, ...). By hooking directly into > the encode/decode code I can efficiently do the transformation on-the-fly. > In my completely unscientific preliminary benchmark I cut the query time > from pgjdbc 650ms to shadow-pgsql 200ms and that is for very simple types > (50k rows) with no optimizations done yet. I expect the difference to be > much larger if you use a timestamp, timestamptz or bytea for example, as the > text format for those types carries a bit more overhead. But once everything > is stable I will do some real benchmarks. Better performance was not the > reason I wrote this, just a pleasant side-effect. > > As for the amount of work: its pretty much done. Some more exotic features > need to be implemented, but those were never available via JDBC anyways (eg. > COPY). I think its stable enough that I will begin moving my projects > "soon", when I release everything to production I'll probably release a > 1.0.0-RC. shadow-pgsql can not be layered on top of JDBC, well technically > thats what I did for the last 2 years > (https://gist.github.com/thheller/de7ecce0db58130ae6b7) BUT it required some > ugly reflection calls since the PGJDBC does not expose all the information I > needed. In the end I decided that I'd feel better to rewrite everything from > scratch as the documentation is quite good and the protocol is simple. > > Since the non-Java world does just fine without JDBC, I think we can do too. > ;) Also, the "Illusion" JDBC provides that you can just switch databases if > you feel like it only holds until you start using postgres-specific > features. Not all databases have arrays, hstore or json types. > > Regards, > /thomas > > > On Saturday, August 23, 2014 5:12:30 PM UTC+2, Kyle Cordes wrote: >> >> On Thursday, August 21, 2014 at 1:00 PM, Thomas Heller wrote: >> > Hey Clojure Folk, >> > >> > I'm close to releasing the first alpha version of >> > https://github.com/thheller/shadow-pgsql a "native" interface to PostgreSQL >> > I wrote. >> > >> > Its an implementation of the native binary protocol without any intent >> > to ever support JDBC. Mostly because that provides a bunch of features I >> > never use, but no support for features I wanted. It is mostly Java but I >> > will probably only use it from Clojure so that is my primary goal going >> > forward. I think the Java bits are close to stable. >> > >> > I'm looking for interested beta testers and feedback. I'm bad at writing >> > docs cause I never know where to start since there are so many features and >> > differences to JDBC. >> > >> >> >> As a user of both Postgres and Clojure, I find this very interesting. It’s >> helps with a couple of pain points around JDBC, such the fact that any >> nonstandard feature ends up hidden behind a untyped interface passing >> strings around. But I also have a couple of bits of feedback that are a >> little more skeptical: >> >> First, the amount of work it will take to get this to a complete enough >> state that large projects could safely switch to it, could be substantial. >> It makes me wonder if, instead, this could be built as a layer up on top of >> the Postgres JDBC driver. This would not be as elegant because it would not >> strip out as much unnecessary code, but it may be quite a lot less work. >> >> Second, it seems to most effectively target people who are both very type >> oriented, yet are using Java or Clojure. It seems to me that folks who are >> so concerned with types that they would step away from the standard way of >> talking to databases generically, might be found over in the community of >> people using more rigidly typed languages like Haskell etc. >> >> Third, although I like the idea of leveraging the features of the tool you >> are using (like Postgres), at the same time experiences taught me that, the >> more firmly a project seems destined to never switch to a different brand of >> database, the more likely some future unexpected opportunity will come up >> where that is exactly what is needed. I suppose this is just Murphy’s Law. >> >> I don’t want to sound discouraging though, I really like this idea. >> >> -- >> Kyle Cordes >> http://kylecordes.com >> >> >> > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
