Re: Proposal: http2 wire format

On Mon, Mar 26, 2018 at 7:51 PM, Craig Ringer wrote: > There's been no visible consideration of overheads and comparison with > existing v3 protocol. Personally I'm fine with adding some protocol overhead > in bytes terms; low latency links have the bandwidth not to care much > compared to payload

Re: Proposal: http2 wire format

On 2018-03-29 17:52:07 -0400, Peter Eisentraut wrote: > On 3/29/18 14:20, Andres Freund wrote: > > On 2018-03-28 20:34:13 -0400, Peter Eisentraut wrote: > >> On 3/28/18 12:09, Andres Freund wrote: > >>> Yea, not the most descriptive... Returning multiple different resultsets > >>> from a function /

Re: Proposal: http2 wire format

On 3/29/18 14:20, Andres Freund wrote: > On 2018-03-28 20:34:13 -0400, Peter Eisentraut wrote: >> On 3/28/18 12:09, Andres Freund wrote: >>> Yea, not the most descriptive... Returning multiple different resultsets >>> from a function / procedure. Inability to do so is a serious limitation >>> of po

Re: Proposal: http2 wire format

On 2018-03-28 20:34:13 -0400, Peter Eisentraut wrote: > On 3/28/18 12:09, Andres Freund wrote: > > Yea, not the most descriptive... Returning multiple different resultsets > > from a function / procedure. Inability to do so is a serious limitation > > of postgres in comparison to some other languag

Re: Proposal: http2 wire format

> > > * room for other resultset formats later. Like Damir, I really want to > add > > protobuf or json serializations of result sets at some point, mainly so > we > > can return "entity graphs" in graph representation rather than left-join > > projection. > > -1. I don't think this belongs in post

Re: Proposal: http2 wire format

On 3/28/18 12:09, Andres Freund wrote: > Yea, not the most descriptive... Returning multiple different resultsets > from a function / procedure. Inability to do so is a serious limitation > of postgres in comparison to some other language with procedures. This is already possible as far as the pro

Re: Proposal: http2 wire format

Hi, On 2018-03-28 09:59:34 -0400, Tom Lane wrote: > Andres Freund writes: > > A few random, very tired, points: > > > - consolidated message for common tasks: > > - (bind, [describe?,] execute) to reduce overhead of prepared > > statement execution (both in messages, as well as branches) >

Re: Proposal: http2 wire format

Hi, On 2018-03-28 16:29:37 +0800, Craig Ringer wrote: > > - allow *streaming* of large datums > > Yes, very much +1 there. That's already on the wiki. Yeah: > > * Permit lazy fetches of large values, at least out-of-line TOASTED values > http://www.postgresql.org/message-id/53ff0ef8@2ndquadr

Re: Proposal: http2 wire format

Andres Freund writes: > A few random, very tired, points: > - consolidated message for common tasks: > - (bind, [describe?,] execute) to reduce overhead of prepared > statement execution (both in messages, as well as branches) > - (anonymous parse, bind, describe, execute) to make it chea

Re: Proposal: http2 wire format

> A few random, very tired, points: > > - consolidated message for common tasks: > - (bind, [describe?,] execute) to reduce overhead of prepared > statement execution (both in messages, as well as branches) > - (anonymous parse, bind, describe, execute) to make it cheaper to > send sta

Re: Proposal: http2 wire format

On 28 March 2018 at 16:02, Andres Freund wrote: > On 2018-03-26 22:44:09 +0200, Damir Simunic wrote: > > > *NONE* of the interesting problems are solved by HTTP2. You *still* > > > need a full blown protocol ontop of it. So no, this doesn't change > that. > > > > If you had to nominate only one o

Re: Proposal: http2 wire format

On 28 March 2018 at 00:42, Damir Simunic wrote: > > > I'm rapidly losing interest. Unless this goes back toward the concrete and > practical I think it's going nowhere. > > > > Your message is exactly what I was hoping for. Thanks for your guidance > and support, really appreciate you. > > Let me

Re: Proposal: http2 wire format

On 2018-03-26 22:44:09 +0200, Damir Simunic wrote: > > *NONE* of the interesting problems are solved by HTTP2. You *still* > > need a full blown protocol ontop of it. So no, this doesn't change that. > > If you had to nominate only one of those problems, which one would you > consider the most int

Re: Proposal: http2 wire format

> > > I'm rapidly losing interest. Unless this goes back toward the concrete and > practical I think it's going nowhere. Your message is exactly what I was hoping for. Thanks for your guidance and support, really appreciate you. Let me now get busy and earn your continued interest and suppo

Re: Proposal: http2 wire format

On 26 March 2018 at 22:56, Tom Lane wrote: > Damir Simunic writes: > >> On 26 Mar 2018, at 11:06, Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> wrote: > >>> If anyone finds the idea of Postgres speaking http2 appealing > > TBH, this sounds like a proposal to expend a whole lot of work (much

Re: Proposal: http2 wire format

Greetings, * Craig Ringer (cr...@2ndquadrant.com) wrote: > On 26 March 2018 at 21:05, Damir Simunic > wrote: > > The same goes for the ‘authorization’ header. Postgres does not support > > Bearer token authorization today. But maybe you’ll be able to define a > > function that knows how to deal w

Re: Proposal: http2 wire format

On 26 March 2018 at 21:05, Damir Simunic wrote: > > On 26 Mar 2018, at 11:06, Vladimir Sitnikov > wrote: > > > > Hi, > > > > >If anyone finds the idea of Postgres speaking http2 appealing > > > > HTTP/2 sounds interesting. > > What do you think of https://grpc.io/ ? > > > > Have you evaluated it

Re: Proposal: http2 wire format

On Mon, Mar 26, 2018 at 1:05 PM, Damir Simunic wrote: > Would it be the only protocol supported? What if I wanted JSON or CSV > returned, or just plain old Postgres v3 binary format, since I already have > the parser written for it? Wouldn’t you need to first solve the problem of > content negoti

Re: Proposal: http2 wire format

> Currently it is implemented via different v3 messages (parse, bind, execute, > row description, data row, etc etc). > > The claim is *any* implementation "on top of HTTP/2" would basically require > to implement those "parse, bind, execute, row data, etc" *messages*. Why? Wouldn’t you be abl

Re: Proposal: http2 wire format

On 26/03/18 21:57, Damir Simunic wrote: On 26 Mar 2018, at 15:42, Alvaro Hernandez > wrote: On 26/03/18 13:11, Damir Simunic wrote: On 26 Mar 2018, at 11:13, Vladimir Sitnikov mailto:sitnikov.vladi...@gmail.com>> wrote: Damir> * What are the criteria for getting

Re: Proposal: http2 wire format

Hi Andres, > > At least I do *NOT* want many protocols in core. We've a hard enough > time to keep up with integrating patches and maintenance to not just > willy nilly integrate multiple new features with unclear lifetimes. Admire your effort in applying all these patches—this commitfest thing

Re: Proposal: http2 wire format

Damir>Wouldn’t that be protocol semantics? Framing is already taken care of by the wire protocol. Apparently I'm using the wrong word. I do mean protocol semantics. Damir>But can you pull off grpc.. Would it be the only protocol supported? Of course there will be lots of "older clients" For inst

Re: Proposal: http2 wire format

It could make sense to arrange a Google Hangouts conversation (or alike). Hangouts allows to record sessions with up to 10 speakers and unlimited listeners. The recording can be shared via YouTube. Damir>Funny you agree with that—for someone having the experience of writing a driver and having a l

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 18:09, Vladimir Sitnikov > wrote: > > Damir>Postgres doesn’t know about grpc, s > > I'm afraid you are missing the point. > I would say PostgreSQL doesn't know about HTTP/2. > It is the same as "PostgreSQL doesn't know about grpc". > > Here's a quote from your pg_h2 repo:

Re: Proposal: http2 wire format

Hi, On 2018-03-26 20:36:09 +0200, Damir Simunic wrote: > If so, I’m not suggesting we get rid of FEBE, but leave it as is and > complement it with a widely understood and supported protocol, that in fact > takes compatibility way more seriously than FEBE. Just leave v3 frozen. Seems > like ulti

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 15:42, Alvaro Hernandez wrote: > > > > On 26/03/18 13:11, Damir Simunic wrote: >>> On 26 Mar 2018, at 11:13, Vladimir Sitnikov >> > wrote: >>> >>> Damir> * What are the criteria for getting this into the core? >>> Craig>Mine would be:

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 18:19, Vladimir Sitnikov > wrote: > > Tom>But starting from the assumption that HTTP2 solves our problems seems to > me to be "Here's a hammer. > > Agree. Funny you agree with that—for someone having the experience of writing a driver and having a long list of things th

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 16:56, Tom Lane wrote: > > Damir Simunic writes: >>> On 26 Mar 2018, at 11:06, Vladimir Sitnikov >>> wrote: If anyone finds the idea of Postgres speaking http2 appealing > > TBH, this sounds like a proposal to expend a whole lot of work (much of it > outside the cor

Re: Proposal: http2 wire format

On 26/03/18 13:11, Damir Simunic wrote: On 26 Mar 2018, at 11:13, Vladimir Sitnikov mailto:sitnikov.vladi...@gmail.com>> wrote: Damir> * What are the criteria for getting this into the core? Craig>Mine would be: +1 There's a relevant list as well: https://github.com/pgjdbc/pgjdbc/blob/mast

Re: Proposal: http2 wire format

Tom>But starting from the assumption that HTTP2 solves our problems seems to me to be "Here's a hammer. Agree. Just a side note: if v4 is ever invented I wish client language support is considered. It does take resources to implement message framing, and data parsing (e.g. int, timestamp, struct,

Re: Proposal: http2 wire format

Damir>Postgres doesn’t know about grpc, s I'm afraid you are missing the point. I would say PostgreSQL doesn't know about HTTP/2. It is the same as "PostgreSQL doesn't know about grpc". Here's a quote from your pg_h2 repo: >What we need is to really build a request object and correctly extract >

Re: Proposal: http2 wire format

Damir Simunic writes: >> On 26 Mar 2018, at 11:06, Vladimir Sitnikov >> wrote: >>> If anyone finds the idea of Postgres speaking http2 appealing TBH, this sounds like a proposal to expend a whole lot of work (much of it outside the core server, and thus not under our control) in order to get fr

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 11:06, Vladimir Sitnikov > wrote: > > Hi, > > >If anyone finds the idea of Postgres speaking http2 appealing > > HTTP/2 sounds interesting. > What do you think of https://grpc.io/ ? > > Have you evaluated it? > It does sound like a ready RPC on top of HTTP/2 with support

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 11:13, Vladimir Sitnikov > wrote: > > Damir> * What are the criteria for getting this into the core? > Craig>Mine would be: > > +1 > > There's a relevant list as well: > https://github.com/pgjdbc/pgjdbc/blob/master/backend_protocol_v4_wanted_features.md > >

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 12:47, Craig Ringer wrote: > > On 26 March 2018 at 17:34, Damir Simunic wrote: > > > > As you move forward with the PoC, consider: even if you decide not to > > become protocol-layer experts, you'll still need to become familiar > > with application-layer security in HTTP

Re: Proposal: http2 wire format

On 26 March 2018 at 17:34, Damir Simunic wrote: > > > As you move forward with the PoC, consider: even if you decide not to > > become protocol-layer experts, you'll still need to become familiar > > with application-layer security in HTTP. > > Good point. Application layer security is indeed a

Re: Proposal: http2 wire format

> On 26 Mar 2018, at 11:34, Craig Ringer wrote: > > On 26 March 2018 at 17:01, Damir Simunic > wrote: > > > > - Doesn't break new clients connecting to old servers > > > > Old server sends “Invalid startup packet” and closes the connection; client’s > T

Re: Proposal: http2 wire format

On 26 March 2018 at 17:01, Damir Simunic wrote: > > > - Doesn't break new clients connecting to old servers > > > > Old server sends “Invalid startup packet” and closes the connection; > client’s TLS layer reports an error. Does that count as not breaking new > clients? > > libpq would have to d

Re: Proposal: http2 wire format

Hi, > On 26 Mar 2018, at 06:47, Jacob Champion wrote: > > On Sun, Mar 25, 2018 at 8:11 PM, Craig Ringer wrote: >> As others have noted, you'll want to find a way to handle this in the least >> SSL-implementation-specific manner possible. IMO if it can't work with >> OpenSSL, Windows's SSL imple

Re: Proposal: http2 wire format

Damir> * What are the criteria for getting this into the core? Craig>Mine would be: +1 There's a relevant list as well: https://github.com/pgjdbc/pgjdbc/blob/master/backend_protocol_v4_wanted_features.md Vladimir

Re: Proposal: http2 wire format

Hi, >If anyone finds the idea of Postgres speaking http2 appealing HTTP/2 sounds interesting. What do you think of https://grpc.io/ ? Have you evaluated it? It does sound like a ready RPC on top of HTTP/2 with support for lots of languages. The idea of reimplementing the protocol for multiple l

Re: Proposal: http2 wire format

Hi, > On 26 Mar 2018, at 05:11, Craig Ringer wrote: > > On 26 March 2018 at 06:00, Damir Simunic wrote: > > > - Overhead for all clients. It may be tiny, but it needs to be > > measured and that cost needs to be weighed against the benefits. > > Maybe a cache miss in the context of a networ

Re: Proposal: http2 wire format

On Sun, Mar 25, 2018 at 8:11 PM, Craig Ringer wrote: > As others have noted, you'll want to find a way to handle this in the least > SSL-implementation-specific manner possible. IMO if it can't work with > OpenSSL, Windows's SSL implementation and OS X's SSL framework it's a > non-starter. +1. >

Re: Proposal: http2 wire format

On 26 March 2018 at 06:00, Damir Simunic wrote: > > - Overhead for all clients. It may be tiny, but it needs to be > > measured and that cost needs to be weighed against the benefits. > > Maybe a cache miss in the context of a network connection is > > negligible, but we do need to know. > >

Re: Proposal: http2 wire format

> On 25 Mar 2018, at 19:42, David Fetter wrote: > > On Sat, Mar 24, 2018 at 06:52:47PM +0100, Damir Simunic wrote: >> Hello hackers, >> >> I’d like to propose the implementation of new wire protocol using http2 >> framing. > > Welcome to the PostgreSQL community! This is a very interesting i

Re: Proposal: http2 wire format

On Sat, Mar 24, 2018 at 06:52:47PM +0100, Damir Simunic wrote: > Hello hackers, > > I’d like to propose the implementation of new wire protocol using http2 > framing. Welcome to the PostgreSQL community! This is a very interesting idea. Please send a patch to this mailing list on this thread.