On Tue, May 21, 2024 at 12:08 PM Jacob Burroughs
wrote:
> I think I thought I was writing about something else when I wrote that
> :sigh:. I think what I really should have written was a version of
> the part below, which is that we use streaming decompression, only
> decompress 8kb at a time, an
On Tue, May 21, 2024 at 1:24 PM Jacob Champion
wrote:
>
> We absolutely have to document the risks and allow clients to be
> written safely. But I think server-side controls on risky behavior
> have proven to be generally more valuable, because the server
> administrator is often in a better spot
On Tue, May 21, 2024 at 1:39 PM Jacob Champion
wrote:
>
> If the server doesn't reject compressed packets pre-authentication,
> then case 2 isn't mitigated. (I haven't proven how risky that case is
> yet, to be clear.) In other words: if the threat model is that a
> client can attack us, we should
On Tue, May 21, 2024 at 9:14 AM Jacob Burroughs
wrote:
> On Tue, May 21, 2024 at 10:43 AM Jelte Fennema-Nio wrote:
> > To help get everyone on the same page I wanted to list all the
> > security concerns in one place:
> >
> > 1. Triggering excessive CPU usage before authentication, by asking for
On Tue, May 21, 2024 at 8:23 AM Jacob Burroughs
wrote:
> As currently implemented, the compression only applies to
> CopyData/DataRow/Query messages, none of which should be involved in
> authentication, unless I've really missed something in my
> understanding.
Right, but Robert has argued that
On Tue, May 21, 2024 at 10:43 AM Jelte Fennema-Nio wrote:
>
> To help get everyone on the same page I wanted to list all the
> security concerns in one place:
>
> 1. Triggering excessive CPU usage before authentication, by asking for
> very high compression levels
> 2. Triggering excessive memory/
On Mon, 20 May 2024 at 21:42, Jacob Champion
wrote:
> As Andrey points out, there was prior work done that started to take
> this into account. I haven't reviewed it to see how good it is -- and
> I think there are probably many use cases in which queries and tables
> contain both private and atta
On Mon, May 20, 2024 at 2:42 PM Jacob Champion
wrote:
>
> I mean... you said it, not me. I'm trying not to rain on the parade
> too much, because compression is clearly very valuable. But it makes
> me really uncomfortable that we're reintroducing the compression
> oracle (especially over the auth
On Mon, May 20, 2024 at 4:12 PM Magnus Hagander wrote:
> That used to be the case in HTTP/1. But header compression was one of the
> headline features of HTTP/2, which isn't exactly new anymore. But there's a
> special algorithm, HPACK, for it. And then http/3 uses QPACK. Cloudflare has
> a pre
On Mon, May 20, 2024 at 9:09 PM Andrey M. Borodin
wrote:
>
>
>
> > On 20 May 2024, at 23:37, Robert Haas wrote:
> >
> > But, does this mean that we should just refuse to offer compression as
> > a feature?
>
> No, absolutely, we need the feature.
>
> > I guess I don't understand why TLS removed
On Mon, May 20, 2024 at 11:37 AM Robert Haas wrote:
> But if that's a practical
> attack, preventing compression prior to the authentication exchange
> probably isn't good enough
I mean... you said it, not me. I'm trying not to rain on the parade
too much, because compression is clearly very valu
On Mon, May 20, 2024 at 11:05 AM Andrey M. Borodin wrote:
> > So, the data would be compressed first, with framing around that, and
> > then transport encryption would happen afterwards. I don't see how
> > that would leak your password, but I have a feeling that might be a
> > sign that I'm about
> On 20 May 2024, at 23:37, Robert Haas wrote:
>
> But if that's a practical
> attack, preventing compression prior to the authentication exchange
> probably isn't good enough: the user could also try to guess what
> queries are being sent on behalf of other users through the same
> pooled c
On Mon, May 20, 2024 at 2:05 PM Andrey M. Borodin wrote:
> Compression defeats encryption. That's why it's not in TLS anymore.
> The thing is compression codecs use data self correlation. And if you mix
> secret data with user's data, user might guess how correlated they are.
Yeah, I'm aware tha
> On 20 May 2024, at 22:48, Robert Haas wrote:
>
> On Mon, May 20, 2024 at 1:23 PM Jacob Champion
> wrote:
>> On Mon, May 20, 2024 at 10:01 AM Robert Haas wrote:
>>> I really hope that you can't poke big enough holes to kill the feature
>>> entirely, though. Because that sounds sad.
>>
>> E
On Mon, May 20, 2024 at 1:23 PM Jacob Champion
wrote:
> On Mon, May 20, 2024 at 10:01 AM Robert Haas wrote:
> > I really hope that you can't poke big enough holes to kill the feature
> > entirely, though. Because that sounds sad.
>
> Even if there are holes, I don't think the situation's going to
On Mon, May 20, 2024 at 10:01 AM Robert Haas wrote:
> I really hope that you can't poke big enough holes to kill the feature
> entirely, though. Because that sounds sad.
Even if there are holes, I don't think the situation's going to be bad
enough to tank everything; otherwise no one would be abl
On Mon, May 20, 2024 at 12:49 PM Jacob Champion
wrote:
> ...and my response was that, no, the proposal doesn't seem to be
> requiring that authentication take place before compression is done.
> (As evidenced by your email. :D) If the claim is that there are no
> security problems with letting una
On Mon, May 20, 2024 at 8:29 AM Robert Haas wrote:
> It does occur to me that if some compression algorithm has a buffer
> overrun bug, restricting its use until after authentication might
> reduce the score of the resulting CVE, because now you have to be able
> to authenticate to make an exploit
On Mon, May 20, 2024 at 10:15 AM Jacob Champion
wrote:
> This is just restating the "you can already do bad things" argument. I
> understand that if your query gets executed, it's going to consume
> resources on the thing that's executing it (for the record, though,
> there are people working on c
On Sat, May 18, 2024 at 1:18 AM Jacob Burroughs
wrote:
> I like this more than what I proposed, and will update the patches to
> reflect this proposal. (I've gotten them locally back into a state of
> applying cleanly and dealing with the changes needed to support direct
> SSL connections, so refa
On Fri, May 17, 2024 at 4:03 PM Jelte Fennema-Nio wrote:
>
> On Fri, 17 May 2024 at 23:10, Jacob Champion
> wrote:
> > We're talking about a transport-level option, though -- I thought the
> > proposal enabled compression before authentication completed? Or has
> > that changed?
>
> I think it wo
On Fri, May 17, 2024 at 11:40 AM Robert Haas wrote:
>
> To be clear, I am not arguing that it should be the receiver's choice.
> I'm arguing it should be the client's choice, which means the client
> decides what it sends and also tells the server what to send to it.
> I'm open to counter-argument
On Fri, 17 May 2024 at 23:10, Jacob Champion
wrote:
> We're talking about a transport-level option, though -- I thought the
> proposal enabled compression before authentication completed? Or has
> that changed?
I think it would make sense to only compress messages after
authentication has complet
On Fri, 17 May 2024 at 23:40, Robert Haas wrote:
> To be clear, I am not arguing that it should be the receiver's choice.
> I'm arguing it should be the client's choice, which means the client
> decides what it sends and also tells the server what to send to it.
> I'm open to counter-arguments, bu
On Fri, May 17, 2024 at 4:53 PM Jacob Burroughs
wrote:
> I think I was more thinking that trying to let both parties control
> the parameter seemed like a recipe for confusion and sadness, and so
> the choice that felt most natural to me was to let the sender control
> it, but I'm definitely open
On Fri, May 17, 2024 at 1:53 PM Jacob Burroughs
wrote:
> New proposal, predicated on the assumption that if you enable
> compression you are ok with the client picking whatever level they
> want. At least with the currently enabled algorithms I don't think
> any of them are so insane that they wo
On Thu, May 16, 2024 at 3:28 AM Robert Haas wrote:
>
> Well, I mean, I don't really know what the right answer is here, but
> right now I can say pg_dump --compress=gzip to compress the dump with
> gzip, or pg_dump --compress=gzip:9 to compress with gzip level 9. Now,
> say that instead of compres
On Wed, May 15, 2024 at 12:50 PM Jacob Burroughs
wrote:
> I think I would agree with that. That said, I don't think the client
> should be in the business of specifying what configuration of the
> compression algorithm the server should use. The server administrator
> (or really most of the time
On Wed, May 15, 2024 at 11:31 AM Robert Haas wrote:
> From my point of view, it's the client who knows what it wants to do
> with the connection. If the client plans to read a lot of data, it
> might want the server to compress that data, especially if it knows
> that it's on a slow link. If the c
On Wed, May 15, 2024 at 12:24 PM Jacob Burroughs
wrote:
> > But now I'm wondering whether these options should really be symmetric
> > on the client and server sides? Isn't it for the server just to
> > specify a list of acceptable algorithms, and the client to set the
> > compression options? If
On Wed, May 15, 2024 at 8:38 AM Robert Haas wrote:
>
> I agree with that goal, but I'm somewhat confused by how your proposal
> achieves it. You had
> libpq_compression=lzma:client_to_server=off;gzip:server_to_client=off,
> so how do we parse that? Is that two completely separate
> specifications,
On Tue, May 14, 2024 at 5:21 PM Jacob Burroughs
wrote:
> The reason for both the semicolons and for not doing this is related
> to using the same specification structure as here:
> https://www.postgresql.org/docs/current/app-pgbasebackup.html
> (specifically the --compress argument).
I agree with
On Tue, May 14, 2024 at 3:24 PM Robert Haas wrote:
>
> IMHO, that's a HUGE improvement. But:
>
> * I would probably change is the name "libpq_compression", because
> even though we have src/backend/libpq, we typically use libpq to refer
> to the client library, not the server's implementation of t
On Tue, May 14, 2024 at 3:22 PM Jacob Burroughs
wrote:
> What if we went with:
> Server side:
> * `libpq_compression=on` (I just want everything the server supports
> available; probably the most common case)
> * `libpq_compression=off` (I don't want any compression ever with this server)
> * `lib
On Tue, May 14, 2024 at 1:35 PM Robert Haas wrote:
>
> Well, in my last response before the thread died, I complained that
> libpq_compression=gzip:compress=off was confusing, and I stand by
> that, because "compress" is used both in the name of the parameter and
> as an option within the value of
On Tue, May 14, 2024 at 12:30 PM Jacob Burroughs
wrote:
> I've withdrawn this patch from the commitfest. I had been waiting for
> some resolution on "Add new protocol message to change GUCs for usage
> with future protocol-only GUCs" before I rebased/refactored this one,
> because this would be i
On Tue, May 14, 2024 at 11:08 AM Robert Haas wrote:
>
> According to https://commitfest.postgresql.org/48/4746/ this patch set
> needs review, but:
>
> 1. Considering that there have been no updates for 5 months, maybe
> it's actually dead?
I've withdrawn this patch from the commitfest. I had be
On Fri, Jan 12, 2024 at 4:11 PM Robert Haas wrote:
> I think that would definitely be better than "compress" and
> "decompress," but I was worried that it might be unclear to the user
> whether the parameter that they specified was from the point of view
> of the client or the server. Perhaps that
On Fri, Jan 12, 2024 at 4:02 PM Jacob Burroughs
wrote:
> > I wonder if we could use "upstream" and "downstream" to be clearer? Or
> > some other terminology?
>
> What about `send` and `receive`?
I think that would definitely be better than "compress" and
"decompress," but I was worried that it mi
> I wonder if we could use "upstream" and "downstream" to be clearer? Or
> some other terminology?
What about `send` and `receive`?
On Sun, Dec 31, 2023 at 2:32 AM Jacob Burroughs
wrote:
> I ended up reworking this to use a version of this option in place of
> the `none` hackery, but naming the parameters `compress` and
> `decompress, so to disable compression but allow decompression you
> would specify `libpq_compression=gzip
On Fri, 29 Dec 2023 at 11:02, Andrey M. Borodin wrote:
> This patchset allows sending CompressionMethod message, which allows to set
> another codec\level picked from the set of negotiated codec sets (during
> startup).
Did you mean to attach a patchset? I don't see the CompressionMethod
messag
> On 21 Dec 2023, at 05:30, Jelte Fennema-Nio wrote:
>
> One thing I'm wondering: should it be possible for the client to change the
> compression it wants mid-connection?
This patchset allows sending CompressionMethod message, which allows to set
another codec\level picked from the set of
Thanks for working on this!
One thing I'm wondering: should it be possible for the client to change the
compression it wants mid-connection? I can think of some scenarios where
that would be useful to connection poolers: if a pooler does plain
forwarding of the compressed messages, then it would n
> I'm having difficulty understanding the details of this handshaking
> algorithm from this description. It seems good that the handshake
> proceeds in each direction somewhat separately from the other, but I
> don't quite understand how the whole thing fits together. If the
> client tells the serv
On Tue, Dec 19, 2023 at 11:41 AM Jacob Burroughs
wrote:
> The compression "handshaking" process now looks as follows:
> 1. Client sends startup packet with `_pq_.libpq_compression = alg1;alg2`
> 2. At this point, the server can immediately begin compressing packets
> to the client with any of the
> I believe this patch series is ready for detailed review/testing, with one
> caveat: as can be seen here https://cirrus-ci.com/build/6732518292979712 ,
> the build is passing on all platforms and all tests except for the primary
> SSL test on Windows.
One correction: I apparently missed a ker
48 matches
Mail list logo
