quire some coordination between the
parent/child analyzes. Essentially, any time a child is analyzed, the
parent should rebuild the stats (to merge from the new child stats).
This is similar to the issue of analyzing the parent too often because
we don't know when exactly the counters get updated,
t; +elog(WARNING, "stats for %d %d not found",
>> +RelationGetRelid(rels[j]),
>> vacattrstats[i]->attr->attnum);
>
> should be %u %d
>
> This code duplication is undesirable:
>> +
On 3/29/21 9:24 PM, Tomas Vondra wrote:
>
>
> On 3/29/21 8:36 PM, Justin Pryzby wrote:
>> Thanks for taking a fresh look at this.
>>
>> As you've written it, this can apply to either/both partitioned or
>> inheritence.
>> I imagine when
ers
and so on).
Haven't tried, but might be worth exploring in the future.
regards
[1] https://www.postgresql.org/message-id/7363.1426537...@sss.pgh.pa.us
[2] https://www.postgresql.org/message-id/22598.1425686...@sss.pgh.pa.us
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
delta before resetting the values to 0). Maybe multi-level
partitioning needs some additional handling, not sure.
regards
[1] https://commitfest.postgresql.org/33/3052/
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 3/30/21 4:09 AM, Tomas Vondra wrote:
> Hi,
>
> ...
>
> We may need to "sync" the counts for individual relations in a couple
> places (e.g. after the worker is done with the leaf, it should propagate
> the remaining delta before resetting the values to 0). M
E and XXX, describing stuff I
ignored to keep it simple, possible future improvement. And so on.
regards
[1]
https://www.postgresql.org/message-id/flat/71d67391-16a9-3e5e-b5e4-8f7fd32cc...@postgrespro.ru
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL C
gnores the mask. But that's a separate
issue.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
rors, which I think might cause
random assert failures (but in practice it's harmless, in the worst case
we'll merge the ranges a bit differently).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/src/backend/ac
; Should the length check precede the addra[i] ?
> Something like:
>
> unsigned char a;
> if (i >= lena) a = 0;
> else a = addra[i];
>
I don't think that makes any difference. We know the bytes are there, we
just want to ignore / r
55'::inet - '1.1.0.255'::inet;
?column?
--
512
(1 row)
test=# select '1.1.2.255'::inet - '1.1.1.0'::inet;
?column?
--
511
(1 row)
So I'll keep the 256.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
l_cmp_value), but the formula in this function
uses 31. So either we can tweak that (seems to fix it for me), or maybe
just switch to interval_cmp_value directly.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
a randomized in various
ways - the scripts are available here:
https://github.com/tvondra/brin-randomized-tests
It was focused on discovering issues in the distance functions, and
comparing results with/without the index. Maybe the next step should be
adding some changes to the data, which might trigg
.102/32'::inet - '10.1.1.142/24'::inet;
?column?
--
-40
(1 row)
That is, (a>b) but then (a-b) < 0. AFAICS it's due to comparator
considering the mask, while the minus ignores it. I find it a bit
strange, but I assume it's intentional.
regards
--
value - the
stats collector might have received other messages modifying those
counters (including e.g. PGSTAT_MTYPE_ANALYZE with a reset), so maybe we
can get into situation with
(changes_since_analyze_reported > changes_since_analyze)
if we just blindly increment the value. I'd bet would l
On 4/4/21 10:05 PM, Alvaro Herrera wrote:
> On 2021-Apr-04, Tomas Vondra wrote:
>
>> 1) I still don't understand why inheritance and declarative partitioning
>> are treated differently. Seems unnecessary nad surprising, but maybe
>> there's a good reason?
&g
On 4/4/21 9:08 PM, Tomas Vondra wrote:
> On 4/3/21 9:42 PM, Alvaro Herrera wrote:
>> Thanks for the quick rework. I like this design much better and I think
>> this is pretty close to committable. Here's a rebased copy with some
>> small cleanups (m
gt; rule.
>
Not sure, but the removal seems unnecessary. I'm worried that this will
significantly reduce the amount of data that we'll be able to prefetch.
How likely it is that we have data that is written but not flushed?
Let's assume the replica is lagging and network band
seems to mostly match your observation - there's a small
performance hit (~2%), although that might be due to changes in the
layout of the binary. And v4+v5 improves that a bit (even compared to
master), although I don't see the same 20% speedup.
I see +25% improvement, but onl
analyzed more times than it should.
> Sounds like we should not send the ancestor list when a column list is
> given to manual analyze. I haven't verified this, however.)
>
Are you sure? I haven't tried, but shouldn't this be prevented by only
sending the delta between
ed 1000, but that's just an arbitrary value and the
actual data might have any other value. And it's unlikely that all three
columns will have the same number of distinct values.
Of course, this also depends on the false positive rate.
FWIW I doubt people are using multi-column BRIN indexes very often.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 4/8/21 1:46 PM, Thomas Munro wrote:
> On Thu, Apr 8, 2021 at 3:27 AM Tomas Vondra
> wrote:
>> On 4/7/21 1:24 PM, Thomas Munro wrote:
>>> I made one other simplifying change: previously, the prefetch module
>>> would read the WAL up to the "written" L
On 4/8/21 4:49 PM, Jaime Casanova wrote:
> On Thu, Apr 08, 2021 at 12:18:36PM +0200, Tomas Vondra wrote:
>> On 4/8/21 9:08 AM, Jaime Casanova wrote:
>>> Hi everyone,
>>>
>>> When testing brin bloom indexes I noted that we need to reduce the
>>> PA
On 4/8/21 5:27 PM, Alvaro Herrera wrote:
> On 2021-Apr-08, Tomas Vondra wrote:
>
>> On 4/8/21 5:22 AM, Alvaro Herrera wrote:
>
>>> However, I just noticed there is a huge problem, which is that the new
>>> code in relation_needs_vacanalyze() is doing find_a
for that plan node, not sure about selectivity of
the other quals, though.
It's also a bit strange that we get speedup for "simple" protocol, while
for "prepared" it gets slower. That seems counter-intuitive, because why
should we see opposite outcomes in those cases?
Thoughts?
>
I think you're right indexes may be a serious bottleneck in some cases,
so exploring ways to improve that seems useful. Ultimately I think we
should be looking for ways to reduce the amount of work we need to do,
but parallelizing it (i.e. doing the same amount of work but in multiple
processes) is a valid approach too.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
;ll re-ATTACH it shortly
> afterwards.
>
> However, I think DROP should be handled ?
>
IMHO we should prefer the default behavior which favors having updated
statistics, and maybe have a way to override it for individual commands.
So ATTACH would update changes_since_analyze by d
On 4/9/21 1:21 AM, David Rowley wrote:
> On Fri, 9 Apr 2021 at 09:32, Tomas Vondra
> wrote:
>>
>> I ran the same set of benchmarks on the v6, which I think should be
>> mostly identical to what was committed. I extended the test a bit to
>> test table with 0, 1,
that's necessary. Repeated attach/detach of the same partition may bloat
the value, but I guess that's an example of "If it hurts don't do it."
What I think we might do is offer some light-weight analyze variant,
e.g. based on the merging of statistics (I've posted a PoC patch a
couple days ago.). That would make the ANALYZEs on parent much cheaper,
so those "unnecessary" analyzes would not be an issue.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 4/11/21 12:03 AM, David Rowley wrote:
> On Sat, 10 Apr 2021 at 10:32, Tomas Vondra
> wrote:
>> But I think the puzzle is not so much about v5 vs v6, but more about v5
>> vs. master. I still don't understand how v5 managed to be faster than
>> master, bu
m master.
>
> As I said I will dig into it and figure it out.
>
Well, plr.h does this:
#define WARNING 19
#define ERROR 20
which seems a bit weird, because elog.h does this (since 1f9158ba481):
#define WARNING 19
#define WARNING_CLIENT_ONLY 20
#define
or
> alternatively just use the schema (see
> https://github.com/swarm64/s64da-benchmark-toolkit/tree/master/benchmarks/tpcds/schemas/psql_native)
>
Thanks, I'll see if I can reproduce that with your schema.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
enerally best to accomplish
> objective X in the same way that existing code does. Deciding
> that you have a better way is often wrong, and even if you
> are right, you should then submit a patch to change all the
> existing cases.)
>
> I was confused here, doe
eability and restricting access to the OS. It's
unfortunate that we tie the this capability to being superuser, so maybe
the right solution would be to introduce a separate role with this
privilege?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
it to the various
selectivity estimators. I don't think anything will be actually looking
at those fields, but initializing them seems easy enough.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On Thu, Jul 23, 2020 at 07:33:45PM -0700, Peter Geoghegan wrote:
On Thu, Jul 23, 2020 at 6:22 PM Tomas Vondra
wrote:
So let me share some fresh I/O statistics collected on the current code
using iosnoop. I've done the tests on two different machines using the
"aggregate part" of
On Fri, Jul 24, 2020 at 10:40:47AM +0200, Tomas Vondra wrote:
On Thu, Jul 23, 2020 at 07:33:45PM -0700, Peter Geoghegan wrote:
On Thu, Jul 23, 2020 at 6:22 PM Tomas Vondra
wrote:
So let me share some fresh I/O statistics collected on the current code
using iosnoop. I've done the tests o
On Fri, Jul 24, 2020 at 11:18:48AM -0400, Robert Haas wrote:
On Thu, Jul 23, 2020 at 9:22 PM Tomas Vondra
wrote:
2MB 4MB8MB64MB256MB
---
hash 6.716.70 6.736.44
constant in all these cases?
Surely more recursive spilling should do more I/O, but the Disk Usage
reported by explain analyze does not show anything like ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
the moment we don't have that capability. So we end up
processing everything in memory, but significantly exceeding work_mem.
FWIW all my tests are done on the same TPC-H data set clustered by
l_shipdate (so probably random with respect to other columns).
regards
--
Tomas Vondra h
On Sat, Jul 25, 2020 at 05:13:00PM -0700, Peter Geoghegan wrote:
On Sat, Jul 25, 2020 at 5:05 PM Tomas Vondra
wrote:
I'm not sure what you mean by "reported memory usage doesn't reflect the
space used for transition state"? Surely it does include that, we've
built th
) look like? Is it particularly
rare / special case, or something reasonable to expect in practice?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Jul 02, 2020 at 03:09:29PM +1200, Thomas Munro wrote:
On Sat, Jun 6, 2020 at 12:36 PM Stephen Frost wrote:
* Tomas Vondra (tomas.von...@2ndquadrant.com) wrote:
> I wonder if we can collect some stats to measure how effective the
> prefetching actually is. Ultimately we want som
cture (MVCC, WAL, backups,...).
SO I hope it can be useful when standard indexes becomes bottleneck.
Isn't it a bit suspicious that with more clients the throughput actually
drops significantly? Is this merely due to PoC stage, or is there some
inherent concurrency bottleneck?
regards
--
On Tue, Aug 04, 2020 at 05:36:51PM +0300, Alexander Korotkov wrote:
Hi, Tomas!
Sorry for the late reply.
On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra
wrote:
I think there's a number of weak points in this approach.
Firstly, it assumes the summaries can be represented as arrays of
bui
d suggest incrementing the version every time an updated version is
submitted, even if it's just a rebased version. It makes it clearer
which version of the code is being discussed, etc.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote
On Tue, Aug 04, 2020 at 08:18:01PM +0300, Konstantin Knizhnik wrote:
On 04.08.2020 18:11, Tomas Vondra wrote:
On Tue, Aug 04, 2020 at 11:22:13AM +0300, Konstantin Knizhnik wrote:
Hi hackers,
I want to share results of my last research of implementing LSM
index in Postgres.
Most of modern
On Thu, Aug 06, 2020 at 02:58:44PM +1200, Thomas Munro wrote:
On Tue, Aug 4, 2020 at 3:47 AM Tomas Vondra
wrote:
On Thu, Jul 02, 2020 at 03:09:29PM +1200, Thomas Munro wrote:
>FYI I am still trying to reproduce and understand the problem Tomas
>reported; more soon.
Any luck try
during recovery in smgrnblocks.
It's not clear to me how would knowing the relation size help reducing
the overhead of this patch?
Can't we somehow identify cases when this optimization might help and
only actually enable it in those cases? Like in a recovery, with a lot
of truncates,
On Fri, Aug 07, 2020 at 10:08:23AM +0300, Konstantin Knizhnik wrote:
On 07.08.2020 00:33, Tomas Vondra wrote:
Unfortunately Konstantin did not share any details about what workloads
he tested, what config etc. But I find the "no regression" hypothesis
rather hard to believe, bec
mance penalty, at worst.
Unless there are any objections, I plan to push this to master only in
about 10 hours time.
Seems OK to me. Thanks.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
column per type is going to scale in the long run.)
I wouldn't expect a huge number of types. I can imagine maybe twice the
current number of types, but not much more. But I'm not sure the output
is easy to read even now ...
regards
--
Tomas Vondra http://www.2ndQu
and what was built is in pg_statistic_ext_data. But if we want the
output to show both what was requested and which types were actually
built, that'll effectively double the number of columns needed :-(
Also, it might be useful to show the size of the statistics built, just
like we show for \d+
sn't apply generally.
Not sure. I doubt it has anything to do with the hardware page size,
that's mostly transparent to the kernel anyway. But it might be that the
prefetching on a particular SSD has more overhead than what it saves.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sat, Aug 29, 2020 at 06:43:47PM -0400, Alvaro Herrera wrote:
On 2020-Aug-29, Tomas Vondra wrote:
But if we want the
output to show both what was requested and which types were actually
built, that'll effectively double the number of columns needed :-(
I was thinking it would be one c
of
partitions/batches was very different, but the query duration was
mostly independent (almost constant).
FWIW I still haven't seen any explanation why the current code spills
more data than the CP_SMALL_TLIST patch (which was reverted).
regards
--
Tomas Vondra http://w
On Sun, Aug 30, 2020 at 02:26:20AM +0200, Tomas Vondra wrote:
On Fri, Aug 28, 2020 at 06:32:38PM -0700, Jeff Davis wrote:
On Thu, 2020-08-27 at 17:28 -0700, Peter Geoghegan wrote:
We have a Postgres 13 open item for Disk-based hash aggregate, which
is the only non-trivial open item. There is a
On Sun, Aug 30, 2020 at 12:33:29PM -0400, Alvaro Herrera wrote:
On 2020-Aug-30, Tomas Vondra wrote:
On Sat, Aug 29, 2020 at 06:43:47PM -0400, Alvaro Herrera wrote:
> On 2020-Aug-29, Tomas Vondra wrote:
> > Also, it might be useful to show the size of the statistics built, just
&g
t (even
if it's just in the \dX+ variant).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Aug 31, 2020 at 10:28:38AM -0400, Alvaro Herrera wrote:
On 2020-Aug-30, Tomas Vondra wrote:
On Sun, Aug 30, 2020 at 12:33:29PM -0400, Alvaro Herrera wrote:
> I wonder how to report that. Knowing that psql \-commands are not meant
> for anything other than human consumption,
On Mon, Aug 31, 2020 at 12:18:09PM -0400, Alvaro Herrera wrote:
On 2020-Aug-31, Tomas Vondra wrote:
I wonder if trying to list info about all stats from the statistics
object in a single line is necessary. Maybe we should split the info
into one line per statistics, so for example
CREATE
On Mon, Aug 31, 2020 at 11:34:34PM -0700, Jeff Davis wrote:
On Sun, 2020-08-30 at 17:03 +0200, Tomas Vondra wrote:
So I'm wondering if the hashagg is not ignoring similar non-I/O costs
for the spilling case. In particular, the initial section computing
startup_cost seems to ignore that w
On Thu, Aug 13, 2020 at 06:57:20PM +1200, Thomas Munro wrote:
On Thu, Aug 6, 2020 at 10:47 PM Tomas Vondra
wrote:
On Thu, Aug 06, 2020 at 02:58:44PM +1200, Thomas Munro wrote:
>On Tue, Aug 4, 2020 at 3:47 AM Tomas Vondra
>> Any luck trying to reproduce thigs? Should I try again an
On Wed, Sep 02, 2020 at 02:05:10AM +1200, Thomas Munro wrote:
On Wed, Sep 2, 2020 at 1:14 AM Tomas Vondra
wrote:
from the archive
Ahh, so perhaps that's the key.
Maybe. For the record, the commands look like this:
archive_command = 'gzip -1 -c %p > /mnt/raid/wal-
e final
patch, but I agree we can add it after improving the costing etc.
* The costing part is pretty rudimentary and looks only at the first group.
It's mostly hand crafted to pass the existing tests.
OK, understood.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
On Tue, Sep 01, 2020 at 12:58:59PM -0700, Jeff Davis wrote:
On Tue, 2020-09-01 at 11:19 +0200, Tomas Vondra wrote:
Why? I don't think we need to change costing of in-memory HashAgg. My
assumption was we'd only tweak startup_cost for cases with spilling
by
adding some
where we don't allow specifying that in CREATE TABLE and you have to do
ALTER TABLE ... ALTER COLUMN ... SET STATISTICS n;
as a separate step.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Sep 01, 2020 at 03:09:14PM -0700, Peter Geoghegan wrote:
On Tue, Sep 1, 2020 at 2:09 PM Tomas Vondra
wrote:
>* Instead of changing the order directly, now patch creates another patch with
> modifier order of clauses. It does so for the normal sort as well as for
> increme
untEntry() -> dynahash that could
be called pretty often I have no idea what kind of pgbench stresstest
could be used to demonstrate the gain (or lack of it).
-Jakub Wartak.
I don't think whit would break a lot of stuff, but I'm kinda dubious
it's a measurable improvement for common workloads ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
er from storing it in a file, or something like that? At that point
it's clear it's up to them to remove the file. So I suggest to keep the
feature as simple as possible - hand the dump over and delete.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Devel
On Thu, Sep 03, 2020 at 05:53:43PM -0700, Jeff Davis wrote:
On Tue, 2020-09-01 at 23:19 +0200, Tomas Vondra wrote:
FWIW any thoughts about the different in temp size compared to
CP_SMALL_TLIST?
Are you referring to results from a while ago? In this thread I don't
see what you're re
statistic the right approach?
5) I doubt it's enough to support relnames - we also need to estimate
joins, so this needs to support plain queries I think. At least that's
what Tom envisioned in his postgres_fdw_support(query text) proposal.
6) I see you've included a
On Fri, Sep 04, 2020 at 11:31:36AM -0700, Jeff Davis wrote:
On Fri, 2020-09-04 at 14:56 +0200, Tomas Vondra wrote:
Those charts show that the CP_SMALL_TLIST resulted in smaller temp
files
(per EXPLAIN ANALYZE the difference is ~25%) and also lower query
durations (also in the ~25% range).
I
costing change is reasonable. It might not go
far enough, but it certainly makes it probably makes it easier to tweak
the rest by changing random_page_cost etc. Not sure if we need/should
tweak the costing to reduce the effect of work_mem (on hashagg).
regards
--
Tomas Vondra
On Mon, Sep 07, 2020 at 01:55:28PM -0700, Jeff Davis wrote:
On Sun, 2020-09-06 at 23:21 +0200, Tomas Vondra wrote:
I've tested the costing changes on the simplified TPC-H query, on two
different machines, and it seems like a clear improvement.
Thank you. Committed.
So yeah, the pa
On Tue, Sep 08, 2020 at 05:55:09PM +0530, Ashutosh Bapat wrote:
On Fri, 4 Sep 2020 at 20:27, Tomas Vondra
wrote
4) I wonder if we actually want/need to simply output pg_statistic data
verbatim like this. Is postgres_fdw actually going to benefit from it? I
kinda doubt that, and my
On Sat, Sep 05, 2020 at 12:05:52PM +1200, Thomas Munro wrote:
On Wed, Sep 2, 2020 at 2:18 AM Tomas Vondra
wrote:
On Wed, Sep 02, 2020 at 02:05:10AM +1200, Thomas Munro wrote:
>On Wed, Sep 2, 2020 at 1:14 AM Tomas Vondra
> wrote:
>> from the archive
>
>Ahh, so perhaps that
Hi,
while looking at the streaming code I noticed two minor issues:
1) logicalrep_read_stream_stop is never defined/called, so the prototype
in logicalproto.h is unnecessary
2) minor typo in one of the comments
Patch attached.
regards
--
Tomas Vondra http://www
ly there is no
argument and we should return false in other cases as well. Basically,
I think we should be consistent in handling the corrupt file case.
FWIW I do agree with this - we should return false here, to make it
return false like in the other data corruption cases. AFAICS that's the
on
On Wed, Sep 09, 2020 at 12:04:28PM -0400, John Naylor wrote:
On Sat, Sep 5, 2020 at 7:21 PM Tomas Vondra
wrote:
OK, here is a rebased version. Most of the breakage was due to changes
to the BRIN sgml docs.
Hi Tomas,
I plan on trying some different queries on different data
distributions to
one, amortizing the serialization costs.
Another option would be to teach add_value to keep the deserialized
summary somewhere, and then force serialization at the end of the BRIN
page range. The end result would be roughly the same, I think.
regards
--
Tomas Vondra http://www.
On Wed, Sep 09, 2020 at 04:53:30PM -0300, Alvaro Herrera wrote:
On 2020-Sep-09, Tomas Vondra wrote:
There are some minor optimizations possible - for example I noticed we
call minmax_multi_get_strategy_procinfo often because it happens in a
loop, and we could easily do it just once. But that
On Wed, Sep 09, 2020 at 10:26:00PM +0200, Tomas Vondra wrote:
On Wed, Sep 09, 2020 at 04:53:30PM -0300, Alvaro Herrera wrote:
On 2020-Sep-09, Tomas Vondra wrote:
There are some minor optimizations possible - for example I noticed we
call minmax_multi_get_strategy_procinfo often because it
breakpoint on the ereport in mdread().
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
a function that "exports" the aggregate
state - similar to serial/deserial functions, but needs to be portable.
I think the trickiest thing here is rewriting the remote query to call
this export function, but maybe we could simply instruct the remote node
to use a different final fun
On 10/15/21 17:05, Alexander Pyhalov wrote:
Tomas Vondra писал 2021-10-15 17:56:
Hi Alexander,
Hi.
And then we should extend this for aggregates with more complex
internal states (e.g. avg), by supporting a function that "exports"
the aggregate state - similar to serial/deserial
block, or something like that.
I'm all for getting security folks and whomever else to come and review
this thread and chime in with their thoughts, but I don't think it's a
reason to hold off on moving forward with the approach that we've been
converging towards.
+1
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
ree
that while using it seems convenient, the consequences of some changes
not incrementing LSN seem potentially disastrous, depending on the
encryption mode.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 10/15/21 21:31, Stephen Frost wrote:
Greetings,
* Tomas Vondra (tomas.von...@enterprisedb.com) wrote:
On 10/15/21 17:05, Alexander Pyhalov wrote:
Tomas Vondra писал 2021-10-15 17:56:
And then we should extend this for aggregates with more complex
internal states (e.g. avg), by supporting
On 10/16/21 16:16, Bruce Momjian wrote:
On Fri, Oct 15, 2021 at 10:57:03PM +0200, Tomas Vondra wrote:
That said, I don't think that's really a huge issue or something that's
a show stopper or a reason to hold off on using XTS. Note that what
those bits actually *are* isn'
processes or whatever?
We already have a way to check integrity by storing page checksum, but
I'm not sure if that's good enough (there's a lot of subtle issues with
building proper AEAD scheme).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
agree.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 10/18/21 17:56, Stephen Frost wrote:
Greetings,
* Tomas Vondra (tomas.von...@enterprisedb.com) wrote:
On 10/15/21 21:22, Stephen Frost wrote:
Now, to address the concern around re-encrypting a block with the same
key+IV but different data and leaking what parts of the page changed, I
do
ity to allow AEAD, only to find out
there are ways around it.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 10/19/21 08:56, Alexander Pyhalov wrote:
Hi.
Tomas Vondra писал 2021-10-15 17:56:
As for the proposed approach, it's probably good enough for the first
version to restrict this to aggregates where the aggregate result is
sufficient, i.e. we don't need any new export/import proced
modified recently, which is a pretty common case.
I think the bigger issue is that people rarely do COUNT(*) on the whole
table. There are usually other conditions and/or GROUP BY, and I'm not
sure how would that work.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 10/20/21 20:33, John Naylor wrote:
On Wed, Oct 20, 2021 at 2:23 PM Tomas Vondra
mailto:tomas.von...@enterprisedb.com>>
wrote:
>
> Couldn't we simply inspect the visibility map, use the index data only
> for fully visible/summarized ranges, and inspect the heap
u either need to explain more clearly what
exactly the adaptive indexes does, or show queries that can't benefit
from these existing features.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
And if you insert a row into the
index, it has to insert the item pointer into each of the small trees,
likely requiring a separate I/O for each.
So I'd bet this is not any different from just having three separate
indexes - it doesn't save space, doesn't save I/O, nothing.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
s also pretty small.
Perhaps I'm entirely wrong and there's a new type of index, better
suited for cases similar to this. The "posting tree" reference actually
made me thinking that maybe btree_gin might be applicable here?
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
401 - 500 of 3960 matches
Mail list logo
