Re: New IndexAM API controlling index vacuum strategies

2021-04-05 Thread Peter Geoghegan
On Mon, Apr 5, 2021 at 4:29 PM Tom Lane wrote: > Peter Geoghegan writes: > > Do you think that it's okay that we rely on the propagation of global > > state to parallel workers on Postgres 13? Don't we need something like > > my fixup commit 49f49def on Postgr

Re: New IndexAM API controlling index vacuum strategies

2021-04-05 Thread Peter Geoghegan
> because most people don't run manual VACUUM with cost limits turned > on. And autovacuum doesn't use parallelism. Oh yeah. "static BufferAccessStrategy vac_strategy" is guaranteed to be initialized to 0, simply because it's static and global. That explains it. -- Peter Geoghegan

Re: New IndexAM API controlling index vacuum strategies

2021-04-05 Thread Peter Geoghegan
On Mon, Apr 5, 2021 at 5:09 PM Peter Geoghegan wrote: > Oh yeah. "static BufferAccessStrategy vac_strategy" is guaranteed to > be initialized to 0, simply because it's static and global. That > explains it. So do we need to allocate a strategy in workers now, or leave

Re: New IndexAM API controlling index vacuum strategies

2021-04-05 Thread Peter Geoghegan
utovacuum_freeze_max_age. How about the following: "... > >VACUUM< will use the higher of this value and 105% of > >guc-autovacuum-freeze-max-age<, so that only ...". It's only slightly > easier to read, but at least it conveys that values lower than 105% of > autovacuum_freeze_max_age are not considered. The same can be said for > the multixact guc documentation. This does need work too. I'm going to push 0002- and 0003- tomorrow morning pacific time. I'll publish a new set of patches tomorrow, once I've finished that up. The last 2 patches will require a lot of focus to get over the line for Postgres 14. -- Peter Geoghegan

Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch

2021-04-05 Thread Peter Eisentraut
On 01.04.21 20:49, Peter Eisentraut wrote: also done I also figured out a way to combine the float8 and numeric implementations so that there is not so much duplication.  Added tests to cover all the edge and overflow cases. I think this is solid now. The extract(julian from timestamp) is

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-04-06 Thread Peter Eisentraut
On 06.04.21 07:24, Japin Li wrote: I think this patch is about ready to commit, but please provide a final version in good time. I took the liberty to address all the review comments and provide a v9 patch on top of Japin's v8 patch-set. (Also, please combine your patches into a single patch.)

Re: psql - add SHOW_ALL_RESULTS option

2021-04-06 Thread Peter Eisentraut
On 14.03.21 10:54, Fabien COELHO wrote: Hello Peter, This reply was two months ago, and nothing has happened, so I have marked the patch as RwF. Given the ongoing work on returning multiple result sets from stored procedures[0], I went to dust off this patch. Based on the feedback, I put

Re: New IndexAM API controlling index vacuum strategies

2021-04-06 Thread Peter Geoghegan
GUCs to reflect the fact that we do everything possible to advance relfrozenxid in the case where the fail safe mechanism kicks in -- not just skipping index vacuuming. It also incorporates your most recent round of feedback. Thanks -- Peter Geoghegan v11-0001-Truncate-line-pointer-arra

Re: SSL SNI

2021-04-07 Thread Peter Eisentraut
On 18.03.21 12:27, Peter Eisentraut wrote: On 25.02.21 19:36, Jacob Champion wrote: On Thu, 2021-02-25 at 17:00 +0100, Peter Eisentraut wrote: Just as additional data points, it has come to my attention that both the Go driver ("lib/pq") and the JDBC environment already send SNI aut

Re: Increase value of OUTER_VAR

2021-04-07 Thread Peter Eisentraut
On 06.03.21 15:59, Tom Lane wrote: Peter Eisentraut writes: On 04.03.21 20:01, Tom Lane wrote: (2) Does that datatype change need to propagate anywhere besides what I touched here? I did not make any effort to search for other places. I think Var.varnosyn CurrentOfExpr.cvarno should

Re: New IndexAM API controlling index vacuum strategies

2021-04-07 Thread Peter Geoghegan
on of the patch that I just pushed. We have run out of time to consider calling PageTruncateLinePointerArray() in more places. I think that the most important thing is that we have *some* protection against line pointer bloat. -- Peter Geoghegan

Re: SQL-standard function body

2021-04-07 Thread Peter Eisentraut
On 03.04.21 05:39, Julien Rouhaud wrote: Are you planning to run pg_indent before committing or would that add too much noise? Yeah, I figured I'd leave that for later, to not bloat the patch so much. Anyway since it's only stylistic issues and the feature freeze is getting closer I'm marking

Re: missing documentation for streaming in-progress transactions

2021-04-07 Thread Peter Smith
streamed transactions)." I think there should be consistent wording used on this page where possible. -- [1] https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html [2] https://www.postgresql.org/docs/devel/protocol-logicalrep-message-formats.html Kind Regards, Peter Smith. Fujitsu Australia

Re: New IndexAM API controlling index vacuum strategies

2021-04-07 Thread Peter Geoghegan
On Wed, Apr 7, 2021 at 8:52 AM Peter Geoghegan wrote: > All of the changes from your fixup patch are clear improvements, and > so I'll include them in the final commit. Thanks! I did change the defaults of the GUCs to 1.6 billion, though. All patches in the patch series have

Re: missing documentation for streaming in-progress transactions

2021-04-07 Thread Peter Smith
On Thu, Apr 8, 2021 at 12:56 PM Amit Kapila wrote: > > On Thu, Apr 8, 2021 at 3:49 AM Peter Smith wrote: > > > > On Wed, Apr 7, 2021 at 10:15 PM Amit Kapila wrote: > > > > > > On Wed, Apr 7, 2021 at 1:11 PM Ajin Cherian wrote: > > > > 3. > &

Re: Support tab completion for upper character inputs in psql

2021-04-08 Thread Peter Eisentraut
On 01.04.21 11:40, tanghy.f...@fujitsu.com wrote: On Wednesday, March 31, 2021 4:05 AM, David Zhang wrote 8 postgres=# update tbl SET DATA = 9 10 postgres=# update TBL SET 11 12 postgres=# So, as you can see the difference is between line 8 and 10 in case 2. It looks like the lowe

Re: Change JOIN tutorial to focus more on explicit joins

2021-04-08 Thread Peter Eisentraut
On 15.03.21 09:06, Jürgen Purtz wrote: +1. All proposed changes integrated. committed

Re: Order dependency in function test

2021-04-08 Thread Peter Eisentraut
On 08.04.21 12:04, Magnus Hagander wrote: Looking at https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2021-04-08%2009%3A43%3A13 which broke with the patch to add pg_wait_backend_termination(). AFAICT the change is that the order of rows coming back from "SELECT routine_name, seq

Re: Dubious coding in nbtinsert.c

2021-04-08 Thread Peter Geoghegan
curitup, while this one is about curitemid. While I have no problem silencing this compiler warning now, I don't see any reason to not just do the same thing again. Which is to initialize the pointer to NULL. -- Peter Geoghegan

Re: [HACKERS] logical decoding of two-phase transactions

2021-04-09 Thread Peter Smith
o you have any objection to me removing them? Otherwise, it might seem strange to document a flag that has no function. -- KInd Regards, Peter Smith. Fujitsu Australia

Postgresql 13 supported on Solaris 11 O/S on SPARC hardware?

2021-04-09 Thread Peter Lee
we looking at the correct website? Thank you very much, -Peter

Re: [HACKERS] logical decoding of two-phase transactions

2021-04-09 Thread Peter Smith
On Fri, Apr 9, 2021 at 6:40 PM Amit Kapila wrote: > > On Fri, Apr 9, 2021 at 12:33 PM Peter Smith wrote: > > > > On Mon, Dec 14, 2020 at 8:27 PM Amit Kapila wrote: > > > > > > 2. > > > + /* > > > + * Flags are determined from the state of th

Re: truncating timestamps on arbitrary intervals

2021-04-09 Thread Peter Eisentraut
On 30.03.21 18:50, John Naylor wrote: On Sat, Mar 27, 2021 at 1:06 PM Justin Pryzby > wrote: > > The current docs seem to be missing a "synopsis", like > > + > +date_trunc(stride, timestamp, origin) > + The attached - adds a synopsis - adds a bit more descrip

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-09 Thread Peter Geoghegan
) -- the failing visibilitymap_clear() call must occur inside a critical section, and all such calls are made within heapam.c (only VACUUM ANALYZE uses a transaction and does writes). It cannot be the two calls to visibilitymap_clear() inside vacuumlazy.c. I suspect that you've figured this much already. Just pointing it out. -- Peter Geoghegan

Re: truncating timestamps on arbitrary intervals

2021-04-10 Thread Peter Eisentraut
On 30.03.21 18:06, John Naylor wrote: Currently, when the origin is after the input, the result is the timestamp at the end of the bin, rather than the beginning as expected. The attached puts the result consistently at the beginning of the bin. In the patch + if (origin > timestamp && stri

Re: truncating timestamps on arbitrary intervals

2021-04-10 Thread Peter Eisentraut
On 10.04.21 14:53, John Naylor wrote: On Sat, Apr 10, 2021 at 7:43 AM Peter Eisentraut <mailto:peter.eisentr...@enterprisedb.com>> wrote: > > On 30.03.21 18:06, John Naylor wrote: > > Currently, when the origin is after the input, the result is the > > time

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-11 Thread Peter Geoghegan
flag won't change underneath it, and if it did, it's clear how this > symptom would ensue. Does this patch seem to fix the problem? -- Peter Geoghegan 0001-Acquire-super-exclusive-lock-in-lazy_vacuum_heap_rel.patch Description: Binary data

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-11 Thread Peter Geoghegan
r a long time. How about temporarily committing this patch, just to review how it affects the buildfarm? -- Peter Geoghegan

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-11 Thread Peter Geoghegan
On Sun, Apr 11, 2021 at 9:10 AM Peter Geoghegan wrote: > I don't have any reason to believe that using a super-exclusive lock > during heap page vacuuming is necessary. My guess is that returning to > doing it that way might make the buildfarm green again. That would at >

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-11 Thread Peter Geoghegan
to maintain safely. We need to simplify it. It is way too complicated. I don't think that I quite understand your first proposal right now, so I'll need to go think about it. -- Peter Geoghegan

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-11 Thread Peter Geoghegan
rcised from vacuumlazy.c (in the same commit that stopped using a superexclusive lock). It was also described as an optimization that wasn't quite worth it. But I don't quite buy that. ISTM that there is a better explanation: it evolved the appearance of being an optimization that might make sense. Which was just camouflage. -- Peter Geoghegan

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-12 Thread Peter Geoghegan
ttps://postgr.es/m/capphfdtseohx8dsk9qp+z++i4bgqoffkip6jdwngea+g7z-...@mail.gmail.com -- Peter Geoghegan

Teaching users how they can get the most out of HOT in Postgres 14

2021-04-12 Thread Peter Geoghegan
new documentation should address. The documentation would still have plenty to say about HOT, though. It would also have something to say about bottom-up index deletion, which can be thought of as avoiding problems when HOT doesn't or can't be applied very often. -- Peter Geoghegan

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-04-12 Thread Peter Geoghegan
, since it's likely to change in Postgres 15 and Postgres 16 anyway. -- Peter Geoghegan

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-04-12 Thread Peter Geoghegan
ns that come with the feature (?): > - A threshold, as an integer, to define a number of pages. > - A scale factor to define a percentage of pages. Why? -- Peter Geoghegan

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-04-12 Thread Peter Geoghegan
ootgun. It might appear to help at first, but risks destabilizing things much later on. -- Peter Geoghegan

Re: PANIC: wrong buffer passed to visibilitymap_clear

2021-04-12 Thread Peter Geoghegan
es, then we must also assume that your fix has negligible risk in the backbranches, because of how it is structured. And so I agree -- I lean towards backpatching. -- Peter Geoghegan

PG Docs - logical decoding output plugins - fix typo

2021-04-12 Thread Peter Smith
PSA a patch to fix a typo found on this page [1], "preapre_end_lsn" -> "prepare_end_lsn" -- [1] https://www.postgresql.org/docs/devel/logicaldecoding-output-plugin.html Kind Regards, Peter Smith. Fujitsu Australia fix_typo.patch Description: Binary data

Re: New IndexAM API controlling index vacuum strategies

2021-04-13 Thread Peter Geoghegan
tion is exactly the kind of thing that risks adding significant, unpredictable delay at a time when we need to advance relfrozenxid as quickly as possible. I pushed a trivial commit that makes the failsafe bypass heap truncation as well just now. Thanks -- Peter Geoghegan

Re: Improving psql slash usage help message

2020-07-21 Thread Peter Eisentraut
all indexes" is easily answered by the help output. Under the other scheme, it's hidden behind a layer of metasyntax. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Using Valgrind to detect faulty buffer accesses (no pin or buffer content lock held)

2020-07-21 Thread Peter Geoghegan
uiring a pin or a lock is made more clear. The added commentary is > descriptive and appears grammatically correct, at least to a non native > speaker. I didn't see this review until now because it ended up in gmail's spam folder. :-( Thanks for taking a look at it! -- Peter Geoghegan

Re: Using Valgrind to detect faulty buffer accesses (no pin or buffer content lock held)

2020-07-21 Thread Peter Geoghegan
On Fri, Jul 17, 2020 at 5:53 PM Peter Geoghegan wrote: > Pushed the first patch just now, and intend to push the other one soon. > Thanks! Pushed the second piece of this (the nbtree patch) just now. Thanks for the review! -- Peter Geoghegan

Re: Index Skip Scan (new UniqueKeys)

2020-07-21 Thread Peter Geoghegan
t_lockbuf() wrapper function instead, so that the new Valgrind instrumentation is used. This shouldn't be hard. Perhaps you can use Valgrind to verify that this patch doesn't have any unsafe buffer accesses. I recall problems like that in earlier versions of the patch series. Valgrind should be able to detect most bugs like that (though see comments within _bt_lockbuf() for details of a bug in this area that Valgrind cannot detect even now). -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-22 Thread Peter Geoghegan
On Tue, Jul 21, 2020 at 1:30 PM Bruce Momjian wrote: > On Tue, Jul 14, 2020 at 03:49:40PM -0700, Peter Geoghegan wrote: > > Maybe I missed your point here. The problem is not so much that we'll > > get HashAggs that spill -- there is nothing intrinsically wrong with > >

Re: Default setting for enable_hashagg_disk

2020-07-23 Thread Peter Geoghegan
ting to hear back from Tomas about that. Tomas? Thanks -- Peter Geoghegan v3-0001-Remove-hashagg_avoid_disk_plan-GUC.patch Description: Binary data v3-0002-Add-hash_mem_multiplier-GUC.patch Description: Binary data

Re: Default setting for enable_hashagg_disk

2020-07-23 Thread Peter Geoghegan
_TLIST. That does make it sound like the costs of the hash agg aren't being represented. I suppose it isn't clear if this is a costing issue because it isn't clear if the execution time performance itself is pathological or is instead something that must be accepted as the cost of spilling the hash agg in a general kind of way. -- Peter Geoghegan

Re: heap_abort_speculative() sets xmin to Invalid* without HEAP_XMIN_INVALID

2020-07-23 Thread Peter Geoghegan
ere > explaining where such tuples would come from. There could be an opportunity to put this on a formal footing by doing something in the amcheck heap checker patch. -- Peter Geoghegan

Re: renaming configure.in to configure.ac

2020-07-24 Thread Peter Eisentraut
On 2020-07-17 10:46, Peter Eisentraut wrote: v1-0001-Rename-configure.in-to-configure.ac.patch I have committed that, and I have sent feedback to the Autoconf developers about our concerns about the slowness of some of the new tests. -- Peter Eisentraut http://www

Re: Default setting for enable_hashagg_disk

2020-07-24 Thread Peter Geoghegan
lot of wasted I/O from repeated spilling of the same tuples. Too + * many will result in lots of memory wasted buffering the spill files (which + * could instead be spent on a larger hash table). + */ +#define HASHAGG_PARTITION_FACTOR 1.50 +#define HASHAGG_MIN_PARTITIONS 4 +#define HASHAGG_MAX_PARTITIONS 1024 -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-24 Thread Peter Geoghegan
isticated cost model for this in cost_agg(). Maybe the "pages_written" calculation could be pessimized. However, it doesn't seem like this is precisely an issue with I/O costs. -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-24 Thread Peter Geoghegan
agg might write out much less data than the total memory used for the equivalent "peak optimal nospill" hash agg case -- or much more. (Again, reiterating what I said in my last e-mail.) -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
On Fri, Jul 24, 2020 at 12:55 PM Peter Geoghegan wrote: > Could that be caused by clustering in the data? > > If the input data is in totally random order then we have a good > chance of never having to spill skewed "common" values. That is, we're > bound to

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
On Sat, Jul 25, 2020 at 9:39 AM Peter Geoghegan wrote: > "Peak Memory Usage: 1605334kB" > > Hash agg avoids spilling entirely (so the planner gets it right this > time around). It even uses notably less memory. I guess that this is because the reported memory usage doesn&#x

Re: Difference for Binary format vs Text format for client-server communication

2020-07-25 Thread Peter Eisentraut
and possibly make changes there. By the time the type's output or send function is invoked, that's already decided. The output/send functions are not the place to make scale or other semantic adjustments. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Deve

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
20MB work_mem to complete without spilling at all -- so it's kind of extreme, though not quite as extreme as many of the similar test results from Tomas.) -- Peter Geoghegan

Re: hashagg slowdown due to spill changes

2020-07-25 Thread Peter Geoghegan
t affects simple in-memory hash aggregation, though.) -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
On Sat, Jul 25, 2020 at 1:10 PM Jeff Davis wrote: > On Sat, 2020-07-25 at 11:05 -0700, Peter Geoghegan wrote: > > What worries me a bit is the sharp discontinuities when spilling with > > significantly less work_mem than the "optimal" amount. For example, > > with

Re: hashagg slowdown due to spill changes

2020-07-25 Thread Peter Geoghegan
On Sat, Jul 25, 2020 at 12:41 PM Peter Geoghegan wrote: > I have added a new open item for this separate > LookupTupleHashEntryHash()/lookup_hash_entry() pipeline-stall issue. Attached is a rebased version of Andres' now-bitrot 2020-06-12 patch ("aggspeed.diff"). I find

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
e it doesn't report this excess), or the random case really doesn't exceed work_mem but has a surprising advantage over the sorted case. -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
ter principled approach is possible. It's hard to judge how much of a problem this really is, though. We'll need to think about this aspect some more. Thanks -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-25 Thread Peter Geoghegan
On Sat, Jul 25, 2020 at 5:31 PM Peter Geoghegan wrote: > I'm glad that this better principled approach is possible. It's hard > to judge how much of a problem this really is, though. We'll need to > think about this aspect some more. BTW, your HLL patch ameliorates the

Re: hashagg slowdown due to spill changes

2020-07-27 Thread Peter Geoghegan
On Sun, Jul 26, 2020 at 4:17 PM Jeff Davis wrote: > I saw less of an improvement than you or Andres (perhaps just more > noise). But given that both you and Andres are reporting a measurable > improvement, then I went ahead and committed it. Thank you. Thanks! -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Peter Geoghegan
. The memory usage is reported on accurately by EXPLAIN ANALYZE. -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Peter Geoghegan
On Mon, Jul 27, 2020 at 10:30 AM Alvaro Herrera wrote: > On 2020-Jul-23, Peter Geoghegan wrote: > I notice you put the prototype for get_hash_mem in nodeHash.h. This > would be fine if not for the fact that optimizer needs to call the > function too, which means now optimizer hav

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Peter Geoghegan
the attached revision. No other changes. The v4-0001-Remove-hashagg_avoid_disk_plan-GUC.patch changes are surprisingly complicated. It would be nice if you could take a look at that aspect (or confirm that it's included in your review). -- Peter Geoghegan v4-0001-Remove-hashagg

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Peter Geoghegan
On Mon, Jul 27, 2020 at 12:52 PM Alvaro Herrera wrote: > On 2020-Jul-27, Peter Geoghegan wrote: > > The v4-0001-Remove-hashagg_avoid_disk_plan-GUC.patch changes are > > surprisingly complicated. It would be nice if you could take a look at > > that aspect (or confirm that

Re: Default setting for enable_hashagg_disk

2020-07-27 Thread Peter Geoghegan
nse. > Anyway, the patch looks good to me. We can have a separate discussion > about pessimizing the costing, if necessary. Pushed the hashagg_avoid_disk_plan patch -- thanks! -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-27 Thread Peter Geoghegan
er on the incremental sort thread, so I thought I'd ask again here, while I was reminded of that issue.) -- Peter Geoghegan

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
h is the kind of thing that could only really happen inside tuplesort.c. This is clear because some of the variables have the tell-tale 0x7f7f7f pattern that we written by CLOBBER_FREED_MEMORY builds when memory is freed. -- Peter Geoghegan

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 8:47 AM Peter Geoghegan wrote: > This is very likely to be related to incremental sort because it's a > use-after-free issue, which is the kind of thing that could only > really happen inside tuplesort.c. This is clear because some of the > variables h

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 10:06 AM Peter Geoghegan wrote: > I wrote the assertion that fails here with the bug that I fixed in > commit 4974d7f87e62a58e80c6524e49677cb25cc10e12 in mind specifically. > That was a bug that involved a scan that returned duplicate tuples due > to

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 10:37 AM Peter Geoghegan wrote: > It's starting to look more like that. I can reproduce the bug by > running the REINDEX in a tight loop while "make installcheck" runs. It > looks as if the two tuples passed to comparetup_index_btree() are > sepa

Re: [BUG] Error in BRIN summarization

2020-07-28 Thread Peter Geoghegan
7nga6e...@mail.gmail.com We see a violation of the HOT invariant in this case, though only for a system catalog index, and only in fairly particular circumstances involving concurrency. -- Peter Geoghegan

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 12:00 PM Peter Geoghegan wrote: > I still don't know what's going on here, but I find it suspicious that > some relevant tuples go through the HEAPTUPLE_INSERT_IN_PROGRESS + > !TransactionIdIsCurrentTransactionId() path of > heapam_index_build_range

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
th (i.e. the path that uses the root_offsets array). I notice that the root tuple of the hot chain is marked HEAP_COMBOCID (and xmin == xmax for the HOT chain tuple). The xmin for the successor (which matches xmin and xmax for root tuple) exactly matches the REINDEX/crashing session's OldestXmin. -- Peter Geoghegan

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 1:26 PM Peter Geoghegan wrote: > On Tue, Jul 28, 2020 at 1:04 PM Tom Lane wrote: > > No, I don't think so. It was designed for the case of unique key X > > being inserted immediately after a deletion of the same key. The > > deleted tuple is p

Re: 13dev failed assert: comparetup_index_btree(): ItemPointer values should never be equal

2020-07-28 Thread Peter Geoghegan
On Tue, Jul 28, 2020 at 2:46 PM Peter Geoghegan wrote: > The fact remains that this function (originally known as > IndexBuildHeapScan(), now heapam_index_build_range_scan()) did not > care about whether or not the index is unique for about 3 years > (excluding the tupleIsAlive stuf

Re: PG 13 release notes, first draft

2020-07-29 Thread Peter Geoghegan
documentation link for the GUC would be very helpful. Thanks -- Peter Geoghegan

Re: HyperLogLog.c and pg_leftmost_one_pos32()

2020-07-29 Thread Peter Geoghegan
e that it's worth backpatching to 13 now. Assuming it is a really measurable cost in practice. -- Peter Geoghegan

Re: Default setting for enable_hashagg_disk

2020-07-29 Thread Peter Geoghegan
On Mon, Jul 27, 2020 at 5:55 PM Peter Geoghegan wrote: > Pushed the hashagg_avoid_disk_plan patch -- thanks! Pushed the hash_mem_multiplier patch as well -- thanks again! As I've said before, I am not totally opposed to adding a true escape hatch. That has not proven truly necessary

Re: PG 13 release notes, first draft

2020-07-29 Thread Peter Geoghegan
f hash_mem_multiplier. This allows hash aggregation to use more memory without affecting competing query operations that are generally less likely to put any additional memory to good use. -- Peter Geoghegan

Re: PG 13 release notes, first draft

2020-07-29 Thread Peter Geoghegan
t it next to the hash agg item itself. It's more likely to be noticed there, and highlighting it a little seems warranted. OTOH, this may not be a problem at all for many individual users. Framing it as a tip rather than a compatibility item gets that across. -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-29 Thread Peter Geoghegan
), which can be either memory or disk space. Technically they don't violate the letter of the law that you cite. (Though admittedly this is a legalistic loophole -- the "space" value presumably has to be interpreted according to different rules for programs that consume non-text EXPLAIN output.) Have I missed something? -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1.

2020-07-29 Thread Peter Geoghegan
On Wed, Jul 29, 2020 at 9:05 PM Justin Pryzby wrote: > So my 2ndary suggestion is to conditionalize based on the method rather than > value of space used. What's the advantage of doing it that way? -- Peter Geoghegan

Re: PG 13 release notes, first draft

2020-07-30 Thread Peter Geoghegan
On Thu, Jul 30, 2020 at 10:32 AM Bruce Momjian wrote: > I came up with a more verbose documentation suggestion, attached. I'm okay with this. Thanks -- Peter Geoghegan

logtape.c stats don't account for unused "prefetched" block numbers

2020-07-30 Thread Peter Geoghegan
"sparse allocation". Although maybe the preallocation stuff should somehow be rolled into the much older nHoleBlocks stuff. Unsure. -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1. (now: incremental sort)

2020-07-30 Thread Peter Geoghegan
something that we have to live with for the foreseeable future. I don't think that this is a bad example -- we don't output maxDiskSpaceUsed or maxMemorySpaceUsed at the conceptual level. -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1. (now: incremental sort)

2020-07-30 Thread Peter Geoghegan
ccess to the sort, and it happens to spill -- we can randomly access the final tape, but cannot do a final on-the-fly merge. Say for a merge join. -- Peter Geoghegan

Re: HashAgg's batching counter starts at 0, but Hash's starts at 1. (now: incremental sort)

2020-07-30 Thread Peter Geoghegan
On Thu, Jul 30, 2020 at 6:40 PM Justin Pryzby wrote: > Feel free to close it out. I'm satisfied that we've had a discussion about > it. Closed it out. -- Peter Geoghegan

Re: Concurrency bug in amcheck

2020-07-31 Thread Peter Geoghegan
On Wed, May 13, 2020 at 4:06 PM Peter Geoghegan wrote: > On Mon, May 11, 2020 at 5:56 AM Alexander Korotkov > wrote: > > Thank you. 2nd patch is proposed for master and makes btree page > > unlink remove all the items from the page being deleted. > > This looks

Re: [PATCH] Btree BackwardScan race condition on Standby during VACUUM

2020-08-01 Thread Peter Geoghegan
ch -- I have only made superficial adjuments. I think that I will be able to commit this next week, barring objections. -- Peter Geoghegan v3-0001-Avoid-backwards-scan-page-deletion-standby-race.patch Description: Binary data

Re: new heapcheck contrib module

2020-08-02 Thread Peter Geoghegan
the table or other indexes) at the first sign of trouble, anyway? It makes sense for the heap structure, but not for indexes. -- Peter Geoghegan

Re: new heapcheck contrib module

2020-08-02 Thread Peter Geoghegan
On Thu, Jul 30, 2020 at 10:59 AM Robert Haas wrote: > I don't really like the name, either. I get that it's probably > inspired by Perl, but I think it should be given a less-clever name > like report_corruption() or something. +1 -- confess() is an awful name for this. -- Peter Geoghegan

Replace remaining StrNCpy() by strlcpy()

2020-08-03 Thread Peter Eisentraut
I propose to replace the remaining uses of StrNCpy() with strlcpy() and remove the former. It's clear that strlcpy() has won the popularity contest, and the presence of the former is just confusing now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: Confusing behavior of create table like

2020-08-03 Thread Peter Eisentraut
x27;s why identity columns were added. You shouldn't use serial columns anymore, especially if you are concerned about behaviors like this. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Confusing behavior of create table like

2020-08-03 Thread Peter Eisentraut
nding effort on patching them up. One thing we could do is change serial/bigserial to expand to identity column definitions instead of the current behavior. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: new heapcheck contrib module

2020-08-03 Thread Peter Geoghegan
uring index verification? That's what it would take to do this in the general case, I guess. More fundamentally, I wonder how many inconsistencies one should imagine that this index has, before we even get into talking about the implementation. -- Peter Geoghegan

Re: Replace remaining StrNCpy() by strlcpy()

2020-08-03 Thread Peter Eisentraut
an to be zeroing the whole destination buffer. That's easy to fix, but it's perhaps wondering briefly why it needs to be zero-padded. hashname() doesn't care, heap_form_tuple() doesn't care. Does anything care? While we're here, shouldn't namestrcpy() do so

Re: public schema default ACL

2020-08-03 Thread Peter Eisentraut
ferently, you can set that up. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: PG 13 release notes, first draft

2020-08-03 Thread Peter Geoghegan
On Thu, Jul 30, 2020 at 10:45 AM Peter Geoghegan wrote: > On Thu, Jul 30, 2020 at 10:32 AM Bruce Momjian wrote: > > I came up with a more verbose documentation suggestion, attached. > > I'm okay with this. Are you going to push this soon, Bruce? -- Peter Geoghegan

<    6   7   8   9   10   11   12   13   14   15   >